most frequent words sometimes don't appear in the topic representation

[avafor]

Hi,
I have noticed sometimes the most frequesnt word which has appeared almost in each member of one topic does not appear in the topic representation. I understood that the topic representation shows the most frequent words in the topic which is a bit confusing in this case. Is there any parameter to change to fix that?
Thank you very much!

[MaartenGr]

Apologies for the late reply! Topic representations actually do not necessarily show the most frequent words in a topic. The most frequent words in most topics are often stopwords. Those are words like the, and, and I. These stopwords are removed by using the c-TF-IDF approach by considering which words contribute most to a topic.

Now I am not entirely sure but I think you might be referring to frequent words that are not stopwords. In that case, if a word appears frequently across most documents, perhaps even across multiple topics, then it might also be considered as a word that does not contribute much to a topic compared to some of the other words.

Lastly, it might be worthwhile to provide a minimal example of what you are running into. It would greatly help in identifying anything that might not be explained with the above.

[avafor]

Thank you very much for your respond.
Exactly, I was referring to non stop words not showing up in the topic names.
Unfortunately, the data is confidentioal and I am not allowed to share examples. Howevert, after playing around with the parameters, I realized with my data it makes sense to first remove stop words and then use the sentence transforermers as the embedding. This shows better results both in clustering as well as the topic names.

[MaartenGr]

Great. Glad to hear that this improved the creation of the topic names!