Is there any model or method to do topic modelling/discovery?

[asharm0662]

Hi,

Is there model or method I can be referred to within haystack that allows topic modelling/discovery?

What I mean by topic modelling/discovery:

1. topic modelling/discovery - unsupervised approach to discovering key themes in a document. For example if the topic in question has the following words 'bees, honey, flowers, water' , the model will give you back that topic is about bees near flower and water.

Thank you in advance.

[julian-risch]

Hi @asharm0662 not directly topic modeling but maybe still helpful for your use case is zero-shot classification of documents. Have a look here:

haystack/haystack/nodes/document_classifier/transformers.py

Line 83 in 00aa1f4

Available models for the task of zero-shot-classification include:

You could use zero shot classification models to group your documents by classes.

Apart from that, you could use an EmbeddingRetriever to calculate an embedding for every document in your document store. After that you could manually loop through all the documents and cluster them by their embeddings. The downside is that the clusters (topics) won't have any label -- just as with any other topic modeling approach, which is why I would recommend the zero-shot classification.

[asharm0662]

thank you so much @julian-risch for your input this helps!