Predefined Topics

[theobouwman]

We observe that Events created in our platform can be classified as Career, Tech, Cultural or Talks when we look at the created topics.

Is it possible to predefine these topics, and let BERTopic classify all documents, existing and new, to these predefined topics?

[MaartenGr]

Yes, you can! If you use zero-shot topic modeling, you can specify these topics and then set the minimum similarity to 0. That will in turn classify all documents with those predefined topics.

[theobouwman]

How specific should these topics be defined. Something like this:

zeroshot_topic_list = [
    "Career opportunities job employer student", 
    "Digital ICT Technology AI Robot",
    "Politics International Relations Talks Lectures Professor University",
    "Culture Arts Performance Geographer Dance Theater Festival Performances",
]

Or use topcis like "Career Jobs", "Digitial ICT AI", "Politics Geo...", "Cultural"?

Also with this code and the latest bertopic version:

umap_model = UMAP(n_neighbors=15, n_components=5, min_dist=0.0, metric='cosine', random_state=42)


topic_model = BERTopic(
    embedding_model="thenlper/gte-small", 
    zeroshot_topic_list=zeroshot_topic_list,
    zeroshot_min_similarity=0,
    representation_model=KeyBERTInspired(),
    umap_model=umap_model # reproducability
)

topics, _ = topic_model.fit_transform(docs_en)

freq = topic_model.get_topic_info()

we get:

The Name column is not set to the topic, and the order is mixed up.

And last question. The Representation column, are that the most common words per topic? If so, is it possible to get this on a per document basis? I am asking this because I have also experimented. with KeyBERT to get keyworks per document, but maybe it is more convenient to use the BERTopic library for this as well.

Thanks

[MaartenGr]

Or use topcis like "Career Jobs", "Digitial ICT AI", "Politics Geo...", "Cultural"?

Either work but note that these texts are embedded, so providing additional detail will help but might make the resulting embedding more specific as a result. It's up to you to decide how abstract/specific you would want a topic to be.

Also with this code and the latest bertopic version:

Which BERTopic version are you using exactly? There was a fix in v0.16.4 and I need to be sure you are using that one.

And last question. The Representation column, are that the most common words per topic? If so, is it possible to get this on a per document basis? I am asking this because I have also experimented. with KeyBERT to get keyworks per document, but maybe it is more convenient to use the BERTopic library for this as well.

Those are not the most common words per topic but the words extracted with KeyBERTInspired. To do this on a per-document level, you would indeed have to use the separate KeyBERT package since BERTopic focus on groups of documents rather than individual documents.