is it possible to run topic representation only after reduction?

[yudelevi]

Hi,

I'm running BertTopic with no_topics="auto"

topic_model = BERTopic(
        embedding_model=embedding_model,
        low_memory=True,
        representation_model=epresentation_model,
        min_topic_size=TOPIC_SIZE,
        top_n_words=20,
        nr_topics="auto",
        verbose=True,
        vectorizer_model=CountVectorizer(stop_words="english"),
        umap_model=umap_model,
    )

and it seems like representation is happening twice:

2024-07-15 21:29:24,956 - BERTopic - Dimensionality - Completed ✓
2024-07-15 21:29:24,959 - BERTopic - Cluster - Start clustering the reduced embeddings
2024-07-15 21:29:25,192 - BERTopic - Cluster - Completed ✓
2024-07-15 21:29:25,192 - BERTopic - Representation - Extracting topics from clusters using representation models.
100%|██████████| 72/72 [00:33<00:00,  2.14it/s]
2024-07-15 21:30:01,202 - BERTopic - Representation - Completed ✓
2024-07-15 21:30:01,208 - BERTopic - Topic reduction - Reducing number of topics
100%|██████████| 62/62 [00:32<00:00,  1.91it/s]
2024-07-15 21:30:35,725 - BERTopic - Topic reduction - Reduced number of topics from 72 to 62

Is it possible to trigger representation will only after topic reduction (should save some time )

Thanks!

[MaartenGr]

Sure, you can do it as follows. First, make sure that during .fit or .fit_transform you do not use the representation_model parameter, like so:

topic_model = BERTopic(
        embedding_model=embedding_model,
        low_memory=True,
        min_topic_size=TOPIC_SIZE,
        top_n_words=20,
        nr_topics="auto",
        verbose=True,
        vectorizer_model=CountVectorizer(stop_words="english"),
        umap_model=umap_model,
    ).fit(MY_DOCS)

Then, you can update the model with the representation model(s) like so:

topic_model.update_topics(
  MY_DOCS, 
  representation_model=representation_model,
  vectorizer_model=CountVectorizer(stop_words="english")
)

[yudelevi]

Thank you!