Update Model version

[SharonC66]

Over the past 12 months, I have developed several bertopic models. Now I am facing issues with version compatibility.
For instance, I can't load the older version model(0.15.0) with new version(0.16.0).
My solution was to used old model's representation and created new model using guided model.
The question is if this is the right an approach? is there any better way to do it?

[MaartenGr]

I would advise using manual topic modeling instead. That allows you to use the topics you have created previously topic_model.topics_ and use those directly as the input for y. It will give you the same topic model with only the underlining dimensionality reduction and clustering algorithms removed.

So essentially do this:

from bertopic import BERTopic
from bertopic.cluster import BaseCluster
from bertopic.vectorizers import ClassTfidfTransformer
from bertopic.dimensionality import BaseDimensionalityReduction

# Prepare our empty sub-models and reduce frequent words while we are at it.
empty_dimensionality_model = BaseDimensionalityReduction()
empty_cluster_model = BaseCluster()

# Fit BERTopic without actually performing any clustering
topic_model= BERTopic(
        embedding_model=MY_EMBEDDING_MODEL,
        umap_model=empty_dimensionality_model,
        hdbscan_model=empty_cluster_model,
)
topics, probs = topic_model.fit_transform(docs, y=MY_PREVIOUS_TOPICS)

[SharonC66]

thanks, MaartenGr. That was my original solution. Then I encountered the problem with matching top 10 representations. Since the previous model had been established and used by users, the representations have to be exact the same. After ran the manual topic modeling, I found it has only 90% similarity. Is there any way to feed or modify each representation in each topic?

[SharonC66]

Also, May I ask how to transform new data if the manual model has dimensionality reduction and clustering algorithms removed?

[MaartenGr]

thanks, MaartenGr. That was my original solution. Then I encountered the problem with matching top 10 representations. Since the previous model had been established and used by users, the representations have to be exact the same. After ran the manual topic modeling, I found it has only 90% similarity. Is there any way to feed or modify each representation in each topic?

That might be a bit more difficult but what you could do is take the representation of each topic and assign them as custom labels. That way, you can get the original representations and use them as custom labels.

If you want the exact same keywords per topic, then it becomes a bit more tricky. You would have to save the original topic_model.topic_representations_ (v0.15) to disk and then assign them to your newly trained model using new_model.topic_representations_ = MY_LOADED_REPRESENTATIONS (v0.16 trained with manual topic modeling).