Access LLM [Keywords] (or just use multi-aspect approach)

[thundergore]

I've made progress with a model that is using openAI representation.

Previously I've found some use in presenting the default representation (high ctfidf scoring words in the topic clusters if I understand correctly) as 'keywords' rather than a topic name.

I'm wondering - if I want to do both - can I access the [keywords] passed to openAI in the prompt? Or do I need to run a multi-aspect approach, generating those representation terms and then separately getting an openAI label?

To me it feels like it must be a quite similar list (if I don't intend to play too much with the representation that's generating a 'default' list of terms from the document clusters).

[MaartenGr]

I'm wondering - if I want to do both - can I access the [keywords] passed to openAI in the prompt? Or do I need to run a multi-aspect approach, generating those representation terms and then separately getting an openAI label?

You can access the keywords passed to the OpenAI prompt by using the .prompts_ variable of your OpenAI representation model. Another option would indeed be to run the representation seperately to more easily access the keywords that are passed to the OpenAI representation. The only thing you would have to do is something like this:

representation_model = {"OpenAI": OpenAI(client, model="gpt-3.5-turbo", chat=True)}

and pass that to BERTopic.