chore: Refactor chat completion creation to a separate method #1197
Conversation
🦋 Changeset detectedLatest commit: 4ee71a2 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
|
| @@ -626,6 +628,16 @@ def create_azure_client( | |||
| tool_choice=tool_choice, | |||
| ) | |||
|
|
|||
| @wraps(AsyncCompletions.create) | |||
| def create_chat_completions_stream(self, **kwargs): | |||
There was a problem hiding this comment.
Is the goal to be able to edit the parameters of the LLM inside the VoicePipelineAgent?
There was a problem hiding this comment.
in general, yes
we'd like to inject custom values to the corresponding openai API in the descendant class, which are either not implemented in the agent for now or that contain customized logic, that would probably not make much sense for this particular (generic) agent implementation
There was a problem hiding this comment.
Could the before_llm_cb callback inside PipelineAgent do the work? You can return an LLMStream from there (so you can use chat with any parameter you want)
There was a problem hiding this comment.
I assume with the recent changes in main this PR becomes obsolete.
The original intention was to customize parameters passed to the OpenAI completion creation API rather than these that the chat one exposes.
For now I think it would make sense to close this PR since we anyway override _main_task (now called _run) of the llm.LLMStream class
There was a problem hiding this comment.
Interesting, I think this PR may solve this too
There was a problem hiding this comment.
most likely it won't because
_extra_optionscannot override existing arguments- We need the change to be done for the openai, while the above one is done to anthropic
This allows to:
chatmethod