All the user to specify the stream
argument of Agent._run_model
somewhere
#371
Labels
enhancement
Enhance an existing feature
Enhancement Description
The default orchestrator calls Agent._run_model without a streaming argument: https://github.com/PrefectHQ/ControlFlow/blob/main/src/controlflow/orchestration/orchestrator.py#L375.
In effect, the default of True is always used unless the user constructs a new Orchestrator.
Use Case
Users may not be able to use a specific LLM API in streaming mode, but may have authorization to it otherwise.
Proposed Implementation
A
stream
parameter can be passed to the agent at construction and set on the object, and thenAgent._run_model
function could then use then use that value. Arguably whether or not an agent should be used in streaming mode is an Agent-level decision rather than a call-level decision, so it may make sense to specify it at construction rather than in the function that triggers that LLM call.https://github.com/PrefectHQ/ControlFlow/blob/main/src/controlflow/agents/agent.py#L284
The text was updated successfully, but these errors were encountered: