[q] Cohere async? Input should be a valid string [type=string_type, input_value=[cohere.Generation {

[llermaly]

Hi , what's the best way to run Guard using cohere async mode?

I tried this with no luck:

async def generate_response(prompt_template: str, prompt_params: dict, guard_model: BaseModel, model_name: str = 'command', max_tokens: int = 1024, temperature: float = 0.0):
    async with cohere.AsyncClient(api_key=os.getenv("COHERE_API_KEY")) as co:
        guard = gd.Guard.from_pydantic(guard_model, prompt=prompt_template)
        response = await guard(
            co.generate,
            prompt_params=prompt_params,
            model=model_name,
            max_tokens=max_tokens,
            temperature=temperature
        )
        return response

This was the error I had:

ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/Users/gustavollermalylarrain/Documents/proyectos/labs/test-cohere/backend-python/.venv/lib/python3.11/site-packages/guardrails/llm_providers.py", line 542, in call
result = await self.invoke_llm(
^^^^^^^^^^^^^^^^^^^^^^
File "/Users/gustavollermalylarrain/Documents/proyectos/labs/test-cohere/backend-python/.venv/lib/python3.11/site-packages/guardrails/llm_providers.py", line 709, in invoke_llm
return LLMResponse(
^^^^^^^^^^^^
File "/Users/gustavollermalylarrain/Documents/proyectos/labs/test-cohere/backend-python/.venv/lib/python3.11/site-packages/pydantic/main.py", line 164, in init
pydantic_self.pydantic_validator.validate_python(data, self_instance=pydantic_self)
pydantic_core._pydantic_core.ValidationError: 1 validation error for LLMResponse
output
Input should be a valid string [type=string_type, input_value=[cohere.Generation {
id:...ken_likelihoods: None
}], input_type=Generations]
For further information visit https://errors.pydantic.dev/2.4/v/string_type

I tried doing this:

async def generate_response(prompt_template: str, prompt_params: dict, guard_model: BaseModel, model_name: str = 'command', max_tokens: int = 1024, temperature: float = 0.0):
    async with cohere.AsyncClient(api_key=os.getenv("COHERE_API_KEY")) as co:
        guard = gd.Guard.from_pydantic(guard_model, prompt=prompt_template)

        async def cohere_generate_wrapper(prompt: str, **kwargs) -> str:
            response = await co.generate(prompt=prompt, **kwargs)
            return response.generations[0].text

        response = await guard(
            cohere_generate_wrapper,
            prompt_params=prompt_params,
            model=model_name,
            max_tokens=max_tokens,
            temperature=temperature
        )
        return response

And works, but I wonder if I'm doing it the right way

Thanks

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 14 days.

[github-actions]

This issue was closed because it has been stalled for 14 days with no activity.