AzureChatOpenAI Assumes a default temperature argument which is incompatible with OpenAI o1 API Call

[SidGurajala]

Checked other resources

• I added a very descriptive title to this issue.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

Please update this with your OpenAI keys from Azure, and a model version in your own deployment for o1-full, not mini or preview.

input = HumanMessage("Hello!")
deployment_name = "o1"

  model = AzureChatOpenAI(
      api_key=AZURE_OPENAI_API_KEY,
      openai_api_version=AZURE_OPENAI_API_VERSION,  
      deployment_name=deployment_name,  
      model_version=model_version,
      streaming=streaming,
 )

response = await model.ainvoke(input)

Error Message and Stack Trace (if applicable)

python3.11/site-packages/langchain_core/language_models/chat_models.py:307, in BaseChatModel.ainvoke(self, input, config, stop, **kwargs)
298 async def ainvoke(
299 self,
300 input: LanguageModelInput,
(...)
304 **kwargs: Any,
305 ) -> BaseMessage:
306 config = ensure_config(config)
--> 307 llm_result = await self.agenerate_prompt(
308 [self._convert_input(input)],
309 stop=stop,
310 callbacks=config.get("callbacks"),
311 tags=config.get("tags"),
312 metadata=config.get("metadata"),
313 run_name=config.get("run_name"),
314 run_id=config.pop("run_id", None),
315 **kwargs,
316 )
317 return cast(ChatGeneration, llm_result.generations[0][0]).message

python3.11/site-packages/langchain_core/language_models/chat_models.py:796, in BaseChatModel.agenerate_prompt(self, prompts, stop, callbacks, **kwargs)
788 async def agenerate_prompt(
789 self,
790 prompts: list[PromptValue],
(...)
793 **kwargs: Any,
794 ) -> LLMResult:
795 prompt_messages = [p.to_messages() for p in prompts]
--> 796 return await self.agenerate(
797 prompt_messages, stop=stop, callbacks=callbacks, **kwargs
798 )

python3.11/site-packages/langchain_core/language_models/chat_models.py:756, in BaseChatModel.agenerate(self, messages, stop, callbacks, tags, metadata, run_name, run_id, **kwargs)
743 if run_managers:
744 await asyncio.gather(
745 *[
746 run_manager.on_llm_end(
(...)
754 ]
755 )
--> 756 raise exceptions[0]
757 flattened_outputs = [
758 LLMResult(generations=[res.generations], llm_output=res.llm_output) # type: ignore[list-item, union-attr]
759 for res in results
760 ]
761 llm_output = self._combine_llm_outputs([res.llm_output for res in results]) # type: ignore[union-attr]

python3.11/site-packages/langchain_core/language_models/chat_models.py:924, in BaseChatModel._agenerate_with_cache(self, messages, stop, run_manager, **kwargs)
922 else:
923 if inspect.signature(self._agenerate).parameters.get("run_manager"):
--> 924 result = await self._agenerate(
925 messages, stop=stop, run_manager=run_manager, **kwargs
926 )
927 else:
928 result = await self._agenerate(messages, stop=stop, **kwargs)

python3.11/site-packages/langchain_openai/chat_models/base.py:825, in BaseChatOpenAI._agenerate(self, messages, stop, run_manager, **kwargs)
823 generation_info = {"headers": dict(raw_response.headers)}
824 else:
--> 825 response = await self.async_client.create(**payload)
826 return await run_in_executor(
827 None, self._create_chat_result, response, generation_info
828 )

python3.11/site-packages/openai/resources/chat/completions.py:1661, in AsyncCompletions.create(self, messages, model, audio, frequency_penalty, function_call, functions, logit_bias, logprobs, max_completion_tokens, max_tokens, metadata, modalities, n, parallel_tool_calls, prediction, presence_penalty, response_format, seed, service_tier, stop, store, stream, stream_options, temperature, tool_choice, tools, top_logprobs, top_p, user, extra_headers, extra_query, extra_body, timeout)
1620 @required_args(["messages", "model"], ["messages", "model", "stream"])
1621 async def create(
1622 self,
(...)
1658 timeout: float | httpx.Timeout | None | NotGiven = NOT_GIVEN,
1659 ) -> ChatCompletion | AsyncStream[ChatCompletionChunk]:
1660 validate_response_format(response_format)
-> 1661 return await self._post(
1662 "/chat/completions",
1663 body=await async_maybe_transform(
1664 {
1665 "messages": messages,
1666 "model": model,
1667 "audio": audio,
1668 "frequency_penalty": frequency_penalty,
1669 "function_call": function_call,
1670 "functions": functions,
1671 "logit_bias": logit_bias,
1672 "logprobs": logprobs,
1673 "max_completion_tokens": max_completion_tokens,
1674 "max_tokens": max_tokens,
1675 "metadata": metadata,
1676 "modalities": modalities,
1677 "n": n,
1678 "parallel_tool_calls": parallel_tool_calls,
1679 "prediction": prediction,
1680 "presence_penalty": presence_penalty,
1681 "response_format": response_format,
1682 "seed": seed,
1683 "service_tier": service_tier,
1684 "stop": stop,
1685 "store": store,
1686 "stream": stream,
1687 "stream_options": stream_options,
1688 "temperature": temperature,
1689 "tool_choice": tool_choice,
1690 "tools": tools,
1691 "top_logprobs": top_logprobs,
1692 "top_p": top_p,
1693 "user": user,
1694 },
1695 completion_create_params.CompletionCreateParams,
1696 ),
1697 options=make_request_options(
1698 extra_headers=extra_headers, extra_query=extra_query, extra_body=extra_body, timeout=timeout
1699 ),
1700 cast_to=ChatCompletion,
1701 stream=stream or False,
1702 stream_cls=AsyncStream[ChatCompletionChunk],
1703 )

python3.11/site-packages/openai/_base_client.py:1843, in AsyncAPIClient.post(self, path, cast_to, body, files, options, stream, stream_cls)
1829 async def post(
1830 self,
1831 path: str,
(...)
1838 stream_cls: type[_AsyncStreamT] | None = None,
1839 ) -> ResponseT | _AsyncStreamT:
1840 opts = FinalRequestOptions.construct(
1841 method="post", url=path, json_data=body, files=await async_to_httpx_files(files), **options
1842 )
-> 1843 return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)

python3.11/site-packages/openai/_base_client.py:1537, in AsyncAPIClient.request(self, cast_to, options, stream, stream_cls, remaining_retries)
1534 else:
1535 retries_taken = 0
-> 1537 return await self._request(
1538 cast_to=cast_to,
1539 options=options,
1540 stream=stream,
1541 stream_cls=stream_cls,
1542 retries_taken=retries_taken,
1543 )

python3.11/site-packages/openai/_base_client.py:1638, in AsyncAPIClient._request(self, cast_to, options, stream, stream_cls, retries_taken)
1635 await err.response.aread()
1637 log.debug("Re-raising status error")
-> 1638 raise self._make_status_error_from_response(err.response) from None
1640 return await self._process_response(
1641 cast_to=cast_to,
1642 options=options,
(...)
1646 retries_taken=retries_taken,
1647 )

BadRequestError: Error code: 400 - {'error': {'message': "Unsupported parameter: 'temperature' is not supported with this model.", 'type': 'invalid_request_error', 'param': 'temperature', 'code': 'unsupported_parameter'}}

Description

I am trying to use openAI's o1 full model with langchain as an agent on a langgraph I've constructed. However, it looks like the call to BaseChatOpenAI._agenerate includes **kwargs with the temperature key word argument. However, the openai package's AsyncCompletions.create() function doesn't accept temperature as an argument for full o1 (not preview or mini) models.

System Info

System Information

OS: Darwin
OS Version: Darwin Kernel Version 24.2.0: Fri Dec 6 19:02:41 PST 2024; root:xnu-11215.61.5~2/RELEASE_ARM64_T6030
Python Version: 3.11.9 (main, Apr 19 2024, 11:43:47) [Clang 14.0.6 ]

Package Information

langchain_core: 0.3.21
langchain: 0.3.9
langchain_community: 0.3.9
langsmith: 0.1.147
langchain_openai: 0.2.11
langchain_text_splitters: 0.3.2
langgraph_sdk: 0.1.43

Optional packages not installed

langserve

Other Dependencies

aiohttp: 3.10.2
async-timeout: Installed. No version info available.
dataclasses-json: 0.6.7
httpx: 0.26.0
httpx-sse: 0.4.0
jsonpatch: 1.33
langsmith-pyo3: Installed. No version info available.
numpy: 1.26.4
openai: 1.60.0
orjson: 3.10.6
packaging: 23.2
pydantic: 2.9.2
pydantic-settings: 2.6.1
PyYAML: 6.0.1
requests: 2.32.3
requests-toolbelt: 1.0.0
SQLAlchemy: 2.0.31
tenacity: 8.5.0
tiktoken: 0.7.0
typing-extensions: 4.12.2

[SidGurajala]

For other users: seems like setting temperature to 1.0 fixes this issue. To be honest I'm unsure of why this works as o1-mini and o1-preview triggers an error where you're told that o1-mini and o1-preview only accept a default value of 1, while o1 full tells you the temperature variable is not accepted unless the value is 1.0.

[ccurme]

The latest versions of langchain-openai (>= 0.3) set a null default temperature. There are some breaking changes associated with that release (hence the minor version bump), but if you can upgrade that may fix the problem.

Your environment information says you are running 0.2.x. For these versions a default temperature of 0.7 is used, but we attempt to parse "o1" in the model name and if recognized, set temperature to 1. This may not work with AzureChatOpenAI, which uses a concept of deployment names to specify models.