How to save reasoning content in LangChain for CoT models

# Saving Reasoning Content in LangChain: Patch for CoT Models

When working with CoT models via LangChain, developers run into the loss of reasoning content—the key block of reasoning generated by the model before the final answer. This isn't a bug but a consequence of an architectural decision: classes like ChatOpenAI strictly adhere to the official OpenAI API, ignoring non-standard fields like reasoning, reasoning_content, or reasoning_details. The result is slower response times and a worse UX, since users don't see the AI's intermediate thinking steps. Here's how to fix it at the code level without waiting for an official update.

Why LangChain Ignores Reasoning Content

LangChain is positioned as a universal framework compatible with a wide range of LLM providers. To avoid fragmentation, its core classes (like ChatOpenAI) implement only the official OpenAI Chat Completion API spec. Fields like reasoning_content, reasoning, and similar ones are extensions introduced by DeepSeek, xAI, vLLM, and others. Each provider uses its own naming and format, creating chaos:

• DeepSeek → reasoning_content
• vLLM → switched from reasoning_content to reasoning
• xAI / OpenRouter → may use reasoning_details

Supporting all variants in ChatOpenAI would require:

• Constant monitoring of API changes from third-party providers.
• Maintaining a growing list of field mappings.
• Turning ChatOpenAI into a universal adapter—which goes against its purpose.

That's why LangChain developers recommend using specialized classes like ChatDeepSeek. But most providers (including polza.ai, StepFun, and others) still offer APIs in OpenAI format—and that's where the issue persists.

How to Apply the Patch in base.py

The solution is to modify two functions in LangChain's source code: _convert_dict_to_message and _convert_delta_to_message_chunk. These handle converting raw LLM responses into Message objects used within the framework. You need to add extraction of the reasoning field and save it to additional_kwargs.

Here's the minimal patch:

def _convert_dict_to_message(_dict):
    ...
    content = _dict.get("content", "") or ""
    reasoning = _dict.get("reasoning", "") or ""  # added
    additional_kwargs: dict = {}
    if reasoning:  # added
        additional_kwargs['reasoning_content'] = reasoning  # added
    if function_call := _dict.get("function_call"):
        additional_kwargs["function_call"] = dict(function_call)
    tool_calls = []
    ...

Similarly for streaming mode:

def _convert_delta_to_message_chunk(_dict):
    ...
    id_ = _dict.get("id")
    role = cast(str, _dict.get("role"))
    content = cast(str, _dict.get("content") or "")
    reasoning = cast(str, _dict.get("reasoning", "") or "")  # added
    additional_kwargs: dict = {}
    if reasoning:  # added
        additional_kwargs['reasoning_content'] = reasoning  # added
    ...

This patch:

• Doesn't break the existing architecture.
• Preserves compatibility with the official API.
• Adds support for reasoning_content from any provider that returns a reasoning field.

Example Usage with Streaming Output

After applying the patch, reasoning content becomes available in additional_kwargs for each chunk. Here's how to set up the output:

from langchain_openai import ChatOpenAI

model = ChatOpenAI(
    model="stepfun/step-3.5-flash",
    openai_api_base="https://api.polza.ai/v1",
    openai_api_key="your_api_key",
)

for chunk in model.stream('Hello!'):
    if "reasoning_content" in chunk.additional_kwargs:
        print(chunk.additional_kwargs['reasoning_content'], end="", flush=True)
    else:
        print(chunk.content, end="", flush=True)

Now users can see the model's reasoning process in real time—this is especially valuable in CoT apps, where intermediate steps like data analysis, hypothesis comparison, and logic checks matter.

Alternative Approaches and Limitations

While the patch is effective, it has limitations:

• Local modification: Changes only affect your copy of LangChain. They'll be lost on package updates.
• Field name incompatibility: If a provider uses reasoning_details instead of reasoning, you'll need to adapt the code.
• Lack of standardization: Without a unified agreement among providers, each case needs manual tweaking.

Clean alternatives:

• Custom MessageConverter—create your own class inheriting from BaseMessageConverter and override the parsing logic.
• Middleware layer—intercept HTTP responses from the provider before they reach LangChain and normalize fields.
• LangChain Expression Language (LCEL)—wrap the model call in a RunnableLambda that extracts reasoning before passing to the chain.

However, these methods are more complex and require a deep understanding of the framework's internals. For an MVP or quick fix, patching base.py is the optimal solution.

Key Takeaways

• Reasoning content is lost not due to a bug, but because LangChain strictly follows the OpenAI API.
• The patch adds support for the reasoning field in additional_kwargs without breaking backward compatibility.
• It works for both regular and streaming calls.
• It requires manual reapplication after each LangChain update.
• It improves UX in CoT apps by making the model's thinking process transparent to users.

— Editorial Team