Skip to content

v2 SDK drops reasoning_content from model run responses (Message + StreamChunk) #985

Description

@aix-ahmet

Summary

The v2 SDK silently drops the reasoning_content field that the platform returns for reasoning LLMs, in both the non-streaming and streaming model-run paths. Consumers (e.g. the agents engine's AiXplainLLM provider) can never see the model's reasoning text — and for some models the entire response is in reasoning_content, so the run appears to have produced empty output.

Evidence

Direct model run on dev (POST {MODELS_RUN_URL}/{model_id} with {"data": "<essay prompt>", "max_tokens": 80}), model Qwen 3.6 35B A3B (69e5f93df018f8050346c1de, alibaba-cloud/qwen-3.6-35b-a3b/deepinfra):

{
  "details": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "",
        "reasoning_content": "Here's a thinking sequence\n\n1.  **Deconstruct the prompt:** ...",
        "name": null,
        "tool_calls": []
      },
      "finish_reason": "length",
      "logprobs": null
    }
  ],
  "status": "SUCCESS",
  "completed": true,
  "data": "",
  "usage": { "prompt_tokens": "26", "completion_tokens": "80", "total_tokens": 106 }
}

All 80 completion tokens went to reasoning_content; content/data are empty. After SDK deserialization, reasoning_content is gone: the Message dataclass in aixplain/v2/model.py only declares role, content, tool_calls, refusal, annotations, and dataclass_json discards unknown keys.

The streaming parser has the same gap: ModelResponseStreamer.__next__ (OpenAI-style branch in aixplain/v2/model.py) reads only delta.content and delta.tool_calls; a delta.reasoning_content key is dropped.

Fix

  1. Message dataclass (aixplain/v2/model.py): add

    reasoning_content: Optional[str] = None

    so details[0].message.reasoning_content survives deserialization.

  2. StreamChunk (aixplain/v2/model.py): add a reasoning_content: Optional[str] = None field, and in the OpenAI-style chunk branch of ModelResponseStreamer.__next__ populate it from delta.get("reasoning_content") (string-typed only, same defensive coercion as content). A chunk that carries only reasoning_content (empty content) must still be yielded, not skipped.

  3. Tests: unit tests deserializing the payload above (non-streaming) and an SSE fixture with {"choices":[{"delta":{"reasoning_content":"..."}}]} chunks (streaming), asserting the field is preserved and that content-only behavior is unchanged.

Notes

  • This mirrors the LiteLLM convention (message.reasoning_content / delta.reasoning_content), which is also what the platform emits, so no renaming — keep the field name as-is.
  • Backwards compatible: field is optional with None default.
  • Downstream consumer change: aixplain/aixplain-agents#222 (the agents engine reads the field once this ships).

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions