Skip to content

bug: thinking_budget kwarg passed to Completions.parse() crashes memory discard in pusher #7898

@beastoin

Description

@beastoin

Summary

utils/llm/trends.py memory discard/trend detection fails for all users in the pusher service. The error repeats continuously in pusher logs.

Error

ERROR:utils.llm.trends:Error determining memory discard: Completions.parse() got an unexpected keyword argument 'thinking_budget'

Observed in

Pusher service (GKE), utils.llm.trends module, memory discard logic.

Code trace

  1. utils/llm/trends.py:68 calls get_llm('trends').with_structured_output(ExpectedOutput).invoke(prompt)
  2. In the basic QoS profile (clients.py:217), 'trends' maps to ('gemini-2.5-flash-lite', 'gemini')
  3. _get_default_client() (clients.py:520) calls _get_or_create_gemini_llm(model, streaming, thinking_budget=0) for all Gemini-routed features
  4. _get_or_create_gemini_llm() (clients.py:492-493) adds thinking_budget to kwargs when model starts with gemini-2.5:
    if thinking_budget is not None and model_name.startswith('gemini-2.5'):
        kwargs['thinking_budget'] = thinking_budget
  5. The thinking_budget kwarg is passed to ChatGoogleGenerativeAI() (clients.py:502), which does not support it — when .with_structured_output() is called, it routes through Completions.parse() which rejects the unknown kwarg

Impact

Memory discard/trend detection fails silently for all users. Every conversation that triggers the memory discard flow in pusher hits this error. The except Exception at trends.py:86-88 catches the error and returns [], so no trends are ever detected.

Repro

Any conversation that triggers the memory discard flow in pusher when the active QoS profile routes trends to a Gemini model.


by AI for @beastoin

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingintelligenceLayer: Summaries, insights, action itemsmaintainerLane: High-risk, cross-system changesp2Priority: Important (score 14-21)

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions