Skip to content

feat: per-model control of which sampling params are sent to the backend - #237

Merged
co-l merged 1 commit into
co-l:developfrom
ikarys:develop
Aug 14, 2026
Merged

feat: per-model control of which sampling params are sent to the backend#237
co-l merged 1 commit into
co-l:developfrom
ikarys:develop

Conversation

@ikarys

@ikarys ikarys commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Summary

Some backends reject sampling parameters they don't support (e.g. temperature, top_p, top_k, reasoning_effort), returning HTTP 400 and blocking the conversation. OpenFox already stripped these via an omitParams field set by auto-config, but users had no manual control over which params are sent.

This PR adds a "Send" checkbox next to each sampling parameter in the provider model editor (Advanced section), so users can explicitly enable/disable individual params per model. Unchecking a param adds it to omitParams and strips it from outgoing requests; re-checking removes it.

What changed

  • ProviderModal: New SamplingParamField component with a "Send" checkbox per param (Temperature, Top P, Top K, Max tokens). Unchecking disables + dims the input and adds the param key to omitParams. Re-checking re-enables it. Pre-existing omitParams entries (e.g. reasoning_effort from auto-config) are preserved on toggle.
  • ProviderModal: Conditional "Re-enable reasoning_effort" checkbox in the Thinking section, shown only when auto-config omitted it. Lets users re-enable reasoning_effort without re-running auto-config.
  • ProviderModal: Omitted fields now show a "Not sent" placeholder instead of a stale dimmed value, making it clear the value won't be sent.
  • Refactor: Collapsed 4 near-identical sampling-param blocks (~80 lines) into one parameterized SamplingParamField component. Net LOC reduction.
  • Comments: Genericized all vendor-specific comments and test fixtures to vendor-neutral wording.

How it works

omitParams: ['temperature', 'top_p']  →  temperature and top_p stripped from request body
omitParams: undefined                  →  all params sent (default)

The omitParams field was already wired through the full stack (ModelConfig → provider-manager → client-pure → request body). This PR exposes it in the UI so users can control it manually, without relying solely on auto-config detection.

Related issues

AI-Enhanced Development

  • AI Models: glm-5.2, claude-opus-5

Cache Impact

  • NoomitParams strips generation-time parameters (temperature, top_p, top_k, max_tokens, reasoning_effort) from the request body. These are not part of the cached prompt prefix (system prompt + tools + messages) or the tool fingerprint. The dynamic context hash and KV cache are unaffected.

Test plan

  • npm run typecheck — clean
  • npm run lint — clean
  • npx prettier --check (modified files) — clean
  • npx vitest run web/src/components/shared/ProviderModal.test.tsx — 18 passed
  • npx vitest run src/server/llm/client-pure.test.ts src/server/providers/auto-config.test.ts src/server/provider-manager.test.ts — 75 passed
  • New tests: Send checkboxes rendered + checked by default, uncheck → omitParams + dim + clear, recheck → removed, preserve pre-existing entries, auto-config omitParams reflected, reasoning_effort re-enable

@co-l
co-l merged commit 2db172b into co-l:develop Aug 14, 2026
6 checks passed
@co-l

co-l commented Aug 14, 2026

Copy link
Copy Markdown
Owner

Merged, will be released in v2.0.119

AI-summary

Adds per-model control over which sampling parameters are sent to the LLM backend. Auto-config now probes each model to detect params the backend rejects (with a control-probe guard added during review to avoid false attribution), and the builder strips them from outgoing requests. The UI exposes per-param "Send" checkboxes plus a recovery control for reasoning_effort.


  • omitParams on ModelConfig — strip rejected top-level sampling params (temperature, top_p, max_tokens, top_k, reasoning_effort) from outgoing requests; runs after all merges so it wins over queryParams additions
  • Auto-detection — auto-config probes each model with an agentic-loop-shaped request (incl. a dummy tool) and reports params rejected via HTTP 400; a control probe skips detection when the backend rejects the probe structurally, and word-boundary matching avoids false positives (e.g. max_tokens_budgetmax_tokens)
  • Accurate stats/retriesmodelParams (turn stats + truncation-retry budget) now reflects the actual wire request, so omitted params are neither sent nor reported as sent
  • UI — per-sampling-param "Send" checkboxes with disabled inputs, a "Re-enable reasoning_effort" recovery checkbox, and auto-config results pre-populating the checkboxes
  • Plumbing — threaded through provider config, provider-manager, the providers API, and the streaming request path

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants