Skip to content

fix: tighten context compression prompt and token budget - #1306

Merged
wsp1911 merged 1 commit into
GCWing:mainfrom
wsp1911:main
Jun 25, 2026
Merged

fix: tighten context compression prompt and token budget#1306
wsp1911 merged 1 commit into
GCWing:mainfrom
wsp1911:main

Conversation

@wsp1911

@wsp1911 wsp1911 commented Jun 25, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adjust context compression so the model no longer spends extra time generating transient analysis, clamp compression-request output tokens with a temporary override, and simplify the retained compression contract wording.

Fixes #

Type and Areas

Type:

bug fix

Areas:

Rust core, AI adapters, runtime ports

Motivation / Impact

Context compression had two practical issues: the prompt required an <analysis> section that was discarded anyway, and compression requests inherited the active model max_tokens without a dedicated cap, which could push some requests past the effective context window and trigger provider-side errors. This change makes compression summaries summary-only, caps compression output tokens to min(model.max_tokens, 8192) with an 8192 fallback when the model has no explicit limit, and removes redundant contract duplication from the compression prompt. The retained contract text is also phrased more naturally for future rounds.

Verification

  • pnpm run fmt:rs
  • cargo test -p bitfun-ai-adapters with_max_tokens_overrides_output_limit -- --nocapture
  • cargo test -p bitfun-core model_summary_prompt_does_not_inline_compaction_contract -- --nocapture
  • cargo test -p bitfun-core model_summary_prompt_requires_summary_only -- --nocapture
  • cargo test -p bitfun-core compression_request_max_tokens_clamps_to_global_cap -- --nocapture
  • cargo test -p bitfun-core build_compression_ai_client_overrides_only_max_tokens -- --nocapture
  • cargo test -p bitfun-runtime-ports compression_contract_renders_model_visible_fields -- --nocapture

Reviewer Notes

  • Compression still retains the contract in the saved compressed history; this change only removes the extra prompt-side duplication.
  • The compression-specific token clamp is intentionally local to the compression request path and does not change normal turn generation limits.

Checklist

  • This PR is focused and does not include secrets, temporary prompts, generated scratch files, or unrelated artifacts.
  • Relevant verification is recorded above, or skipped checks are explained.
  • User-facing strings, docs, and locales are updated where applicable.

@wsp1911
wsp1911 marked this pull request as ready for review June 25, 2026 03:20
@wsp1911
wsp1911 merged commit fb96968 into GCWing:main Jun 25, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant