Skip to content

Fix prune calibration with cache token usage#296

Open
romy651 wants to merge 2 commits into
danny-avila:mainfrom
romy651:fix/prune-cache-calibration
Open

Fix prune calibration with cache token usage#296
romy651 wants to merge 2 commits into
danny-avila:mainfrom
romy651:fix/prune-cache-calibration

Conversation

@romy651

@romy651 romy651 commented Jul 10, 2026

Copy link
Copy Markdown

Summary

Fixes prune calibration when provider usage includes prompt-cache detail fields that are not directly comparable with the active context window.

The pruner now prefers the provider-reported usageMetadata.input_tokens for calibration and skips calibration for turns where the reported input is clearly not comparable, such as Bedrock cache-artifact turns with tiny input_tokens and very large cache creation/read detail fields.

Root Cause

Cache detail fields can represent provider billing/cache accounting rather than the actual prompt window size. Folding those details into calibration could inflate the observed input and poison the running calibration ratio or resolved instruction overhead.

Impact

This keeps pruning budget estimates stable when prompt caching is active, while preserving raw token maps and existing provider-specific cache semantics.

Validation

  • npx tsc --noEmit
  • npx eslint src/messages/prune.ts src/specs/token-accounting-pipeline.test.ts
  • git diff --check HEAD^ HEAD
  • npx jest src/specs/token-accounting-pipeline.test.ts
  • npx jest src/specs/summarize-prune.test.ts src/specs/token-distribution-edge-case.test.ts src/specs/thinking-prune.test.ts

@romy651 romy651 marked this pull request as ready for review July 10, 2026 09:55
@danny-avila

Copy link
Copy Markdown
Owner

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Swish!

Reviewed commit: 2d39f268e5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants