feat(context): token-budget accounting for context assembly (bo-ne9k)#32
Merged
Conversation
Budgets were counted in source LINES everywhere, but agents have TOKEN budgets and a 50-line dense-code chunk costs far more tokens than 50 lines of comments. Add a `[context] budget_unit = "line" | "token"` toggle. In token mode each chunk's cost is estimated from its content (~chars/4, deterministic, no tokenizer files) instead of its line span, so injection size is predictable against the model window. `budget_lines` and all derived caps (max_chunk_lines, pin reserve, bridged/knowledge budgets) are then interpreted in tokens. Default stays `line` — zero behavior change unless opted in. - BudgetUnit enum (serde lowercase, FromStr/Display) + estimate_tokens() + chunk_cost() helper in search::context - budget_unit threaded through ContextConfig and the 3 live builders (cli/context, http/handlers/context, cli/hook); calibrate/review keep line mode via ..default() - config: [context].budget_unit knob + docs/configuration.md - tests: estimate_tokens, unit roundtrip, token-mode budget enforcement (huge-span/tiny-content chunk admitted; over-budget chunk dropped), config parse + default assertions Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UHPGrbayqXRFaMHuvs6yLX
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds a
[context] budget_unit = "line" | "token"toggle so the context-assembly budget can be enforced in tokens instead of source lines.Why (bo-ne9k)
Budgets were counted in LINES everywhere, but agents have TOKEN budgets — a 50-line dense-code chunk costs far more tokens than 50 lines of comments. Line-counting makes injection size unpredictable against the model window.
How
BudgetUnitenum (serde lowercase,FromStr/Display) +estimate_tokens()(deterministic ~chars/4, no tokenizer files) +chunk_cost()helper insearch::context.budget_linesand all derived caps (max_chunk_lines, pin reserve, bridged/knowledge budgets) are interpreted in tokens.budget_unitthreaded throughContextConfigand the 3 live builders (CLI context, HTTP context, hook).calibrate/reviewkeep line mode via..default().docs/configuration.md.Default stays
line→ zero behavior change unless opted in.Tests
test_estimate_tokens,test_budget_unit_roundtriptest_token_budget_enforcement— a huge-line-span / tiny-content chunk is admitted (proving tokens, not lines, are counted) and an over-budget chunk is droppedbudget_unit = "token") + default (line) assertionscargo build/clippyclean (0 errors); all context + config tests pass.🤖 Generated with Claude Code