Add opt-in Anthropic compaction for long threads#287
Conversation
Adds an npm run verify:compaction script that checks the compaction summary survives the storage round-trip and is re-applied on resend.
Set contextOptions.compaction to summarize earlier context via Anthropic's compact_20260112, and widen the recent-message window so history reaches the trigger.
|
Warning Review limit reached
More reviews will be available in 12 minutes and 5 seconds. Learn how PR review limits work. Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file). ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits. 🚦 How do rate limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Repository: get-convex/coderabbit/.coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthroughThis PR adds opt-in Anthropic server-side context compaction to convex-agent. A new 🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/verify-compaction.ts`:
- Around line 132-135: The round-trip comparison in verify-compaction.ts is only
logged and does not stop the script on mismatch, so make the check fail fast
when the JSON strings differ. Update the round-trip verification around the
byte-identical comparison of compactionPart and restoredCompaction so it throws
or exits non-zero on a mismatch, keeping the existing console output as context
if needed, and ensure the behavior is enforced by the main verification flow in
verify-compaction.ts.
- Around line 157-163: The recompaction detection in verify-compaction uses a
hardcoded threshold and a strict greater-than check, which can miss boundary
cases. Update the logic around bigRecompaction and reApplied to use the
TRIGGER_TOKENS constant instead of 50_000, and make the comparison inclusive so
a recompaction at the trigger boundary is detected correctly.
In `@src/client/search.ts`:
- Around line 146-150: Gate the expanded recent-message window in search.ts so
it only applies when Anthropic is the active provider; the current numItems
fallback under opts.compaction widens reads for all providers. Update the logic
around the numItems selection to check the provider/model state used by this
search path, and keep non-Anthropic runs on DEFAULT_RECENT_MESSAGES unless
opts.recentMessages is explicitly set.
In `@src/client/start.ts`:
- Around line 64-69: `withCompaction` is passing `compaction.triggerTokens`
directly into the `compact_20260112` trigger without validating it first, which
can lead to provider-side failures. Update the `withCompaction` logic in
`start.ts` to reject non-integer values and anything below 50_000 before
building the compaction edit, and keep the fallback to
`DEFAULT_COMPACTION_TRIGGER_TOKENS` only after validation passes.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository: get-convex/coderabbit/.coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 7da3f39e-b43f-4915-96fe-28c97ee419c7
⛔ Files ignored due to path filters (1)
package-lock.jsonis excluded by!**/package-lock.json
📒 Files selected for processing (7)
package.jsonscripts/verify-compaction.tssrc/client/compaction.test.tssrc/client/search.tssrc/client/start.tssrc/client/types.tssrc/shared.ts
- Only widen the recent-message window for Anthropic models (decided in startGeneration where the provider is known), keeping search.ts provider-agnostic. - Validate compaction.triggerTokens is an integer >= 50000 before building the edit. - Make verify:compaction fail on a round-trip mismatch and use the trigger constant for recompaction detection.
ai 6.0.209 needs @ai-sdk/provider-utils 4.0.30 and @ai-sdk/provider 3.0.10; bumping ai alone duplicates provider-utils and breaks the build with mismatched schema types. Dev deps only — peer ranges are unchanged.
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/client/compaction.test.ts`:
- Around line 68-72: Tighten the withCompaction test for the triggerTokens lower
bound: the current expectation is too broad because it matches an integer-only
validation error and can hide a regression in the 50k minimum check. Update the
assertion in the rejects a triggerTokens below the 50k minimum test to match
only the specific lower-bound validation message for triggerTokens, using the
withCompaction helper as the target behavior and avoiding any fallback that
would pass on a generic integer error.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository: get-convex/coderabbit/.coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: f66bae2a-e4bd-42d0-a7d7-98dbd449ffb7
⛔ Files ignored due to path filters (1)
package-lock.jsonis excluded by!**/package-lock.json
📒 Files selected for processing (4)
package.jsonscripts/verify-compaction.tssrc/client/compaction.test.tssrc/client/start.ts
🚧 Files skipped from review as they are similar to previous changes (1)
- scripts/verify-compaction.ts
Lets the unit tests assert the specific failure mode instead of matching a shared message.
1 participant
Hey! 👋
Long threads on Anthropic models eventually run past the context window, and right now the only real lever is
recentMessagestruncation — which just throws old context away instead of summarizing it.Turns out Anthropic's server-side compaction (
compact_20260112) plugs into the agent's existing storage really cleanly, so this ended up being a small change! The summary comes back as a text part taggedproviderOptions.anthropic.type === "compaction", andserializeMessage/toModelMessagealready round-trip that untouched — nothing to change in the mapping/validators layer.What's in here
@ai-sdk/anthropic3.0.13 → 3.0.62 (compaction landed in 3.0.41).contextOptions.compactionknob:providerOptions.anthropic(preserving anything you already pass, and it won't double-add), and widens the recent-message window so history actually reaches the trigger and the stored summary gets replayed instead of re-summarized.How I checked it (live, not hand-wavy)
npm run verify:compactiondrives a ~115k-token thread past the trigger, round-trips the response through the realserializeMessage/toModelMessage, resends, and readsusage.iterations: turn 2 shows no re-compaction (the message pass stays tiny), proving the round-tripped summary was honored by the API. Also added unit tests for the merge logic.Totally happy to adjust naming or placement (e.g. whether
compactionbelongs oncontextOptionsor its own option) — let me know what fits the codebase best! 🙏