feat: context compaction — summarize-and-truncate per the merged design#74
Conversation
Implements docs/design/context-compaction.md's engine-layer mechanism:
- Session.Compact(ctx, CompactOptions) folds a contiguous prefix of whole
turns into one synthetic summary message, keeping the most recent
keep_turns (default 2, hard floor 1) verbatim.
- The summary is produced via the session's own provider (or an override),
banner-marked as synthesized, and spliced in via spliceCompact — shared
by the live path and LoadSession's replay so they can never drift apart.
- Automatic trigger (maybeAutoCompact) runs at the top of every Prompt call,
before the incoming user message is appended, gated on
Config.ContextWindowTokens (opt-in) and Config.CompactionThreshold
(default 0.8), with churn-guard hysteresis so a hot kept-region tool
result cannot cause thrashing.
- Usage accounting: the summarization call's spend lands in cumulative
Usage() only, never LastUsage() — live or replayed.
- New recCompact journal record carries the summary inline plus the
summarization call's own usage, replayed cumulative-only by LoadSession.
- EventHistoryCompacted/EventCompactionFailed engine events; the summary
flows through the ordinary EventMessage path first.
- Session.CompactionCount()/LastCompactedAt() for durable observability.
Regression test TestIncidentRecoverableByCompaction encodes the production
incident (a goal session died at 205102 tokens > 200000 maximum,
unrecoverable) and is red-verified against the pre-fix code.
Also verifies the double-RoleUser splice shape (compacted summary followed
by the first kept turn's prompt) merges correctly through the Anthropic
transcoder's existing same-role alternation handling.
No server-layer wiring yet (POST /session/{id}/compact, GET /session
fields, openapi.yaml) — follow-up commit.
Wires engine/compact.go's Session.Compact into the HTTP surface, per
docs/design/context-compaction.md §1/§4:
- POST /session/{id}/compact: run-token auth, claims the session's single
run slot exactly like prompt_async/goal (409 busy, 503 draining, 404
unknown), runs synchronously, optional {keep_turns, model} body.
keep_turns has a hard floor of 1 — 0 or negative is 400, never clamped.
Returns {turns_folded, first_id, last_id, summary}; turns_folded: 0 is a
200, not an error.
- Publish now routes engine.EventHistoryCompacted to a durable
history.compacted journal record (publishHistoryCompacted) and
engine.EventCompactionFailed to a live-only compaction.failed event.
Because Session.Compact emits the summary's EventMessage before
EventHistoryCompacted, and OnEvent is synchronous, the journal/SSE order
is always: summary message, then history.compacted.
- Session.compaction_count/last_compacted_at on GET /session (and
buildSession), sourced from the engine (durable across restart).
- openapi.yaml: the new path, CompactRequest/CompactResponse schemas,
Session.compaction_count/last_compacted_at, and the two new Event types
with their compact_first_id/compact_last_id/compact_turns_folded/
compact_summary_id fields.
Tests cover the happy path, the keep_turns floor, the no-op/200 case, 409
on a busy session, 404/401, and the message-before-history.compacted event
ordering.
…ion_threshold, compaction_keep_turns) Wires docs/design/context-compaction.md's opt-in engine.Config fields through the CLI/serve config layer, following the same non-zero-overrides merge rule as goal_evaluator_model: a project .harness.json value wins over the user config when set, otherwise the user value (or the engine's own zero-fills-a-default) applies. Both harness run and harness serve wire these into every session's engine.Config.
TestAutoCompactionAcrossRestart is the e2e keystone docs/design/context-
compaction.md calls for: a real `harness serve` binary, driven by a
scripted fake Anthropic provider (completeTurnWithUsage, scriptedUsageAnthropic),
is pushed past a small test-configured context_window_tokens/
compaction_keep_turns threshold. It asserts, end to end over HTTP:
- automatic compaction fires before the triggering turn's own request
(exactly one extra summarization call, in the right position in the
request sequence)
- the session keeps working afterward, with a visibly smaller
last_input_tokens than the value that triggered compaction
- GET /session's compaction_count/last_compacted_at flip
- the durable journal shows the summary's message event strictly before
history.compacted, and history.compacted names the fold
- after a SIGKILL + fresh process on the same session directory, GET
/session and GET /session/{id}/message replay the compacted, trimmed
history exactly, the event journal stays contiguous, and a further
prompt succeeds — a post-compaction session restarts exactly as cleanly
as an ordinary one.
Also extracts completeTurnWithUsage from completeTurn (explicit input/
output token counts) and adds the compaction fields to the shared apiEvent
test type.
…hanges TestPursueGoalAutoCompactsMidLoop proves docs/design/context-compaction.md §1's central claim end to end: PursueGoal drives every worker turn through the ordinary Prompt path, so maybeAutoCompact's check at the top of Prompt fires mid-goal-loop with no separate scheduler and no goal.go wiring at all. goalCompactProvider extends goal_test.go's worker/evaluator-by-tools- presence classification with a third case (compaction's tool-less summarization call, identified by its unique system prompt) so a single scripted provider can serve all three request shapes a goal session now makes.
CompactRequest.model now $ref's the shared ModelRef schema (a plain "provider/model" string) instead of a bespoke string field, consistent with PromptRequest.model.
📦 Previous Review (superseded by newer review)Claude finished @andybons's task in 5m 37s —— View job Review: context compaction (PR #74)
This is a careful, well-documented PR — the durability model (inline summary in the 🔴 Auto-trigger undercounts context when prompt caching is active (
|
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
The threshold check read lastUsage.InputTokens alone, but harness injects cache_control by default: on a warm session the Anthropic adapter reports the bulk of the prompt in CacheReadTokens (new prefix growth in CacheWriteTokens) and InputTokens is only the uncached tail — so automatic compaction never fired in exactly the long-cached-session shape it exists for. Sum all three input components. Red-first test encodes the original incident under caching: a 205,102-token warm prompt (3,102 input + 190,000 cache read + 12,000 cache write) now triggers; it did not before.
📦 Previous Review (superseded by newer review)Claude finished @andybons's task in 3m 22s —— View job Review: context compaction (PR #74) — re-review after
|
📦 Previous Review (superseded by newer review)Claude finished @andybons's task in 3m 20s —— View job Review: context compaction (PR #74) — re-review after
|
provider.Usage had provider-inconsistent semantics: Anthropic reports input_tokens EXCLUSIVE of cache reads/writes (disjoint components) while the OpenAI Responses adapter passed through input_tokens INCLUSIVE of input_tokens_details.cached_tokens — so summing the components (as the auto-compaction threshold now does) double-counted the cached portion on OpenAI and fired early. Document the disjoint contract on Usage and make the OpenAI adapter subtract: InputTokens is the uncached remainder, the sum reconstructs the wire total. openaicompat already conforms (total in InputTokens, zero cache fields). Adapter tests updated to the contract: 100 wire total with 25 cached now reports 75+25, 7 with 2 reports 5+2.
|
Claude finished @andybons's task in 2m 30s —— View job Review: context compaction (PR #74) — re-review after
|
| Adapter | InputTokens source |
Cache fields | Sum = true prompt? |
|---|---|---|---|
Anthropic (anthropic.go:291-293) |
input_tokens (already cache-exclusive) |
cache_creation / cache_read separate |
✅ |
OpenAI (openai.go:296-300) |
input_tokens - cached_tokens (now subtracted, floored at 0) |
CacheReadTokens = cached_tokens |
✅ |
openaicompat (openaicompat.go:409) |
prompt_tokens (cache-inclusive total) |
none set (0) | ✅ (sum = total) |
engine/compact.go:327's InputTokens + CacheReadTokens + CacheWriteTokens sum is now correct on every adapter, and the adapter tests (stream_test.go:124 75+25, :219 5+2) encode the contract red-first. The negative guard (if uncached < 0) is a sensible defense against a malformed wire total.
Verified sound (re-confirmed this round)
- Durability — inline summary in
recCompact(no dangling reference across a torn write), sharedspliceCompactbetween live (compact.go:278) and replay (store.go:407) paths, cumulative-only usage on replay so the small summarization call can't mask trigger pressure,compactCount/lastCompactedAtrestored on load. - Concurrency — explicit path claims the single run slot (
handleCompact→claimForPrompt, balancedwg.Done); auto path runs insidePromptbefore the user message is appended (engine.go:642); the network summarization runs unlocked and re-acquiress.mufor one splice+persist critical section — a reader sees pre- or post-state, never half-spliced. - Churn guard —
compactHysteresisset on fold, cleared when summed usage next dips below threshold; the one-turn lag resolves correctly. - Journal ordering — summary
EventMessageemitted beforeEventHistoryCompacted(compact.go:199-206), so a tailer never holds a dangling summary id.
🟢 Trivial (non-blocking)
- Duplicate doc line in
provider/provider.go:50-51.84ca426added the new contract comment but left the original one-liner above it, so the type now has two lead sentences:Drop line 50. Fix this →// Usage is token accounting for one request. // Usage reports token accounting for one provider call. // // CONTRACT: ...
🟡 Minor (carried from prior rounds — still open, non-blocking)
CompactionModelhas no config-file knob.1a860dfwiredcontext_window_tokens/compaction_threshold/compaction_keep_turnsintoconfig.Configandcmd/harness/main.go, butengine.Config.CompactionModelstill has nocompaction_modelcounterpart — it's reachable only via the per-callPOST /compactbody and is always zero from config. Add it alongside the other three, or note in the doc comment that it's request-only.CompactionThresholdisn't range-validated. Only<= 0gets the 0.8 default; a value> 1.0silently means "never auto-compact" — a fat-fingeredcompaction_threshold: 80(meaning 80%) disables the feature silently, inconsistent withkeep_turns's explicit floor. Consider clamping to(0,1]or warning.
Overall: the core is correct and well-tested, the durability and concurrency models hold, and the one blocking item from the prior round is closed at the contract level. The remaining items are config-polish, not correctness blockers — I'd merge on these once the two minor knobs are either addressed or explicitly deferred.
1 participant
Implements
docs/design/context-compaction.md(merged as the build spec in #64-era work):EventMessagepath before thehistory.compactedevent;keep_turnsfloor of 1; churn-guard hysteresis so compaction cannot thrash; the double-RoleUser splice preserved and tested (the transcoder merges adjacent same-role messages — the splice shape is load-bearing).LastUsageagainst the model context threshold, including mid-goal-loop with zero changes togoal.go(tested).POST /session/{id}/compact(409 when busy), compaction observability onGET /session, openapi updated.context_window_tokens,compaction_threshold,compaction_keep_turns.go test -race,go vet,GOOS=windows go buildall green.