epic(context): Context-Adaptive Memory (CAM) — full implementation

[bug-ops]

Spec

specs/062-context-adaptive-memory/ — added in PR #4546.

Overview

CAM replaces the binary keep/discard compaction with a three-level fidelity model (Full / Compressed / Placeholder) for context messages. A heuristic scorer evaluates each message on four signals (temporal decay, role importance, keyword/semantic overlap, plan relevance) and assigns the tightest fidelity level that preserves turn coherence. At high budget pressure a proactive regrade pass fires before the hard compaction tier. Target: 40–60% token reduction in long sessions with no coherence regression.

Three research issues are unified: AgeMem (#4016), Adaptive Focus Memory (#4017), PAACE (#4018).

Issue Map

#4547  Phase 1 MVP — heuristic scorer, proactive regrade, config
   │
   ├──► #4549  Phase 2-A — PAACE: live DAG lookahead wired into planned_next_tools
   │             (needs: zeph-orchestration lookahead_tools() API)
   │
   ├──► #4550  Phase 2-B — Fidelity persistence in SQLite (cross-turn stability)
   │             (needs: zeph-db schema migration, zeph-agent-persistence r/w)
   │
   ├──► #4551  Phase 2-C — LLM-assisted Compressed rendering (fidelity_compress_provider)
   │             (needs: scorer async path, deferred_summary caching)
   │
   └──► #4552  Phase 2-D — Embedding-based semantic scoring (replaces keyword_overlap)
                 (needs: embed_one() from zeph-llm, MessageMetadata embedding cache)

Dependency graph

#4547 (Phase 1)
  ├─ T-01  ContextFidelity + PlannedToolHint       zeph-common         (unblocked)
  ├─ T-02  MessageMetadata.fidelity_tag            zeph-llm            (← T-01)
  ├─ T-03  FidelityConfig + FidelityScorer         zeph-context        (← T-01, T-02)
  ├─ T-04  ContextAssemblyInput extension          zeph-context        (← T-01, T-03)
  ├─ T-05  ContextManager regrade guard            zeph-context        (← T-01)
  ├─ T-06  Wire scorer in service.rs               zeph-agent-context  (← T-03, T-04, T-05)
  ├─ T-07  Proactive regrade call site             zeph-agent-context  (← T-05, T-06)
  ├─ T-08  Placeholder exclusion in compaction     zeph-agent-context  (← T-02, T-06)
  ├─ T-09  Config integration (zeph-config)        zeph-config         (← T-03)
  └─ T-10  Benchmark + playbook                   zeph-bench + docs   (← T-06, T-07, T-08)

#4549 (Phase 2-A)  ← #4547
  └─ Add lookahead_tools(depth) to zeph-orchestration, populate planned_next_tools at turn start

#4550 (Phase 2-B)  ← #4547
  └─ SQLite column fidelity_tag, zeph-agent-persistence read/write, additive migration

#4551 (Phase 2-C)  ← #4547
  └─ compress_provider config, async scorer path, deferred_summary cache

#4552 (Phase 2-D)  ← #4547
  └─ semantic_scoring_provider, embed_one() reuse, MessageMetadata embedding cache
       Phase 2-D is independent of 2-A/2-B/2-C but benefits from 2-B (persistence of embeddings)

Critical invariants (all phases)

• INV-01 Fidelity scoring only after apply_prepared_context() returns — prevents index invalidation
• INV-02 Placeholder messages excluded from hard compaction summarizer input
• INV-03 Tool-use/tool-result pairs downgrade atomically (never split a pair)
• INV-04 No consecutive same-role Placeholder messages in final window
• INV-05 Fidelity scores normalized by sum of active weights
• INV-06 regraded_this_turn guard prevents double-regrade per turn

Milestone targets

Phase	Issue	Target version	Gate
1 — MVP	#4547	v0.21	enabled = false default, all 12 ACs in spec.md pass
2-A — PAACE	#4549	v0.22+	orchestration DAG exposes lookahead_tools()
2-B — Persistence	#4550	v0.22+	additive DB migration, no data loss
2-C — LLM compress	#4551	v0.23+	async scorer path stable
2-D — Embeddings	#4552	v0.23+	embedding infra stable, Phase 2-B landed

Definition of done

Epic is complete when:

• Implement: Context-Adaptive Memory (CAM) — Phase 1 MVP #4547 merged and all 12 ACs pass in live session
• feat(context): CAM Phase 2-A — wire PAACE lookahead from orchestration DAG into ContextAssemblyInput #4549 merged — plan-relevance signal produces non-zero scores in orchestrated sessions
• feat(context): CAM Phase 2-B — persist ContextFidelity per message in SQLite for cross-turn stability #4550 merged — fidelity tags survive session reload
• feat(context): CAM Phase 2-C — LLM-assisted Compressed rendering via fidelity_compress_provider #4551 merged — Compressed messages use LLM summary when provider configured
• feat(context): CAM Phase 2-D — embedding-based semantic scoring to replace keyword_overlap #4552 merged — keyword_overlap fully replaced by cosine similarity when provider configured
• Coverage-status rows for all CAM components show Tested
• Token reduction ≥ 40% confirmed in benchmark vs. baseline without CAM