feat(eval): memory eval foundation — research survey + L1/L2/L3 harness + frozen baselines by fasdwcx · Pull Request #2 · relytcloud/engram

docs(spec): memory eval & optimization program design (uplift x2, injected tokens /2)

fasdwcx

committed

docs(plan): memory eval foundation implementation plan (Phase 0 + Phase 1)

fasdwcx

committed

feat(eval): metrics package (recall@k, MRR, approx tokens, keyword-group hits)

fasdwcx

committed

feat(eval): retrieval QA dataset schema and JSONL loader

fasdwcx

committed

feat(eval): scorecard JSON persistence and markdown comparison

fasdwcx

committed

feat(eval): L1 retrieval runner and evalrun CLI skeleton

fasdwcx

committed

docs(research): memory SOTA survey with engram-mapped priority matrix

fasdwcx

committed

feat(eval): freeze phoenix-retrieval-v1 dataset (dump-facts tooling + 50-100 QA cases)

fasdwcx

committed

fix(eval): de-parrot retrieval dataset keyword groups (pre-freeze construction fix)

fasdwcx

committed

feat(eval): L2 injected-token meter (static protocol + context + search composite)

fasdwcx

committed

feat(eval): L3 task schema and frozen phoenix-e2e-v1 task set

fasdwcx

committed

fix(eval): de-leak small-fix task prompts and correct fix-002 provenance (pre-freeze)

fasdwcx

committed

fix(cli-tests): force sqlite routing in tests so live MemoryLake enablement never receives test writes

fasdwcx

committed

fix(cli-tests): isolate HOME so tests can never touch the real ~/.engram enablement/config

fasdwcx

committed

Revert "fix(cli-tests): isolate HOME so tests can never touch the real ~/.engram enablement/config"

fasdwcx

committed

Revert "fix(cli-tests): force sqlite routing in tests so live MemoryLake enablement never receives test writes"

fasdwcx

committed

feat(eval): L3 runner with isolated arms, claude driver, and calibrated LLM judge

fasdwcx

committed

fix(eval): L3 runner resilience — judge retry with stderr, non-fatal judge failures, checkpoint/resume

fasdwcx

committed

style(eval): gofmt evalrun main.go

fasdwcx

committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(eval): memory eval foundation — research survey + L1/L2/L3 harness + frozen baselines - #2

feat(eval): memory eval foundation — research survey + L1/L2/L3 harness + frozen baselines#2
fasdwcx merged 25 commits into
mainfrom
feat/memory-eval-foundation

Commits on Jul 24, 2026

docs(spec): memory eval & optimization program design (uplift x2, injected tokens /2)

docs(plan): memory eval foundation implementation plan (Phase 0 + Phase 1)

feat(eval): metrics package (recall@k, MRR, approx tokens, keyword-group hits)

feat(eval): retrieval QA dataset schema and JSONL loader

feat(eval): scorecard JSON persistence and markdown comparison

feat(eval): L1 retrieval runner and evalrun CLI skeleton

docs(research): memory SOTA survey with engram-mapped priority matrix

feat(eval): freeze phoenix-retrieval-v1 dataset (dump-facts tooling + 50-100 QA cases)

fix(eval): de-parrot retrieval dataset keyword groups (pre-freeze construction fix)

feat(eval): L2 injected-token meter (static protocol + context + search composite)

feat(eval): L3 task schema and frozen phoenix-e2e-v1 task set

fix(eval): de-leak small-fix task prompts and correct fix-002 provenance (pre-freeze)

fix(cli-tests): force sqlite routing in tests so live MemoryLake enablement never receives test writes

fix(cli-tests): isolate HOME so tests can never touch the real ~/.engram enablement/config

Revert "fix(cli-tests): isolate HOME so tests can never touch the real ~/.engram enablement/config"

Revert "fix(cli-tests): force sqlite routing in tests so live MemoryLake enablement never receives test writes"

feat(eval): L3 runner with isolated arms, claude driver, and calibrated LLM judge

fix(eval): L3 runner resilience — judge retry with stderr, non-fatal judge failures, checkpoint/resume

style(eval): gofmt evalrun main.go

Commits on Jul 27, 2026

fix(eval): disable tools for judge calls so single-turn judging cannot abort

feat(eval): baseline scorecards and report (Phase 2 acceptance anchors)

docs(eval): correct tokenizer comment and L3 isolation/arm-protocol claims to match shipped behavior

fix(docs): restore cloud upgrade command examples and local-first wording in README

fix(cli-tests): update MemoryLake routing fixtures for the direct fact-add endpoint

chore: retrigger CI checks