meta-audit: fix doc-audit scope rot, close openspec/README.md gap#708
Open
stphung wants to merge 3 commits into
Open
meta-audit: fix doc-audit scope rot, close openspec/README.md gap#708stphung wants to merge 3 commits into
stphung wants to merge 3 commits into
Conversation
Records this run's findings (PR #706) in the meta-audit history log.
Agent 1's Scope: line still named 6 docs/*.md files (system-design, core-systems, secondary-systems, balancing, challenge-minigames, infrastructure) that were deleted on 2026-07-04 when OpenSpec became the source of truth (#675) -- predating 4 of the last 5 evaluated doc-audit runs. Updated it to reflect the current docs/*.md contents. Also closes a scope gap: openspec/README.md carries its own copy of the capability/requirement/scenario counts (independent of CLAUDE.md's copy), and PR #702 already caught it drifting on its own -- formalized it into the scope-universe and Agent 1's Scope: line so coverage isn't incidental. Re-verified all 20 sampled findings from the 5 runs since the last eval marker (PR666, PR693, PR697, PR702, PR706) against source as of each run's commit -- zero inaccuracies found across the sample.
Marks PR666/693/697/702/706 as evaluated -- 20/20 sampled findings confirmed accurate, scope rot and a scope gap fixed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
meta-auditfordoc-audit, triggered by the 5-run threshold (runs since the last eval marker: PR #666, #693, #697, #702, #706).Phase 2 — adversarial re-verification: Sampled the 20 most recent findings across those 5 runs (of 38 total; 18 skipped — the 11 from PR #666 and 6 from PR #693, the two oldest runs, plus 1 low-value repeat finding from PR #697), each independently re-derived from source at its own commit by a fresh agent blind to the logged answer. Result: 20/20 confirmed — zero inaccuracies found.
doc-audit's findings across this window are reliable.Phase 3 — scope-staleness check, using the
scope-universecomment at the top ofdoc-audit/SKILL.md:Scope:line still named 6docs/*.mdfiles —system-design.md,core-systems.md,secondary-systems.md,balancing.md,challenge-minigames.md,infrastructure.md— that were deleted on 2026-07-04 (Adopt OpenSpec as source of truth; archive the old design docs #675, "Adopt OpenSpec as source of truth"). That deletion predates 4 of the 5 evaluated runs (only PR docs: fix stale constants and missing entries found by doc-audit #666's audit commit predates it). The line now reflects what's actually underdocs/*.mdtoday (decisions.md,README.md) with a pointer to why the others are gone.openspec/README.mdcarries its own copy of the OpenSpec capability/requirement/scenario counts, independent ofCLAUDE.md's copy of the same numbers — and PR doc-audit: fix stale OpenSpec counts, facade note, file attribution #702's run already caught it drifting on its own (197/500+ → actual 208/498), without it being named anywhere in the scope. Added it to both thescope-universecomment and Agent 1'sScope:line so coverage isn't incidental.No confirmed-inaccuracy pattern emerged from Phase 2, so there's nothing to add to the anti-patterns table this round.
Test plan
git log/timestamps that the OpenSpec-adoption deletion (974bdbb, 2026-07-04 17:04) predates 4 of 5 evaluated runs' audit-base commits (PR693 2026-07-05, PR697 2026-07-06, PR702 2026-07-07, PR706 2026-07-08), while PR666's audit base (2026-07-04 12:43) predates itopenspec/README.md's count claims are not covered by any existingScope:glob (CLAUDE.md,docs/*.md,docs/dossiers/*.md,src/*/CLAUDE.md,README.md)Generated by Claude Code