docs: run the three required graph audits against the live #51-#57 graph - #80
Merged
Conversation
## Summary - Add review-suite/evals/v2/audits/scope-completeness-audit.md, dependency-sequencing-audit.md, and shovel-readiness-audit.md, each an independent pass over the live GitHub issue graph after #59's approved #51-#57 body edits and native-edge read-back. - Fix two stale cross-references the scope/completeness pass found in the live tracker: #56 referenced a #52 changed-surface concept that #52's narrowing dropped, and #54 referenced a #52 risk profile that no longer exists anywhere in the graph. Neither ticket's disposition changed. - Record a summary of all three audits' findings in DECISION-RECORD.md, and point both v2/README.md and the top-level evals/README.md at the finished audit files. ## Why - #59 requires all three audits to run independently against the live graph, with findings and fixes recorded separately per pass, before the ticket can close. All three passed clean, with one flagged-but-non-blocking open question (whether #56's blockedBy #52 edge still has a real technical justification after #52's narrowing) reported for the owner rather than unilaterally resolved.
## Summary - The scope-completeness audit's cross-reference sweep checked only issue bodies. A follow-up title sweep found #52's and #53's live titles still named exactly the concepts ("risk evidence", "specialist routing") their own bodies say were dropped. Both titles are corrected via updateIssue (title only, no body change), and the audit file now records this second sweep and its findings. - The dependency-sequencing audit's #58/#50 precedent argument previously claimed #50 "closed since before #58 started," which live createdAt/ closedAt data contradicts (#58 was created before #50 closed). Replaced with the accurate timeline; the precedent's substance is unaffected. - DECISION-RECORD.md's audit summary now mentions the two title fixes alongside the two body fixes already recorded. ## Why - review-code-change's pass on this candidate returned `changes_required`: the scope-completeness audit's "no other stale cross-reference" claim was incomplete (titles unchecked), and a supporting factual claim in the dependency-sequencing audit was independently disprovable from GitHub's own timestamps.
## Summary - shovel-readiness-audit.md's #52 and #53 section headings still used the pre-fix issue titles ("Evaluate compact review coverage, impact, and risk evidence", "...and specialist routing"). Updated to match the corrected live titles. - DECISION-RECORD.md's #52 and #53 section headings had the same staleness (they predate the title fix commit); updated to match as well. ## Why - The second review-code-change pass on this candidate found the title fix from the prior commit had not propagated to two more headings inside this same delivered set, reproducing the exact staleness class the audits exist to catch.
## Summary - DECISION-RECORD.md's #57 entry no longer says the "approved by #58" cross- reference correction is "still pending... not by this repository candidate." That fix, plus two further #58-instead-of-#59 corrections in the same body (thresholds and gate-manifest references), were already applied to the live tracker after PR #79 merged. The entry now says so, and notes scope-completeness-audit.md independently confirms no stale cross-reference remains in #57's live body. ## Why - The third review-code-change pass on this candidate found the document contradicted itself: one section called a fix "still pending," while this candidate's own audit-results section and the live tracker showed it had already happened before these audits ran.
## Summary - dependency-sequencing-audit.md's live graph read-back table now includes the updatedAt column its own intro line already claimed to have queried, with the actual re-fetched timestamps for #51-#59, so scope-completeness-audit.md's pointer to "the exact updatedAt timestamps" in this table resolves to real data instead of a table that never had that column. ## Why - The fourth review-code-change pass on this candidate found a citation in scope-completeness-audit.md that pointed at data dependency-sequencing- audit.md did not actually contain, even though the underlying claim (the fixes landed when stated) was independently verifiable and true.
13 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
review-suite/evals/v2/audits/scope-completeness-audit.md,dependency-sequencing-audit.md, andshovel-readiness-audit.md, each anindependent pass run against the live GitHub issue graph for Make clean verdicts require passing validation and current-head lens evidence #51-Run the preregistered review v2 ablation and integration closeout #57 after
Use the v1 baseline to finalize and preregister the review v2 graph #59's approved body/title edits and native-edge read-back (applied
out-of-band via GraphQL in this same session, ahead of this PR).
DECISION-RECORD.md, and pointREADME.md(bothv2/and the top-levelevals/) at the finished audit files.Why
with findings and fixes recorded separately per pass, before the ticket can
close. All three passed clean. One flagged-but-non-blocking open question
(whether Operationalize adjudicated connector outcomes as review regressions #56's
blockedBy #52edge still has a real technicaljustification after Add consumer/impact-traversal evidence to the shared review contract #52's narrowing) is reported for the owner rather than
unilaterally resolved, per Use the v1 baseline to finalize and preregister the review v2 graph #59's own instruction to report a discrepancy
rather than quietly reconcile it.
Test plan
just format— all checks passed, no reformatting neededjust lint— all checks passedjust test— 214 review-suite tests plus every skill's own tests, all OKreview-code-changesuite (5rounds; 4 found small, genuine issues, all fixed; final round clean)
Refs #59