Skip to content

fix(overnight-multi-issue-implementation): drop two figures 1.3.0 could not source (1.3.1) - #26

Merged
wan-huiyan merged 1 commit into
mainfrom
fix/unsourceable-figures-1.3.1
Aug 6, 2026
Merged

fix(overnight-multi-issue-implementation): drop two figures 1.3.0 could not source (1.3.1)#26
wan-huiyan merged 1 commit into
mainfrom
fix/unsourceable-figures-1.3.1

Conversation

@wan-huiyan

@wan-huiyan wan-huiyan commented Aug 6, 2026

Copy link
Copy Markdown
Owner

Follow-up to #25. That release shipped, inside the "never truncate a findings payload" lesson, a figure of exactly the kind the same release's reviewer line exists to catch — a number quoted from a document, with no vintage, that a reader of the skill cannot re-derive. It was found by opening the artifact, which is the only way any of these are ever found.

1. "in a 553-line pre-registration" — dropped

The pre-registration is 1,109 lines, not 553.

Both numbers were true. 553 was the document the reviewer read; the PR then had five blocking findings fixed on the branch before it merged, roughly doubling it. So the figure was accurate when the finding was written and wrong about the artifact as it stands, with nothing to tell a reader which is which.

Dropped rather than dated. The load-bearing claim is "no power statement anywhere" — the length was decoration, and dating it would only invite the same rot again in a document the skill's readers cannot open. Now reads "in a pre-registration that had no power statement anywhere", past tense, because the document has had one since the fix.

2. "About one in four, on documents." — retracted, because the source retracted it

This one the audit did not expect. The host repo's own later audit reopened its count of five: a sixth instance surfaced in a PR body, and two of the five were documents written from scratch, where "the defect it existed to repair" is a stretch. Its ruling, verbatim, is to treat this as "common enough to budget for, not as a measured rate — the useful part is not the number, it is that the diff never catches it."

Quoting a rate the source has withdrawn is the same defect one layer up, so the rate is gone. The count of five stays, attributed as the run's own count and immediately qualified. What was always load-bearing is untouched and now stated plainly: the diff never catches these; only re-derivation does.

Every other figure was checked and holds

Figure Source
six findings returned / five arrived the run's own PR record — "a three-lens review returned six blocking findings and the orchestrator truncated the payload so five arrived"
"a margin about five times finer than the data could resolve" ±0.03 against a smallest detectable difference of ≈0.17 (≈5.7×), recorded as held on re-derivation
server 269314 three separate handoffs record that exact transition
slice(0, 9000) the mechanism itself, not a measurement

Also checked and left alone, with nothing contradicting them: eight replayed PRs, two of three board tasks already claimed, and the run scale in the worked example (17 items, 4 workflows, 91 subagents, ~11 hours, 20 PRs).

One incidental note on why dating the figure would not have helped: the pre-registration measures 1,109 lines on one checkout and about 1,100 on another. A nine-line spread between two working copies of the same document, on the same day, is the argument for dropping the count rather than pinning it.

Versions — all four places, for a two-phrase change

SKILL.md frontmatter                    1.3.0 -> 1.3.1
plugins/.../.claude-plugin/plugin.json  1.3.0 -> 1.3.1
.claude-plugin/marketplace.json         1.3.0 -> 1.3.1
VERSION (bundle)                        1.3.0 -> 1.3.1

A one-line content change still needs the bump; this repo has come back for that three times (#20, #21, #24). Bumping VERSION cuts a v1.3.1 release, same as #25.

The README carries the correction as its own version-history entry, naming both figures. A correction that reaches only the SKILL leaves the changelog still asserting the withdrawn rate — and the changelog is a rendered surface.

Gates

python3 scripts/check_skill_descriptions.py . --no-color --triggers   exit 0
scripts/leak_scan.sh .                                    LEAK GATE: clean, exit 0
python3 .github/scripts/validate_plugins.py    OK: marketplace + plugins valid, exit 0

Listing budget, --context 1000000: needed 30,504 chars (~7,626 tok) · budget 40,000 (~10,000 tok) · fits, 9,496 chars to spare.

No description changed — --compare reports 1,463 → 1,463 chars · 8 triggers unchanged · No trigger dropped or narrowed. SKILL.md 742 → 747 lines.

🤖 Generated with Claude Code

https://claude.ai/code/session_013JVMGELnoA2vM56Va4gLXE

…ld not source (1.3.1)

1.3.0 shipped, inside the "never truncate a findings payload" lesson, a figure
of exactly the kind the same release's reviewer line exists to catch. Found by
checking it against the artifact, which is the only way any of these are ever
found.

1. "in a 553-line pre-registration with no power statement anywhere"

   The pre-registration is 1,109 lines, not 553. Both numbers were true: 553
   was the document the reviewer read, and the PR then had five blocking
   findings fixed on the branch before it merged, roughly doubling it. So the
   figure was accurate when the finding was written and wrong about the
   artifact as it now stands, with no vintage attached to tell a reader which
   is which -- in a document the skill's readers cannot open.

   Dropped rather than dated. The load-bearing claim is "no power statement
   anywhere"; the length was decoration, and dating it would only invite the
   same rot again. Now reads "in a pre-registration that had no power
   statement anywhere" -- past tense, because the document has had one since
   the fix.

2. "About one in four, on documents."

   Retracted at the source. The host repo's own later audit reopened the count
   of five: a sixth instance surfaced in a PR body, and two of the five were
   documents written from scratch, where "the defect it existed to repair" is
   a stretch. Its ruling is to treat this as "common enough to budget for, not
   a measured rate", and quoting a rate the source has withdrawn is the same
   defect one layer up.

   The count of five stays, attributed as the run's own count and immediately
   qualified. What was always load-bearing is untouched and now stated
   plainly: the diff never catches these, only re-derivation does.

Every other figure in the two blocks was checked against the run's handoffs
and HOLDS, so nothing else moved:

    six findings / five arrived   the run's PR record: "a three-lens review
                                  returned six blocking findings and the
                                  orchestrator truncated the payload so five
                                  arrived"
    "about five times finer"      +/-0.03 margin against a smallest detectable
                                  difference of ~0.17, recorded as held on
                                  re-derivation
    server 269 -> 314             three separate handoffs record that exact
                                  transition
    slice(0, 9000)                the mechanism itself, not a measurement

Also checked and left alone: eight replayed PRs, two of three board tasks
claimed, and the run scale in the worked example (17 items, 4 workflows, 91
subagents, ~11 hours, 20 PRs). Nothing contradicts them.

Versions, all four places, for a two-phrase change -- the repo's known failure
mode is a content change that never ships because no version moved:

    SKILL.md frontmatter                    1.3.0 -> 1.3.1
    plugins/.../.claude-plugin/plugin.json  1.3.0 -> 1.3.1
    .claude-plugin/marketplace.json         1.3.0 -> 1.3.1
    VERSION (bundle)                        1.3.0 -> 1.3.1

README carries the correction too, as its own version-history entry naming
both figures. A correction that reaches only the SKILL leaves the changelog
asserting the withdrawn rate.

No description changed: the gate reports 1,463 chars with 73 to spare and
--compare reports 8 triggers unchanged, none dropped or narrowed. SKILL.md
742 -> 747 lines. Gates: description-cap exit 0, leak exit 0,
validate_plugins exit 0, listing budget fits at 1M with 9,496 chars spare.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013JVMGELnoA2vM56Va4gLXE
@wan-huiyan
wan-huiyan force-pushed the fix/unsourceable-figures-1.3.1 branch from 717f53e to cca187b Compare August 6, 2026 11:05
@wan-huiyan
wan-huiyan merged commit 102d3d4 into main Aug 6, 2026
1 check passed
@wan-huiyan
wan-huiyan deleted the fix/unsourceable-figures-1.3.1 branch August 6, 2026 11:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant