Add confidence signals, human-review gate, no-op removal, and file-list targeting to docs sweeps - #251
Merged
Conversation
…st targeting to docs sweeps Implements four docs-quality-sweep improvements from the AI & Docs Challenge cohort (June 2026): - Confidence signals (elastic/docs-eng-team#621): new shared gh-aw-fragments/findings-contract.md adds a per-finding `confidence` (high/medium/low) field with explicit tier criteria to all seven sweeps; strengthens docs-issue-scope so unverified terminology from low-integrity input is attributed rather than asserted and rated Low. - Human-review gate (elastic/docs-eng-team#622): sweeps conditionally apply a `needs-human-review` label and a review-before-acting callout when any finding is medium/low confidence; SizeBot no longer applies `good-for-ai` to needs-human-review issues; intended agent workflow documented. - Finding-type allowlist + no-op removal (elastic/docs-eng-team#623): removes the `missing-h1-anchor` category from the openings sweep (docs-builder auto-generates a heading anchor and H1s are unique, so the fix changes nothing rendered); adds a closed category allowlist to every sweep. - File-list targeting (elastic/docs-eng-team#625): new `target-files` input on the orchestrator and all seven sub-sweeps overrides target-path/scope-mode to process exactly the listed files. Recompiled all lock files with gh-aw v0.83.1 (0 errors, 0 warnings). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
reakaleek
approved these changes
Jul 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements four docs-quality-sweep improvements surfaced by the AI & Docs Challenge cohort (June 2026). New shared fragment
gh-aw-fragments/findings-contract.mdcarries the cross-cutting output contract; all seven sweeps import it. Lock files recompiled with gh-aw v0.83.1 (0 errors, 0 warnings).Confidence signals —
elastic/docs-eng-team#621confidencefield (high/medium/low) with explicit tier criteria (a separate axis fromseverity) to every sweep, surfaced in the fix-issue YAML.docs-issue-scope: strengthens the existing Confidence rating so terminology or capabilities that appear only in an issue/PR (this runs withmin-integrity: none) are attributed rather than asserted and rated Low, with a caveat line under the table — directly targeting the "wrong terminology propagated into Security docs" incident.Human-review gate —
elastic/docs-eng-team#622medium/low-confidence finding, the sweep adds aneeds-human-reviewlabel and a review-before-acting> [!WARNING]callout above the findings.size-logic.md) will not applygood-for-aito an issue labeledneeds-human-review, keeping uncertain output off the auto-delegation track.Finding-type allowlist + no-op removal —
elastic/docs-eng-team#623missing-h1-anchorcategory from the openings sweep. Verified against the docs-builder syntax reference: every heading auto-generates a default anchor and an H1 is unique per page, so adding a custom[anchor-id]to an H1 changes nothing rendered — the confirmed no-op from the cohort.File-list targeting —
elastic/docs-eng-team#625target-filesinput (newline- or comma-separated, docs-root-relative) on the orchestrator and all seven sub-sweeps. When set, it overridestarget-path/scope-modeand processes exactly those files — the precise, low-cost path for post-merge "check only what changed" runs. Cost scales with the file count, not directory size.Testing
make compile→ 12 workflows, 0 errors / 0 warnings.bash -non the injected pre-step scripts (shared + typos variant).target-filesparser against a temp docs tree: comma/newline mixing, leading-slash stripping, and missing-file handling. Caught and fixed a last-entry-dropped bug (while readwithout trailing newline) — now useswhile IFS= read -r raw || [ -n "$raw" ].Closes elastic/docs-eng-team#621
Closes elastic/docs-eng-team#622
Closes elastic/docs-eng-team#623
Closes elastic/docs-eng-team#625
🤖 Generated with Claude Code