feat(patch): structured error context for self-correction (Closes #996)#1007
Closed
Lexus2016 wants to merge 2 commits into
Closed
feat(patch): structured error context for self-correction (Closes #996)#1007Lexus2016 wants to merge 2 commits into
Lexus2016 wants to merge 2 commits into
Conversation
added 2 commits
July 14, 2026 18:04
Enriches patch-failure error output with structured context so the model can self-correct on its next turn without re-reading the file, replacing the need for an architectural LLM re-invocation from the tool handler. Changes: - fuzzy_match.py: add classify_error(), _format_file_context_snippet(), format_structured_error() — builds a structured error package with error-type classification, closest matching region (±5 lines with line numbers), and recovery suggestion matched to error type - file_operations.py: add structured_error field to PatchResult; patch_replace error path now builds structured error context and passes it through; legacy 'Did you mean?' hint suppressed when structured context is available (strictly more useful) - file_tools.py: patch_tool consumes _diagnostic from PatchResult; config-gated retry threshold replaces hardcoded 3; 'PERMANENT FAILURE' classification when threshold exceeded; generic _hint suppressed when structured diagnostic present - config.py: add patch.self_correction_retries config key (default 3, max 5) to DEFAULT_CONFIG - New tests: test_patch_self_correction.py (34 tests covering classify_error, format_structured_error, _format_file_context_snippet, PatchResult.structured_error, config gate, integration) - Updated test: test_patch_failure_tracking.py asserts PERMANENT FAILURE classification in escalating hint Closes #996
Enriches patch-failure error output with structured context so the model can self-correct on its next turn without re-reading the file. Changes: - fuzzy_match.py: classify_error(), _format_file_context_snippet(), format_structured_error() — structured error package with error-type classification, closest matching region (±5 lines with line numbers), and recovery suggestion matched to error type - file_operations.py: structured_error field on PatchResult; error path builds structured context; legacy 'Did you mean?' hint suppressed when structured context is available - file_tools.py: patch_tool consumes _diagnostic; config-gated retry threshold; PERMANENT FAILURE classification - config.py: patch.self_correction_retries in DEFAULT_CONFIG - tests: 34 new tests for self-correction, updated escalation assertion - uv.lock: regenerated for v0.18.3 version bump (fixes PR #997 CI failure) Rework of closed PR #997 — the only issue was a stale uv.lock after the version bump from 0.18.2 to 0.18.3. Code changes reviewed as solid by owner. Closes #996 Co-Authored-By: Hermes Evolution <evolution@hermes.ai>
This was referenced Jul 14, 2026
Owner
Author
|
CI failed on two test slices after re-run — real bugs, not flakes. See rework brief on issue #996. |
This was referenced Jul 15, 2026
Lexus2016
added a commit
that referenced
this pull request
Jul 15, 2026
…lures (#1017) Implements structured error context and automatic guided retry for patch/write_file failures (issue #996). Builds on existing fuzzy_match.py infrastructure. Changes: - tools/fuzzy_match.py: Add classify_error() and format_structured_error() to package rich error context (error type, closest matches, surrounding content, line numbers) for the model - tools/file_tools.py: Add _get_self_correction_retries() config-gated retry threshold, integrate structured error feedback into patch failures, escalate to 'permanent' classification after threshold exhausted - tools/file_operations.py: Integrate structured error context into write_file and patch paths, lint-after-patch integration - hermes_cli/config.py: Add patch.self_correction_retries (default 3, max 5) and patch.lint_after_patch (default True) config options - acp_registry/agent.json: Bump version 0.18.2 → 0.18.3 - pyproject.toml: Bump version 0.18.2 → 0.18.3 - uv.lock: Regenerate for v0.18.3 - tests/tools/test_patch_self_correction.py: New test suite (20 tests) - tests/tools/test_patch_failure_tracking.py: Updated tests Fixes two bugs from closed PR #1007: 1. Config schema: 'patch' category now has 2 fields (self_correction_retries + lint_after_patch), satisfying dashboard invariant ≥2 fields/category 2. Version mismatch: agent.json version and uvx.package pin updated to 0.18.3 to match pyproject.toml Closes #996 Co-authored-by: Hermes Evolution <evolution@hermes.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rework of closed PR #997 — enriches patch-failure error output with structured error context so the model can self-correct on its next turn without re-reading the file.
Problem (Issue #996)
When a patch fails, the model receives a bare error string like
Could not find match for old_stringwith no context about what's actually in the file. This causes retry loops where the model makes slight variations of the same incorrect old_string, wasting turns and tokens.Solution
Instead of re-invoking the LLM (which would violate the narrow-waist principle and break prompt caching), we enrich the tool-result error so the model naturally self-corrects on its next turn.
Error package contents
no_match,ambiguous,identical,escape_drift,permission, etc.Config gate
Controls how many consecutive patch failures on the same file before the error is classified as
PERMANENT FAILUREand the model is instructed to stop retrying.Rework Note
PR #997 was closed because
uv.lockwas not regenerated after the version bump0.18.2 → 0.18.3, causing all CI test slices to fail atuv sync --locked. This PR regeneratesuv.lockand also appliesruff formatto the changed files.Files changed
tools/fuzzy_match.py—classify_error(),_format_file_context_snippet(),format_structured_error()tools/file_operations.py—PatchResult.structured_errorfield;patch_replacebuilds structured context on failuretools/file_tools.py—patch_toolconsumes_diagnostic; config-gated retry threshold;PERMANENT FAILUREclassificationhermes_cli/config.py—patch.self_correction_retriesinDEFAULT_CONFIGtests/tools/test_patch_self_correction.py— 34 new teststests/tools/test_patch_failure_tracking.py— updated escalation assertionuv.lock— regenerated for v0.18.3Test results
Closes #996