Skip to content

fix(review-fix-loop): unify duplicated JSON-schema validation engine - #116

Merged
shaug merged 1 commit into
mainfrom
claude/115-unify-json-schema-validation
Jul 31, 2026
Merged

fix(review-fix-loop): unify duplicated JSON-schema validation engine#116
shaug merged 1 commit into
mainfrom
claude/115-unify-json-schema-validation

Conversation

@shaug

@shaug shaug commented Jul 31, 2026

Copy link
Copy Markdown
Owner

Summary

Unifies the duplicated generic JSON-schema-subset validation engine
(_path/_is_type/validate_schema) so review-fix-loop's schema validator
reuses the canonical implementation instead of hand-duplicating it:

  • Extends the canonical review-suite/scripts/validate.py's _is_type/
    validate_schema to support "type": "integer" and minimum/maximum
    numeric checks, as a backward-compatible superset. No current
    review-packet/review-result schema field uses integer, so this is
    behavior-preserving for every existing consumer (proved by a new test that
    re-runs every existing fixture packet/result through the extended engine and
    asserts identical outcomes).
  • Refreshes every bundled references/review-suite/validate.py copy via
    just sync-contracts so the extended engine reaches every consuming skill
    (babysit-pr, implement-ticket, review-code-change,
    review-code-simplicity, review-correctness, review-fix-loop,
    review-solution-simplicity).
  • Updates skills/review-fix-loop/scripts/validate.py to import
    _path/_is_type/validate_schema from its bundled
    references/review-suite/validate.py copy via the same
    importlib.util.spec_from_file_location pattern review_gate.py already
    uses for evaluate_bound, instead of hand-duplicating them. Only its own
    schema-specific validate_invocation/validate_checkpoint/
    validate_terminal_result and cross-document checks remain in this file.
  • Adds generic-engine unit tests (integer type, minimum, maximum,
    behavior-preservation) to review-suite/scripts/tests/test_contracts.py
    and a regression test in
    skills/review-fix-loop/scripts/tests/test_validate.py proving
    max_fix_cycles non-integer rejection now flows through the shared engine.

This is a follow-up ticket for epic #95, discovered during the epic's own
closeout review: skills/review-fix-loop/scripts/validate.py (added by #96)
contained a near-verbatim copy of the generic engine already canonical in
review-suite/scripts/validate.py. Unlike every other bundled contract copy in
this repository (kept in sync via just sync-contracts and
test_bundled_contracts.py), this duplicated engine had no drift-detection
test.

Non-goals

  • No behavior change to review-fix-loop's public validation semantics
    (validate_invocation, validate_checkpoint, validate_terminal_result, or
    the cross-document checks) — this is a pure internal deduplication.
  • No change to review-packet/review-result schemas or their existing
    consumers.

Acceptance criteria

  • review-suite/scripts/validate.py's generic engine supports "integer"
    type and minimum/maximum checks without changing behavior for any
    existing schema.
  • skills/review-fix-loop/scripts/validate.py no longer hand-duplicates
    _path/_is_type/validate_schema; it imports and reuses the canonical
    engine.
  • All existing review-fix-loop tests (273) and review-suite tests
    (318) continue to pass unchanged in behavior.
  • A regression test demonstrates the shared engine now enforces
    integer/minimum/maximum for at least one review-fix-loop schema
    field (max_fix_cycles).

Validation

  • just format — pass, 1 file reformatted (new test wrapped by ruff), no
    behavior change.
  • just lint — pass (ruff, mdformat, skills-ref validate on every skill,
    validate_plugins.py).
  • just test — full repository suite, pass, 0 failures.
  • python3 -m unittest discover -s skills/review-fix-loop/scripts/tests -p 'test_*.py' — 274 tests (273 pre-existing + 1 new), pass.
  • python3 -m unittest discover -s review-suite/scripts/tests -p 'test_*.py'
    — 326 tests (318 pre-existing + 8 new), pass.

Review

One full review-code-change pass (fresh, isolated, read-only reviewer given
only raw evidence) returned a clean aggregate verdict on the first pass — no
blocking or strong-recommendation findings from solution-simplicity,
correctness, or code-simplicity. The reviewer independently confirmed: the
canonical engine's additions are structurally inert for every current
review-packet/review-result field (grepped both schemas for integer/
minimum/maximum, found none); the deleted review-fix-loop engine code was
byte-for-byte identical in logic to the canonical engine before this change
(zero behavior drift); all 7 bundled copies are byte-identical to canonical;
and both invocation.schema.json's max_fix_cycles and
terminal-result.schema.json's original_max_fix_cycles declare
{"type": "integer", "minimum": 1, "maximum": 10}, so the new regression test
genuinely exercises the new shared code path.

Guardrail evidence

Diff is 169 insertions / 72 deletions across 12 files — well within
carve-changesets' "a few hundred new or changed lines" preferred range. One
cohesive deliverable (a single internal deduplication), largely mechanical
(bundled-copy resync via just sync-contracts), reviewed clean as one unit on
the first pass. Ordinary single-PR path, no decomposition needed.

Fixes #115
Refs #95

## Summary
- Extend the canonical `review-suite/scripts/validate.py`'s `_is_type`/
  `validate_schema` to support `"type": "integer"` and `minimum`/`maximum`
  numeric checks, as a backward-compatible superset (no current
  `review-packet`/`review-result` schema field uses `integer`, so this is
  behavior-preserving for every existing consumer).
- Refresh every bundled `references/review-suite/validate.py` copy via
  `just sync-contracts` so the extended engine reaches every consuming
  skill.
- Update `skills/review-fix-loop/scripts/validate.py` to import
  `_path`/`_is_type`/`validate_schema` from its bundled
  `references/review-suite/validate.py` copy via the same
  `importlib.util.spec_from_file_location` pattern `review_gate.py` already
  uses for `evaluate_bound`, instead of hand-duplicating them. Only its own
  schema-specific `validate_invocation`/`validate_checkpoint`/
  `validate_terminal_result` and cross-document checks remain in this file.
- Add generic-engine unit tests (integer type, minimum, maximum) to
  `review-suite/scripts/tests/test_contracts.py` and a regression test in
  `skills/review-fix-loop/scripts/tests/test_validate.py` proving
  `max_fix_cycles` non-integer rejection now flows through the shared
  engine.

## Why
An exact-head closeout review of epic #95 found that
`skills/review-fix-loop/scripts/validate.py` (added by #96) contained a
near-verbatim copy of the generic engine already canonical in
`skills/review-suite/scripts/validate.py`, extended only with `"integer"`
type support and `minimum`/`maximum` numeric checks. Unlike every other
bundled contract copy in this repository, this duplicated engine had no
drift-detection test, so a future fix to shared JSON-schema-subset
semantics applied to one copy would silently not apply to the other.

All 273 pre-existing review-fix-loop tests and 318 pre-existing
review-suite tests continue to pass unchanged, plus the 9 new tests this
change adds (274 and 326 respectively).

Fixes #115
@shaug
shaug merged commit c400d77 into main Jul 31, 2026
1 check passed
@shaug
shaug deleted the claude/115-unify-json-schema-validation branch July 31, 2026 10:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unify duplicated JSON-schema validation engine in review-fix-loop

1 participant