Summary
Polish verification failure output to make mismatches faster to diagnose, especially for large stdout/stderr snapshots.
Problem
Current failures can be noisy for large outputs and make it harder to identify the highest-signal differences quickly.
Proposal
Improve VerificationError messaging and diff presentation:
-
Field-level summary first
- e.g.
stdout changed, stderr unchanged, status changed
-
Show diffs only for changed fields
- skip unchanged fields entirely
-
Use compact unified diff with context limit
- configurable context lines (default small, e.g. 3)
-
Truncate very large diffs with clear marker
- include "truncated" indicator and hint for full output option
-
Optional full diff toggle
- env var or config option to force complete diff output
Example Failure Shape
Backspin verification failed!
Record: fixtures/backspin/example.yml
Summary:
- stdout: changed
- stderr: unchanged
- status: changed (expected 0, actual 1)
[stdout diff]
@@ ... @@
- old line
+ new line
...
(diff truncated, set BACKSPIN_FULL_DIFF=1 for full output)
Acceptance Criteria
- changed fields are clearly enumerated before diff blocks
- unchanged fields are omitted from diff section
- large diffs are truncated deterministically with explicit notice
- docs include guidance for full diff mode
Summary
Polish verification failure output to make mismatches faster to diagnose, especially for large stdout/stderr snapshots.
Problem
Current failures can be noisy for large outputs and make it harder to identify the highest-signal differences quickly.
Proposal
Improve
VerificationErrormessaging and diff presentation:Field-level summary first
stdout changed,stderr unchanged,status changedShow diffs only for changed fields
Use compact unified diff with context limit
Truncate very large diffs with clear marker
Optional full diff toggle
Example Failure Shape
Acceptance Criteria