Skip to content

fix(fast-mode): gate seed close on clean lint + passed tests; tester fails on collection errors#21

Draft
ryanholtschneider2 wants to merge 1 commit into
mainfrom
agentic-po-formulas-software-dev-7fl
Draft

fix(fast-mode): gate seed close on clean lint + passed tests; tester fails on collection errors#21
ryanholtschneider2 wants to merge 1 commit into
mainfrom
agentic-po-formulas-software-dev-7fl

Conversation

@ryanholtschneider2

Copy link
Copy Markdown
Owner

What & why

Fixes po-formulas-software-dev-7fl. Two independent gates let 6 real test regressions slip through 7 agent-step gates during the pd-17lv CDR-A epic series.

Root cause

  1. software_dev_fast auto-closed the seed regardless of verdict — including when a verdict was empty (the lint/test step never wrote a verdict file, i.e. didn't run / unknown). It only logged a warning, never gated the close.
  2. The tester treated pytest collection errors (a module failing to import) as silent skips — it reported passed off the tests that ran while 9 modules dropped on ModuleNotFoundError.

Fix

  • Fast-mode close (parent/po_formulas/software_dev.py): close the seed only when lint == "clean" AND unit == "passed". Anything else — empty, non-clean, or non-passed — leaves the seed open and returns status="needs-review" so the caller can re-dispatch or escalate to software-dev-full. This makes the code match the flow's own docstring.
  • Tester contract (parent/po_formulas/agents/tester/task.md): run pytest --collect-only first, report a collection_errors count, and treat collection_errors >= 1 as a hard FAILED even if every collected test passed.
  • README updated to document the tighter close gate + collection-error rule.

Tests

New parent/tests/test_software_dev_fast.py:

  • empty lint verdict → seed NOT closed, status == needs-review (the exact incident)
  • failed unit verdict (how a collection error now surfaces) → seed NOT closed
  • empty unit verdict → seed NOT closed
  • clean + passed → seed closed, status == completed
  • tester prompt-contract: --collect-only + collection_errors-gates-on-fail present

Full pack unit suite: green except one pre-existing, order-dependent failure in test_software_dev_pack_path_metadata.py that fails identically on base main in a full-suite run and passes standalone (unrelated to this change).

Acceptance mapping

  • ✅ Tester verdict reports collection_errors; ≥1 fails the run.
  • ✅ Empty lint_verdict blocks fast-mode auto-close (downgrades to needs-review).
  • ✅ Repro: an ImportError surfaces as a FAILED unit verdict → fast-formula does NOT close the seed (covered by test_failed_unit_verdict_blocks_close).

…fails on collection errors (po-formulas-software-dev-7fl)

software_dev_fast auto-closed the seed regardless of verdict — including
when a verdict was *empty* (the step never wrote a verdict file). That
blind spot let 6 real test regressions through the pd-17lv CDR-A series:
9 test modules failed to import while the tester still reported 'passed',
and every child closed with an empty lint verdict.

- Fast-mode now closes the seed only when lint == 'clean' AND unit ==
  'passed'. Empty or non-clean/non-passed verdicts leave the seed open
  and return status='needs-review'. Matches the flow's own docstring.
- Tester task.md: run 'pytest --collect-only' first, report a
  collection_errors count, and treat collection_errors >= 1 (a module
  that won't import) as a hard FAILED rather than a silent skip.
- New tests cover the close gate (empty/failed verdicts block close) and
  the tester prompt contract. README updated.
@ryanholtschneider2
ryanholtschneider2 marked this pull request as draft June 14, 2026 18:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant