Skip to content

Improve iterative design-validation loop for highly technical/impactful changes #259

Description

@JRedeker

Context

During the improveCrossProject change (a highly technical, high-impact change touching cross-project durable state, Temporal workflow semantics, and multi-store consistency), the design phase required 6 iterations of independent validator (adv-researcher) review before reaching convergence. Each iteration surfaced 2-6 genuine architectural findings.

Iteration finding-count trend: 5 → 6 → 4 → 3 → 4 → 2 (slowly converging).

Problem

For highly technical / impactful changes, the current design-validation loop has these friction points:

  1. No upfront classification of "high-technical-impact" — the loop topology (single validator pass → approve) is identical for trivial and structural changes. High-impact changes need more rigorous upfront scoping.
  2. Validator finds new concerns each round — each iteration is genuinely valuable but expensive in agent context. No mechanism to bound iteration count or escalate to human architect.
  3. No "design debt" surface — when iterations hit practical limits, there is no first-class way to defer findings as fast-follow changes while proceeding with documented design debt.
  4. No multi-validator triangulation — single adv-researcher pass has one perspective. High-impact changes might benefit from adv-tron (structural) + adv-researcher (external) + adv-reviewer (implementation feasibility) cross-check.
  5. Agreement artifact evolution is implicit — the agreement changed mid-session (added AC11-AC15 + avoidance 6), invalidating earlier design assumptions. No versioning signal to the design phase.

Proposal

Add an "impact tier" classification at proposal/agreement time that gates design-phase rigor:

Tier Trigger Design-phase path
standard Single-component, well-understood surface Single validator pass; approve on PASS/CAUTION
high-impact Cross-cutting, durable-state, authn/authz, workflow semantics, multi-store consistency Multi-validator triangulation + bounded iteration budget (default 3) + design-debt escape hatch
critical Security-bearing, data-loss-bearing, core-contract changes Above + mandatory human architect review at design gate

Specific additions:

  • Impact classification field in proposal artifact (impact_tier: standard | high-impact | critical); machine-enforced; human override allowed.
  • Bounded iteration budget — after N validator rounds (default 3 for high-impact, 5 for critical), if still CONFLICT, auto-pause and offer: continue / accept-with-debt / descope / human-architect.
  • Design debt first-classadv_change_update_design_debt tool that records deferred findings as fast-follow ADV child changes linked from parent.
  • Multi-validator triangulationadv_design_validate_multi that runs adv-researcher + adv-tron + adv-reviewer in parallel and synthesizes.
  • Agreement versioning — agreement artifact carries revision field; design artifacts reference agreement_revision; mismatch triggers re-validation.

Evidence

Observed directly on improveCrossProject (2026-07-20):

  • 6 validator iterations required
  • Each iteration consumed meaningful agent context (design body grew from ~2k words to ~5k words)
  • Several findings were genuinely load-bearing (Signal-cannot-return, cross-store atomic rollback impossible, CWE-367 race) and would have shipped structural debt if skipped
  • No mechanism to bound iteration or escalate short of user-initiated pause
  • Agreement artifact was updated mid-session; design assumed stale snapshot for one round

Out of scope

  • Replacing the validator (adv-researcher works well; this is about loop topology)
  • Changing the gate sequence (proposal/discovery/design/planning/etc unchanged)
  • Auto-approving designs (always human checkpoint at design gate)

Related

  • Change: improveCrossProject
  • Iteration logs: improveCrossProject|change:researcher:design-validation-{v1..v6}|adv-researcher|{1..6}

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions