diff --git a/.agents/skills/map-efficient/SKILL.md b/.agents/skills/map-efficient/SKILL.md index 7ab5e4e4..e92c4e17 100644 --- a/.agents/skills/map-efficient/SKILL.md +++ b/.agents/skills/map-efficient/SKILL.md @@ -17,7 +17,10 @@ section when the workflow below points to it. Under `isolation_active` (Slice 5a the wave-loop creates per-member worktrees, dispatches actor subagents **sequentially** (one per turn), verifies via `concurrency_ready`, then accepts atomically via `merge_wave_worktrees`; concurrent fan-out is Slice 5b -(`dispatch_mode==concurrent`). +(`dispatch_mode==concurrent`). Under `dispatch_mode==concurrent` (opt-in via +`execution.concurrent_dispatch: true`), call `run_concurrent_wave`: dispatch N +actor subagents in **one turn** per sub-batch; on any failure `abort_wave_group` +discards the whole group and reruns from base (bounded by `max_wave_retries`). ## Mutation Boundary Constraints diff --git a/.agents/skills/map-efficient/efficient-reference.md b/.agents/skills/map-efficient/efficient-reference.md index fb3597db..80f57aa4 100644 --- a/.agents/skills/map-efficient/efficient-reference.md +++ b/.agents/skills/map-efficient/efficient-reference.md @@ -115,7 +115,9 @@ wave-loop on every run. The wave-loop engages **only when ALL THREE hold** `mapify init` config always runs the legacy sequential walker. Even when the wave-loop engages, dispatch remains **sequential** in Slice 5a (`isolation_active=True`, `dispatch_mode` from `get_wave_step` keyed to `sequential`); concurrent fan-out -is Slice 5b (`dispatch_mode==concurrent`, `concurrency_enabled=True`, not yet shipped). +is Slice 5b (`dispatch_mode==concurrent`, `concurrency_enabled=True`), +**ACTIVE when opted in** via `execution.concurrent_dispatch: true` +(gate: `dispatch_mode == 'concurrent'`). ### Sequential walker @@ -158,24 +160,26 @@ to base on any conflict or gate failure (worktrees kept for retry). On a single subtask's Monitor failure, `discard_subtask_worktree` that subtask and retry it before calling `merge_wave_worktrees`. -### Concurrent actor dispatch — **Slice 5b only** (`dispatch_mode == 'concurrent'`) — GATED EXAMPLE +### Concurrent actor dispatch — **Slice 5b** (`dispatch_mode == 'concurrent'`) — **ACTIVE when opted in** -> **IMPORTANT — read before using this example.** -> Concurrent fan-out (dispatching multiple actor subagents in a single turn) is -> **Slice 5b** and is enabled **only when `concurrency_enabled: true` / -> `parallel_ready` flag set / `dispatch_mode == 'concurrent'`**. In the **current -> framework** (`concurrency_enabled=False`, Slice 5a), dispatch stays **SEQUENTIAL +> **IMPORTANT — read before using this section.** +> Concurrent fan-out (dispatching N actor subagents in a single turn) is +> **ACTIVE when opted in** via `execution.concurrent_dispatch: true` +> (gate: `dispatch_mode == 'concurrent'`). With the **default config** +> (`concurrent_dispatch=false`, Slice 5a), dispatch stays **SEQUENTIAL > even when a wave has `mode=="parallel"`** — one actor subagent per turn, each -> pinned to its own worktree. The example below is reference material for when -> Slice 5b ships; do NOT treat it as an active instruction now. Use your runtime's +> pinned to its own worktree. Act on the instructions below **only** when +> `get_wave_step` returns `dispatch_mode == 'concurrent'`. Use your runtime's > own parallel actor-subagent dispatch mechanism — this is the provider-neutral > shape, not a literal API call. -When Slice 5b concurrency is enabled, a parallel wave with N subtasks dispatches -all N actor subagents in **one turn**: +**Runtime wiring:** when `get_wave_step` returns `dispatch_mode == 'concurrent'`, +call `run_concurrent_wave` (runner), which batch-splits the wave by `max_actors` +and merges each sub-batch atomically via `merge_wave_worktrees`. For each sub-batch, +dispatch all N actor subagents **in one turn** — not one per turn: ```text -# CORRECT (Slice 5b / concurrency_enabled=True / dispatch_mode=='concurrent' only) — one turn, N actor subagents: +# CORRECT (dispatch_mode=='concurrent' only) — N actor subagents in one turn: dispatch actor subagent -> ST-003 (pinned to its own worktree) dispatch actor subagent -> ST-004 (pinned to its own worktree) @@ -189,6 +193,13 @@ dispatch actor subagent -> ST-004 (pinned to its own worktree) **`max_actors` cap:** Default 4–8 per wave. Groups larger than `max_actors` are pre-split into sequential batches before dispatch. +**Retry-discard on failure:** on any actor failure, timeout, or Monitor-reject +within a concurrent group, the runner calls `abort_wave_group`, which discards +the **entire group** (cancels siblings, resets all worktrees to base SHA, removes +group branches) and reruns from base. Retries are bounded by `max_wave_retries` +(default 3); on exhaustion the runner escalates to a human and does **not** +auto-restart. Never merges a successful subset — discard-all-or-merge-all (HC-4). + ### Anti-patterns — Slice 5b concurrent dispatch only > These apply **only** under Slice 5b concurrent dispatch (`dispatch_mode == 'concurrent'`). In Slice 5a and the default sequential walker, one actor dispatch per turn **is** the correct behavior. diff --git a/.claude/skills/map-efficient/SKILL.md b/.claude/skills/map-efficient/SKILL.md index efb7b7fc..9d199bc2 100644 --- a/.claude/skills/map-efficient/SKILL.md +++ b/.claude/skills/map-efficient/SKILL.md @@ -207,7 +207,7 @@ else fi ``` -**Execution strategy:** `select_execution_strategy` chooses between the legacy sequential walker and the wave-loop. The wave-loop (`get_wave_step` / `validate_wave_step` / `advance_wave`) engages only when `execution.wave_mode ∈ {on, auto}` AND a color group has ≥2 members; otherwise `get_next_step` (sequential walker) runs. Under `isolation_active` (Slice 5a), the wave-loop creates per-member worktrees, dispatches Actors **sequentially** (one per turn, `HC-3`), verifies via `concurrency_ready`, then accepts atomically via `merge_wave_worktrees`; concurrent fan-out is Slice 5b (`dispatch_mode==concurrent`). See [efficient-reference.md](efficient-reference.md#wave-execution) for the decision table and full wave loop. +**Execution strategy:** `select_execution_strategy` chooses between the legacy sequential walker and the wave-loop. The wave-loop (`get_wave_step` / `validate_wave_step` / `advance_wave`) engages only when `execution.wave_mode ∈ {on, auto}` AND a color group has ≥2 members; otherwise `get_next_step` (sequential walker) runs. Under `isolation_active` (Slice 5a), the wave-loop creates per-member worktrees, dispatches Actors **sequentially** (one per turn, `HC-3`), verifies via `concurrency_ready`, then accepts atomically via `merge_wave_worktrees`; concurrent fan-out is Slice 5b (`dispatch_mode==concurrent`). Under `dispatch_mode==concurrent` (opt-in via `execution.concurrent_dispatch: true`), call `run_concurrent_wave`: emit N `Task(actor)` blocks in **one message** per sub-batch; on any failure `abort_wave_group` discards the whole group and reruns from base (bounded by `max_wave_retries`). See [efficient-reference.md](efficient-reference.md#wave-execution) for the decision table and full wave loop. **Note on resume:** `resume_from_plan` (Step 0) now auto-invokes `set_waves` when `blueprint.json` is present, so resumed workflows do not need a manual diff --git a/.claude/skills/map-efficient/efficient-reference.md b/.claude/skills/map-efficient/efficient-reference.md index f855be02..1c118702 100644 --- a/.claude/skills/map-efficient/efficient-reference.md +++ b/.claude/skills/map-efficient/efficient-reference.md @@ -136,7 +136,7 @@ including clean passes — must carry concrete evidence references. | `auto` / `on` | `auto` / `required` | no (all groups size 1) | Legacy sequential walker (`get_next_step`) | | `auto` / `on` | `auto` / `required` | yes | Wave-loop (`get_wave_step` / `validate_wave_step` / `advance_wave`) | -**Defaults (canonical MapConfig):** `execution.wave_mode=auto`, `worktree.isolation=off`. Because the isolation gate (#2) fails by default, a stock `mapify init` config always runs the legacy sequential walker — byte-identical to pre-Slice-3. Even when the wave-loop does engage, dispatch remains **sequential** in Slice 5a (`isolation_active=True`, `dispatch_mode` from `get_wave_step` keyed to `sequential`); concurrent fan-out is Slice 5b (`dispatch_mode==concurrent`, `concurrency_enabled=True`, not yet shipped). +**Defaults (canonical MapConfig):** `execution.wave_mode=auto`, `worktree.isolation=off`. Because the isolation gate (#2) fails by default, a stock `mapify init` config always runs the legacy sequential walker — byte-identical to pre-Slice-3. Even when the wave-loop does engage, dispatch remains **sequential** in Slice 5a (`isolation_active=True`, `dispatch_mode` from `get_wave_step` keyed to `sequential`); concurrent fan-out is Slice 5b (`dispatch_mode==concurrent`, `concurrency_enabled=True`), **ACTIVE when opted in** via `execution.concurrent_dispatch: true` (gate: `dispatch_mode == 'concurrent'`). ### Sequential walker @@ -148,21 +148,24 @@ Use `get_wave_step`, `validate_wave_step`, and `advance_wave` when the wave-loop When the wave-loop engages AND `isolation_active` is true (`worktree.isolation` ∈ {`auto`, `required`}), the Slice 5a flow applies: (a) create a worktree per wave member via `create_subtask_worktree`; (b) dispatch the member Actors **sequentially** — one per turn, each pinned to its own worktree path (`HC-3`); (c) call `concurrency_ready` (ST-003) to verify all member worktrees before merge; (d) accept the whole wave atomically via `merge_wave_worktrees` — never one-at-a-time, with whole-wave rollback on any failure. See [Parallel waves](#worktree-isolation) under Worktree isolation for the full protocol. Concurrent fan-out (dispatching all Actors in one message) is Slice 5b (`dispatch_mode==concurrent`) and is not yet active. -### Concurrent Actor dispatch — **Slice 5b only** (`dispatch_mode == 'concurrent'`) — GATED EXAMPLE +### Concurrent Actor dispatch — **Slice 5b** (`dispatch_mode == 'concurrent'`) — **ACTIVE when opted in** -> **IMPORTANT — read before using this example.** +> **IMPORTANT — read before using this section.** > Concurrent fan-out (emitting multiple `Task(actor)` calls in a single message) is -> **Slice 5b** and is enabled **only when `concurrency_enabled: true` / -> `parallel_ready` flag set / `dispatch_mode == 'concurrent'`**. In the **current -> framework** (`concurrency_enabled=False`, Slice 5a), dispatch stays **SEQUENTIAL +> **ACTIVE when opted in** via `execution.concurrent_dispatch: true` +> (gate: `dispatch_mode == 'concurrent'`). With the **default config** +> (`concurrent_dispatch=false`, Slice 5a), dispatch stays **SEQUENTIAL > even when a wave has `mode=="parallel"`** — one Actor per turn, each pinned to -> its own worktree. The example below is reference material for when Slice 5b ships; -> do NOT treat it as an active instruction now. +> its own worktree. Act on the instructions below **only** when `get_wave_step` +> returns `dispatch_mode == 'concurrent'`. -When Slice 5b concurrency is enabled, a parallel wave with N subtasks dispatches all N Actors in **one message** with N `Task` calls — not one per turn: +**Runtime wiring:** when `get_wave_step` returns `dispatch_mode == 'concurrent'`, +call `run_concurrent_wave` (runner), which batch-splits the wave by `max_actors` +and merges each sub-batch atomically via `merge_wave_worktrees`. For each sub-batch, +emit all N `Task(actor)` calls in **one assistant message** — not one per turn: ```text -# CORRECT (Slice 5b / concurrency_enabled=True / dispatch_mode=='concurrent' only) — N Task calls in one message: +# CORRECT (dispatch_mode=='concurrent' only) — N Task calls in one message: Task( subagent_type="actor", description="Implement ST-003", @@ -185,6 +188,8 @@ Task( **`max_actors` cap:** Default 4–8 concurrent actors per wave. Groups larger than `max_actors` are pre-split into sequential batches of `max_actors` before dispatch; do not emit more than `max_actors` Task calls in a single message. +**Retry-discard on failure:** on any actor failure, timeout, or Monitor-reject within a concurrent group, the runner calls `abort_wave_group`, which discards the **entire group** (cancels siblings, resets all worktrees to base SHA, removes group branches) and reruns from base. Retries are bounded by `max_wave_retries` (default 3); on exhaustion the runner escalates to a human and does **not** auto-restart. Never merges a successful subset — discard-all-or-merge-all (HC-4). + ### Anti-patterns — Slice 5b concurrent dispatch only > These apply **only** under Slice 5b concurrent dispatch (`dispatch_mode == 'concurrent'`). In Slice 5a and the default sequential walker, one Task per turn **is** the correct behavior — the first three below are NOT anti-patterns there. diff --git a/.map/scripts/map_orchestrator.py b/.map/scripts/map_orchestrator.py index e2dad2c4..f88a2fc0 100755 --- a/.map/scripts/map_orchestrator.py +++ b/.map/scripts/map_orchestrator.py @@ -255,6 +255,19 @@ def _extract_subtask_ids_from_plan_artifacts( WAVE_REASON_NO_WAVES = "no_waves" WAVE_REASON_WAVE_COMPLETE = "wave_complete" WAVE_REASON_DISPATCH_SEQUENTIAL = "dispatch_sequential_5a" +# Stable reason codes for compute_dispatch_gate (ST-001, Slice 5b). +WAVE_REASON_CONCURRENT_GATED = "concurrent_gated" +WAVE_REASON_GATE_NOT_PARALLELIZABLE = "gate_not_parallelizable" +# Current wave is width-1 even though a later wave is parallel; dispatch sequentially +# for this wave — not an error, just the natural plan structure. +WAVE_REASON_CURRENT_WAVE_SEQUENTIAL = "current_wave_sequential" + + +class DispatchGateError(RuntimeError): + """Raised when concurrent_dispatch=true but a required prerequisite is missing. + + HC-3: never silent-degrade — callers must handle explicitly. + """ def _read_map_config_scalars(project_dir: Path) -> dict[str, str]: @@ -2303,8 +2316,16 @@ def get_wave_step(branch: str) -> dict: state_file = Path(f".map/{branch}/step_state.json") state = StepState.load(state_file) - # Compute structured dispatch signal fields (ST-002). - dispatch_mode = "concurrent" if WAVE_CONCURRENCY_ENABLED else "sequential" + # Compute structured dispatch signal via config-driven gate (ST-001, Slice 5b). + # compute_dispatch_gate short-circuits to sequential on the first line when + # concurrent_dispatch=false (default), touching no new code/probe/import (HC-1). + gate = compute_dispatch_gate(branch, Path(".")) + dispatch_mode = gate["dispatch_mode"] + dispatch_reason = gate["reason"] + # concurrency_enabled alias: True iff dispatch_mode resolved to "concurrent". + # WAVE_CONCURRENCY_ENABLED is kept as a dormant unused const (backward compat). + concurrency_enabled = dispatch_mode == "concurrent" + try: from map_step_runner import ( # pyright: ignore[reportMissingImports] _worktree_isolation_mode, @@ -2319,8 +2340,8 @@ def get_wave_step(branch: str) -> dict: "wave_index": 0, "subtasks": [], "is_complete": True, - "concurrency_enabled": dispatch_mode == "concurrent", - "dispatch_mode": dispatch_mode, + "concurrency_enabled": concurrency_enabled, + "dispatch_mode": "sequential", "isolation_active": isolation_active, "reason": WAVE_REASON_NO_WAVES, "message": "No execution waves configured. Use sequential mode.", @@ -2332,8 +2353,8 @@ def get_wave_step(branch: str) -> dict: "wave_index": state.current_wave_index, "subtasks": [], "is_complete": True, - "concurrency_enabled": dispatch_mode == "concurrent", - "dispatch_mode": dispatch_mode, + "concurrency_enabled": concurrency_enabled, + "dispatch_mode": "sequential", "isolation_active": isolation_active, "reason": WAVE_REASON_WAVE_COMPLETE, } @@ -2372,12 +2393,10 @@ def get_wave_step(branch: str) -> dict: "wave_total": len(state.execution_waves), "subtasks": subtask_infos, "is_complete": False, - # concurrency_enabled=False: even when mode=="parallel" (width>=2 wave), - # dispatch is strictly sequential this slice. Slice 5 flips WAVE_CONCURRENCY_ENABLED. - "concurrency_enabled": dispatch_mode == "concurrent", + "concurrency_enabled": concurrency_enabled, "dispatch_mode": dispatch_mode, "isolation_active": isolation_active, - "reason": WAVE_REASON_DISPATCH_SEQUENTIAL, + "reason": dispatch_reason, } @@ -2546,6 +2565,113 @@ def select_execution_strategy( } +def compute_dispatch_gate( + branch: str, project_dir: Optional[Path] = None +) -> dict: + """Compute the dispatch mode for the current wave, fail-closed on config contradiction. + + Gate logic (evaluated in order): + + 1. If concurrent_dispatch is False (default): return sequential immediately. + FIRST executable line — no probe, no select_execution_strategy call, no import + of any concurrency primitive (HC-1 byte-identity). + + 2. If concurrent_dispatch is True AND worktree.isolation == 'off': + raise DispatchGateError — config contradiction, HC-3 never silent-degrade. + + 3. If concurrent_dispatch is True AND isolation != 'off' AND NOT concurrency_allowed: + return sequential with WAVE_REASON_GATE_NOT_PARALLELIZABLE (not an error — + the plan has no parallelizable groups). + + 4. If concurrent_dispatch is True AND isolation != 'off' AND concurrency_allowed + AND the CURRENT wave (execution_waves[current_wave_index]) has width < 2: + return sequential with WAVE_REASON_CURRENT_WAVE_SEQUENTIAL (not an error — + the current wave is width-1 even though a later wave is parallel). + + 5. If concurrent_dispatch is True AND isolation != 'off' AND concurrency_allowed + AND the CURRENT wave has width >= 2: + return concurrent with WAVE_REASON_CONCURRENT_GATED. + + Args: + branch: Git branch name (sanitized). + project_dir: Project root containing .map/config.yaml. + Defaults to Path('.'). + + Returns: + {"dispatch_mode": "sequential" | "concurrent", "reason": } + + Raises: + DispatchGateError: When concurrent_dispatch=true but worktree.isolation='off' + (HC-3: config contradiction must never be silently degraded). + """ + if project_dir is None: + project_dir = Path(".") + + # Step 1: short-circuit on flag=false — first gate check after parameter + # normalization. No concurrency probe, no select_execution_strategy call, no + # _worktree_isolation_mode/concurrency_ready call, no dispatcher import runs on + # this path (HC-1 byte-identity). The project_dir None-guard above is a safe + # default-arg normalization, not a concurrency primitive. + try: + from map_step_runner import ( # pyright: ignore[reportMissingImports] + _concurrent_dispatch_enabled, + ) + flag_on = _concurrent_dispatch_enabled(project_dir) + except ImportError: + flag_on = False + + if not flag_on: + return { + "dispatch_mode": "sequential", + "reason": WAVE_REASON_DISPATCH_SEQUENTIAL, + } + + # Step 2: flag is on — check isolation config. + try: + from map_step_runner import ( # pyright: ignore[reportMissingImports] + _worktree_isolation_mode as _wt_iso, + ) + isolation = _wt_iso(project_dir) + except ImportError: + isolation = "off" + + if isolation == "off": + raise DispatchGateError( + "concurrent_dispatch=true requires worktree.isolation != 'off', " + f"but worktree.isolation is 'off' in {project_dir}. " + "Set worktree.isolation to 'auto' or 'required' to enable concurrent dispatch." + ) + + # Step 3: check whether the plan is actually parallelizable (any wave has width>=2). + strategy_result = select_execution_strategy(branch, project_dir) + concurrency_allowed = strategy_result.get("concurrency_allowed", False) + + if not concurrency_allowed: + return { + "dispatch_mode": "sequential", + "reason": WAVE_REASON_GATE_NOT_PARALLELIZABLE, + } + + # Step 4: plan has at least one parallel wave, but gate on the ACTIVE wave. + # select_execution_strategy checks any wave (has_parallel_groups), not the + # current wave index. A width-1 current wave must dispatch sequentially even + # if a later wave is parallel — dispatch_mode is per-wave, not per-plan. + state_file = Path(f".map/{branch}/step_state.json") + state = StepState.load(state_file) + waves = state.execution_waves + idx = state.current_wave_index + if idx >= len(waves) or len(waves[idx]) < 2: + return { + "dispatch_mode": "sequential", + "reason": WAVE_REASON_CURRENT_WAVE_SEQUENTIAL, + } + + return { + "dispatch_mode": "concurrent", + "reason": WAVE_REASON_CONCURRENT_GATED, + } + + def _write_feedback_file( branch: str, filename: str, header: str, feedback: str ) -> Optional[str]: diff --git a/.map/scripts/map_step_runner.py b/.map/scripts/map_step_runner.py index a343f61a..a7d3fe4f 100755 --- a/.map/scripts/map_step_runner.py +++ b/.map/scripts/map_step_runner.py @@ -148,6 +148,20 @@ def _map_config_int(project_dir: Path, key: str, default: int) -> int: _WAVE_MODE_VALID = frozenset({"off", "auto", "on"}) +# Truthy string values for boolean-style config flags. +_CONCURRENT_DISPATCH_TRUTHY = frozenset({"true", "yes", "y", "1", "on"}) + + +def _concurrent_dispatch_enabled(project_dir: Path) -> bool: + """Return True when execution.concurrent_dispatch is explicitly enabled. + + Mirrors the _wt_isolation_enabled pattern. Default is False (off) so the + sequential path stays byte-identical to Slice 5a by default (HC-1). Never + raises. + """ + raw = _map_config_str(project_dir, "execution.concurrent_dispatch", "false") + return raw.strip().lower() in _CONCURRENT_DISPATCH_TRUTHY + def _execution_wave_mode(project_dir: Path) -> str: """Return the execution.wave_mode setting: 'off' | 'auto' | 'on'. @@ -15348,6 +15362,23 @@ def _wt_force_remove(path: Path, branch_ref: str) -> None: _WT_REASON_NOT_REGISTERED: str = "not_registered" _WT_REASON_HEAD_MISMATCH: str = "head_mismatch" _WT_REASON_DIRTY: str = "dirty" +# group lifecycle reason codes (5b.1) +_WT_REASON_GROUP_HEAD_MISMATCH: str = "group_head_mismatch" +_WT_REASON_GROUP_DIRTY_TREE: str = "group_dirty_tree" +_WT_REASON_GROUP_WORKTREES_REMAIN: str = "group_worktrees_remain" +# group lifecycle event codes (5b.1) — stable set; classifier replays these +_WT_GROUP_EVENT_CREATED: str = "created" +_WT_GROUP_EVENT_STARTED: str = "started" +_WT_GROUP_EVENT_FINISHED: str = "finished" +_WT_GROUP_EVENT_MERGED: str = "merged" +_WT_GROUP_EVENT_ABORTED: str = "aborted" +_WT_GROUP_VALID_EVENTS: frozenset[str] = frozenset({ + _WT_GROUP_EVENT_CREATED, + _WT_GROUP_EVENT_STARTED, + _WT_GROUP_EVENT_FINISHED, + _WT_GROUP_EVENT_MERGED, + _WT_GROUP_EVENT_ABORTED, +}) _WT_ISOLATION_VALID = frozenset({"off", "auto", "required"}) @@ -15818,6 +15849,308 @@ def cleanup_orphan_worktrees(branch: str) -> dict[str, object]: return {"removed": removed, "kept_active": kept_active, "ok": True} +# --------------------------------------------------------------------------- +# Group lifecycle verbs (5b.1) — coordinator-owned, idempotent +# --------------------------------------------------------------------------- + + +def begin_wave_group( + group_ids: list[str], + branch: Optional[str] = None, +) -> dict[str, object]: + """Record the base_sha anchor + per-subtask lifecycle skeleton for a parallel group. + + Stores state under ``wave_groups[group_id]`` in the branch-scoped worktree-state + sidecar. Idempotent: re-invoking with the same group_ids does not duplicate + entries or overwrite ``base_sha`` if already set (crash-safe recovery). + + Returns:: + + { + "ok": True, + "group_id": , + "base_sha": , + "subtask_ids": [...], + } + """ + if not _wt_is_git_repo(): + return _wt_error(_WT_REASON_NOT_GIT_REPO, "not inside a git work tree") + + branch_name = branch or get_branch_name() + base_sha = _wt_head_sha() + if base_sha is None: + return _wt_error("no_head_sha", "could not resolve HEAD sha") + + # Canonical group key: sorted ids joined so order-of-call doesn't vary the key. + ids_sorted = sorted(str(s) for s in group_ids if str(s).strip()) + group_key = "|".join(ids_sorted) + + state = _read_worktree_state(branch_name) + if not isinstance(state.get("wave_groups"), dict): + state["wave_groups"] = {} + wave_groups = state["wave_groups"] + if not isinstance(wave_groups, dict): + wave_groups = {} + state["wave_groups"] = wave_groups + + if group_key not in wave_groups: + wave_groups[group_key] = { + "base_sha": base_sha, + "subtask_ids": ids_sorted, + "lifecycle": {}, # subtask_id -> list of {seq, event, ts} + } + else: + # Idempotent: fill in missing skeleton fields without overwriting base_sha. + existing = wave_groups[group_key] + if not isinstance(existing, dict): + existing = {} + wave_groups[group_key] = existing + existing.setdefault("base_sha", base_sha) + existing.setdefault("subtask_ids", ids_sorted) + if not isinstance(existing.get("lifecycle"), dict): + existing["lifecycle"] = {} + # Ensure every id has a slot + for sid in ids_sorted: + existing["lifecycle"].setdefault(sid, []) + + _write_worktree_state(branch_name, state) + return { + "ok": True, + "group_key": group_key, + "base_sha": base_sha, + "subtask_ids": ids_sorted, + } + + +def record_group_lifecycle( + group_key: str, + subtask_id: str, + event: str, + branch: Optional[str] = None, +) -> dict[str, object]: + """Append a lifecycle event for *subtask_id* inside *group_key*. + + Events are appended with a monotonically-increasing sequence number so + replaying the list in ``seq`` order reconstructs a deterministic in-flight + timeline (used by the classifier in ST-003 to derive ``max_in_flight``). + + *event* must be one of the stable codes in ``_WT_GROUP_VALID_EVENTS`` + (created / started / finished / merged / aborted). + + Returns:: + + { + "ok": True, + "group_key": ..., + "subtask_id": ..., + "event": ..., + "seq": , + } + """ + if event not in _WT_GROUP_VALID_EVENTS: + return _wt_error( + "invalid_event", + f"event {event!r} not in valid set {sorted(_WT_GROUP_VALID_EVENTS)}", + ) + + branch_name = branch or get_branch_name() + + # Serialize concurrent record_group_lifecycle calls from SEPARATE PROCESSES + # (actor worktrees) with an advisory file lock. threading.Lock is insufficient + # because actors are separate OS processes. Reuses the same fcntl pattern as + # _wt_acquire_merge_lock/_wt_release_merge_lock (wave-lifecycle.lock is a + # distinct lock from wave-merge.lock so lifecycle appends never block merges). + _lc_lock = _wt_acquire_lifecycle_lock() + try: + state = _read_worktree_state(branch_name) + + wave_groups = state.get("wave_groups") + if not isinstance(wave_groups, dict) or group_key not in wave_groups: + return _wt_error("unknown_group", f"group {group_key!r} not found; call begin_wave_group first") + + group = wave_groups[group_key] + if not isinstance(group, dict): + return _wt_error("corrupt_group", f"group record for {group_key!r} is malformed") + + lifecycle = group.get("lifecycle") + if not isinstance(lifecycle, dict): + lifecycle = {} + group["lifecycle"] = lifecycle + + sid = str(subtask_id).strip() + if sid not in lifecycle: + lifecycle[sid] = [] + events_list = lifecycle[sid] + if not isinstance(events_list, list): + events_list = [] + lifecycle[sid] = events_list + + # Monotonic seq: max of all existing seqs across ALL subtasks in this group + 1. + max_seq = 0 + for ev_list in lifecycle.values(): + if isinstance(ev_list, list): + for ev in ev_list: + if isinstance(ev, dict): + max_seq = max(max_seq, int(ev.get("seq", 0))) + seq = max_seq + 1 + + import time as _time # local import — keeps module-level imports minimal + events_list.append({"seq": seq, "event": event, "ts": _time.time()}) + + _write_worktree_state(branch_name, state) + finally: + _wt_release_lifecycle_lock(_lc_lock) + + return {"ok": True, "group_key": group_key, "subtask_id": sid, "event": event, "seq": seq} + + +def verify_group_clean( + branch: Optional[str] = None, +) -> dict[str, object]: + """Read-only check: repo is in a clean state after a wave group completes. + + Returns ``clean=True`` iff ALL of: + 1. HEAD sha == the recorded ``base_sha`` for every group (HEAD not diverged). + 2. Working tree is clean (``git status --porcelain`` minus runtime-state paths). + 3. Zero group worktrees remain (``wave_groups`` dict is empty or all groups + have been removed from the sidecar). + + Returns:: + + { + "clean": bool, + "reason": | None, + "head_sha": ..., + "base_sha": ..., + } + """ + if not _wt_is_git_repo(): + return { + "clean": False, + "reason": _WT_REASON_NOT_GIT_REPO, + "head_sha": None, + "base_sha": None, + } + + branch_name = branch or get_branch_name() + head_sha = _wt_head_sha() + state = _read_worktree_state(branch_name) + + # Collect base_shas from all groups + wave_groups = state.get("wave_groups") + if isinstance(wave_groups, dict) and wave_groups: + # Any group that has a recorded base_sha must match HEAD. + for _gk, grp in wave_groups.items(): + if not isinstance(grp, dict): + continue + recorded_base = grp.get("base_sha") + if recorded_base and head_sha != recorded_base: + return { + "clean": False, + "reason": _WT_REASON_GROUP_HEAD_MISMATCH, + "head_sha": head_sha, + "base_sha": recorded_base, + } + # Groups still present means group worktrees remain. + return { + "clean": False, + "reason": _WT_REASON_GROUP_WORKTREES_REMAIN, + "head_sha": head_sha, + "base_sha": None, + } + + # No groups remain — check tree cleanliness. + status = _wt_git(["status", "--porcelain"], timeout=15) + if status.returncode != 0: + return { + "clean": False, + "reason": _WT_REASON_DIRTY, + "head_sha": head_sha, + "base_sha": None, + } + dirty_lines = [ + ln for ln in status.stdout.splitlines() + if ln.strip() and not _wt_is_runtime_state_path(_wt_porcelain_path(ln)) + ] + if dirty_lines: + return { + "clean": False, + "reason": _WT_REASON_GROUP_DIRTY_TREE, + "head_sha": head_sha, + "base_sha": None, + } + + return {"clean": True, "reason": None, "head_sha": head_sha, "base_sha": head_sha} + + +def reconcile_orphan_groups( + branch: Optional[str] = None, +) -> dict[str, object]: + """Startup sweep: find groups left mid-flight and invoke cleanup. + + Composes the existing ``cleanup_orphan_worktrees`` to remove physical worktrees, + then removes stale group entries from the wave_groups sidecar. Idempotent: + a second call after everything is clean returns ``swept=0``. + + Returns:: + + { + "ok": True, + "swept": , + "cleanup": , + } + """ + branch_name = branch or get_branch_name() + + # Step 1: remove physical orphan worktrees (existing helper). + cleanup_result = cleanup_orphan_worktrees(branch_name) + + # Step 2: remove stale wave_group entries from sidecar. + state = _read_worktree_state(branch_name) + wave_groups = state.get("wave_groups") + swept = 0 + if isinstance(wave_groups, dict) and wave_groups: + # A group is stale if all its lifecycle events include a terminal event + # (merged or aborted) OR if it has no lifecycle events at all (never started). + _terminal = {_WT_GROUP_EVENT_MERGED, _WT_GROUP_EVENT_ABORTED} + stale_keys: list[str] = [] + for gk, grp in list(wave_groups.items()): + if not isinstance(grp, dict): + stale_keys.append(gk) + continue + lifecycle = grp.get("lifecycle", {}) + if not isinstance(lifecycle, dict) or not lifecycle: + # No lifecycle events recorded — orphan from a crash before start. + stale_keys.append(gk) + continue + # Check if EVERY declared subtask has at least one terminal event. + # Iterate over declared subtask_ids (recorded by begin_wave_group), NOT + # lifecycle.values(): a partially-recorded group (one subtask missing its + # events slot) must NOT be swept — missing slot → NOT terminal. + declared_sids = grp.get("subtask_ids", []) + if not isinstance(declared_sids, list) or not declared_sids: + # No declared subtasks — treat as orphan (begin_wave_group not called). + stale_keys.append(gk) + continue + all_terminal = all( + isinstance(lifecycle.get(sid), list) + and any( + isinstance(ev, dict) and ev.get("event") in _terminal + for ev in lifecycle[sid] + ) + for sid in declared_sids + ) + if all_terminal: + stale_keys.append(gk) + for key in stale_keys: + del wave_groups[key] + swept += 1 + if swept: + _write_worktree_state(branch_name, state) + + return {"ok": True, "swept": swept, "cleanup": cleanup_result} + + def create_subtask_worktree( subtask_id: str, attempt: int = 0, @@ -16214,6 +16547,56 @@ def _wt_release_merge_lock(handle: Optional[Any]) -> None: pass +def _wt_lifecycle_lock_path() -> Optional[Path]: + """Advisory lock path for the lifecycle sidecar (separate from the merge lock).""" + common = _wt_git_common_dir() + if common is None: + return None + return common / "map-framework" / "wave-lifecycle.lock" + + +def _wt_acquire_lifecycle_lock() -> Optional[Any]: + """Advisory file lock for lifecycle sidecar read-modify-write. + + Reuses the same fcntl pattern as _wt_acquire_merge_lock so concurrent + actor processes (separate OS processes) cannot clobber each other's events. + Returns a file handle holding the lock, or None when unavailable. + """ + lock_path = _wt_lifecycle_lock_path() + if lock_path is None: + return None + lock_path.parent.mkdir(parents=True, exist_ok=True) + handle = lock_path.open("w") + try: + import fcntl # noqa: PLC0415 + except ImportError: + return handle # best-effort on non-POSIX + try: + fcntl.flock(handle.fileno(), fcntl.LOCK_EX) # blocking — sidecar writes are fast + except OSError: + handle.close() + return None + return handle + + +def _wt_release_lifecycle_lock(handle: Optional[Any]) -> None: + if handle is None: + return + try: + import fcntl # noqa: PLC0415 + + try: + fcntl.flock(handle.fileno(), fcntl.LOCK_UN) + except OSError: + pass + except ImportError: + pass + try: + handle.close() + except OSError: + pass + + def merge_subtask_worktree( subtask_id: str, attempt: int = 0, @@ -16906,6 +17289,323 @@ def concurrency_ready( } +# --------------------------------------------------------------------------- +# Concurrent-wave COORDINATOR (5b.4, ST-005) + group-abort (5b.5, ST-006) +# --------------------------------------------------------------------------- + +_MAX_ACTORS_MIN: int = 1 +_MAX_ACTORS_MAX: int = 8 +_MAX_ACTORS_DEFAULT: int = 4 + +_MAX_WAVE_RETRIES_MIN: int = 1 +_MAX_WAVE_RETRIES_MAX: int = 10 +_MAX_WAVE_RETRIES_DEFAULT: int = 3 + + +def _max_actors(project_dir: Optional[Path] = None) -> int: + """Read ``execution.max_actors`` from config and clamp to [1, 8]. + + Mirrors ``clamp_max_actors()`` from ``MapConfig`` without importing it. + Non-int / bool / absent values fall back to the default 4. + """ + raw = _map_config_int( + project_dir or (_wt_project_dir() or Path(".")), + "execution.max_actors", + _MAX_ACTORS_DEFAULT, + ) + # _map_config_int already returns > 0 or default; clamp to [1, 8]. + return max(_MAX_ACTORS_MIN, min(_MAX_ACTORS_MAX, raw)) + + +def _max_wave_retries(project_dir: Optional[Path] = None) -> int: + """Read ``execution.max_wave_retries`` from config and clamp to [1, 10]. + + Default is 3. Non-int / bool / absent values fall back to the default. + Mirrors the _max_actors pattern. + """ + raw = _map_config_int( + project_dir or (_wt_project_dir() or Path(".")), + "execution.max_wave_retries", + _MAX_WAVE_RETRIES_DEFAULT, + ) + return max(_MAX_WAVE_RETRIES_MIN, min(_MAX_WAVE_RETRIES_MAX, raw)) + + +def _chunk(items: list[str], size: int) -> list[list[str]]: + """Split *items* into ordered sub-lists each of width <= *size*.""" + if size < 1: + size = 1 + return [items[i : i + size] for i in range(0, len(items), size)] + + +def abort_wave_group( + group_id: str, + branch: Optional[str] = None, +) -> dict[str, object]: + """Idempotent, runner-owned group-abort verb (HC-4, ST-006). + + On ANY pre-merge actor failure / timeout / cancel / Monitor-reject, discard + the WHOLE group and return to base. NEVER partially merges a subset. + + Steps (each is idempotent on re-entry): + + 1. Read the group's recorded ``base_sha`` from the lifecycle sidecar + (written by ``begin_wave_group``). + 2. If ``_wt_active_git_operation()`` reports a mid-merge, run + ``git merge --abort`` first. + 3. **Reuse** ``_wt_rollback(base_sha)`` — hard-reset to base_sha + clean + with ``-e .map -e .codex -e .agents`` so the gitignored runtime state + (step_state.json, worktree sidecar) is NEVER deleted. + DO NOT call ``git clean -fdx`` or ``git clean -x`` directly. + 4. Discard every group worktree + branch via ``discard_subtask_worktree`` + (which uses ``_wt_force_remove`` internally). + 5. Mark the group ``aborted`` in the lifecycle sidecar and remove the + group entry from ``wave_groups`` so ``verify_group_clean`` sees zero + groups. + 6. Call ``verify_group_clean`` and return its verdict. + + Idempotent: a second invocation after a partial abort converges to + ``verify_group_clean == True`` without error. + + Returns ``verify_group_clean`` dict augmented with ``aborted_group_id``. + """ + if not _wt_is_git_repo(): + return _wt_error(_WT_REASON_NOT_GIT_REPO, "not inside a git work tree") + + branch_name = branch or get_branch_name() + state = _read_worktree_state(branch_name) + wave_groups = state.get("wave_groups") + + # Resolve the canonical group key from the sidecar. The caller may pass + # the raw group_id string (could be the canonical key, or a single subtask id). + group_key: Optional[str] = None + base_sha: Optional[str] = None + subtask_ids: list[str] = [] + if isinstance(wave_groups, dict): + if group_id in wave_groups: + group_key = group_id + else: + # Tolerate a caller passing one member subtask id — scan for it. + for gk, grp in wave_groups.items(): + if isinstance(grp, dict): + sids = grp.get("subtask_ids", []) + if isinstance(sids, list) and group_id in sids: + group_key = gk + break + if group_key is not None: + grp = wave_groups[group_key] + if isinstance(grp, dict): + base_sha = str(grp.get("base_sha", "")) or None + raw_sids = grp.get("subtask_ids", []) + subtask_ids = list(raw_sids) if isinstance(raw_sids, list) else [] + + # Step 2: abort any in-progress merge (idempotent — fails safely if none). + active_op = _wt_active_git_operation() + if active_op == "merge": + _wt_git(["merge", "--abort"]) + + # Step 3: rollback to base_sha if we have one. + # MUST reuse _wt_rollback — it excludes .map/.codex/.agents from the clean. + rollback_verified = True # optimistic; no base_sha → nothing to verify + if base_sha: + _wt_rollback(base_sha) + # Verify rollback landed BEFORE touching group state (F5). + # If HEAD != base_sha the rollback failed; keep the group entry so that + # verify_group_clean still has base_sha and a wrong HEAD cannot pass as clean. + actual_head = _wt_head_sha() + if actual_head != base_sha: + rollback_verified = False + result: dict[str, object] = { + "ok": False, + "clean": False, + "reason": "rollback_head_mismatch", + "base_sha": base_sha, + "actual_head": actual_head, + "aborted_group_id": group_id, + } + return result + + # Step 4: discard every group worktree + branch. + for sid in subtask_ids: + discard_subtask_worktree(sid, branch=branch_name) + + # Step 5: mark aborted in sidecar and remove the group entry so + # verify_group_clean sees zero groups. Only reached when rollback is verified. + if group_key is not None and rollback_verified: + # Record aborted event for every subtask (best-effort). + for sid in subtask_ids: + record_group_lifecycle(group_key, sid, _WT_GROUP_EVENT_ABORTED, branch_name) + # Re-read state (record_group_lifecycle writes it). + state2 = _read_worktree_state(branch_name) + wg2 = state2.get("wave_groups") + if isinstance(wg2, dict) and group_key in wg2: + del wg2[group_key] + _write_worktree_state(branch_name, state2) + + # Step 6: verify clean and return. + verdict = verify_group_clean(branch_name) + final_result: dict[str, object] = dict(verdict) + final_result["aborted_group_id"] = group_id + return final_result + + +def run_concurrent_wave( + group_ids: list[str], + branch: Optional[str] = None, + project_dir: Optional[Path] = None, +) -> dict[str, object]: + """Coordinate an N-way concurrent wave: batch-split + atomic sub-batch merge. + + This function is the COORDINATOR side of concurrent dispatch. It does NOT + spawn actor agents — the skill emits N Task blocks to start actors (ST-007). + Its responsibilities are: + + 1. **Default-off guard** (HC-1, defense-in-depth): if ``concurrent_dispatch`` + is not explicitly enabled in config, return ``CONCURRENT_DISPATCH_DISABLED`` + immediately so a direct CLI or coordinator call cannot trigger concurrent + merging under the default-off configuration. + + 2. **Batch-split**: read ``max_actors`` from config (via ``_max_actors()``), + clamp to [1, 8], then split the sorted ``group_ids`` into sequential + sub-batches each of width <= cap. A group already within the cap is one + batch (no split). + + 3. **Atomic sub-batch merge**: for each sub-batch call the existing + ``merge_wave_worktrees(sub_batch, branch=...)`` which is all-or-nothing + (HC-4, #284 invariant). The NEXT sub-batch branches from the prior + sub-batch's post-merge HEAD. Do NOT re-implement merge. + + 4. **Telemetry / lifecycle**: call ``record_dispatch_actual`` CLI verb (via + ``begin_wave_group`` + ``record_group_lifecycle`` — these must be called + by the skill/coordinator before the actors run; this function only reads + state and calls merge). Telemetry is emitted exactly ONCE per wave via + the ``record_dispatch_actual`` CLI (the skill wires that in ST-007). + + 5. **Abort-once on failure, return needs_redispatch** (ST-006, architectural): + on a ``merge_wave_worktrees`` error, invoke ``abort_wave_group`` ONCE to + discard the WHOLE group and reset to base (HC-4). Do NOT retry internally — + the group worktrees are gone after abort and cannot be re-merged without the + skill re-dispatching actors. Return a structured result with + ``needs_redispatch: True`` and ``attempts_remaining`` (read from the group + sidecar so successive calls can track exhaustion). The SKILL owns the retry + loop: re-dispatch actors, then call run_concurrent_wave again. + + Returns on full success:: + + { + "status": "success", + "ok": True, + "group_ids": [...], # original sorted ids + "sub_batches": [[...], ...], # how ids were split + "max_actors": int, # cap used + "batches_merged": int, # number of sub-batches atomically merged + "merged_ids": [...], # all ids that landed (may be < group_ids if no-change) + "no_changes": [...], # ids with no-change-in-worktree + } + + Returns when concurrent dispatch is disabled (HC-1 defense-in-depth):: + + { + "status": "error", + "ok": False, + "kind": "CONCURRENT_DISPATCH_DISABLED", + ... + } + + Returns on merge failure (abort-once, needs_redispatch):: + + { + "status": "error", + "ok": False, + "kind": "WAVE_ABORTED", + "needs_redispatch": True, + "attempts_remaining": int, # decremented in group sidecar; 0 → escalate + "escalate_to_human": bool, # True when attempts_remaining == 0 + "group_ids": [...], + "merge_error": {...}, # merge_wave_worktrees error dict + } + """ + if not _wt_is_git_repo(): + return _wt_error(_WT_REASON_NOT_GIT_REPO, "not inside a git work tree") + + pd = project_dir or _wt_project_dir() or Path(".") + branch_name = branch or get_branch_name() + + # F6: Default-off guard (HC-1, defense-in-depth). A direct CLI call or + # coordinator misconfiguration cannot trigger concurrent merging unless the + # flag is explicitly set. This is a second line of defense after compute_dispatch_gate. + if not _concurrent_dispatch_enabled(pd): + return _wt_error( + "CONCURRENT_DISPATCH_DISABLED", + "run_concurrent_wave requires execution.concurrent_dispatch=true in config; " + "default is off (HC-1). Set the flag explicitly to enable concurrent dispatch.", + ) + + # Deterministic sorted list — order-of-call must not vary group membership. + ids_sorted = sorted(str(s) for s in group_ids if str(s).strip()) + if not ids_sorted: + return _wt_error("NO_SUBTASKS", "no subtask ids supplied to run_concurrent_wave") + + cap = _max_actors(pd) + max_retries = _max_wave_retries(pd) + sub_batches = _chunk(ids_sorted, cap) + group_key = "|".join(ids_sorted) + + merged_ids: list[str] = [] + no_changes: list[str] = [] + batches_merged = 0 + + for batch in sub_batches: + merge_result = merge_wave_worktrees(batch, branch=branch_name, skip_post_wave=True) + if merge_result.get("status") == "error" or not merge_result.get("ok"): + # F7: Abort ONCE — worktrees are gone after abort; the skill must + # re-dispatch actors before calling run_concurrent_wave again. + # Track attempt count in the group sidecar so successive calls can + # decrement and escalate when exhausted. + abort_wave_group(group_key, branch_name) + + # Read attempt count from sidecar (written by begin_wave_group / prior calls). + _st2 = _read_worktree_state(branch_name) + _wg2 = _st2.get("wave_groups") or {} + _grp2 = _wg2.get(group_key) if isinstance(_wg2, dict) else None + _attempts_used = 1 + if isinstance(_grp2, dict): + _attempts_used = int(_grp2.get("abort_attempts", 0)) + 1 + _grp2["abort_attempts"] = _attempts_used + _write_worktree_state(branch_name, _st2) + + attempts_remaining = max(0, max_retries - _attempts_used) + return { + "status": "error", + "ok": False, + "kind": "WAVE_ABORTED", + "needs_redispatch": True, + "attempts_remaining": attempts_remaining, + "escalate_to_human": attempts_remaining == 0, + "group_ids": ids_sorted, + "merge_error": dict(merge_result), + "failed_batch": batch, + "batches_merged_before_failure": batches_merged, + } + + raw_merged = merge_result.get("merged", []) + raw_no_changes = merge_result.get("no_changes", []) + merged_ids.extend(list(raw_merged) if isinstance(raw_merged, list) else []) + no_changes.extend(list(raw_no_changes) if isinstance(raw_no_changes, list) else []) + batches_merged += 1 + + return { + "status": "success", + "ok": True, + "group_ids": ids_sorted, + "sub_batches": sub_batches, + "max_actors": cap, + "batches_merged": batches_merged, + "merged_ids": merged_ids, + "no_changes": no_changes, + } + + if __name__ == "__main__": # Simple CLI interface for testing import sys @@ -18419,6 +19119,259 @@ def _flag_val(name: str) -> Optional[str]: if _wt_r.get("status") == "error": sys.exit(1) + elif func_name == "begin_wave_group": + # CLI: begin_wave_group [ ...] [--branch B] + # Record the group base_sha + per-subtask lifecycle skeleton (5b.1). + # Idempotent: re-running with the same ids does not duplicate state. + import argparse as _ap + + _p = _ap.ArgumentParser(prog="map_step_runner.py begin_wave_group") + _p.add_argument("group_ids", nargs="+") + _p.add_argument("--branch", default=None) + _a = _p.parse_args(sys.argv[2:]) + _wt_r = begin_wave_group(_a.group_ids, _a.branch) + print(json.dumps(_wt_r, indent=2)) + if not _wt_r.get("ok"): + sys.exit(1) + + elif func_name == "record_group_lifecycle": + # CLI: record_group_lifecycle [--branch B] + # Append a lifecycle event (created/started/finished/merged/aborted) (5b.1). + import argparse as _ap + + _p = _ap.ArgumentParser(prog="map_step_runner.py record_group_lifecycle") + _p.add_argument("group_key") + _p.add_argument("subtask_id") + _p.add_argument("event") + _p.add_argument("--branch", default=None) + _a = _p.parse_args(sys.argv[2:]) + _wt_r = record_group_lifecycle(_a.group_key, _a.subtask_id, _a.event, _a.branch) + print(json.dumps(_wt_r, indent=2)) + if not _wt_r.get("ok"): + sys.exit(1) + + elif func_name == "verify_group_clean": + # CLI: verify_group_clean [--branch B] + # Read-only: clean iff HEAD==base_sha AND tree clean AND zero group worktrees. + # Exits 0 when clean=True; exits 1 when clean=False (usable as a gate). + import argparse as _ap + + _p = _ap.ArgumentParser(prog="map_step_runner.py verify_group_clean") + _p.add_argument("--branch", default=None) + _a = _p.parse_args(sys.argv[2:]) + _wt_r = verify_group_clean(_a.branch) + print(json.dumps(_wt_r, indent=2)) + if not _wt_r.get("clean"): + sys.exit(1) + + elif func_name == "reconcile_orphan_groups": + # CLI: reconcile_orphan_groups [--branch B] + # Startup sweep: remove stale wave_group sidecar entries + orphan worktrees. + import argparse as _ap + + _p = _ap.ArgumentParser(prog="map_step_runner.py reconcile_orphan_groups") + _p.add_argument("--branch", default=None) + _a = _p.parse_args(sys.argv[2:]) + _wt_r = reconcile_orphan_groups(_a.branch) + print(json.dumps(_wt_r, indent=2)) + if not _wt_r.get("ok"): + sys.exit(1) + + elif func_name == "record_dispatch_actual": + # CLI: record_dispatch_actual + # [--branch B] [--same-turn-count N] [--skill-reported-concurrent] + # + # Coordinator-owned dispatch telemetry (5b.2 / ST-003). + # Replays the ST-002 lifecycle events recorded under wave_groups to + # compute max_in_flight deterministically (sorted-seq sweep, no wall-clock), + # then calls classify_dispatch() with the typed evidence inputs, and + # persists a ParallelismReport via record_dispatch_actual() ONLY when + # the outcome is concurrent_observed. All other outcomes are no-ops. + # + # Producer-owns-parse: the runner parses lifecycle/sidecar here; the + # classifier (in parallelism_observability) receives only typed ints. + import argparse as _ap + import sys as _sys + + _p = _ap.ArgumentParser(prog="map_step_runner.py record_dispatch_actual") + _p.add_argument("group_key", help="Canonical group key (sorted subtask IDs joined by '|')") + _p.add_argument("run_id", help="Run identifier for the parallelism.json path") + _p.add_argument("out_path", help="Destination path for parallelism.json") + _p.add_argument("--branch", default=None) + _p.add_argument( + "--same-turn-count", + type=int, + default=0, + dest="same_turn_count", + help="Number of Task tool calls in the same turn (from coordinator transcript)", + ) + _p.add_argument( + "--skill-reported-concurrent", + action="store_true", + dest="skill_reported_concurrent", + help="Set when the skill or Actor self-reported concurrent dispatch", + ) + _a = _p.parse_args(sys.argv[2:]) + + # --- Step 1: read the wave_groups sidecar for this branch --- + _branch_name = _a.branch or get_branch_name() + _state = _read_worktree_state(_branch_name) + _wave_groups = _state.get("wave_groups") or {} + + # --- Step 2: extract base_shas and lifecycle events for this group --- + # F8: collect per-subtask base SHAs from each worktree record so + # classify_dispatch can detect isolation_violation (len(set(base_shas))>1). + # Appending only the group-level base_sha means all subtasks share one SHA + # and the isolation_violation path is unreachable. + _group_data = _wave_groups.get(_a.group_key) if isinstance(_wave_groups, dict) else None + _base_shas: list[str] = [] + _all_events: list[dict] = [] + + if isinstance(_group_data, dict): + _lifecycle = _group_data.get("lifecycle") or {} + if isinstance(_lifecycle, dict): + for _sid_events in _lifecycle.values(): + if isinstance(_sid_events, list): + for _ev in _sid_events: + if isinstance(_ev, dict): + _all_events.append(_ev) + + # Collect per-subtask base SHAs from each worktree sidecar record. + # Falls back to the group-level base_sha for subtasks whose worktree + # record is absent (partial registration or pre-begin_wave_group crash). + _group_sids = _group_data.get("subtask_ids", []) + _worktrees = _state.get("worktrees") or {} + _group_level_sha = _group_data.get("base_sha") + if isinstance(_group_sids, list) and _group_sids: + for _sid in _group_sids: + _slug = _wt_slug(_sid) + _wt_rec = _worktrees.get(_slug) if isinstance(_worktrees, dict) and _slug else None + if isinstance(_wt_rec, dict): + _per_sha = _wt_rec.get("base_sha") + if isinstance(_per_sha, str) and _per_sha: + _base_shas.append(_per_sha) + continue + # Fallback: use group-level SHA when per-subtask record missing. + if isinstance(_group_level_sha, str) and _group_level_sha: + _base_shas.append(_group_level_sha) + else: + # No declared subtask_ids — fall back to group-level SHA. + if isinstance(_group_level_sha, str) and _group_level_sha: + _base_shas.append(_group_level_sha) + + # --- Step 3: compute max_in_flight by replaying sorted lifecycle events --- + # Sweep events sorted by monotonic seq number. Only "started" and + # "finished" affect the in-flight counter. This is deterministic and + # completely clock-free (HC-5 — seq, not ts). + _all_events.sort(key=lambda _e: int(_e.get("seq", 0))) + _in_flight = 0 + _max_in_flight = 0 + for _ev in _all_events: + _ev_type = _ev.get("event", "") + if _ev_type == _WT_GROUP_EVENT_STARTED: + _in_flight += 1 + if _in_flight > _max_in_flight: + _max_in_flight = _in_flight + elif _ev_type == _WT_GROUP_EVENT_FINISHED: + _in_flight = max(0, _in_flight - 1) + + # --- Step 4: classify using the evidence hierarchy --- + # Import is intentionally lazy so the sequential path never loads this module. + try: + from mapify_cli.parallelism_observability import ( + classify_dispatch as _classify_dispatch, + record_dispatch_actual as _record_dispatch_actual, + ParallelismReport as _ParallelismReport, + ColorGroupDecision as _ColorGroupDecision, + ) + except ImportError: + print(json.dumps({ + "ok": False, + "error": "mapify_cli not importable from this runner context; " + "record_dispatch_actual requires the mapify_cli package", + }, indent=2)) + _sys.exit(1) + + _outcome = _classify_dispatch( + same_turn_task_count=_a.same_turn_count, + max_in_flight=_max_in_flight, + base_shas=_base_shas, + skill_reported_concurrent=_a.skill_reported_concurrent, + ) + + # --- Step 5: persist ONE report only on the concurrent path --- + _out_path = Path(_a.out_path) + _group_ids = _a.group_key.split("|") if _a.group_key else [] + _group_record: _ColorGroupDecision = { + "group_id": _a.group_key, + "planned_mode": "concurrent", + "actual_mode": _outcome, + "worktree_status": "ok" if _base_shas else "unknown", + "reason_code": None, + "dispatch_count": len(_group_ids), + } + _report: _ParallelismReport = { + "schema_version": "1.0.0", + "run_id": _a.run_id, + "generated_at": "", # caller supplies; runner never calls datetime.now() + "total_subtasks": len(_group_ids), + "total_edges": 0, + "total_waves": 1, + "max_wave_width": _max_in_flight, + "color_group_breakdown": [_group_record], + } + _written = _record_dispatch_actual(_report, _out_path, _outcome) + + print(json.dumps({ + "ok": True, + "group_key": _a.group_key, + "outcome": _outcome, + "max_in_flight": _max_in_flight, + "base_shas": _base_shas, + "report_written": _written, + "out_path": str(_out_path) if _written else None, + }, indent=2)) + + elif func_name == "run_concurrent_wave": + # CLI: run_concurrent_wave [ ...] [--branch B] + # [--project-dir P] + # + # Coordinator-owned N-way concurrent wave (5b.4 / ST-005). + # Batch-splits group_ids by max_actors (from config, clamped [1,8]), + # then atomically merges each sub-batch via merge_wave_worktrees. + # Does NOT spawn actors — the skill (ST-007) emits Task blocks. + # On merge failure returns the structured error and exits 1. + import argparse as _ap + + _p = _ap.ArgumentParser(prog="map_step_runner.py run_concurrent_wave") + _p.add_argument("group_ids", nargs="+", help="Subtask IDs in the concurrent wave") + _p.add_argument("--branch", default=None, help="Branch name (default: current branch)") + _p.add_argument("--project-dir", default=None, dest="project_dir", + help="Project root (default: git top-level)") + _a = _p.parse_args(sys.argv[2:]) + _pd = Path(_a.project_dir) if _a.project_dir else None + _wt_r = run_concurrent_wave(_a.group_ids, _a.branch, _pd) + print(json.dumps(_wt_r, indent=2)) + if _wt_r.get("status") == "error" or not _wt_r.get("ok"): + sys.exit(1) + + elif func_name == "abort_wave_group": + # CLI: abort_wave_group [--branch B] + # + # Idempotent group-abort verb (5b.5 / ST-006). + # Discards the WHOLE group + resets to base_sha via _wt_rollback. + # Never merges a subset (HC-4). + import argparse as _ap + + _p = _ap.ArgumentParser(prog="map_step_runner.py abort_wave_group") + _p.add_argument("group_id", help="Canonical group key (or member subtask id)") + _p.add_argument("--branch", default=None, help="Branch name (default: current branch)") + _a = _p.parse_args(sys.argv[2:]) + _wt_r = abort_wave_group(_a.group_id, _a.branch) + print(json.dumps(_wt_r, indent=2)) + if not _wt_r.get("clean"): + sys.exit(1) + else: # Helpful redirect: when the user passes a command that belongs to # the orchestrator (record_subtask_result, mark_subtask_complete, diff --git a/CHANGELOG.md b/CHANGELOG.md index c9b6b4a8..68f8fc90 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] ### Added +- **Concurrent Actor dispatch for parallel waves (`execution.concurrent_dispatch`, part of #303 Slice 5b).** Activates same-turn concurrent dispatch of Actor subagents within a parallel wave, previously scheduled but always run sequentially. Flag-gated (`execution.concurrent_dispatch: false` by default) so the default code-path is byte-identical to Slice 5a; defaults flip only in Slice 6. Key components: `compute_dispatch_gate` (strict conjunction — `concurrent_dispatch` AND `concurrency_allowed` AND `concurrency_ready` AND `isolation != off`; hard-aborts with a `ConfigError`-equivalent on any config contradiction, fail-closed rather than degrading silently), `run_concurrent_wave` (splits the wave into sub-batches of `execution.max_actors` and dispatches each sub-batch; atomic per-sub-batch merge via `merge_wave_worktrees`), `abort_wave_group` (whole-group rollback — reverts every worktree in the group back to wave base, bounded by `execution.max_wave_retries`), `record_dispatch_actual` (clock-free phantom-parallelism classifier using `max_in_flight` replay — worktree SHA proves isolation but NOT concurrency; emits `phantom_parallel` evidence when actors ran but no concurrent overlap is detectable). Test harness uses barrier-based determinism (no wall-clock sleeps); HC-1 leak-guard suite validates no cross-subtask state leaks under concurrent dispatch. Council review split this into 5a (infrastructure) and 5b (activation); this entry covers 5b. - **`deferred_nondeterministic` wired into the core Monitor verdict path (completes #252).** The flaky-triage primitives (`run_/record_/validate_flaky_test_triage`) and the `defer_flaky_subtask` close+advance command already existed, but were **disjoint** from the Monitor verdict path: Monitor could only emit `valid:true`/`valid:false`, had no field to signal a flaky defer, and `validate_step 2.4` could only pass or hard-stop — so a confirmed flake forced an out-of-band manual `defer_flaky_subtask`. The third Monitor outcome is now part of the structured verdict. (1) The Monitor schema gains an OPTIONAL structured `disposition: {kind, check_id}` field (`kind` enum currently `{deferred_nondeterministic}`), absent for normal verdicts, with guidance to emit it on confirmed mixed pass/fail evidence instead of demanding a fake Actor fix. (2) `validate_step 2.4 --disposition deferred_nondeterministic --check-id --monitor-envelope -` routes to the existing `defer_flaky_subtask` **in-process** (the single owner of the close+advance transaction), placed BEFORE the recommendation gates so a defer carrying `recommendation=needs_investigation` is not hard-stopped. (3) **Anti-gaming** (llm-council-reviewed, conv `d3ddca63`): the deferral is honored ONLY when the Monitor envelope structurally backs it — `valid:false`, non-empty `failed_checks`, and a structured `disposition` whose kind + `check_id` match the flags — AND the sidecar holds mixed pass/fail evidence for that `check_id` (re-validated from disk by `defer_flaky_subtask`). A Monitor cannot dodge a real deterministic failure or a green check by merely claiming "flaky"; `recommendation in {revise, block}` together with a disposition is rejected as a contradiction. (Note: the Monitor schema's `failed_checks` lists failed quality *dimensions*, a different namespace from a flaky check id, so the binding is "Monitor admits a dimension failure + dispositions match" rather than "check_id ∈ failed_checks".) (4) **Verdict vs routing:** a deferred run returns `valid:false` + `deferred:true` + `non_green_outcome:true` (a deferral is NOT green — it is a routing decision, not a clean pass); the CLI exits `0` on a deferral (not a hard-stop) and `1` only on a true invalid verdict. A single source-of-truth `MONITOR_DISPOSITIONS` policy dict drives the routing, the CLI `--disposition` surface, and a drift-guard test (the Monitor prompt must name every supported disposition). Closes the last core slice of #252. - **Context-budget statusline for all MAP sessions (`map-statusline.py`, completes #284 Phase 3).** A Claude Code `statusLine` render command that shows live context-window usage at a glance: `[Opus] MAP ctx 47% (94k/200k) · feature-x · ST-003 ACTOR`. It reads the usage Claude Code **pre-computes** on stdin (`context_window.used_percentage` / `context_window_size` / `total_input_tokens`), so it does NO transcript parsing, no token counting, and no network — it formats already-available numbers plus the git branch (read directly from `.git/HEAD`, no `git` subprocess; handles the linked-worktree `.git`-as-file case) and the active MAP subtask (best-effort `.map//step_state.json`). Output is **never blank and never crashes** — any error degrades to a minimal safe line; it shows `--%` before the first API response (instead of a misleading `0%`) and a `200k?` uncertainty marker when the harness omits the window size. It is wired **non-destructively** at install time by `ensure_map_statusline`: the `statusLine` entry is merged into the user-owned `.claude/settings.local.json` ONLY when no status line already exists in the local/project/user scope — so MAP never overrides a status line the user configured. Writing to `settings.local.json` (not the MAP-managed `settings.json`) avoids all managed-file drift/`.bak` churn and stays idempotent across upgrades; remove the `statusLine` key there to disable. Claude provider only (`statusLine` is a Claude Code concept; the Codex install path never wires it). The other Phase 3 item — threshold warnings — already shipped via the `context-meter.py` `/compact` nudge; the heartbeat/SSE-keepalive item is closed as **harness-owned** (MAP's orchestrator is prompt-driven and dispatches subagents through Claude Code's Task tool, which the harness keeps alive — MAP ships no bespoke keepalive). Design was llm-council-reviewed (conv `585f773b`). Completes #284 Phase 3. - **Parallel-wave merge coordinator for worktree isolation (`merge_wave_worktrees`, part of #284 Phase 2).** Wires the existing wave/DAG scheduler to per-subtask worktree isolation so a parallel wave's independent subtasks each run in their own worktree and are accepted **atomically**. Every worktree of a wave is cut off the same base (HEAD at wave start), so they cannot be merged one at a time — the first `merge_subtask_worktree` advances HEAD and the next trips `BASE_DIVERGED`. The new coordinator relaxes *only* that guard to a wave-scoped form: it refuses **external** HEAD movement (`EXTERNAL_HEAD_MOVED`) but allows the sibling divergence each in-wave squash-merge creates. It derives `wave_base_sha` from the sidecar (never a caller parameter), preflights every worktree (commit + per-worktree guards + pre-merge verify) BEFORE touching the working branch, then squash-merges each accepted worktree **by frozen SHA in sorted id order** (one runner commit per subtask — the one-commit-per-subtask contract holds), then runs **one post-wave full gate on the merged tree inside the same transaction**. It is **all-or-nothing** (council-reviewed, conv `c29d6fa9`): any textual conflict, commit failure, or post-wave-gate failure rolls the whole working branch back to the wave base via `git reset --hard` + `git clean -fd` (squash leaves no `MERGE_HEAD`, so `git merge --abort` is never used; MAP runtime state is excluded from the clean) and leaves **every** worktree intact for retry — no partial-wave state ever survives. Safety extras: an advisory `flock` serializes coordinators (`MERGE_IN_PROGRESS`); attached-/clean-target preconditions; conflicted paths are attributed back to the subtasks that touched them (declared-disjoint `affected_files` is only a scheduler hint, so actual changed-file overlap is reported as advisory telemetry while git's textual conflict stays the hard guard). The shared `_wt_freeze_and_verify` primitive (commit + guards + pre-merge verify) is extracted once and reused by both the single-subtask and wave merge paths. CLI: `merge_wave_worktrees [--branch B] [--verify-cmd CMD…] [--skip-verify] [--post-wave-cmd CMD…] [--skip-post-wave]`. Phase 3 (context-budget hooks) remains open on #284. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 08814878..69f64ef6 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -186,6 +186,52 @@ divergence each in-wave squash-merge creates. Design (llm-council-reviewed, conv reported as advisory telemetry while git's textual conflict stays the hard guard). +### Phase 3 / Slice 5b: concurrent Actor dispatch (flag-gated, default off) + +Slice 5b activates same-turn concurrent dispatch of Actor subagents within a +parallel wave. The entire path is guarded by `execution.concurrent_dispatch` +(default `false`); with the flag off the code path is byte-identical to Slice 5a. + +**Dispatch gate (`compute_dispatch_gate`).** A strict conjunction of four +conditions: `concurrent_dispatch` is true AND `concurrency_allowed` (platform +supports parallel Task dispatch) AND `concurrency_ready` (runner state is +consistent) AND `worktree.isolation != off`. Any single condition false → gate +returns `disabled`. A config contradiction (e.g. `concurrent_dispatch: true` with +isolation off) is a hard abort (`ConfigError`-equivalent) — fail-closed, never +silent degradation. + +**Group lifecycle.** `begin_wave_group` opens a dispatch group and records the +base SHA from the sidecar. `record_group_lifecycle` appends structured events +(started, dispatched, merged, aborted). `verify_group_clean` asserts no group is +open before a new wave starts. `reconcile_orphan_groups` detects and cleans up +groups that were opened but never closed (e.g. runner crash mid-wave). + +**`record_dispatch_actual` — clock-free classifier.** Determines whether actors +ran with actual concurrency or only phantom parallelism (dispatched concurrently +but serialized by the harness). Uses `max_in_flight` replay over dispatch +timestamps recorded in the sidecar — no wall-clock reads, no `time.sleep`. A +worktree SHA proves isolation (each actor worked on its own tree) but does NOT +prove concurrency; the classifier emits `phantom_parallel: true` when the +evidence is isolation-only. Evidence hierarchy: overlapping dispatch windows → +concurrent; non-overlapping with worktree SHAs → isolated-sequential; no +worktree SHAs → unverifiable. + +**`run_concurrent_wave`.** Splits the wave's subtask list into sub-batches of +`execution.max_actors` (clamped `[1, 8]`). Dispatches each sub-batch as +concurrent Actor Tasks, then calls `merge_wave_worktrees` atomically for that +sub-batch. A sub-batch failure triggers `abort_wave_group` for the whole group. + +**`abort_wave_group` — bounded rollback.** On any sub-batch failure, reverts +every worktree in the group back to wave base (`discard_subtask_worktree` for +each member). Retries the whole group up to `execution.max_wave_retries` (clamped +`[1, 10]`, default 3). Exhausted retries → escalation (same path as +`build_escalation_outcome`). + +**Test harness.** Uses barrier-based determinism — actors synchronise on a shared +`threading.Barrier` rather than sleeping, so tests are deterministic regardless of +scheduling. The HC-1 leak-guard suite asserts no cross-subtask state leaks +(worktree files, MAP sidecar entries, git refs) under concurrent dispatch. + Phase 3 (context-budget hooks) is the final slice of #284 and is now complete. **Threshold warnings** already ship via the `context-meter.py` `UserPromptSubmit` hook (a `/compact` nudge when `compression_threshold_tokens` is crossed) and the diff --git a/docs/USAGE.md b/docs/USAGE.md index 36bc4f16..56290361 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -293,6 +293,29 @@ A concurrent second coordinator is blocked by an advisory lock. Phase 2's wave-merge coordinator (`merge_wave_worktrees`) has landed; Phase 3 (context-budget hooks) remains open on #284. +### Concurrent dispatch (Slice 5b, opt-in) + +Concurrent Actor dispatch within a parallel wave is an **opt-in, off-by-default** +feature controlled by three config keys. With `execution.concurrent_dispatch` +unset or `false` (the default), wave dispatch is sequential — identical behavior +to before Slice 5b. + +```yaml +# .map/config.yaml +execution.concurrent_dispatch: false # set to true to enable same-turn concurrent dispatch +execution.max_actors: 4 # max parallel Actor agents per sub-batch (clamp [1,8]) +execution.max_wave_retries: 3 # max whole-group rollback+restart attempts (clamp [1,10]) +``` + +**Requirements when enabling:** `worktree.isolation` must be `auto` or `required`. +Setting `execution.concurrent_dispatch: true` with isolation off (or `disabled`) +produces a hard `ConfigError` abort — the gate fails closed rather than degrading +silently. + +**Sequential default.** With `concurrent_dispatch: false` (or absent), every wave +runs the same sequential Actor→merge loop as before; no config error is raised and +none of the concurrent-dispatch code paths are exercised. + ## Stack Overflow for Agents (SOFA) SOFA integration is an **opt-in, off-by-default, read-only** prior-art search. diff --git a/src/mapify_cli/config/project_config.py b/src/mapify_cli/config/project_config.py index 7c2d9de9..9b6d4fdb 100644 --- a/src/mapify_cli/config/project_config.py +++ b/src/mapify_cli/config/project_config.py @@ -207,6 +207,23 @@ class MapConfig: # DORMANT in Slice 5a — parsed and validated but no execution path reads it yet. retry_degraded_once: bool = False + # Enable same-turn concurrent Actor dispatch in a parallel wave (#303 Slice 5b). + # The ONLY user-facing switch that activates concurrent dispatch; `False` by + # default — Slice 6 flips this once the rollback path is proven stable. + # Dotted YAML alias: `execution.concurrent_dispatch`. + # YAML 1.1 bare off/on arrive as Python bool, which matches this field type — + # no coercion needed (unlike the string enum fields). + # DORMANT in 5b.0 — consumed by ST-001's gate; no execution path reads it yet. + concurrent_dispatch: bool = False + + # Bounded retry cap for whole-wave rollback/restart in Slice 5b (#303). + # Range 1–10; values outside the range are clamped by clamp_max_wave_retries(). + # Non-int / bool values fall back to the default 3. + # Dotted YAML alias: `execution.max_wave_retries`. + # DORMANT in 5b.0 — consumed by ST-006's rollback path; no execution path reads + # it yet. + max_wave_retries: int = 3 + def clamp_max_actors(n: object) -> int: """Clamp max_actors to the valid range [1, 8], or return the default 4. @@ -224,6 +241,22 @@ def clamp_max_actors(n: object) -> int: return max(1, min(8, n)) +def clamp_max_wave_retries(n: object) -> int: + """Clamp max_wave_retries to the valid range [1, 10], or return the default 3. + + Non-int values (including bool, str, None) return the default 3. + int values are clamped: below 1 → 1, above 10 → 10. + + Note: bool is explicitly excluded (isinstance(True, int) is True in Python) + because a YAML boolean arriving here is a misconfiguration, not an int. + The floor is 1 — a valid-but-low int (e.g. 0) is clamped to 1 (minimum + legal value), while a non-int/bool/None falls back to the default 3. + """ + if isinstance(n, bool) or not isinstance(n, int): + return 3 + return max(1, min(10, n)) + + def load_map_config(project_path: Path) -> MapConfig: """Load MAP config from .map/config.yaml with fallback to defaults. @@ -295,6 +328,8 @@ def load_map_config(project_path: Path) -> MapConfig: ("execution.wave_mode", "execution_wave_mode"), ("execution.max_actors", "max_actors"), ("execution.retry_degraded_once", "retry_degraded_once"), + ("execution.concurrent_dispatch", "concurrent_dispatch"), + ("execution.max_wave_retries", "max_wave_retries"), ): if dotted in data and field_name not in data: data[field_name] = data.pop(dotted) @@ -436,8 +471,11 @@ def load_map_config(project_path: Path) -> MapConfig: cfg.execution_wave_mode = "auto" # Clamp max_actors to [1, 8]; non-int/bool → default 4. - # retry_degraded_once is a plain bool handled by the generic type-check loop. + # retry_degraded_once and concurrent_dispatch are plain bools handled by + # the generic type-check loop. cfg.max_actors = clamp_max_actors(cfg.max_actors) + # Clamp max_wave_retries to [1, 10]; non-int/bool → default 3. + cfg.max_wave_retries = clamp_max_wave_retries(cfg.max_wave_retries) return cfg @@ -613,6 +651,16 @@ def generate_default_config(include_comments: bool = True) -> str: # DORMANT until Slice 5b — setting this in Slice 5a has no effect. # execution.retry_degraded_once: false +# Enable same-turn concurrent Actor dispatch (#303 Slice 5b). The ONLY switch +# that activates concurrent dispatch; false by default — Slice 6 flips this once +# the rollback path is proven stable. YAML 1.1 bare off/on arrive as bool. +# execution.concurrent_dispatch: false + +# Bounded retry cap for whole-wave rollback/restart (#303 Slice 5b). +# Valid range 1–10; values outside the range are clamped (0→1, 99→10). +# Non-int / bool values fall back to the default 3. +# execution.max_wave_retries: 3 + # Strip MAP-internal workflow IDs (ST-/AC-/VC-/INV-/HC-) from the code a run # changed, at workflow completion (Stop hook). On by default; uncomment and set # to false to keep the IDs the framework wrote into comments/strings/test names. diff --git a/src/mapify_cli/parallelism_observability.py b/src/mapify_cli/parallelism_observability.py index 1f85beeb..1eee54b1 100644 --- a/src/mapify_cli/parallelism_observability.py +++ b/src/mapify_cli/parallelism_observability.py @@ -7,13 +7,10 @@ ``.map/runs//parallelism.json``. - A ``write_parallelism_report`` writer that is NO-OP by default (gated on an explicit ``enabled=True`` argument or the ST-003 observability toggle). - -Slice 5 will add the runtime writer/detection logic and must resolve the -runner-vs-CLI runtime-locality concern: the runner (.jinja template installed -in user repos) cannot import this CLI-side module in installed repos. Slice 5 -should either (a) duplicate the write path inside the runner .jinja with this -module as the canonical schema reference, or (b) expose a subprocess-callable -entry point that the runner shells out to. Decide in the Slice 5 spike. + - Classification-outcome constants and ``classify_dispatch`` (5b.2) which + maps typed evidence signals to a single outcome string. + - ``record_dispatch_actual`` (5b.2) which persists exactly one report per + wave on the concurrent path; is a no-op on the default/sequential path. """ from __future__ import annotations @@ -56,6 +53,42 @@ } ) +# --------------------------------------------------------------------------- +# Classification-outcome constants (5b.2 — ST-003) +# --------------------------------------------------------------------------- +# These are the classifier verdict strings returned by classify_dispatch(). +# Style mirrors the worktree-fallback codes above (SC-1). + +DISPATCH_OUTCOME_CONCURRENT_OBSERVED: str = "concurrent_observed" +"""Evidence confirms truly concurrent execution (same-turn count ≥2, max_in_flight ≥2).""" + +DISPATCH_OUTCOME_SAME_TURN_BUT_HOST_SEQUENTIAL: str = "same_turn_but_host_sequential" +"""Tasks were dispatched same-turn but host ran them serially (max_in_flight ≤1).""" + +DISPATCH_OUTCOME_PHANTOM_PARALLEL: str = "phantom_parallel" +"""Skill claimed concurrency but evidence shows at most one task (possible self-report error).""" + +DISPATCH_OUTCOME_SEQUENTIAL_OBSERVED: str = "sequential_observed" +"""Normal sequential execution confirmed (same-turn count ≤1, max_in_flight ≤1).""" + +DISPATCH_OUTCOME_ISOLATION_VIOLATION: str = "isolation_violation" +"""Multiple distinct base-SHAs detected across group worktrees — isolation breach.""" + +DISPATCH_OUTCOME_UNKNOWN: str = "unknown" +"""Evidence is insufficient or contradictory — cannot classify.""" + +# Validation set for all 6 classifier outcomes. +ALL_DISPATCH_OUTCOMES: frozenset[str] = frozenset( + { + DISPATCH_OUTCOME_CONCURRENT_OBSERVED, + DISPATCH_OUTCOME_SAME_TURN_BUT_HOST_SEQUENTIAL, + DISPATCH_OUTCOME_PHANTOM_PARALLEL, + DISPATCH_OUTCOME_SEQUENTIAL_OBSERVED, + DISPATCH_OUTCOME_ISOLATION_VIOLATION, + DISPATCH_OUTCOME_UNKNOWN, + } +) + # --------------------------------------------------------------------------- # Schema: ParallelismReport # --------------------------------------------------------------------------- @@ -115,7 +148,114 @@ class ParallelismReport(TypedDict): # --------------------------------------------------------------------------- -# Dormant writer — NO-OP by default (, SC-1) +# Classifier (5b.2 — ST-003) +# --------------------------------------------------------------------------- + + +def classify_dispatch( + same_turn_task_count: int, + max_in_flight: int, + base_shas: list[str], + skill_reported_concurrent: bool, +) -> str: + """Return a classification-outcome string for a dispatched wave. + + Evidence hierarchy (evaluated top-to-bottom; first match wins): + 1. ``len(set(base_shas)) > 1`` → isolation_violation + (different base SHAs across group worktrees: isolation breach) + 2. ``same_turn_task_count >= 2 and max_in_flight >= 2`` → concurrent_observed + (both transcript count AND in-flight overlap confirm concurrency) + 3. ``same_turn_task_count >= 2 and max_in_flight <= 1`` → same_turn_but_host_sequential + (tasks queued same-turn but host ran them serially) + 4. ``skill_reported_concurrent and same_turn_task_count <= 1 and max_in_flight <= 1`` + → phantom_parallel (skill self-reported concurrency but ALL objective evidence + shows ≤1 task; contradictory evidence same_turn<=1 BUT max_in_flight>=2 → unknown) + 5. ``same_turn_task_count <= 1 and max_in_flight <= 1`` → sequential_observed + (normal sequential execution) + 6. else → unknown + + Contract (HC-5): NO wall-clock timing. Inputs are pre-computed typed ints + supplied by the runner (producer-owns-parse). Skill self-report is consulted + ONLY in rule 4 and is NEVER authoritative for a positive concurrency claim. + + Args: + same_turn_task_count: Number of Task tool calls dispatched in the same + turn (parsed from the coordinator transcript by the runner). + max_in_flight: Maximum simultaneously-running tasks derived from + replayed sorted lifecycle events (runner computes via sweep). + base_shas: Base-SHA recorded for each group worktree. A healthy group + has all identical SHAs; >1 distinct SHA indicates isolation drift. + skill_reported_concurrent: Whether the skill or Actor self-reported + that concurrent dispatch was used. + + Returns: + One of the ``DISPATCH_OUTCOME_*`` constants. + """ + # Rule 1: isolation violation supersedes everything else. + if len(set(base_shas)) > 1: + return DISPATCH_OUTCOME_ISOLATION_VIOLATION + + # Rule 2: strongest positive-concurrency evidence (both signals agree). + if same_turn_task_count >= 2 and max_in_flight >= 2: + return DISPATCH_OUTCOME_CONCURRENT_OBSERVED + + # Rule 3: same-turn dispatch but host ran them serially. + if same_turn_task_count >= 2 and max_in_flight <= 1: + return DISPATCH_OUTCOME_SAME_TURN_BUT_HOST_SEQUENTIAL + + # Rule 4: skill self-report present but ALL objective evidence says ≤1 task. + # Both same_turn_task_count AND max_in_flight must confirm ≤1; if max_in_flight>=2 + # the evidence is contradictory (skill_reported says concurrent but same-turn says + # ≤1 while in-flight says ≥2) → fall through to unknown (rule 6). + # Self-report is consulted ONLY here; never treated as authoritative. + if skill_reported_concurrent and same_turn_task_count <= 1 and max_in_flight <= 1: + return DISPATCH_OUTCOME_PHANTOM_PARALLEL + + # Rule 5: normal sequential execution confirmed. + if same_turn_task_count <= 1 and max_in_flight <= 1: + return DISPATCH_OUTCOME_SEQUENTIAL_OBSERVED + + # Rule 6: evidence is contradictory or incomplete. + return DISPATCH_OUTCOME_UNKNOWN + + +# --------------------------------------------------------------------------- +# Coordinator writer (5b.2 — ST-003) +# --------------------------------------------------------------------------- + + +def record_dispatch_actual( + report: ParallelismReport, + out_path: Path, + outcome: str, +) -> bool: + """Persist exactly ONE ParallelismReport per wave — ONLY on the concurrent path. + + Decision rule: + - ``outcome == DISPATCH_OUTCOME_CONCURRENT_OBSERVED`` → write (enabled=True) + - any other outcome → no-op (returns False, no file created/touched) + + This is the *only* activation site for ``write_parallelism_report(enabled=True)`` + in 5b.2. All other callers must keep ``enabled=False`` (the dormant default). + + Args: + report: Fully-populated ParallelismReport dict (caller supplies run_id, + generated_at, etc.; this function never calls datetime.now()). + out_path: Destination path for ``parallelism.json``; parent dirs are + created by the underlying writer. + outcome: Result of ``classify_dispatch(...)``; determines whether the + writer fires. + + Returns: + ``True`` if the report was written; ``False`` on the no-op path. + """ + if outcome != DISPATCH_OUTCOME_CONCURRENT_OBSERVED: + return False + return write_parallelism_report(report, out_path, enabled=True) + + +# --------------------------------------------------------------------------- +# Dormant writer — NO-OP by default (SC-1) # --------------------------------------------------------------------------- diff --git a/src/mapify_cli/templates/codex/skills/map-efficient/SKILL.md b/src/mapify_cli/templates/codex/skills/map-efficient/SKILL.md index 7ab5e4e4..e92c4e17 100644 --- a/src/mapify_cli/templates/codex/skills/map-efficient/SKILL.md +++ b/src/mapify_cli/templates/codex/skills/map-efficient/SKILL.md @@ -17,7 +17,10 @@ section when the workflow below points to it. Under `isolation_active` (Slice 5a the wave-loop creates per-member worktrees, dispatches actor subagents **sequentially** (one per turn), verifies via `concurrency_ready`, then accepts atomically via `merge_wave_worktrees`; concurrent fan-out is Slice 5b -(`dispatch_mode==concurrent`). +(`dispatch_mode==concurrent`). Under `dispatch_mode==concurrent` (opt-in via +`execution.concurrent_dispatch: true`), call `run_concurrent_wave`: dispatch N +actor subagents in **one turn** per sub-batch; on any failure `abort_wave_group` +discards the whole group and reruns from base (bounded by `max_wave_retries`). ## Mutation Boundary Constraints diff --git a/src/mapify_cli/templates/codex/skills/map-efficient/efficient-reference.md b/src/mapify_cli/templates/codex/skills/map-efficient/efficient-reference.md index fb3597db..80f57aa4 100644 --- a/src/mapify_cli/templates/codex/skills/map-efficient/efficient-reference.md +++ b/src/mapify_cli/templates/codex/skills/map-efficient/efficient-reference.md @@ -115,7 +115,9 @@ wave-loop on every run. The wave-loop engages **only when ALL THREE hold** `mapify init` config always runs the legacy sequential walker. Even when the wave-loop engages, dispatch remains **sequential** in Slice 5a (`isolation_active=True`, `dispatch_mode` from `get_wave_step` keyed to `sequential`); concurrent fan-out -is Slice 5b (`dispatch_mode==concurrent`, `concurrency_enabled=True`, not yet shipped). +is Slice 5b (`dispatch_mode==concurrent`, `concurrency_enabled=True`), +**ACTIVE when opted in** via `execution.concurrent_dispatch: true` +(gate: `dispatch_mode == 'concurrent'`). ### Sequential walker @@ -158,24 +160,26 @@ to base on any conflict or gate failure (worktrees kept for retry). On a single subtask's Monitor failure, `discard_subtask_worktree` that subtask and retry it before calling `merge_wave_worktrees`. -### Concurrent actor dispatch — **Slice 5b only** (`dispatch_mode == 'concurrent'`) — GATED EXAMPLE +### Concurrent actor dispatch — **Slice 5b** (`dispatch_mode == 'concurrent'`) — **ACTIVE when opted in** -> **IMPORTANT — read before using this example.** -> Concurrent fan-out (dispatching multiple actor subagents in a single turn) is -> **Slice 5b** and is enabled **only when `concurrency_enabled: true` / -> `parallel_ready` flag set / `dispatch_mode == 'concurrent'`**. In the **current -> framework** (`concurrency_enabled=False`, Slice 5a), dispatch stays **SEQUENTIAL +> **IMPORTANT — read before using this section.** +> Concurrent fan-out (dispatching N actor subagents in a single turn) is +> **ACTIVE when opted in** via `execution.concurrent_dispatch: true` +> (gate: `dispatch_mode == 'concurrent'`). With the **default config** +> (`concurrent_dispatch=false`, Slice 5a), dispatch stays **SEQUENTIAL > even when a wave has `mode=="parallel"`** — one actor subagent per turn, each -> pinned to its own worktree. The example below is reference material for when -> Slice 5b ships; do NOT treat it as an active instruction now. Use your runtime's +> pinned to its own worktree. Act on the instructions below **only** when +> `get_wave_step` returns `dispatch_mode == 'concurrent'`. Use your runtime's > own parallel actor-subagent dispatch mechanism — this is the provider-neutral > shape, not a literal API call. -When Slice 5b concurrency is enabled, a parallel wave with N subtasks dispatches -all N actor subagents in **one turn**: +**Runtime wiring:** when `get_wave_step` returns `dispatch_mode == 'concurrent'`, +call `run_concurrent_wave` (runner), which batch-splits the wave by `max_actors` +and merges each sub-batch atomically via `merge_wave_worktrees`. For each sub-batch, +dispatch all N actor subagents **in one turn** — not one per turn: ```text -# CORRECT (Slice 5b / concurrency_enabled=True / dispatch_mode=='concurrent' only) — one turn, N actor subagents: +# CORRECT (dispatch_mode=='concurrent' only) — N actor subagents in one turn: dispatch actor subagent -> ST-003 (pinned to its own worktree) dispatch actor subagent -> ST-004 (pinned to its own worktree) @@ -189,6 +193,13 @@ dispatch actor subagent -> ST-004 (pinned to its own worktree) **`max_actors` cap:** Default 4–8 per wave. Groups larger than `max_actors` are pre-split into sequential batches before dispatch. +**Retry-discard on failure:** on any actor failure, timeout, or Monitor-reject +within a concurrent group, the runner calls `abort_wave_group`, which discards +the **entire group** (cancels siblings, resets all worktrees to base SHA, removes +group branches) and reruns from base. Retries are bounded by `max_wave_retries` +(default 3); on exhaustion the runner escalates to a human and does **not** +auto-restart. Never merges a successful subset — discard-all-or-merge-all (HC-4). + ### Anti-patterns — Slice 5b concurrent dispatch only > These apply **only** under Slice 5b concurrent dispatch (`dispatch_mode == 'concurrent'`). In Slice 5a and the default sequential walker, one actor dispatch per turn **is** the correct behavior. diff --git a/src/mapify_cli/templates/map/scripts/map_orchestrator.py b/src/mapify_cli/templates/map/scripts/map_orchestrator.py index e2dad2c4..f88a2fc0 100755 --- a/src/mapify_cli/templates/map/scripts/map_orchestrator.py +++ b/src/mapify_cli/templates/map/scripts/map_orchestrator.py @@ -255,6 +255,19 @@ def _extract_subtask_ids_from_plan_artifacts( WAVE_REASON_NO_WAVES = "no_waves" WAVE_REASON_WAVE_COMPLETE = "wave_complete" WAVE_REASON_DISPATCH_SEQUENTIAL = "dispatch_sequential_5a" +# Stable reason codes for compute_dispatch_gate (ST-001, Slice 5b). +WAVE_REASON_CONCURRENT_GATED = "concurrent_gated" +WAVE_REASON_GATE_NOT_PARALLELIZABLE = "gate_not_parallelizable" +# Current wave is width-1 even though a later wave is parallel; dispatch sequentially +# for this wave — not an error, just the natural plan structure. +WAVE_REASON_CURRENT_WAVE_SEQUENTIAL = "current_wave_sequential" + + +class DispatchGateError(RuntimeError): + """Raised when concurrent_dispatch=true but a required prerequisite is missing. + + HC-3: never silent-degrade — callers must handle explicitly. + """ def _read_map_config_scalars(project_dir: Path) -> dict[str, str]: @@ -2303,8 +2316,16 @@ def get_wave_step(branch: str) -> dict: state_file = Path(f".map/{branch}/step_state.json") state = StepState.load(state_file) - # Compute structured dispatch signal fields (ST-002). - dispatch_mode = "concurrent" if WAVE_CONCURRENCY_ENABLED else "sequential" + # Compute structured dispatch signal via config-driven gate (ST-001, Slice 5b). + # compute_dispatch_gate short-circuits to sequential on the first line when + # concurrent_dispatch=false (default), touching no new code/probe/import (HC-1). + gate = compute_dispatch_gate(branch, Path(".")) + dispatch_mode = gate["dispatch_mode"] + dispatch_reason = gate["reason"] + # concurrency_enabled alias: True iff dispatch_mode resolved to "concurrent". + # WAVE_CONCURRENCY_ENABLED is kept as a dormant unused const (backward compat). + concurrency_enabled = dispatch_mode == "concurrent" + try: from map_step_runner import ( # pyright: ignore[reportMissingImports] _worktree_isolation_mode, @@ -2319,8 +2340,8 @@ def get_wave_step(branch: str) -> dict: "wave_index": 0, "subtasks": [], "is_complete": True, - "concurrency_enabled": dispatch_mode == "concurrent", - "dispatch_mode": dispatch_mode, + "concurrency_enabled": concurrency_enabled, + "dispatch_mode": "sequential", "isolation_active": isolation_active, "reason": WAVE_REASON_NO_WAVES, "message": "No execution waves configured. Use sequential mode.", @@ -2332,8 +2353,8 @@ def get_wave_step(branch: str) -> dict: "wave_index": state.current_wave_index, "subtasks": [], "is_complete": True, - "concurrency_enabled": dispatch_mode == "concurrent", - "dispatch_mode": dispatch_mode, + "concurrency_enabled": concurrency_enabled, + "dispatch_mode": "sequential", "isolation_active": isolation_active, "reason": WAVE_REASON_WAVE_COMPLETE, } @@ -2372,12 +2393,10 @@ def get_wave_step(branch: str) -> dict: "wave_total": len(state.execution_waves), "subtasks": subtask_infos, "is_complete": False, - # concurrency_enabled=False: even when mode=="parallel" (width>=2 wave), - # dispatch is strictly sequential this slice. Slice 5 flips WAVE_CONCURRENCY_ENABLED. - "concurrency_enabled": dispatch_mode == "concurrent", + "concurrency_enabled": concurrency_enabled, "dispatch_mode": dispatch_mode, "isolation_active": isolation_active, - "reason": WAVE_REASON_DISPATCH_SEQUENTIAL, + "reason": dispatch_reason, } @@ -2546,6 +2565,113 @@ def select_execution_strategy( } +def compute_dispatch_gate( + branch: str, project_dir: Optional[Path] = None +) -> dict: + """Compute the dispatch mode for the current wave, fail-closed on config contradiction. + + Gate logic (evaluated in order): + + 1. If concurrent_dispatch is False (default): return sequential immediately. + FIRST executable line — no probe, no select_execution_strategy call, no import + of any concurrency primitive (HC-1 byte-identity). + + 2. If concurrent_dispatch is True AND worktree.isolation == 'off': + raise DispatchGateError — config contradiction, HC-3 never silent-degrade. + + 3. If concurrent_dispatch is True AND isolation != 'off' AND NOT concurrency_allowed: + return sequential with WAVE_REASON_GATE_NOT_PARALLELIZABLE (not an error — + the plan has no parallelizable groups). + + 4. If concurrent_dispatch is True AND isolation != 'off' AND concurrency_allowed + AND the CURRENT wave (execution_waves[current_wave_index]) has width < 2: + return sequential with WAVE_REASON_CURRENT_WAVE_SEQUENTIAL (not an error — + the current wave is width-1 even though a later wave is parallel). + + 5. If concurrent_dispatch is True AND isolation != 'off' AND concurrency_allowed + AND the CURRENT wave has width >= 2: + return concurrent with WAVE_REASON_CONCURRENT_GATED. + + Args: + branch: Git branch name (sanitized). + project_dir: Project root containing .map/config.yaml. + Defaults to Path('.'). + + Returns: + {"dispatch_mode": "sequential" | "concurrent", "reason": } + + Raises: + DispatchGateError: When concurrent_dispatch=true but worktree.isolation='off' + (HC-3: config contradiction must never be silently degraded). + """ + if project_dir is None: + project_dir = Path(".") + + # Step 1: short-circuit on flag=false — first gate check after parameter + # normalization. No concurrency probe, no select_execution_strategy call, no + # _worktree_isolation_mode/concurrency_ready call, no dispatcher import runs on + # this path (HC-1 byte-identity). The project_dir None-guard above is a safe + # default-arg normalization, not a concurrency primitive. + try: + from map_step_runner import ( # pyright: ignore[reportMissingImports] + _concurrent_dispatch_enabled, + ) + flag_on = _concurrent_dispatch_enabled(project_dir) + except ImportError: + flag_on = False + + if not flag_on: + return { + "dispatch_mode": "sequential", + "reason": WAVE_REASON_DISPATCH_SEQUENTIAL, + } + + # Step 2: flag is on — check isolation config. + try: + from map_step_runner import ( # pyright: ignore[reportMissingImports] + _worktree_isolation_mode as _wt_iso, + ) + isolation = _wt_iso(project_dir) + except ImportError: + isolation = "off" + + if isolation == "off": + raise DispatchGateError( + "concurrent_dispatch=true requires worktree.isolation != 'off', " + f"but worktree.isolation is 'off' in {project_dir}. " + "Set worktree.isolation to 'auto' or 'required' to enable concurrent dispatch." + ) + + # Step 3: check whether the plan is actually parallelizable (any wave has width>=2). + strategy_result = select_execution_strategy(branch, project_dir) + concurrency_allowed = strategy_result.get("concurrency_allowed", False) + + if not concurrency_allowed: + return { + "dispatch_mode": "sequential", + "reason": WAVE_REASON_GATE_NOT_PARALLELIZABLE, + } + + # Step 4: plan has at least one parallel wave, but gate on the ACTIVE wave. + # select_execution_strategy checks any wave (has_parallel_groups), not the + # current wave index. A width-1 current wave must dispatch sequentially even + # if a later wave is parallel — dispatch_mode is per-wave, not per-plan. + state_file = Path(f".map/{branch}/step_state.json") + state = StepState.load(state_file) + waves = state.execution_waves + idx = state.current_wave_index + if idx >= len(waves) or len(waves[idx]) < 2: + return { + "dispatch_mode": "sequential", + "reason": WAVE_REASON_CURRENT_WAVE_SEQUENTIAL, + } + + return { + "dispatch_mode": "concurrent", + "reason": WAVE_REASON_CONCURRENT_GATED, + } + + def _write_feedback_file( branch: str, filename: str, header: str, feedback: str ) -> Optional[str]: diff --git a/src/mapify_cli/templates/map/scripts/map_step_runner.py b/src/mapify_cli/templates/map/scripts/map_step_runner.py index a343f61a..a7d3fe4f 100755 --- a/src/mapify_cli/templates/map/scripts/map_step_runner.py +++ b/src/mapify_cli/templates/map/scripts/map_step_runner.py @@ -148,6 +148,20 @@ def _map_config_int(project_dir: Path, key: str, default: int) -> int: _WAVE_MODE_VALID = frozenset({"off", "auto", "on"}) +# Truthy string values for boolean-style config flags. +_CONCURRENT_DISPATCH_TRUTHY = frozenset({"true", "yes", "y", "1", "on"}) + + +def _concurrent_dispatch_enabled(project_dir: Path) -> bool: + """Return True when execution.concurrent_dispatch is explicitly enabled. + + Mirrors the _wt_isolation_enabled pattern. Default is False (off) so the + sequential path stays byte-identical to Slice 5a by default (HC-1). Never + raises. + """ + raw = _map_config_str(project_dir, "execution.concurrent_dispatch", "false") + return raw.strip().lower() in _CONCURRENT_DISPATCH_TRUTHY + def _execution_wave_mode(project_dir: Path) -> str: """Return the execution.wave_mode setting: 'off' | 'auto' | 'on'. @@ -15348,6 +15362,23 @@ def _wt_force_remove(path: Path, branch_ref: str) -> None: _WT_REASON_NOT_REGISTERED: str = "not_registered" _WT_REASON_HEAD_MISMATCH: str = "head_mismatch" _WT_REASON_DIRTY: str = "dirty" +# group lifecycle reason codes (5b.1) +_WT_REASON_GROUP_HEAD_MISMATCH: str = "group_head_mismatch" +_WT_REASON_GROUP_DIRTY_TREE: str = "group_dirty_tree" +_WT_REASON_GROUP_WORKTREES_REMAIN: str = "group_worktrees_remain" +# group lifecycle event codes (5b.1) — stable set; classifier replays these +_WT_GROUP_EVENT_CREATED: str = "created" +_WT_GROUP_EVENT_STARTED: str = "started" +_WT_GROUP_EVENT_FINISHED: str = "finished" +_WT_GROUP_EVENT_MERGED: str = "merged" +_WT_GROUP_EVENT_ABORTED: str = "aborted" +_WT_GROUP_VALID_EVENTS: frozenset[str] = frozenset({ + _WT_GROUP_EVENT_CREATED, + _WT_GROUP_EVENT_STARTED, + _WT_GROUP_EVENT_FINISHED, + _WT_GROUP_EVENT_MERGED, + _WT_GROUP_EVENT_ABORTED, +}) _WT_ISOLATION_VALID = frozenset({"off", "auto", "required"}) @@ -15818,6 +15849,308 @@ def cleanup_orphan_worktrees(branch: str) -> dict[str, object]: return {"removed": removed, "kept_active": kept_active, "ok": True} +# --------------------------------------------------------------------------- +# Group lifecycle verbs (5b.1) — coordinator-owned, idempotent +# --------------------------------------------------------------------------- + + +def begin_wave_group( + group_ids: list[str], + branch: Optional[str] = None, +) -> dict[str, object]: + """Record the base_sha anchor + per-subtask lifecycle skeleton for a parallel group. + + Stores state under ``wave_groups[group_id]`` in the branch-scoped worktree-state + sidecar. Idempotent: re-invoking with the same group_ids does not duplicate + entries or overwrite ``base_sha`` if already set (crash-safe recovery). + + Returns:: + + { + "ok": True, + "group_id": , + "base_sha": , + "subtask_ids": [...], + } + """ + if not _wt_is_git_repo(): + return _wt_error(_WT_REASON_NOT_GIT_REPO, "not inside a git work tree") + + branch_name = branch or get_branch_name() + base_sha = _wt_head_sha() + if base_sha is None: + return _wt_error("no_head_sha", "could not resolve HEAD sha") + + # Canonical group key: sorted ids joined so order-of-call doesn't vary the key. + ids_sorted = sorted(str(s) for s in group_ids if str(s).strip()) + group_key = "|".join(ids_sorted) + + state = _read_worktree_state(branch_name) + if not isinstance(state.get("wave_groups"), dict): + state["wave_groups"] = {} + wave_groups = state["wave_groups"] + if not isinstance(wave_groups, dict): + wave_groups = {} + state["wave_groups"] = wave_groups + + if group_key not in wave_groups: + wave_groups[group_key] = { + "base_sha": base_sha, + "subtask_ids": ids_sorted, + "lifecycle": {}, # subtask_id -> list of {seq, event, ts} + } + else: + # Idempotent: fill in missing skeleton fields without overwriting base_sha. + existing = wave_groups[group_key] + if not isinstance(existing, dict): + existing = {} + wave_groups[group_key] = existing + existing.setdefault("base_sha", base_sha) + existing.setdefault("subtask_ids", ids_sorted) + if not isinstance(existing.get("lifecycle"), dict): + existing["lifecycle"] = {} + # Ensure every id has a slot + for sid in ids_sorted: + existing["lifecycle"].setdefault(sid, []) + + _write_worktree_state(branch_name, state) + return { + "ok": True, + "group_key": group_key, + "base_sha": base_sha, + "subtask_ids": ids_sorted, + } + + +def record_group_lifecycle( + group_key: str, + subtask_id: str, + event: str, + branch: Optional[str] = None, +) -> dict[str, object]: + """Append a lifecycle event for *subtask_id* inside *group_key*. + + Events are appended with a monotonically-increasing sequence number so + replaying the list in ``seq`` order reconstructs a deterministic in-flight + timeline (used by the classifier in ST-003 to derive ``max_in_flight``). + + *event* must be one of the stable codes in ``_WT_GROUP_VALID_EVENTS`` + (created / started / finished / merged / aborted). + + Returns:: + + { + "ok": True, + "group_key": ..., + "subtask_id": ..., + "event": ..., + "seq": , + } + """ + if event not in _WT_GROUP_VALID_EVENTS: + return _wt_error( + "invalid_event", + f"event {event!r} not in valid set {sorted(_WT_GROUP_VALID_EVENTS)}", + ) + + branch_name = branch or get_branch_name() + + # Serialize concurrent record_group_lifecycle calls from SEPARATE PROCESSES + # (actor worktrees) with an advisory file lock. threading.Lock is insufficient + # because actors are separate OS processes. Reuses the same fcntl pattern as + # _wt_acquire_merge_lock/_wt_release_merge_lock (wave-lifecycle.lock is a + # distinct lock from wave-merge.lock so lifecycle appends never block merges). + _lc_lock = _wt_acquire_lifecycle_lock() + try: + state = _read_worktree_state(branch_name) + + wave_groups = state.get("wave_groups") + if not isinstance(wave_groups, dict) or group_key not in wave_groups: + return _wt_error("unknown_group", f"group {group_key!r} not found; call begin_wave_group first") + + group = wave_groups[group_key] + if not isinstance(group, dict): + return _wt_error("corrupt_group", f"group record for {group_key!r} is malformed") + + lifecycle = group.get("lifecycle") + if not isinstance(lifecycle, dict): + lifecycle = {} + group["lifecycle"] = lifecycle + + sid = str(subtask_id).strip() + if sid not in lifecycle: + lifecycle[sid] = [] + events_list = lifecycle[sid] + if not isinstance(events_list, list): + events_list = [] + lifecycle[sid] = events_list + + # Monotonic seq: max of all existing seqs across ALL subtasks in this group + 1. + max_seq = 0 + for ev_list in lifecycle.values(): + if isinstance(ev_list, list): + for ev in ev_list: + if isinstance(ev, dict): + max_seq = max(max_seq, int(ev.get("seq", 0))) + seq = max_seq + 1 + + import time as _time # local import — keeps module-level imports minimal + events_list.append({"seq": seq, "event": event, "ts": _time.time()}) + + _write_worktree_state(branch_name, state) + finally: + _wt_release_lifecycle_lock(_lc_lock) + + return {"ok": True, "group_key": group_key, "subtask_id": sid, "event": event, "seq": seq} + + +def verify_group_clean( + branch: Optional[str] = None, +) -> dict[str, object]: + """Read-only check: repo is in a clean state after a wave group completes. + + Returns ``clean=True`` iff ALL of: + 1. HEAD sha == the recorded ``base_sha`` for every group (HEAD not diverged). + 2. Working tree is clean (``git status --porcelain`` minus runtime-state paths). + 3. Zero group worktrees remain (``wave_groups`` dict is empty or all groups + have been removed from the sidecar). + + Returns:: + + { + "clean": bool, + "reason": | None, + "head_sha": ..., + "base_sha": ..., + } + """ + if not _wt_is_git_repo(): + return { + "clean": False, + "reason": _WT_REASON_NOT_GIT_REPO, + "head_sha": None, + "base_sha": None, + } + + branch_name = branch or get_branch_name() + head_sha = _wt_head_sha() + state = _read_worktree_state(branch_name) + + # Collect base_shas from all groups + wave_groups = state.get("wave_groups") + if isinstance(wave_groups, dict) and wave_groups: + # Any group that has a recorded base_sha must match HEAD. + for _gk, grp in wave_groups.items(): + if not isinstance(grp, dict): + continue + recorded_base = grp.get("base_sha") + if recorded_base and head_sha != recorded_base: + return { + "clean": False, + "reason": _WT_REASON_GROUP_HEAD_MISMATCH, + "head_sha": head_sha, + "base_sha": recorded_base, + } + # Groups still present means group worktrees remain. + return { + "clean": False, + "reason": _WT_REASON_GROUP_WORKTREES_REMAIN, + "head_sha": head_sha, + "base_sha": None, + } + + # No groups remain — check tree cleanliness. + status = _wt_git(["status", "--porcelain"], timeout=15) + if status.returncode != 0: + return { + "clean": False, + "reason": _WT_REASON_DIRTY, + "head_sha": head_sha, + "base_sha": None, + } + dirty_lines = [ + ln for ln in status.stdout.splitlines() + if ln.strip() and not _wt_is_runtime_state_path(_wt_porcelain_path(ln)) + ] + if dirty_lines: + return { + "clean": False, + "reason": _WT_REASON_GROUP_DIRTY_TREE, + "head_sha": head_sha, + "base_sha": None, + } + + return {"clean": True, "reason": None, "head_sha": head_sha, "base_sha": head_sha} + + +def reconcile_orphan_groups( + branch: Optional[str] = None, +) -> dict[str, object]: + """Startup sweep: find groups left mid-flight and invoke cleanup. + + Composes the existing ``cleanup_orphan_worktrees`` to remove physical worktrees, + then removes stale group entries from the wave_groups sidecar. Idempotent: + a second call after everything is clean returns ``swept=0``. + + Returns:: + + { + "ok": True, + "swept": , + "cleanup": , + } + """ + branch_name = branch or get_branch_name() + + # Step 1: remove physical orphan worktrees (existing helper). + cleanup_result = cleanup_orphan_worktrees(branch_name) + + # Step 2: remove stale wave_group entries from sidecar. + state = _read_worktree_state(branch_name) + wave_groups = state.get("wave_groups") + swept = 0 + if isinstance(wave_groups, dict) and wave_groups: + # A group is stale if all its lifecycle events include a terminal event + # (merged or aborted) OR if it has no lifecycle events at all (never started). + _terminal = {_WT_GROUP_EVENT_MERGED, _WT_GROUP_EVENT_ABORTED} + stale_keys: list[str] = [] + for gk, grp in list(wave_groups.items()): + if not isinstance(grp, dict): + stale_keys.append(gk) + continue + lifecycle = grp.get("lifecycle", {}) + if not isinstance(lifecycle, dict) or not lifecycle: + # No lifecycle events recorded — orphan from a crash before start. + stale_keys.append(gk) + continue + # Check if EVERY declared subtask has at least one terminal event. + # Iterate over declared subtask_ids (recorded by begin_wave_group), NOT + # lifecycle.values(): a partially-recorded group (one subtask missing its + # events slot) must NOT be swept — missing slot → NOT terminal. + declared_sids = grp.get("subtask_ids", []) + if not isinstance(declared_sids, list) or not declared_sids: + # No declared subtasks — treat as orphan (begin_wave_group not called). + stale_keys.append(gk) + continue + all_terminal = all( + isinstance(lifecycle.get(sid), list) + and any( + isinstance(ev, dict) and ev.get("event") in _terminal + for ev in lifecycle[sid] + ) + for sid in declared_sids + ) + if all_terminal: + stale_keys.append(gk) + for key in stale_keys: + del wave_groups[key] + swept += 1 + if swept: + _write_worktree_state(branch_name, state) + + return {"ok": True, "swept": swept, "cleanup": cleanup_result} + + def create_subtask_worktree( subtask_id: str, attempt: int = 0, @@ -16214,6 +16547,56 @@ def _wt_release_merge_lock(handle: Optional[Any]) -> None: pass +def _wt_lifecycle_lock_path() -> Optional[Path]: + """Advisory lock path for the lifecycle sidecar (separate from the merge lock).""" + common = _wt_git_common_dir() + if common is None: + return None + return common / "map-framework" / "wave-lifecycle.lock" + + +def _wt_acquire_lifecycle_lock() -> Optional[Any]: + """Advisory file lock for lifecycle sidecar read-modify-write. + + Reuses the same fcntl pattern as _wt_acquire_merge_lock so concurrent + actor processes (separate OS processes) cannot clobber each other's events. + Returns a file handle holding the lock, or None when unavailable. + """ + lock_path = _wt_lifecycle_lock_path() + if lock_path is None: + return None + lock_path.parent.mkdir(parents=True, exist_ok=True) + handle = lock_path.open("w") + try: + import fcntl # noqa: PLC0415 + except ImportError: + return handle # best-effort on non-POSIX + try: + fcntl.flock(handle.fileno(), fcntl.LOCK_EX) # blocking — sidecar writes are fast + except OSError: + handle.close() + return None + return handle + + +def _wt_release_lifecycle_lock(handle: Optional[Any]) -> None: + if handle is None: + return + try: + import fcntl # noqa: PLC0415 + + try: + fcntl.flock(handle.fileno(), fcntl.LOCK_UN) + except OSError: + pass + except ImportError: + pass + try: + handle.close() + except OSError: + pass + + def merge_subtask_worktree( subtask_id: str, attempt: int = 0, @@ -16906,6 +17289,323 @@ def concurrency_ready( } +# --------------------------------------------------------------------------- +# Concurrent-wave COORDINATOR (5b.4, ST-005) + group-abort (5b.5, ST-006) +# --------------------------------------------------------------------------- + +_MAX_ACTORS_MIN: int = 1 +_MAX_ACTORS_MAX: int = 8 +_MAX_ACTORS_DEFAULT: int = 4 + +_MAX_WAVE_RETRIES_MIN: int = 1 +_MAX_WAVE_RETRIES_MAX: int = 10 +_MAX_WAVE_RETRIES_DEFAULT: int = 3 + + +def _max_actors(project_dir: Optional[Path] = None) -> int: + """Read ``execution.max_actors`` from config and clamp to [1, 8]. + + Mirrors ``clamp_max_actors()`` from ``MapConfig`` without importing it. + Non-int / bool / absent values fall back to the default 4. + """ + raw = _map_config_int( + project_dir or (_wt_project_dir() or Path(".")), + "execution.max_actors", + _MAX_ACTORS_DEFAULT, + ) + # _map_config_int already returns > 0 or default; clamp to [1, 8]. + return max(_MAX_ACTORS_MIN, min(_MAX_ACTORS_MAX, raw)) + + +def _max_wave_retries(project_dir: Optional[Path] = None) -> int: + """Read ``execution.max_wave_retries`` from config and clamp to [1, 10]. + + Default is 3. Non-int / bool / absent values fall back to the default. + Mirrors the _max_actors pattern. + """ + raw = _map_config_int( + project_dir or (_wt_project_dir() or Path(".")), + "execution.max_wave_retries", + _MAX_WAVE_RETRIES_DEFAULT, + ) + return max(_MAX_WAVE_RETRIES_MIN, min(_MAX_WAVE_RETRIES_MAX, raw)) + + +def _chunk(items: list[str], size: int) -> list[list[str]]: + """Split *items* into ordered sub-lists each of width <= *size*.""" + if size < 1: + size = 1 + return [items[i : i + size] for i in range(0, len(items), size)] + + +def abort_wave_group( + group_id: str, + branch: Optional[str] = None, +) -> dict[str, object]: + """Idempotent, runner-owned group-abort verb (HC-4, ST-006). + + On ANY pre-merge actor failure / timeout / cancel / Monitor-reject, discard + the WHOLE group and return to base. NEVER partially merges a subset. + + Steps (each is idempotent on re-entry): + + 1. Read the group's recorded ``base_sha`` from the lifecycle sidecar + (written by ``begin_wave_group``). + 2. If ``_wt_active_git_operation()`` reports a mid-merge, run + ``git merge --abort`` first. + 3. **Reuse** ``_wt_rollback(base_sha)`` — hard-reset to base_sha + clean + with ``-e .map -e .codex -e .agents`` so the gitignored runtime state + (step_state.json, worktree sidecar) is NEVER deleted. + DO NOT call ``git clean -fdx`` or ``git clean -x`` directly. + 4. Discard every group worktree + branch via ``discard_subtask_worktree`` + (which uses ``_wt_force_remove`` internally). + 5. Mark the group ``aborted`` in the lifecycle sidecar and remove the + group entry from ``wave_groups`` so ``verify_group_clean`` sees zero + groups. + 6. Call ``verify_group_clean`` and return its verdict. + + Idempotent: a second invocation after a partial abort converges to + ``verify_group_clean == True`` without error. + + Returns ``verify_group_clean`` dict augmented with ``aborted_group_id``. + """ + if not _wt_is_git_repo(): + return _wt_error(_WT_REASON_NOT_GIT_REPO, "not inside a git work tree") + + branch_name = branch or get_branch_name() + state = _read_worktree_state(branch_name) + wave_groups = state.get("wave_groups") + + # Resolve the canonical group key from the sidecar. The caller may pass + # the raw group_id string (could be the canonical key, or a single subtask id). + group_key: Optional[str] = None + base_sha: Optional[str] = None + subtask_ids: list[str] = [] + if isinstance(wave_groups, dict): + if group_id in wave_groups: + group_key = group_id + else: + # Tolerate a caller passing one member subtask id — scan for it. + for gk, grp in wave_groups.items(): + if isinstance(grp, dict): + sids = grp.get("subtask_ids", []) + if isinstance(sids, list) and group_id in sids: + group_key = gk + break + if group_key is not None: + grp = wave_groups[group_key] + if isinstance(grp, dict): + base_sha = str(grp.get("base_sha", "")) or None + raw_sids = grp.get("subtask_ids", []) + subtask_ids = list(raw_sids) if isinstance(raw_sids, list) else [] + + # Step 2: abort any in-progress merge (idempotent — fails safely if none). + active_op = _wt_active_git_operation() + if active_op == "merge": + _wt_git(["merge", "--abort"]) + + # Step 3: rollback to base_sha if we have one. + # MUST reuse _wt_rollback — it excludes .map/.codex/.agents from the clean. + rollback_verified = True # optimistic; no base_sha → nothing to verify + if base_sha: + _wt_rollback(base_sha) + # Verify rollback landed BEFORE touching group state (F5). + # If HEAD != base_sha the rollback failed; keep the group entry so that + # verify_group_clean still has base_sha and a wrong HEAD cannot pass as clean. + actual_head = _wt_head_sha() + if actual_head != base_sha: + rollback_verified = False + result: dict[str, object] = { + "ok": False, + "clean": False, + "reason": "rollback_head_mismatch", + "base_sha": base_sha, + "actual_head": actual_head, + "aborted_group_id": group_id, + } + return result + + # Step 4: discard every group worktree + branch. + for sid in subtask_ids: + discard_subtask_worktree(sid, branch=branch_name) + + # Step 5: mark aborted in sidecar and remove the group entry so + # verify_group_clean sees zero groups. Only reached when rollback is verified. + if group_key is not None and rollback_verified: + # Record aborted event for every subtask (best-effort). + for sid in subtask_ids: + record_group_lifecycle(group_key, sid, _WT_GROUP_EVENT_ABORTED, branch_name) + # Re-read state (record_group_lifecycle writes it). + state2 = _read_worktree_state(branch_name) + wg2 = state2.get("wave_groups") + if isinstance(wg2, dict) and group_key in wg2: + del wg2[group_key] + _write_worktree_state(branch_name, state2) + + # Step 6: verify clean and return. + verdict = verify_group_clean(branch_name) + final_result: dict[str, object] = dict(verdict) + final_result["aborted_group_id"] = group_id + return final_result + + +def run_concurrent_wave( + group_ids: list[str], + branch: Optional[str] = None, + project_dir: Optional[Path] = None, +) -> dict[str, object]: + """Coordinate an N-way concurrent wave: batch-split + atomic sub-batch merge. + + This function is the COORDINATOR side of concurrent dispatch. It does NOT + spawn actor agents — the skill emits N Task blocks to start actors (ST-007). + Its responsibilities are: + + 1. **Default-off guard** (HC-1, defense-in-depth): if ``concurrent_dispatch`` + is not explicitly enabled in config, return ``CONCURRENT_DISPATCH_DISABLED`` + immediately so a direct CLI or coordinator call cannot trigger concurrent + merging under the default-off configuration. + + 2. **Batch-split**: read ``max_actors`` from config (via ``_max_actors()``), + clamp to [1, 8], then split the sorted ``group_ids`` into sequential + sub-batches each of width <= cap. A group already within the cap is one + batch (no split). + + 3. **Atomic sub-batch merge**: for each sub-batch call the existing + ``merge_wave_worktrees(sub_batch, branch=...)`` which is all-or-nothing + (HC-4, #284 invariant). The NEXT sub-batch branches from the prior + sub-batch's post-merge HEAD. Do NOT re-implement merge. + + 4. **Telemetry / lifecycle**: call ``record_dispatch_actual`` CLI verb (via + ``begin_wave_group`` + ``record_group_lifecycle`` — these must be called + by the skill/coordinator before the actors run; this function only reads + state and calls merge). Telemetry is emitted exactly ONCE per wave via + the ``record_dispatch_actual`` CLI (the skill wires that in ST-007). + + 5. **Abort-once on failure, return needs_redispatch** (ST-006, architectural): + on a ``merge_wave_worktrees`` error, invoke ``abort_wave_group`` ONCE to + discard the WHOLE group and reset to base (HC-4). Do NOT retry internally — + the group worktrees are gone after abort and cannot be re-merged without the + skill re-dispatching actors. Return a structured result with + ``needs_redispatch: True`` and ``attempts_remaining`` (read from the group + sidecar so successive calls can track exhaustion). The SKILL owns the retry + loop: re-dispatch actors, then call run_concurrent_wave again. + + Returns on full success:: + + { + "status": "success", + "ok": True, + "group_ids": [...], # original sorted ids + "sub_batches": [[...], ...], # how ids were split + "max_actors": int, # cap used + "batches_merged": int, # number of sub-batches atomically merged + "merged_ids": [...], # all ids that landed (may be < group_ids if no-change) + "no_changes": [...], # ids with no-change-in-worktree + } + + Returns when concurrent dispatch is disabled (HC-1 defense-in-depth):: + + { + "status": "error", + "ok": False, + "kind": "CONCURRENT_DISPATCH_DISABLED", + ... + } + + Returns on merge failure (abort-once, needs_redispatch):: + + { + "status": "error", + "ok": False, + "kind": "WAVE_ABORTED", + "needs_redispatch": True, + "attempts_remaining": int, # decremented in group sidecar; 0 → escalate + "escalate_to_human": bool, # True when attempts_remaining == 0 + "group_ids": [...], + "merge_error": {...}, # merge_wave_worktrees error dict + } + """ + if not _wt_is_git_repo(): + return _wt_error(_WT_REASON_NOT_GIT_REPO, "not inside a git work tree") + + pd = project_dir or _wt_project_dir() or Path(".") + branch_name = branch or get_branch_name() + + # F6: Default-off guard (HC-1, defense-in-depth). A direct CLI call or + # coordinator misconfiguration cannot trigger concurrent merging unless the + # flag is explicitly set. This is a second line of defense after compute_dispatch_gate. + if not _concurrent_dispatch_enabled(pd): + return _wt_error( + "CONCURRENT_DISPATCH_DISABLED", + "run_concurrent_wave requires execution.concurrent_dispatch=true in config; " + "default is off (HC-1). Set the flag explicitly to enable concurrent dispatch.", + ) + + # Deterministic sorted list — order-of-call must not vary group membership. + ids_sorted = sorted(str(s) for s in group_ids if str(s).strip()) + if not ids_sorted: + return _wt_error("NO_SUBTASKS", "no subtask ids supplied to run_concurrent_wave") + + cap = _max_actors(pd) + max_retries = _max_wave_retries(pd) + sub_batches = _chunk(ids_sorted, cap) + group_key = "|".join(ids_sorted) + + merged_ids: list[str] = [] + no_changes: list[str] = [] + batches_merged = 0 + + for batch in sub_batches: + merge_result = merge_wave_worktrees(batch, branch=branch_name, skip_post_wave=True) + if merge_result.get("status") == "error" or not merge_result.get("ok"): + # F7: Abort ONCE — worktrees are gone after abort; the skill must + # re-dispatch actors before calling run_concurrent_wave again. + # Track attempt count in the group sidecar so successive calls can + # decrement and escalate when exhausted. + abort_wave_group(group_key, branch_name) + + # Read attempt count from sidecar (written by begin_wave_group / prior calls). + _st2 = _read_worktree_state(branch_name) + _wg2 = _st2.get("wave_groups") or {} + _grp2 = _wg2.get(group_key) if isinstance(_wg2, dict) else None + _attempts_used = 1 + if isinstance(_grp2, dict): + _attempts_used = int(_grp2.get("abort_attempts", 0)) + 1 + _grp2["abort_attempts"] = _attempts_used + _write_worktree_state(branch_name, _st2) + + attempts_remaining = max(0, max_retries - _attempts_used) + return { + "status": "error", + "ok": False, + "kind": "WAVE_ABORTED", + "needs_redispatch": True, + "attempts_remaining": attempts_remaining, + "escalate_to_human": attempts_remaining == 0, + "group_ids": ids_sorted, + "merge_error": dict(merge_result), + "failed_batch": batch, + "batches_merged_before_failure": batches_merged, + } + + raw_merged = merge_result.get("merged", []) + raw_no_changes = merge_result.get("no_changes", []) + merged_ids.extend(list(raw_merged) if isinstance(raw_merged, list) else []) + no_changes.extend(list(raw_no_changes) if isinstance(raw_no_changes, list) else []) + batches_merged += 1 + + return { + "status": "success", + "ok": True, + "group_ids": ids_sorted, + "sub_batches": sub_batches, + "max_actors": cap, + "batches_merged": batches_merged, + "merged_ids": merged_ids, + "no_changes": no_changes, + } + + if __name__ == "__main__": # Simple CLI interface for testing import sys @@ -18419,6 +19119,259 @@ def _flag_val(name: str) -> Optional[str]: if _wt_r.get("status") == "error": sys.exit(1) + elif func_name == "begin_wave_group": + # CLI: begin_wave_group [ ...] [--branch B] + # Record the group base_sha + per-subtask lifecycle skeleton (5b.1). + # Idempotent: re-running with the same ids does not duplicate state. + import argparse as _ap + + _p = _ap.ArgumentParser(prog="map_step_runner.py begin_wave_group") + _p.add_argument("group_ids", nargs="+") + _p.add_argument("--branch", default=None) + _a = _p.parse_args(sys.argv[2:]) + _wt_r = begin_wave_group(_a.group_ids, _a.branch) + print(json.dumps(_wt_r, indent=2)) + if not _wt_r.get("ok"): + sys.exit(1) + + elif func_name == "record_group_lifecycle": + # CLI: record_group_lifecycle [--branch B] + # Append a lifecycle event (created/started/finished/merged/aborted) (5b.1). + import argparse as _ap + + _p = _ap.ArgumentParser(prog="map_step_runner.py record_group_lifecycle") + _p.add_argument("group_key") + _p.add_argument("subtask_id") + _p.add_argument("event") + _p.add_argument("--branch", default=None) + _a = _p.parse_args(sys.argv[2:]) + _wt_r = record_group_lifecycle(_a.group_key, _a.subtask_id, _a.event, _a.branch) + print(json.dumps(_wt_r, indent=2)) + if not _wt_r.get("ok"): + sys.exit(1) + + elif func_name == "verify_group_clean": + # CLI: verify_group_clean [--branch B] + # Read-only: clean iff HEAD==base_sha AND tree clean AND zero group worktrees. + # Exits 0 when clean=True; exits 1 when clean=False (usable as a gate). + import argparse as _ap + + _p = _ap.ArgumentParser(prog="map_step_runner.py verify_group_clean") + _p.add_argument("--branch", default=None) + _a = _p.parse_args(sys.argv[2:]) + _wt_r = verify_group_clean(_a.branch) + print(json.dumps(_wt_r, indent=2)) + if not _wt_r.get("clean"): + sys.exit(1) + + elif func_name == "reconcile_orphan_groups": + # CLI: reconcile_orphan_groups [--branch B] + # Startup sweep: remove stale wave_group sidecar entries + orphan worktrees. + import argparse as _ap + + _p = _ap.ArgumentParser(prog="map_step_runner.py reconcile_orphan_groups") + _p.add_argument("--branch", default=None) + _a = _p.parse_args(sys.argv[2:]) + _wt_r = reconcile_orphan_groups(_a.branch) + print(json.dumps(_wt_r, indent=2)) + if not _wt_r.get("ok"): + sys.exit(1) + + elif func_name == "record_dispatch_actual": + # CLI: record_dispatch_actual + # [--branch B] [--same-turn-count N] [--skill-reported-concurrent] + # + # Coordinator-owned dispatch telemetry (5b.2 / ST-003). + # Replays the ST-002 lifecycle events recorded under wave_groups to + # compute max_in_flight deterministically (sorted-seq sweep, no wall-clock), + # then calls classify_dispatch() with the typed evidence inputs, and + # persists a ParallelismReport via record_dispatch_actual() ONLY when + # the outcome is concurrent_observed. All other outcomes are no-ops. + # + # Producer-owns-parse: the runner parses lifecycle/sidecar here; the + # classifier (in parallelism_observability) receives only typed ints. + import argparse as _ap + import sys as _sys + + _p = _ap.ArgumentParser(prog="map_step_runner.py record_dispatch_actual") + _p.add_argument("group_key", help="Canonical group key (sorted subtask IDs joined by '|')") + _p.add_argument("run_id", help="Run identifier for the parallelism.json path") + _p.add_argument("out_path", help="Destination path for parallelism.json") + _p.add_argument("--branch", default=None) + _p.add_argument( + "--same-turn-count", + type=int, + default=0, + dest="same_turn_count", + help="Number of Task tool calls in the same turn (from coordinator transcript)", + ) + _p.add_argument( + "--skill-reported-concurrent", + action="store_true", + dest="skill_reported_concurrent", + help="Set when the skill or Actor self-reported concurrent dispatch", + ) + _a = _p.parse_args(sys.argv[2:]) + + # --- Step 1: read the wave_groups sidecar for this branch --- + _branch_name = _a.branch or get_branch_name() + _state = _read_worktree_state(_branch_name) + _wave_groups = _state.get("wave_groups") or {} + + # --- Step 2: extract base_shas and lifecycle events for this group --- + # F8: collect per-subtask base SHAs from each worktree record so + # classify_dispatch can detect isolation_violation (len(set(base_shas))>1). + # Appending only the group-level base_sha means all subtasks share one SHA + # and the isolation_violation path is unreachable. + _group_data = _wave_groups.get(_a.group_key) if isinstance(_wave_groups, dict) else None + _base_shas: list[str] = [] + _all_events: list[dict] = [] + + if isinstance(_group_data, dict): + _lifecycle = _group_data.get("lifecycle") or {} + if isinstance(_lifecycle, dict): + for _sid_events in _lifecycle.values(): + if isinstance(_sid_events, list): + for _ev in _sid_events: + if isinstance(_ev, dict): + _all_events.append(_ev) + + # Collect per-subtask base SHAs from each worktree sidecar record. + # Falls back to the group-level base_sha for subtasks whose worktree + # record is absent (partial registration or pre-begin_wave_group crash). + _group_sids = _group_data.get("subtask_ids", []) + _worktrees = _state.get("worktrees") or {} + _group_level_sha = _group_data.get("base_sha") + if isinstance(_group_sids, list) and _group_sids: + for _sid in _group_sids: + _slug = _wt_slug(_sid) + _wt_rec = _worktrees.get(_slug) if isinstance(_worktrees, dict) and _slug else None + if isinstance(_wt_rec, dict): + _per_sha = _wt_rec.get("base_sha") + if isinstance(_per_sha, str) and _per_sha: + _base_shas.append(_per_sha) + continue + # Fallback: use group-level SHA when per-subtask record missing. + if isinstance(_group_level_sha, str) and _group_level_sha: + _base_shas.append(_group_level_sha) + else: + # No declared subtask_ids — fall back to group-level SHA. + if isinstance(_group_level_sha, str) and _group_level_sha: + _base_shas.append(_group_level_sha) + + # --- Step 3: compute max_in_flight by replaying sorted lifecycle events --- + # Sweep events sorted by monotonic seq number. Only "started" and + # "finished" affect the in-flight counter. This is deterministic and + # completely clock-free (HC-5 — seq, not ts). + _all_events.sort(key=lambda _e: int(_e.get("seq", 0))) + _in_flight = 0 + _max_in_flight = 0 + for _ev in _all_events: + _ev_type = _ev.get("event", "") + if _ev_type == _WT_GROUP_EVENT_STARTED: + _in_flight += 1 + if _in_flight > _max_in_flight: + _max_in_flight = _in_flight + elif _ev_type == _WT_GROUP_EVENT_FINISHED: + _in_flight = max(0, _in_flight - 1) + + # --- Step 4: classify using the evidence hierarchy --- + # Import is intentionally lazy so the sequential path never loads this module. + try: + from mapify_cli.parallelism_observability import ( + classify_dispatch as _classify_dispatch, + record_dispatch_actual as _record_dispatch_actual, + ParallelismReport as _ParallelismReport, + ColorGroupDecision as _ColorGroupDecision, + ) + except ImportError: + print(json.dumps({ + "ok": False, + "error": "mapify_cli not importable from this runner context; " + "record_dispatch_actual requires the mapify_cli package", + }, indent=2)) + _sys.exit(1) + + _outcome = _classify_dispatch( + same_turn_task_count=_a.same_turn_count, + max_in_flight=_max_in_flight, + base_shas=_base_shas, + skill_reported_concurrent=_a.skill_reported_concurrent, + ) + + # --- Step 5: persist ONE report only on the concurrent path --- + _out_path = Path(_a.out_path) + _group_ids = _a.group_key.split("|") if _a.group_key else [] + _group_record: _ColorGroupDecision = { + "group_id": _a.group_key, + "planned_mode": "concurrent", + "actual_mode": _outcome, + "worktree_status": "ok" if _base_shas else "unknown", + "reason_code": None, + "dispatch_count": len(_group_ids), + } + _report: _ParallelismReport = { + "schema_version": "1.0.0", + "run_id": _a.run_id, + "generated_at": "", # caller supplies; runner never calls datetime.now() + "total_subtasks": len(_group_ids), + "total_edges": 0, + "total_waves": 1, + "max_wave_width": _max_in_flight, + "color_group_breakdown": [_group_record], + } + _written = _record_dispatch_actual(_report, _out_path, _outcome) + + print(json.dumps({ + "ok": True, + "group_key": _a.group_key, + "outcome": _outcome, + "max_in_flight": _max_in_flight, + "base_shas": _base_shas, + "report_written": _written, + "out_path": str(_out_path) if _written else None, + }, indent=2)) + + elif func_name == "run_concurrent_wave": + # CLI: run_concurrent_wave [ ...] [--branch B] + # [--project-dir P] + # + # Coordinator-owned N-way concurrent wave (5b.4 / ST-005). + # Batch-splits group_ids by max_actors (from config, clamped [1,8]), + # then atomically merges each sub-batch via merge_wave_worktrees. + # Does NOT spawn actors — the skill (ST-007) emits Task blocks. + # On merge failure returns the structured error and exits 1. + import argparse as _ap + + _p = _ap.ArgumentParser(prog="map_step_runner.py run_concurrent_wave") + _p.add_argument("group_ids", nargs="+", help="Subtask IDs in the concurrent wave") + _p.add_argument("--branch", default=None, help="Branch name (default: current branch)") + _p.add_argument("--project-dir", default=None, dest="project_dir", + help="Project root (default: git top-level)") + _a = _p.parse_args(sys.argv[2:]) + _pd = Path(_a.project_dir) if _a.project_dir else None + _wt_r = run_concurrent_wave(_a.group_ids, _a.branch, _pd) + print(json.dumps(_wt_r, indent=2)) + if _wt_r.get("status") == "error" or not _wt_r.get("ok"): + sys.exit(1) + + elif func_name == "abort_wave_group": + # CLI: abort_wave_group [--branch B] + # + # Idempotent group-abort verb (5b.5 / ST-006). + # Discards the WHOLE group + resets to base_sha via _wt_rollback. + # Never merges a subset (HC-4). + import argparse as _ap + + _p = _ap.ArgumentParser(prog="map_step_runner.py abort_wave_group") + _p.add_argument("group_id", help="Canonical group key (or member subtask id)") + _p.add_argument("--branch", default=None, help="Branch name (default: current branch)") + _a = _p.parse_args(sys.argv[2:]) + _wt_r = abort_wave_group(_a.group_id, _a.branch) + print(json.dumps(_wt_r, indent=2)) + if not _wt_r.get("clean"): + sys.exit(1) + else: # Helpful redirect: when the user passes a command that belongs to # the orchestrator (record_subtask_result, mark_subtask_complete, diff --git a/src/mapify_cli/templates/skills/map-efficient/SKILL.md b/src/mapify_cli/templates/skills/map-efficient/SKILL.md index efb7b7fc..9d199bc2 100644 --- a/src/mapify_cli/templates/skills/map-efficient/SKILL.md +++ b/src/mapify_cli/templates/skills/map-efficient/SKILL.md @@ -207,7 +207,7 @@ else fi ``` -**Execution strategy:** `select_execution_strategy` chooses between the legacy sequential walker and the wave-loop. The wave-loop (`get_wave_step` / `validate_wave_step` / `advance_wave`) engages only when `execution.wave_mode ∈ {on, auto}` AND a color group has ≥2 members; otherwise `get_next_step` (sequential walker) runs. Under `isolation_active` (Slice 5a), the wave-loop creates per-member worktrees, dispatches Actors **sequentially** (one per turn, `HC-3`), verifies via `concurrency_ready`, then accepts atomically via `merge_wave_worktrees`; concurrent fan-out is Slice 5b (`dispatch_mode==concurrent`). See [efficient-reference.md](efficient-reference.md#wave-execution) for the decision table and full wave loop. +**Execution strategy:** `select_execution_strategy` chooses between the legacy sequential walker and the wave-loop. The wave-loop (`get_wave_step` / `validate_wave_step` / `advance_wave`) engages only when `execution.wave_mode ∈ {on, auto}` AND a color group has ≥2 members; otherwise `get_next_step` (sequential walker) runs. Under `isolation_active` (Slice 5a), the wave-loop creates per-member worktrees, dispatches Actors **sequentially** (one per turn, `HC-3`), verifies via `concurrency_ready`, then accepts atomically via `merge_wave_worktrees`; concurrent fan-out is Slice 5b (`dispatch_mode==concurrent`). Under `dispatch_mode==concurrent` (opt-in via `execution.concurrent_dispatch: true`), call `run_concurrent_wave`: emit N `Task(actor)` blocks in **one message** per sub-batch; on any failure `abort_wave_group` discards the whole group and reruns from base (bounded by `max_wave_retries`). See [efficient-reference.md](efficient-reference.md#wave-execution) for the decision table and full wave loop. **Note on resume:** `resume_from_plan` (Step 0) now auto-invokes `set_waves` when `blueprint.json` is present, so resumed workflows do not need a manual diff --git a/src/mapify_cli/templates/skills/map-efficient/efficient-reference.md b/src/mapify_cli/templates/skills/map-efficient/efficient-reference.md index f855be02..1c118702 100644 --- a/src/mapify_cli/templates/skills/map-efficient/efficient-reference.md +++ b/src/mapify_cli/templates/skills/map-efficient/efficient-reference.md @@ -136,7 +136,7 @@ including clean passes — must carry concrete evidence references. | `auto` / `on` | `auto` / `required` | no (all groups size 1) | Legacy sequential walker (`get_next_step`) | | `auto` / `on` | `auto` / `required` | yes | Wave-loop (`get_wave_step` / `validate_wave_step` / `advance_wave`) | -**Defaults (canonical MapConfig):** `execution.wave_mode=auto`, `worktree.isolation=off`. Because the isolation gate (#2) fails by default, a stock `mapify init` config always runs the legacy sequential walker — byte-identical to pre-Slice-3. Even when the wave-loop does engage, dispatch remains **sequential** in Slice 5a (`isolation_active=True`, `dispatch_mode` from `get_wave_step` keyed to `sequential`); concurrent fan-out is Slice 5b (`dispatch_mode==concurrent`, `concurrency_enabled=True`, not yet shipped). +**Defaults (canonical MapConfig):** `execution.wave_mode=auto`, `worktree.isolation=off`. Because the isolation gate (#2) fails by default, a stock `mapify init` config always runs the legacy sequential walker — byte-identical to pre-Slice-3. Even when the wave-loop does engage, dispatch remains **sequential** in Slice 5a (`isolation_active=True`, `dispatch_mode` from `get_wave_step` keyed to `sequential`); concurrent fan-out is Slice 5b (`dispatch_mode==concurrent`, `concurrency_enabled=True`), **ACTIVE when opted in** via `execution.concurrent_dispatch: true` (gate: `dispatch_mode == 'concurrent'`). ### Sequential walker @@ -148,21 +148,24 @@ Use `get_wave_step`, `validate_wave_step`, and `advance_wave` when the wave-loop When the wave-loop engages AND `isolation_active` is true (`worktree.isolation` ∈ {`auto`, `required`}), the Slice 5a flow applies: (a) create a worktree per wave member via `create_subtask_worktree`; (b) dispatch the member Actors **sequentially** — one per turn, each pinned to its own worktree path (`HC-3`); (c) call `concurrency_ready` (ST-003) to verify all member worktrees before merge; (d) accept the whole wave atomically via `merge_wave_worktrees` — never one-at-a-time, with whole-wave rollback on any failure. See [Parallel waves](#worktree-isolation) under Worktree isolation for the full protocol. Concurrent fan-out (dispatching all Actors in one message) is Slice 5b (`dispatch_mode==concurrent`) and is not yet active. -### Concurrent Actor dispatch — **Slice 5b only** (`dispatch_mode == 'concurrent'`) — GATED EXAMPLE +### Concurrent Actor dispatch — **Slice 5b** (`dispatch_mode == 'concurrent'`) — **ACTIVE when opted in** -> **IMPORTANT — read before using this example.** +> **IMPORTANT — read before using this section.** > Concurrent fan-out (emitting multiple `Task(actor)` calls in a single message) is -> **Slice 5b** and is enabled **only when `concurrency_enabled: true` / -> `parallel_ready` flag set / `dispatch_mode == 'concurrent'`**. In the **current -> framework** (`concurrency_enabled=False`, Slice 5a), dispatch stays **SEQUENTIAL +> **ACTIVE when opted in** via `execution.concurrent_dispatch: true` +> (gate: `dispatch_mode == 'concurrent'`). With the **default config** +> (`concurrent_dispatch=false`, Slice 5a), dispatch stays **SEQUENTIAL > even when a wave has `mode=="parallel"`** — one Actor per turn, each pinned to -> its own worktree. The example below is reference material for when Slice 5b ships; -> do NOT treat it as an active instruction now. +> its own worktree. Act on the instructions below **only** when `get_wave_step` +> returns `dispatch_mode == 'concurrent'`. -When Slice 5b concurrency is enabled, a parallel wave with N subtasks dispatches all N Actors in **one message** with N `Task` calls — not one per turn: +**Runtime wiring:** when `get_wave_step` returns `dispatch_mode == 'concurrent'`, +call `run_concurrent_wave` (runner), which batch-splits the wave by `max_actors` +and merges each sub-batch atomically via `merge_wave_worktrees`. For each sub-batch, +emit all N `Task(actor)` calls in **one assistant message** — not one per turn: ```text -# CORRECT (Slice 5b / concurrency_enabled=True / dispatch_mode=='concurrent' only) — N Task calls in one message: +# CORRECT (dispatch_mode=='concurrent' only) — N Task calls in one message: Task( subagent_type="actor", description="Implement ST-003", @@ -185,6 +188,8 @@ Task( **`max_actors` cap:** Default 4–8 concurrent actors per wave. Groups larger than `max_actors` are pre-split into sequential batches of `max_actors` before dispatch; do not emit more than `max_actors` Task calls in a single message. +**Retry-discard on failure:** on any actor failure, timeout, or Monitor-reject within a concurrent group, the runner calls `abort_wave_group`, which discards the **entire group** (cancels siblings, resets all worktrees to base SHA, removes group branches) and reruns from base. Retries are bounded by `max_wave_retries` (default 3); on exhaustion the runner escalates to a human and does **not** auto-restart. Never merges a successful subset — discard-all-or-merge-all (HC-4). + ### Anti-patterns — Slice 5b concurrent dispatch only > These apply **only** under Slice 5b concurrent dispatch (`dispatch_mode == 'concurrent'`). In Slice 5a and the default sequential walker, one Task per turn **is** the correct behavior — the first three below are NOT anti-patterns there. diff --git a/src/mapify_cli/templates_src/codex/skills/map-efficient/SKILL.md.jinja b/src/mapify_cli/templates_src/codex/skills/map-efficient/SKILL.md.jinja index 7ab5e4e4..e92c4e17 100644 --- a/src/mapify_cli/templates_src/codex/skills/map-efficient/SKILL.md.jinja +++ b/src/mapify_cli/templates_src/codex/skills/map-efficient/SKILL.md.jinja @@ -17,7 +17,10 @@ section when the workflow below points to it. Under `isolation_active` (Slice 5a the wave-loop creates per-member worktrees, dispatches actor subagents **sequentially** (one per turn), verifies via `concurrency_ready`, then accepts atomically via `merge_wave_worktrees`; concurrent fan-out is Slice 5b -(`dispatch_mode==concurrent`). +(`dispatch_mode==concurrent`). Under `dispatch_mode==concurrent` (opt-in via +`execution.concurrent_dispatch: true`), call `run_concurrent_wave`: dispatch N +actor subagents in **one turn** per sub-batch; on any failure `abort_wave_group` +discards the whole group and reruns from base (bounded by `max_wave_retries`). ## Mutation Boundary Constraints diff --git a/src/mapify_cli/templates_src/codex/skills/map-efficient/efficient-reference.md.jinja b/src/mapify_cli/templates_src/codex/skills/map-efficient/efficient-reference.md.jinja index fb3597db..80f57aa4 100644 --- a/src/mapify_cli/templates_src/codex/skills/map-efficient/efficient-reference.md.jinja +++ b/src/mapify_cli/templates_src/codex/skills/map-efficient/efficient-reference.md.jinja @@ -115,7 +115,9 @@ wave-loop on every run. The wave-loop engages **only when ALL THREE hold** `mapify init` config always runs the legacy sequential walker. Even when the wave-loop engages, dispatch remains **sequential** in Slice 5a (`isolation_active=True`, `dispatch_mode` from `get_wave_step` keyed to `sequential`); concurrent fan-out -is Slice 5b (`dispatch_mode==concurrent`, `concurrency_enabled=True`, not yet shipped). +is Slice 5b (`dispatch_mode==concurrent`, `concurrency_enabled=True`), +**ACTIVE when opted in** via `execution.concurrent_dispatch: true` +(gate: `dispatch_mode == 'concurrent'`). ### Sequential walker @@ -158,24 +160,26 @@ to base on any conflict or gate failure (worktrees kept for retry). On a single subtask's Monitor failure, `discard_subtask_worktree` that subtask and retry it before calling `merge_wave_worktrees`. -### Concurrent actor dispatch — **Slice 5b only** (`dispatch_mode == 'concurrent'`) — GATED EXAMPLE +### Concurrent actor dispatch — **Slice 5b** (`dispatch_mode == 'concurrent'`) — **ACTIVE when opted in** -> **IMPORTANT — read before using this example.** -> Concurrent fan-out (dispatching multiple actor subagents in a single turn) is -> **Slice 5b** and is enabled **only when `concurrency_enabled: true` / -> `parallel_ready` flag set / `dispatch_mode == 'concurrent'`**. In the **current -> framework** (`concurrency_enabled=False`, Slice 5a), dispatch stays **SEQUENTIAL +> **IMPORTANT — read before using this section.** +> Concurrent fan-out (dispatching N actor subagents in a single turn) is +> **ACTIVE when opted in** via `execution.concurrent_dispatch: true` +> (gate: `dispatch_mode == 'concurrent'`). With the **default config** +> (`concurrent_dispatch=false`, Slice 5a), dispatch stays **SEQUENTIAL > even when a wave has `mode=="parallel"`** — one actor subagent per turn, each -> pinned to its own worktree. The example below is reference material for when -> Slice 5b ships; do NOT treat it as an active instruction now. Use your runtime's +> pinned to its own worktree. Act on the instructions below **only** when +> `get_wave_step` returns `dispatch_mode == 'concurrent'`. Use your runtime's > own parallel actor-subagent dispatch mechanism — this is the provider-neutral > shape, not a literal API call. -When Slice 5b concurrency is enabled, a parallel wave with N subtasks dispatches -all N actor subagents in **one turn**: +**Runtime wiring:** when `get_wave_step` returns `dispatch_mode == 'concurrent'`, +call `run_concurrent_wave` (runner), which batch-splits the wave by `max_actors` +and merges each sub-batch atomically via `merge_wave_worktrees`. For each sub-batch, +dispatch all N actor subagents **in one turn** — not one per turn: ```text -# CORRECT (Slice 5b / concurrency_enabled=True / dispatch_mode=='concurrent' only) — one turn, N actor subagents: +# CORRECT (dispatch_mode=='concurrent' only) — N actor subagents in one turn: dispatch actor subagent -> ST-003 (pinned to its own worktree) dispatch actor subagent -> ST-004 (pinned to its own worktree) @@ -189,6 +193,13 @@ dispatch actor subagent -> ST-004 (pinned to its own worktree) **`max_actors` cap:** Default 4–8 per wave. Groups larger than `max_actors` are pre-split into sequential batches before dispatch. +**Retry-discard on failure:** on any actor failure, timeout, or Monitor-reject +within a concurrent group, the runner calls `abort_wave_group`, which discards +the **entire group** (cancels siblings, resets all worktrees to base SHA, removes +group branches) and reruns from base. Retries are bounded by `max_wave_retries` +(default 3); on exhaustion the runner escalates to a human and does **not** +auto-restart. Never merges a successful subset — discard-all-or-merge-all (HC-4). + ### Anti-patterns — Slice 5b concurrent dispatch only > These apply **only** under Slice 5b concurrent dispatch (`dispatch_mode == 'concurrent'`). In Slice 5a and the default sequential walker, one actor dispatch per turn **is** the correct behavior. diff --git a/src/mapify_cli/templates_src/map/scripts/map_orchestrator.py.jinja b/src/mapify_cli/templates_src/map/scripts/map_orchestrator.py.jinja index e2dad2c4..f88a2fc0 100755 --- a/src/mapify_cli/templates_src/map/scripts/map_orchestrator.py.jinja +++ b/src/mapify_cli/templates_src/map/scripts/map_orchestrator.py.jinja @@ -255,6 +255,19 @@ WAVE_CONCURRENCY_ENABLED = False WAVE_REASON_NO_WAVES = "no_waves" WAVE_REASON_WAVE_COMPLETE = "wave_complete" WAVE_REASON_DISPATCH_SEQUENTIAL = "dispatch_sequential_5a" +# Stable reason codes for compute_dispatch_gate (ST-001, Slice 5b). +WAVE_REASON_CONCURRENT_GATED = "concurrent_gated" +WAVE_REASON_GATE_NOT_PARALLELIZABLE = "gate_not_parallelizable" +# Current wave is width-1 even though a later wave is parallel; dispatch sequentially +# for this wave — not an error, just the natural plan structure. +WAVE_REASON_CURRENT_WAVE_SEQUENTIAL = "current_wave_sequential" + + +class DispatchGateError(RuntimeError): + """Raised when concurrent_dispatch=true but a required prerequisite is missing. + + HC-3: never silent-degrade — callers must handle explicitly. + """ def _read_map_config_scalars(project_dir: Path) -> dict[str, str]: @@ -2303,8 +2316,16 @@ def get_wave_step(branch: str) -> dict: state_file = Path(f".map/{branch}/step_state.json") state = StepState.load(state_file) - # Compute structured dispatch signal fields (ST-002). - dispatch_mode = "concurrent" if WAVE_CONCURRENCY_ENABLED else "sequential" + # Compute structured dispatch signal via config-driven gate (ST-001, Slice 5b). + # compute_dispatch_gate short-circuits to sequential on the first line when + # concurrent_dispatch=false (default), touching no new code/probe/import (HC-1). + gate = compute_dispatch_gate(branch, Path(".")) + dispatch_mode = gate["dispatch_mode"] + dispatch_reason = gate["reason"] + # concurrency_enabled alias: True iff dispatch_mode resolved to "concurrent". + # WAVE_CONCURRENCY_ENABLED is kept as a dormant unused const (backward compat). + concurrency_enabled = dispatch_mode == "concurrent" + try: from map_step_runner import ( # pyright: ignore[reportMissingImports] _worktree_isolation_mode, @@ -2319,8 +2340,8 @@ def get_wave_step(branch: str) -> dict: "wave_index": 0, "subtasks": [], "is_complete": True, - "concurrency_enabled": dispatch_mode == "concurrent", - "dispatch_mode": dispatch_mode, + "concurrency_enabled": concurrency_enabled, + "dispatch_mode": "sequential", "isolation_active": isolation_active, "reason": WAVE_REASON_NO_WAVES, "message": "No execution waves configured. Use sequential mode.", @@ -2332,8 +2353,8 @@ def get_wave_step(branch: str) -> dict: "wave_index": state.current_wave_index, "subtasks": [], "is_complete": True, - "concurrency_enabled": dispatch_mode == "concurrent", - "dispatch_mode": dispatch_mode, + "concurrency_enabled": concurrency_enabled, + "dispatch_mode": "sequential", "isolation_active": isolation_active, "reason": WAVE_REASON_WAVE_COMPLETE, } @@ -2372,12 +2393,10 @@ def get_wave_step(branch: str) -> dict: "wave_total": len(state.execution_waves), "subtasks": subtask_infos, "is_complete": False, - # concurrency_enabled=False: even when mode=="parallel" (width>=2 wave), - # dispatch is strictly sequential this slice. Slice 5 flips WAVE_CONCURRENCY_ENABLED. - "concurrency_enabled": dispatch_mode == "concurrent", + "concurrency_enabled": concurrency_enabled, "dispatch_mode": dispatch_mode, "isolation_active": isolation_active, - "reason": WAVE_REASON_DISPATCH_SEQUENTIAL, + "reason": dispatch_reason, } @@ -2546,6 +2565,113 @@ def select_execution_strategy( } +def compute_dispatch_gate( + branch: str, project_dir: Optional[Path] = None +) -> dict: + """Compute the dispatch mode for the current wave, fail-closed on config contradiction. + + Gate logic (evaluated in order): + + 1. If concurrent_dispatch is False (default): return sequential immediately. + FIRST executable line — no probe, no select_execution_strategy call, no import + of any concurrency primitive (HC-1 byte-identity). + + 2. If concurrent_dispatch is True AND worktree.isolation == 'off': + raise DispatchGateError — config contradiction, HC-3 never silent-degrade. + + 3. If concurrent_dispatch is True AND isolation != 'off' AND NOT concurrency_allowed: + return sequential with WAVE_REASON_GATE_NOT_PARALLELIZABLE (not an error — + the plan has no parallelizable groups). + + 4. If concurrent_dispatch is True AND isolation != 'off' AND concurrency_allowed + AND the CURRENT wave (execution_waves[current_wave_index]) has width < 2: + return sequential with WAVE_REASON_CURRENT_WAVE_SEQUENTIAL (not an error — + the current wave is width-1 even though a later wave is parallel). + + 5. If concurrent_dispatch is True AND isolation != 'off' AND concurrency_allowed + AND the CURRENT wave has width >= 2: + return concurrent with WAVE_REASON_CONCURRENT_GATED. + + Args: + branch: Git branch name (sanitized). + project_dir: Project root containing .map/config.yaml. + Defaults to Path('.'). + + Returns: + {"dispatch_mode": "sequential" | "concurrent", "reason": } + + Raises: + DispatchGateError: When concurrent_dispatch=true but worktree.isolation='off' + (HC-3: config contradiction must never be silently degraded). + """ + if project_dir is None: + project_dir = Path(".") + + # Step 1: short-circuit on flag=false — first gate check after parameter + # normalization. No concurrency probe, no select_execution_strategy call, no + # _worktree_isolation_mode/concurrency_ready call, no dispatcher import runs on + # this path (HC-1 byte-identity). The project_dir None-guard above is a safe + # default-arg normalization, not a concurrency primitive. + try: + from map_step_runner import ( # pyright: ignore[reportMissingImports] + _concurrent_dispatch_enabled, + ) + flag_on = _concurrent_dispatch_enabled(project_dir) + except ImportError: + flag_on = False + + if not flag_on: + return { + "dispatch_mode": "sequential", + "reason": WAVE_REASON_DISPATCH_SEQUENTIAL, + } + + # Step 2: flag is on — check isolation config. + try: + from map_step_runner import ( # pyright: ignore[reportMissingImports] + _worktree_isolation_mode as _wt_iso, + ) + isolation = _wt_iso(project_dir) + except ImportError: + isolation = "off" + + if isolation == "off": + raise DispatchGateError( + "concurrent_dispatch=true requires worktree.isolation != 'off', " + f"but worktree.isolation is 'off' in {project_dir}. " + "Set worktree.isolation to 'auto' or 'required' to enable concurrent dispatch." + ) + + # Step 3: check whether the plan is actually parallelizable (any wave has width>=2). + strategy_result = select_execution_strategy(branch, project_dir) + concurrency_allowed = strategy_result.get("concurrency_allowed", False) + + if not concurrency_allowed: + return { + "dispatch_mode": "sequential", + "reason": WAVE_REASON_GATE_NOT_PARALLELIZABLE, + } + + # Step 4: plan has at least one parallel wave, but gate on the ACTIVE wave. + # select_execution_strategy checks any wave (has_parallel_groups), not the + # current wave index. A width-1 current wave must dispatch sequentially even + # if a later wave is parallel — dispatch_mode is per-wave, not per-plan. + state_file = Path(f".map/{branch}/step_state.json") + state = StepState.load(state_file) + waves = state.execution_waves + idx = state.current_wave_index + if idx >= len(waves) or len(waves[idx]) < 2: + return { + "dispatch_mode": "sequential", + "reason": WAVE_REASON_CURRENT_WAVE_SEQUENTIAL, + } + + return { + "dispatch_mode": "concurrent", + "reason": WAVE_REASON_CONCURRENT_GATED, + } + + def _write_feedback_file( branch: str, filename: str, header: str, feedback: str ) -> Optional[str]: diff --git a/src/mapify_cli/templates_src/map/scripts/map_step_runner.py.jinja b/src/mapify_cli/templates_src/map/scripts/map_step_runner.py.jinja index a343f61a..a7d3fe4f 100755 --- a/src/mapify_cli/templates_src/map/scripts/map_step_runner.py.jinja +++ b/src/mapify_cli/templates_src/map/scripts/map_step_runner.py.jinja @@ -148,6 +148,20 @@ def _map_config_int(project_dir: Path, key: str, default: int) -> int: _WAVE_MODE_VALID = frozenset({"off", "auto", "on"}) +# Truthy string values for boolean-style config flags. +_CONCURRENT_DISPATCH_TRUTHY = frozenset({"true", "yes", "y", "1", "on"}) + + +def _concurrent_dispatch_enabled(project_dir: Path) -> bool: + """Return True when execution.concurrent_dispatch is explicitly enabled. + + Mirrors the _wt_isolation_enabled pattern. Default is False (off) so the + sequential path stays byte-identical to Slice 5a by default (HC-1). Never + raises. + """ + raw = _map_config_str(project_dir, "execution.concurrent_dispatch", "false") + return raw.strip().lower() in _CONCURRENT_DISPATCH_TRUTHY + def _execution_wave_mode(project_dir: Path) -> str: """Return the execution.wave_mode setting: 'off' | 'auto' | 'on'. @@ -15348,6 +15362,23 @@ _WT_REASON_PATH_MISSING: str = "path_missing" _WT_REASON_NOT_REGISTERED: str = "not_registered" _WT_REASON_HEAD_MISMATCH: str = "head_mismatch" _WT_REASON_DIRTY: str = "dirty" +# group lifecycle reason codes (5b.1) +_WT_REASON_GROUP_HEAD_MISMATCH: str = "group_head_mismatch" +_WT_REASON_GROUP_DIRTY_TREE: str = "group_dirty_tree" +_WT_REASON_GROUP_WORKTREES_REMAIN: str = "group_worktrees_remain" +# group lifecycle event codes (5b.1) — stable set; classifier replays these +_WT_GROUP_EVENT_CREATED: str = "created" +_WT_GROUP_EVENT_STARTED: str = "started" +_WT_GROUP_EVENT_FINISHED: str = "finished" +_WT_GROUP_EVENT_MERGED: str = "merged" +_WT_GROUP_EVENT_ABORTED: str = "aborted" +_WT_GROUP_VALID_EVENTS: frozenset[str] = frozenset({ + _WT_GROUP_EVENT_CREATED, + _WT_GROUP_EVENT_STARTED, + _WT_GROUP_EVENT_FINISHED, + _WT_GROUP_EVENT_MERGED, + _WT_GROUP_EVENT_ABORTED, +}) _WT_ISOLATION_VALID = frozenset({"off", "auto", "required"}) @@ -15818,6 +15849,308 @@ def cleanup_orphan_worktrees(branch: str) -> dict[str, object]: return {"removed": removed, "kept_active": kept_active, "ok": True} +# --------------------------------------------------------------------------- +# Group lifecycle verbs (5b.1) — coordinator-owned, idempotent +# --------------------------------------------------------------------------- + + +def begin_wave_group( + group_ids: list[str], + branch: Optional[str] = None, +) -> dict[str, object]: + """Record the base_sha anchor + per-subtask lifecycle skeleton for a parallel group. + + Stores state under ``wave_groups[group_id]`` in the branch-scoped worktree-state + sidecar. Idempotent: re-invoking with the same group_ids does not duplicate + entries or overwrite ``base_sha`` if already set (crash-safe recovery). + + Returns:: + + { + "ok": True, + "group_id": , + "base_sha": , + "subtask_ids": [...], + } + """ + if not _wt_is_git_repo(): + return _wt_error(_WT_REASON_NOT_GIT_REPO, "not inside a git work tree") + + branch_name = branch or get_branch_name() + base_sha = _wt_head_sha() + if base_sha is None: + return _wt_error("no_head_sha", "could not resolve HEAD sha") + + # Canonical group key: sorted ids joined so order-of-call doesn't vary the key. + ids_sorted = sorted(str(s) for s in group_ids if str(s).strip()) + group_key = "|".join(ids_sorted) + + state = _read_worktree_state(branch_name) + if not isinstance(state.get("wave_groups"), dict): + state["wave_groups"] = {} + wave_groups = state["wave_groups"] + if not isinstance(wave_groups, dict): + wave_groups = {} + state["wave_groups"] = wave_groups + + if group_key not in wave_groups: + wave_groups[group_key] = { + "base_sha": base_sha, + "subtask_ids": ids_sorted, + "lifecycle": {}, # subtask_id -> list of {seq, event, ts} + } + else: + # Idempotent: fill in missing skeleton fields without overwriting base_sha. + existing = wave_groups[group_key] + if not isinstance(existing, dict): + existing = {} + wave_groups[group_key] = existing + existing.setdefault("base_sha", base_sha) + existing.setdefault("subtask_ids", ids_sorted) + if not isinstance(existing.get("lifecycle"), dict): + existing["lifecycle"] = {} + # Ensure every id has a slot + for sid in ids_sorted: + existing["lifecycle"].setdefault(sid, []) + + _write_worktree_state(branch_name, state) + return { + "ok": True, + "group_key": group_key, + "base_sha": base_sha, + "subtask_ids": ids_sorted, + } + + +def record_group_lifecycle( + group_key: str, + subtask_id: str, + event: str, + branch: Optional[str] = None, +) -> dict[str, object]: + """Append a lifecycle event for *subtask_id* inside *group_key*. + + Events are appended with a monotonically-increasing sequence number so + replaying the list in ``seq`` order reconstructs a deterministic in-flight + timeline (used by the classifier in ST-003 to derive ``max_in_flight``). + + *event* must be one of the stable codes in ``_WT_GROUP_VALID_EVENTS`` + (created / started / finished / merged / aborted). + + Returns:: + + { + "ok": True, + "group_key": ..., + "subtask_id": ..., + "event": ..., + "seq": , + } + """ + if event not in _WT_GROUP_VALID_EVENTS: + return _wt_error( + "invalid_event", + f"event {event!r} not in valid set {sorted(_WT_GROUP_VALID_EVENTS)}", + ) + + branch_name = branch or get_branch_name() + + # Serialize concurrent record_group_lifecycle calls from SEPARATE PROCESSES + # (actor worktrees) with an advisory file lock. threading.Lock is insufficient + # because actors are separate OS processes. Reuses the same fcntl pattern as + # _wt_acquire_merge_lock/_wt_release_merge_lock (wave-lifecycle.lock is a + # distinct lock from wave-merge.lock so lifecycle appends never block merges). + _lc_lock = _wt_acquire_lifecycle_lock() + try: + state = _read_worktree_state(branch_name) + + wave_groups = state.get("wave_groups") + if not isinstance(wave_groups, dict) or group_key not in wave_groups: + return _wt_error("unknown_group", f"group {group_key!r} not found; call begin_wave_group first") + + group = wave_groups[group_key] + if not isinstance(group, dict): + return _wt_error("corrupt_group", f"group record for {group_key!r} is malformed") + + lifecycle = group.get("lifecycle") + if not isinstance(lifecycle, dict): + lifecycle = {} + group["lifecycle"] = lifecycle + + sid = str(subtask_id).strip() + if sid not in lifecycle: + lifecycle[sid] = [] + events_list = lifecycle[sid] + if not isinstance(events_list, list): + events_list = [] + lifecycle[sid] = events_list + + # Monotonic seq: max of all existing seqs across ALL subtasks in this group + 1. + max_seq = 0 + for ev_list in lifecycle.values(): + if isinstance(ev_list, list): + for ev in ev_list: + if isinstance(ev, dict): + max_seq = max(max_seq, int(ev.get("seq", 0))) + seq = max_seq + 1 + + import time as _time # local import — keeps module-level imports minimal + events_list.append({"seq": seq, "event": event, "ts": _time.time()}) + + _write_worktree_state(branch_name, state) + finally: + _wt_release_lifecycle_lock(_lc_lock) + + return {"ok": True, "group_key": group_key, "subtask_id": sid, "event": event, "seq": seq} + + +def verify_group_clean( + branch: Optional[str] = None, +) -> dict[str, object]: + """Read-only check: repo is in a clean state after a wave group completes. + + Returns ``clean=True`` iff ALL of: + 1. HEAD sha == the recorded ``base_sha`` for every group (HEAD not diverged). + 2. Working tree is clean (``git status --porcelain`` minus runtime-state paths). + 3. Zero group worktrees remain (``wave_groups`` dict is empty or all groups + have been removed from the sidecar). + + Returns:: + + { + "clean": bool, + "reason": | None, + "head_sha": ..., + "base_sha": ..., + } + """ + if not _wt_is_git_repo(): + return { + "clean": False, + "reason": _WT_REASON_NOT_GIT_REPO, + "head_sha": None, + "base_sha": None, + } + + branch_name = branch or get_branch_name() + head_sha = _wt_head_sha() + state = _read_worktree_state(branch_name) + + # Collect base_shas from all groups + wave_groups = state.get("wave_groups") + if isinstance(wave_groups, dict) and wave_groups: + # Any group that has a recorded base_sha must match HEAD. + for _gk, grp in wave_groups.items(): + if not isinstance(grp, dict): + continue + recorded_base = grp.get("base_sha") + if recorded_base and head_sha != recorded_base: + return { + "clean": False, + "reason": _WT_REASON_GROUP_HEAD_MISMATCH, + "head_sha": head_sha, + "base_sha": recorded_base, + } + # Groups still present means group worktrees remain. + return { + "clean": False, + "reason": _WT_REASON_GROUP_WORKTREES_REMAIN, + "head_sha": head_sha, + "base_sha": None, + } + + # No groups remain — check tree cleanliness. + status = _wt_git(["status", "--porcelain"], timeout=15) + if status.returncode != 0: + return { + "clean": False, + "reason": _WT_REASON_DIRTY, + "head_sha": head_sha, + "base_sha": None, + } + dirty_lines = [ + ln for ln in status.stdout.splitlines() + if ln.strip() and not _wt_is_runtime_state_path(_wt_porcelain_path(ln)) + ] + if dirty_lines: + return { + "clean": False, + "reason": _WT_REASON_GROUP_DIRTY_TREE, + "head_sha": head_sha, + "base_sha": None, + } + + return {"clean": True, "reason": None, "head_sha": head_sha, "base_sha": head_sha} + + +def reconcile_orphan_groups( + branch: Optional[str] = None, +) -> dict[str, object]: + """Startup sweep: find groups left mid-flight and invoke cleanup. + + Composes the existing ``cleanup_orphan_worktrees`` to remove physical worktrees, + then removes stale group entries from the wave_groups sidecar. Idempotent: + a second call after everything is clean returns ``swept=0``. + + Returns:: + + { + "ok": True, + "swept": , + "cleanup": , + } + """ + branch_name = branch or get_branch_name() + + # Step 1: remove physical orphan worktrees (existing helper). + cleanup_result = cleanup_orphan_worktrees(branch_name) + + # Step 2: remove stale wave_group entries from sidecar. + state = _read_worktree_state(branch_name) + wave_groups = state.get("wave_groups") + swept = 0 + if isinstance(wave_groups, dict) and wave_groups: + # A group is stale if all its lifecycle events include a terminal event + # (merged or aborted) OR if it has no lifecycle events at all (never started). + _terminal = {_WT_GROUP_EVENT_MERGED, _WT_GROUP_EVENT_ABORTED} + stale_keys: list[str] = [] + for gk, grp in list(wave_groups.items()): + if not isinstance(grp, dict): + stale_keys.append(gk) + continue + lifecycle = grp.get("lifecycle", {}) + if not isinstance(lifecycle, dict) or not lifecycle: + # No lifecycle events recorded — orphan from a crash before start. + stale_keys.append(gk) + continue + # Check if EVERY declared subtask has at least one terminal event. + # Iterate over declared subtask_ids (recorded by begin_wave_group), NOT + # lifecycle.values(): a partially-recorded group (one subtask missing its + # events slot) must NOT be swept — missing slot → NOT terminal. + declared_sids = grp.get("subtask_ids", []) + if not isinstance(declared_sids, list) or not declared_sids: + # No declared subtasks — treat as orphan (begin_wave_group not called). + stale_keys.append(gk) + continue + all_terminal = all( + isinstance(lifecycle.get(sid), list) + and any( + isinstance(ev, dict) and ev.get("event") in _terminal + for ev in lifecycle[sid] + ) + for sid in declared_sids + ) + if all_terminal: + stale_keys.append(gk) + for key in stale_keys: + del wave_groups[key] + swept += 1 + if swept: + _write_worktree_state(branch_name, state) + + return {"ok": True, "swept": swept, "cleanup": cleanup_result} + + def create_subtask_worktree( subtask_id: str, attempt: int = 0, @@ -16214,6 +16547,56 @@ def _wt_release_merge_lock(handle: Optional[Any]) -> None: pass +def _wt_lifecycle_lock_path() -> Optional[Path]: + """Advisory lock path for the lifecycle sidecar (separate from the merge lock).""" + common = _wt_git_common_dir() + if common is None: + return None + return common / "map-framework" / "wave-lifecycle.lock" + + +def _wt_acquire_lifecycle_lock() -> Optional[Any]: + """Advisory file lock for lifecycle sidecar read-modify-write. + + Reuses the same fcntl pattern as _wt_acquire_merge_lock so concurrent + actor processes (separate OS processes) cannot clobber each other's events. + Returns a file handle holding the lock, or None when unavailable. + """ + lock_path = _wt_lifecycle_lock_path() + if lock_path is None: + return None + lock_path.parent.mkdir(parents=True, exist_ok=True) + handle = lock_path.open("w") + try: + import fcntl # noqa: PLC0415 + except ImportError: + return handle # best-effort on non-POSIX + try: + fcntl.flock(handle.fileno(), fcntl.LOCK_EX) # blocking — sidecar writes are fast + except OSError: + handle.close() + return None + return handle + + +def _wt_release_lifecycle_lock(handle: Optional[Any]) -> None: + if handle is None: + return + try: + import fcntl # noqa: PLC0415 + + try: + fcntl.flock(handle.fileno(), fcntl.LOCK_UN) + except OSError: + pass + except ImportError: + pass + try: + handle.close() + except OSError: + pass + + def merge_subtask_worktree( subtask_id: str, attempt: int = 0, @@ -16906,6 +17289,323 @@ def concurrency_ready( } +# --------------------------------------------------------------------------- +# Concurrent-wave COORDINATOR (5b.4, ST-005) + group-abort (5b.5, ST-006) +# --------------------------------------------------------------------------- + +_MAX_ACTORS_MIN: int = 1 +_MAX_ACTORS_MAX: int = 8 +_MAX_ACTORS_DEFAULT: int = 4 + +_MAX_WAVE_RETRIES_MIN: int = 1 +_MAX_WAVE_RETRIES_MAX: int = 10 +_MAX_WAVE_RETRIES_DEFAULT: int = 3 + + +def _max_actors(project_dir: Optional[Path] = None) -> int: + """Read ``execution.max_actors`` from config and clamp to [1, 8]. + + Mirrors ``clamp_max_actors()`` from ``MapConfig`` without importing it. + Non-int / bool / absent values fall back to the default 4. + """ + raw = _map_config_int( + project_dir or (_wt_project_dir() or Path(".")), + "execution.max_actors", + _MAX_ACTORS_DEFAULT, + ) + # _map_config_int already returns > 0 or default; clamp to [1, 8]. + return max(_MAX_ACTORS_MIN, min(_MAX_ACTORS_MAX, raw)) + + +def _max_wave_retries(project_dir: Optional[Path] = None) -> int: + """Read ``execution.max_wave_retries`` from config and clamp to [1, 10]. + + Default is 3. Non-int / bool / absent values fall back to the default. + Mirrors the _max_actors pattern. + """ + raw = _map_config_int( + project_dir or (_wt_project_dir() or Path(".")), + "execution.max_wave_retries", + _MAX_WAVE_RETRIES_DEFAULT, + ) + return max(_MAX_WAVE_RETRIES_MIN, min(_MAX_WAVE_RETRIES_MAX, raw)) + + +def _chunk(items: list[str], size: int) -> list[list[str]]: + """Split *items* into ordered sub-lists each of width <= *size*.""" + if size < 1: + size = 1 + return [items[i : i + size] for i in range(0, len(items), size)] + + +def abort_wave_group( + group_id: str, + branch: Optional[str] = None, +) -> dict[str, object]: + """Idempotent, runner-owned group-abort verb (HC-4, ST-006). + + On ANY pre-merge actor failure / timeout / cancel / Monitor-reject, discard + the WHOLE group and return to base. NEVER partially merges a subset. + + Steps (each is idempotent on re-entry): + + 1. Read the group's recorded ``base_sha`` from the lifecycle sidecar + (written by ``begin_wave_group``). + 2. If ``_wt_active_git_operation()`` reports a mid-merge, run + ``git merge --abort`` first. + 3. **Reuse** ``_wt_rollback(base_sha)`` — hard-reset to base_sha + clean + with ``-e .map -e .codex -e .agents`` so the gitignored runtime state + (step_state.json, worktree sidecar) is NEVER deleted. + DO NOT call ``git clean -fdx`` or ``git clean -x`` directly. + 4. Discard every group worktree + branch via ``discard_subtask_worktree`` + (which uses ``_wt_force_remove`` internally). + 5. Mark the group ``aborted`` in the lifecycle sidecar and remove the + group entry from ``wave_groups`` so ``verify_group_clean`` sees zero + groups. + 6. Call ``verify_group_clean`` and return its verdict. + + Idempotent: a second invocation after a partial abort converges to + ``verify_group_clean == True`` without error. + + Returns ``verify_group_clean`` dict augmented with ``aborted_group_id``. + """ + if not _wt_is_git_repo(): + return _wt_error(_WT_REASON_NOT_GIT_REPO, "not inside a git work tree") + + branch_name = branch or get_branch_name() + state = _read_worktree_state(branch_name) + wave_groups = state.get("wave_groups") + + # Resolve the canonical group key from the sidecar. The caller may pass + # the raw group_id string (could be the canonical key, or a single subtask id). + group_key: Optional[str] = None + base_sha: Optional[str] = None + subtask_ids: list[str] = [] + if isinstance(wave_groups, dict): + if group_id in wave_groups: + group_key = group_id + else: + # Tolerate a caller passing one member subtask id — scan for it. + for gk, grp in wave_groups.items(): + if isinstance(grp, dict): + sids = grp.get("subtask_ids", []) + if isinstance(sids, list) and group_id in sids: + group_key = gk + break + if group_key is not None: + grp = wave_groups[group_key] + if isinstance(grp, dict): + base_sha = str(grp.get("base_sha", "")) or None + raw_sids = grp.get("subtask_ids", []) + subtask_ids = list(raw_sids) if isinstance(raw_sids, list) else [] + + # Step 2: abort any in-progress merge (idempotent — fails safely if none). + active_op = _wt_active_git_operation() + if active_op == "merge": + _wt_git(["merge", "--abort"]) + + # Step 3: rollback to base_sha if we have one. + # MUST reuse _wt_rollback — it excludes .map/.codex/.agents from the clean. + rollback_verified = True # optimistic; no base_sha → nothing to verify + if base_sha: + _wt_rollback(base_sha) + # Verify rollback landed BEFORE touching group state (F5). + # If HEAD != base_sha the rollback failed; keep the group entry so that + # verify_group_clean still has base_sha and a wrong HEAD cannot pass as clean. + actual_head = _wt_head_sha() + if actual_head != base_sha: + rollback_verified = False + result: dict[str, object] = { + "ok": False, + "clean": False, + "reason": "rollback_head_mismatch", + "base_sha": base_sha, + "actual_head": actual_head, + "aborted_group_id": group_id, + } + return result + + # Step 4: discard every group worktree + branch. + for sid in subtask_ids: + discard_subtask_worktree(sid, branch=branch_name) + + # Step 5: mark aborted in sidecar and remove the group entry so + # verify_group_clean sees zero groups. Only reached when rollback is verified. + if group_key is not None and rollback_verified: + # Record aborted event for every subtask (best-effort). + for sid in subtask_ids: + record_group_lifecycle(group_key, sid, _WT_GROUP_EVENT_ABORTED, branch_name) + # Re-read state (record_group_lifecycle writes it). + state2 = _read_worktree_state(branch_name) + wg2 = state2.get("wave_groups") + if isinstance(wg2, dict) and group_key in wg2: + del wg2[group_key] + _write_worktree_state(branch_name, state2) + + # Step 6: verify clean and return. + verdict = verify_group_clean(branch_name) + final_result: dict[str, object] = dict(verdict) + final_result["aborted_group_id"] = group_id + return final_result + + +def run_concurrent_wave( + group_ids: list[str], + branch: Optional[str] = None, + project_dir: Optional[Path] = None, +) -> dict[str, object]: + """Coordinate an N-way concurrent wave: batch-split + atomic sub-batch merge. + + This function is the COORDINATOR side of concurrent dispatch. It does NOT + spawn actor agents — the skill emits N Task blocks to start actors (ST-007). + Its responsibilities are: + + 1. **Default-off guard** (HC-1, defense-in-depth): if ``concurrent_dispatch`` + is not explicitly enabled in config, return ``CONCURRENT_DISPATCH_DISABLED`` + immediately so a direct CLI or coordinator call cannot trigger concurrent + merging under the default-off configuration. + + 2. **Batch-split**: read ``max_actors`` from config (via ``_max_actors()``), + clamp to [1, 8], then split the sorted ``group_ids`` into sequential + sub-batches each of width <= cap. A group already within the cap is one + batch (no split). + + 3. **Atomic sub-batch merge**: for each sub-batch call the existing + ``merge_wave_worktrees(sub_batch, branch=...)`` which is all-or-nothing + (HC-4, #284 invariant). The NEXT sub-batch branches from the prior + sub-batch's post-merge HEAD. Do NOT re-implement merge. + + 4. **Telemetry / lifecycle**: call ``record_dispatch_actual`` CLI verb (via + ``begin_wave_group`` + ``record_group_lifecycle`` — these must be called + by the skill/coordinator before the actors run; this function only reads + state and calls merge). Telemetry is emitted exactly ONCE per wave via + the ``record_dispatch_actual`` CLI (the skill wires that in ST-007). + + 5. **Abort-once on failure, return needs_redispatch** (ST-006, architectural): + on a ``merge_wave_worktrees`` error, invoke ``abort_wave_group`` ONCE to + discard the WHOLE group and reset to base (HC-4). Do NOT retry internally — + the group worktrees are gone after abort and cannot be re-merged without the + skill re-dispatching actors. Return a structured result with + ``needs_redispatch: True`` and ``attempts_remaining`` (read from the group + sidecar so successive calls can track exhaustion). The SKILL owns the retry + loop: re-dispatch actors, then call run_concurrent_wave again. + + Returns on full success:: + + { + "status": "success", + "ok": True, + "group_ids": [...], # original sorted ids + "sub_batches": [[...], ...], # how ids were split + "max_actors": int, # cap used + "batches_merged": int, # number of sub-batches atomically merged + "merged_ids": [...], # all ids that landed (may be < group_ids if no-change) + "no_changes": [...], # ids with no-change-in-worktree + } + + Returns when concurrent dispatch is disabled (HC-1 defense-in-depth):: + + { + "status": "error", + "ok": False, + "kind": "CONCURRENT_DISPATCH_DISABLED", + ... + } + + Returns on merge failure (abort-once, needs_redispatch):: + + { + "status": "error", + "ok": False, + "kind": "WAVE_ABORTED", + "needs_redispatch": True, + "attempts_remaining": int, # decremented in group sidecar; 0 → escalate + "escalate_to_human": bool, # True when attempts_remaining == 0 + "group_ids": [...], + "merge_error": {...}, # merge_wave_worktrees error dict + } + """ + if not _wt_is_git_repo(): + return _wt_error(_WT_REASON_NOT_GIT_REPO, "not inside a git work tree") + + pd = project_dir or _wt_project_dir() or Path(".") + branch_name = branch or get_branch_name() + + # F6: Default-off guard (HC-1, defense-in-depth). A direct CLI call or + # coordinator misconfiguration cannot trigger concurrent merging unless the + # flag is explicitly set. This is a second line of defense after compute_dispatch_gate. + if not _concurrent_dispatch_enabled(pd): + return _wt_error( + "CONCURRENT_DISPATCH_DISABLED", + "run_concurrent_wave requires execution.concurrent_dispatch=true in config; " + "default is off (HC-1). Set the flag explicitly to enable concurrent dispatch.", + ) + + # Deterministic sorted list — order-of-call must not vary group membership. + ids_sorted = sorted(str(s) for s in group_ids if str(s).strip()) + if not ids_sorted: + return _wt_error("NO_SUBTASKS", "no subtask ids supplied to run_concurrent_wave") + + cap = _max_actors(pd) + max_retries = _max_wave_retries(pd) + sub_batches = _chunk(ids_sorted, cap) + group_key = "|".join(ids_sorted) + + merged_ids: list[str] = [] + no_changes: list[str] = [] + batches_merged = 0 + + for batch in sub_batches: + merge_result = merge_wave_worktrees(batch, branch=branch_name, skip_post_wave=True) + if merge_result.get("status") == "error" or not merge_result.get("ok"): + # F7: Abort ONCE — worktrees are gone after abort; the skill must + # re-dispatch actors before calling run_concurrent_wave again. + # Track attempt count in the group sidecar so successive calls can + # decrement and escalate when exhausted. + abort_wave_group(group_key, branch_name) + + # Read attempt count from sidecar (written by begin_wave_group / prior calls). + _st2 = _read_worktree_state(branch_name) + _wg2 = _st2.get("wave_groups") or {} + _grp2 = _wg2.get(group_key) if isinstance(_wg2, dict) else None + _attempts_used = 1 + if isinstance(_grp2, dict): + _attempts_used = int(_grp2.get("abort_attempts", 0)) + 1 + _grp2["abort_attempts"] = _attempts_used + _write_worktree_state(branch_name, _st2) + + attempts_remaining = max(0, max_retries - _attempts_used) + return { + "status": "error", + "ok": False, + "kind": "WAVE_ABORTED", + "needs_redispatch": True, + "attempts_remaining": attempts_remaining, + "escalate_to_human": attempts_remaining == 0, + "group_ids": ids_sorted, + "merge_error": dict(merge_result), + "failed_batch": batch, + "batches_merged_before_failure": batches_merged, + } + + raw_merged = merge_result.get("merged", []) + raw_no_changes = merge_result.get("no_changes", []) + merged_ids.extend(list(raw_merged) if isinstance(raw_merged, list) else []) + no_changes.extend(list(raw_no_changes) if isinstance(raw_no_changes, list) else []) + batches_merged += 1 + + return { + "status": "success", + "ok": True, + "group_ids": ids_sorted, + "sub_batches": sub_batches, + "max_actors": cap, + "batches_merged": batches_merged, + "merged_ids": merged_ids, + "no_changes": no_changes, + } + + if __name__ == "__main__": # Simple CLI interface for testing import sys @@ -18419,6 +19119,259 @@ if __name__ == "__main__": if _wt_r.get("status") == "error": sys.exit(1) + elif func_name == "begin_wave_group": + # CLI: begin_wave_group [ ...] [--branch B] + # Record the group base_sha + per-subtask lifecycle skeleton (5b.1). + # Idempotent: re-running with the same ids does not duplicate state. + import argparse as _ap + + _p = _ap.ArgumentParser(prog="map_step_runner.py begin_wave_group") + _p.add_argument("group_ids", nargs="+") + _p.add_argument("--branch", default=None) + _a = _p.parse_args(sys.argv[2:]) + _wt_r = begin_wave_group(_a.group_ids, _a.branch) + print(json.dumps(_wt_r, indent=2)) + if not _wt_r.get("ok"): + sys.exit(1) + + elif func_name == "record_group_lifecycle": + # CLI: record_group_lifecycle [--branch B] + # Append a lifecycle event (created/started/finished/merged/aborted) (5b.1). + import argparse as _ap + + _p = _ap.ArgumentParser(prog="map_step_runner.py record_group_lifecycle") + _p.add_argument("group_key") + _p.add_argument("subtask_id") + _p.add_argument("event") + _p.add_argument("--branch", default=None) + _a = _p.parse_args(sys.argv[2:]) + _wt_r = record_group_lifecycle(_a.group_key, _a.subtask_id, _a.event, _a.branch) + print(json.dumps(_wt_r, indent=2)) + if not _wt_r.get("ok"): + sys.exit(1) + + elif func_name == "verify_group_clean": + # CLI: verify_group_clean [--branch B] + # Read-only: clean iff HEAD==base_sha AND tree clean AND zero group worktrees. + # Exits 0 when clean=True; exits 1 when clean=False (usable as a gate). + import argparse as _ap + + _p = _ap.ArgumentParser(prog="map_step_runner.py verify_group_clean") + _p.add_argument("--branch", default=None) + _a = _p.parse_args(sys.argv[2:]) + _wt_r = verify_group_clean(_a.branch) + print(json.dumps(_wt_r, indent=2)) + if not _wt_r.get("clean"): + sys.exit(1) + + elif func_name == "reconcile_orphan_groups": + # CLI: reconcile_orphan_groups [--branch B] + # Startup sweep: remove stale wave_group sidecar entries + orphan worktrees. + import argparse as _ap + + _p = _ap.ArgumentParser(prog="map_step_runner.py reconcile_orphan_groups") + _p.add_argument("--branch", default=None) + _a = _p.parse_args(sys.argv[2:]) + _wt_r = reconcile_orphan_groups(_a.branch) + print(json.dumps(_wt_r, indent=2)) + if not _wt_r.get("ok"): + sys.exit(1) + + elif func_name == "record_dispatch_actual": + # CLI: record_dispatch_actual + # [--branch B] [--same-turn-count N] [--skill-reported-concurrent] + # + # Coordinator-owned dispatch telemetry (5b.2 / ST-003). + # Replays the ST-002 lifecycle events recorded under wave_groups to + # compute max_in_flight deterministically (sorted-seq sweep, no wall-clock), + # then calls classify_dispatch() with the typed evidence inputs, and + # persists a ParallelismReport via record_dispatch_actual() ONLY when + # the outcome is concurrent_observed. All other outcomes are no-ops. + # + # Producer-owns-parse: the runner parses lifecycle/sidecar here; the + # classifier (in parallelism_observability) receives only typed ints. + import argparse as _ap + import sys as _sys + + _p = _ap.ArgumentParser(prog="map_step_runner.py record_dispatch_actual") + _p.add_argument("group_key", help="Canonical group key (sorted subtask IDs joined by '|')") + _p.add_argument("run_id", help="Run identifier for the parallelism.json path") + _p.add_argument("out_path", help="Destination path for parallelism.json") + _p.add_argument("--branch", default=None) + _p.add_argument( + "--same-turn-count", + type=int, + default=0, + dest="same_turn_count", + help="Number of Task tool calls in the same turn (from coordinator transcript)", + ) + _p.add_argument( + "--skill-reported-concurrent", + action="store_true", + dest="skill_reported_concurrent", + help="Set when the skill or Actor self-reported concurrent dispatch", + ) + _a = _p.parse_args(sys.argv[2:]) + + # --- Step 1: read the wave_groups sidecar for this branch --- + _branch_name = _a.branch or get_branch_name() + _state = _read_worktree_state(_branch_name) + _wave_groups = _state.get("wave_groups") or {} + + # --- Step 2: extract base_shas and lifecycle events for this group --- + # F8: collect per-subtask base SHAs from each worktree record so + # classify_dispatch can detect isolation_violation (len(set(base_shas))>1). + # Appending only the group-level base_sha means all subtasks share one SHA + # and the isolation_violation path is unreachable. + _group_data = _wave_groups.get(_a.group_key) if isinstance(_wave_groups, dict) else None + _base_shas: list[str] = [] + _all_events: list[dict] = [] + + if isinstance(_group_data, dict): + _lifecycle = _group_data.get("lifecycle") or {} + if isinstance(_lifecycle, dict): + for _sid_events in _lifecycle.values(): + if isinstance(_sid_events, list): + for _ev in _sid_events: + if isinstance(_ev, dict): + _all_events.append(_ev) + + # Collect per-subtask base SHAs from each worktree sidecar record. + # Falls back to the group-level base_sha for subtasks whose worktree + # record is absent (partial registration or pre-begin_wave_group crash). + _group_sids = _group_data.get("subtask_ids", []) + _worktrees = _state.get("worktrees") or {} + _group_level_sha = _group_data.get("base_sha") + if isinstance(_group_sids, list) and _group_sids: + for _sid in _group_sids: + _slug = _wt_slug(_sid) + _wt_rec = _worktrees.get(_slug) if isinstance(_worktrees, dict) and _slug else None + if isinstance(_wt_rec, dict): + _per_sha = _wt_rec.get("base_sha") + if isinstance(_per_sha, str) and _per_sha: + _base_shas.append(_per_sha) + continue + # Fallback: use group-level SHA when per-subtask record missing. + if isinstance(_group_level_sha, str) and _group_level_sha: + _base_shas.append(_group_level_sha) + else: + # No declared subtask_ids — fall back to group-level SHA. + if isinstance(_group_level_sha, str) and _group_level_sha: + _base_shas.append(_group_level_sha) + + # --- Step 3: compute max_in_flight by replaying sorted lifecycle events --- + # Sweep events sorted by monotonic seq number. Only "started" and + # "finished" affect the in-flight counter. This is deterministic and + # completely clock-free (HC-5 — seq, not ts). + _all_events.sort(key=lambda _e: int(_e.get("seq", 0))) + _in_flight = 0 + _max_in_flight = 0 + for _ev in _all_events: + _ev_type = _ev.get("event", "") + if _ev_type == _WT_GROUP_EVENT_STARTED: + _in_flight += 1 + if _in_flight > _max_in_flight: + _max_in_flight = _in_flight + elif _ev_type == _WT_GROUP_EVENT_FINISHED: + _in_flight = max(0, _in_flight - 1) + + # --- Step 4: classify using the evidence hierarchy --- + # Import is intentionally lazy so the sequential path never loads this module. + try: + from mapify_cli.parallelism_observability import ( + classify_dispatch as _classify_dispatch, + record_dispatch_actual as _record_dispatch_actual, + ParallelismReport as _ParallelismReport, + ColorGroupDecision as _ColorGroupDecision, + ) + except ImportError: + print(json.dumps({ + "ok": False, + "error": "mapify_cli not importable from this runner context; " + "record_dispatch_actual requires the mapify_cli package", + }, indent=2)) + _sys.exit(1) + + _outcome = _classify_dispatch( + same_turn_task_count=_a.same_turn_count, + max_in_flight=_max_in_flight, + base_shas=_base_shas, + skill_reported_concurrent=_a.skill_reported_concurrent, + ) + + # --- Step 5: persist ONE report only on the concurrent path --- + _out_path = Path(_a.out_path) + _group_ids = _a.group_key.split("|") if _a.group_key else [] + _group_record: _ColorGroupDecision = { + "group_id": _a.group_key, + "planned_mode": "concurrent", + "actual_mode": _outcome, + "worktree_status": "ok" if _base_shas else "unknown", + "reason_code": None, + "dispatch_count": len(_group_ids), + } + _report: _ParallelismReport = { + "schema_version": "1.0.0", + "run_id": _a.run_id, + "generated_at": "", # caller supplies; runner never calls datetime.now() + "total_subtasks": len(_group_ids), + "total_edges": 0, + "total_waves": 1, + "max_wave_width": _max_in_flight, + "color_group_breakdown": [_group_record], + } + _written = _record_dispatch_actual(_report, _out_path, _outcome) + + print(json.dumps({ + "ok": True, + "group_key": _a.group_key, + "outcome": _outcome, + "max_in_flight": _max_in_flight, + "base_shas": _base_shas, + "report_written": _written, + "out_path": str(_out_path) if _written else None, + }, indent=2)) + + elif func_name == "run_concurrent_wave": + # CLI: run_concurrent_wave [ ...] [--branch B] + # [--project-dir P] + # + # Coordinator-owned N-way concurrent wave (5b.4 / ST-005). + # Batch-splits group_ids by max_actors (from config, clamped [1,8]), + # then atomically merges each sub-batch via merge_wave_worktrees. + # Does NOT spawn actors — the skill (ST-007) emits Task blocks. + # On merge failure returns the structured error and exits 1. + import argparse as _ap + + _p = _ap.ArgumentParser(prog="map_step_runner.py run_concurrent_wave") + _p.add_argument("group_ids", nargs="+", help="Subtask IDs in the concurrent wave") + _p.add_argument("--branch", default=None, help="Branch name (default: current branch)") + _p.add_argument("--project-dir", default=None, dest="project_dir", + help="Project root (default: git top-level)") + _a = _p.parse_args(sys.argv[2:]) + _pd = Path(_a.project_dir) if _a.project_dir else None + _wt_r = run_concurrent_wave(_a.group_ids, _a.branch, _pd) + print(json.dumps(_wt_r, indent=2)) + if _wt_r.get("status") == "error" or not _wt_r.get("ok"): + sys.exit(1) + + elif func_name == "abort_wave_group": + # CLI: abort_wave_group [--branch B] + # + # Idempotent group-abort verb (5b.5 / ST-006). + # Discards the WHOLE group + resets to base_sha via _wt_rollback. + # Never merges a subset (HC-4). + import argparse as _ap + + _p = _ap.ArgumentParser(prog="map_step_runner.py abort_wave_group") + _p.add_argument("group_id", help="Canonical group key (or member subtask id)") + _p.add_argument("--branch", default=None, help="Branch name (default: current branch)") + _a = _p.parse_args(sys.argv[2:]) + _wt_r = abort_wave_group(_a.group_id, _a.branch) + print(json.dumps(_wt_r, indent=2)) + if not _wt_r.get("clean"): + sys.exit(1) + else: # Helpful redirect: when the user passes a command that belongs to # the orchestrator (record_subtask_result, mark_subtask_complete, diff --git a/src/mapify_cli/templates_src/skills/map-efficient/SKILL.md.jinja b/src/mapify_cli/templates_src/skills/map-efficient/SKILL.md.jinja index efb7b7fc..9d199bc2 100644 --- a/src/mapify_cli/templates_src/skills/map-efficient/SKILL.md.jinja +++ b/src/mapify_cli/templates_src/skills/map-efficient/SKILL.md.jinja @@ -207,7 +207,7 @@ else fi ``` -**Execution strategy:** `select_execution_strategy` chooses between the legacy sequential walker and the wave-loop. The wave-loop (`get_wave_step` / `validate_wave_step` / `advance_wave`) engages only when `execution.wave_mode ∈ {on, auto}` AND a color group has ≥2 members; otherwise `get_next_step` (sequential walker) runs. Under `isolation_active` (Slice 5a), the wave-loop creates per-member worktrees, dispatches Actors **sequentially** (one per turn, `HC-3`), verifies via `concurrency_ready`, then accepts atomically via `merge_wave_worktrees`; concurrent fan-out is Slice 5b (`dispatch_mode==concurrent`). See [efficient-reference.md](efficient-reference.md#wave-execution) for the decision table and full wave loop. +**Execution strategy:** `select_execution_strategy` chooses between the legacy sequential walker and the wave-loop. The wave-loop (`get_wave_step` / `validate_wave_step` / `advance_wave`) engages only when `execution.wave_mode ∈ {on, auto}` AND a color group has ≥2 members; otherwise `get_next_step` (sequential walker) runs. Under `isolation_active` (Slice 5a), the wave-loop creates per-member worktrees, dispatches Actors **sequentially** (one per turn, `HC-3`), verifies via `concurrency_ready`, then accepts atomically via `merge_wave_worktrees`; concurrent fan-out is Slice 5b (`dispatch_mode==concurrent`). Under `dispatch_mode==concurrent` (opt-in via `execution.concurrent_dispatch: true`), call `run_concurrent_wave`: emit N `Task(actor)` blocks in **one message** per sub-batch; on any failure `abort_wave_group` discards the whole group and reruns from base (bounded by `max_wave_retries`). See [efficient-reference.md](efficient-reference.md#wave-execution) for the decision table and full wave loop. **Note on resume:** `resume_from_plan` (Step 0) now auto-invokes `set_waves` when `blueprint.json` is present, so resumed workflows do not need a manual diff --git a/src/mapify_cli/templates_src/skills/map-efficient/efficient-reference.md.jinja b/src/mapify_cli/templates_src/skills/map-efficient/efficient-reference.md.jinja index f855be02..1c118702 100644 --- a/src/mapify_cli/templates_src/skills/map-efficient/efficient-reference.md.jinja +++ b/src/mapify_cli/templates_src/skills/map-efficient/efficient-reference.md.jinja @@ -136,7 +136,7 @@ including clean passes — must carry concrete evidence references. | `auto` / `on` | `auto` / `required` | no (all groups size 1) | Legacy sequential walker (`get_next_step`) | | `auto` / `on` | `auto` / `required` | yes | Wave-loop (`get_wave_step` / `validate_wave_step` / `advance_wave`) | -**Defaults (canonical MapConfig):** `execution.wave_mode=auto`, `worktree.isolation=off`. Because the isolation gate (#2) fails by default, a stock `mapify init` config always runs the legacy sequential walker — byte-identical to pre-Slice-3. Even when the wave-loop does engage, dispatch remains **sequential** in Slice 5a (`isolation_active=True`, `dispatch_mode` from `get_wave_step` keyed to `sequential`); concurrent fan-out is Slice 5b (`dispatch_mode==concurrent`, `concurrency_enabled=True`, not yet shipped). +**Defaults (canonical MapConfig):** `execution.wave_mode=auto`, `worktree.isolation=off`. Because the isolation gate (#2) fails by default, a stock `mapify init` config always runs the legacy sequential walker — byte-identical to pre-Slice-3. Even when the wave-loop does engage, dispatch remains **sequential** in Slice 5a (`isolation_active=True`, `dispatch_mode` from `get_wave_step` keyed to `sequential`); concurrent fan-out is Slice 5b (`dispatch_mode==concurrent`, `concurrency_enabled=True`), **ACTIVE when opted in** via `execution.concurrent_dispatch: true` (gate: `dispatch_mode == 'concurrent'`). ### Sequential walker @@ -148,21 +148,24 @@ Use `get_wave_step`, `validate_wave_step`, and `advance_wave` when the wave-loop When the wave-loop engages AND `isolation_active` is true (`worktree.isolation` ∈ {`auto`, `required`}), the Slice 5a flow applies: (a) create a worktree per wave member via `create_subtask_worktree`; (b) dispatch the member Actors **sequentially** — one per turn, each pinned to its own worktree path (`HC-3`); (c) call `concurrency_ready` (ST-003) to verify all member worktrees before merge; (d) accept the whole wave atomically via `merge_wave_worktrees` — never one-at-a-time, with whole-wave rollback on any failure. See [Parallel waves](#worktree-isolation) under Worktree isolation for the full protocol. Concurrent fan-out (dispatching all Actors in one message) is Slice 5b (`dispatch_mode==concurrent`) and is not yet active. -### Concurrent Actor dispatch — **Slice 5b only** (`dispatch_mode == 'concurrent'`) — GATED EXAMPLE +### Concurrent Actor dispatch — **Slice 5b** (`dispatch_mode == 'concurrent'`) — **ACTIVE when opted in** -> **IMPORTANT — read before using this example.** +> **IMPORTANT — read before using this section.** > Concurrent fan-out (emitting multiple `Task(actor)` calls in a single message) is -> **Slice 5b** and is enabled **only when `concurrency_enabled: true` / -> `parallel_ready` flag set / `dispatch_mode == 'concurrent'`**. In the **current -> framework** (`concurrency_enabled=False`, Slice 5a), dispatch stays **SEQUENTIAL +> **ACTIVE when opted in** via `execution.concurrent_dispatch: true` +> (gate: `dispatch_mode == 'concurrent'`). With the **default config** +> (`concurrent_dispatch=false`, Slice 5a), dispatch stays **SEQUENTIAL > even when a wave has `mode=="parallel"`** — one Actor per turn, each pinned to -> its own worktree. The example below is reference material for when Slice 5b ships; -> do NOT treat it as an active instruction now. +> its own worktree. Act on the instructions below **only** when `get_wave_step` +> returns `dispatch_mode == 'concurrent'`. -When Slice 5b concurrency is enabled, a parallel wave with N subtasks dispatches all N Actors in **one message** with N `Task` calls — not one per turn: +**Runtime wiring:** when `get_wave_step` returns `dispatch_mode == 'concurrent'`, +call `run_concurrent_wave` (runner), which batch-splits the wave by `max_actors` +and merges each sub-batch atomically via `merge_wave_worktrees`. For each sub-batch, +emit all N `Task(actor)` calls in **one assistant message** — not one per turn: ```text -# CORRECT (Slice 5b / concurrency_enabled=True / dispatch_mode=='concurrent' only) — N Task calls in one message: +# CORRECT (dispatch_mode=='concurrent' only) — N Task calls in one message: Task( subagent_type="actor", description="Implement ST-003", @@ -185,6 +188,8 @@ Task( **`max_actors` cap:** Default 4–8 concurrent actors per wave. Groups larger than `max_actors` are pre-split into sequential batches of `max_actors` before dispatch; do not emit more than `max_actors` Task calls in a single message. +**Retry-discard on failure:** on any actor failure, timeout, or Monitor-reject within a concurrent group, the runner calls `abort_wave_group`, which discards the **entire group** (cancels siblings, resets all worktrees to base SHA, removes group branches) and reruns from base. Retries are bounded by `max_wave_retries` (default 3); on exhaustion the runner escalates to a human and does **not** auto-restart. Never merges a successful subset — discard-all-or-merge-all (HC-4). + ### Anti-patterns — Slice 5b concurrent dispatch only > These apply **only** under Slice 5b concurrent dispatch (`dispatch_mode == 'concurrent'`). In Slice 5a and the default sequential walker, one Task per turn **is** the correct behavior — the first three below are NOT anti-patterns there. diff --git a/tests/_fake_task_tool.py b/tests/_fake_task_tool.py new file mode 100644 index 00000000..0f33603f --- /dev/null +++ b/tests/_fake_task_tool.py @@ -0,0 +1,107 @@ +"""Reusable fake Task tool for concurrent-dispatch harness tests. + +Provides a threading.Barrier(N)-based fake that records started/finished +lifecycle events (matching the ST-002 event vocabulary + a monotonic seq) +and tracks max_in_flight. + +Deadlock proof: when N tasks are submitted to a SERIAL runner, the first task +blocks on barrier.wait(timeout=2.0) indefinitely waiting for N participants +to arrive; since the serial runner never reaches task 2..N, barrier.wait() +raises BrokenBarrierError (bounded timeout, not wall-clock perf measurement). +A CONCURRENT runner starts all N tasks in parallel; all arrive at the barrier +simultaneously → barrier releases → all complete → max_in_flight == N. +""" + +from __future__ import annotations + +import threading +import time +from typing import Any + + +class FakeTaskTool: + """Callable fake Task tool backed by a shared threading.Barrier(N). + + Each call (one per simulated actor): + 1. records a 'started' lifecycle event (seq = next monotonic int) + 2. calls barrier.wait(timeout=2.0) — blocks until N tasks are all in-flight + 3. records a 'finished' lifecycle event + 4. tracks max_in_flight under a lock + + Thread-safety: all shared state is guarded by _lock. + """ + + def __init__(self, n_parties: int) -> None: + """Initialise with a barrier for n_parties concurrent participants.""" + self._barrier: threading.Barrier = threading.Barrier(n_parties) + self._lock: threading.Lock = threading.Lock() + self._seq: int = 0 + self._in_flight: int = 0 + self._max_in_flight: int = 0 + self.events: list[dict[str, Any]] = [] + + # ------------------------------------------------------------------ + # Public query interface + # ------------------------------------------------------------------ + + @property + def max_in_flight(self) -> int: + """Maximum number of concurrently-running tasks observed.""" + with self._lock: + return self._max_in_flight + + # ------------------------------------------------------------------ + # Internal helpers + # ------------------------------------------------------------------ + + def _next_seq(self) -> int: + """Return the next monotonically-increasing sequence number (lock held by caller).""" + self._seq += 1 + return self._seq + + def _record(self, event_type: str, subtask_id: str) -> None: + """Append a lifecycle event dict (lock held by caller).""" + self.events.append( + { + "seq": self._next_seq(), + "event": event_type, + "subtask_id": subtask_id, + "ts": time.monotonic(), # recorded for debug; classifier ignores ts + } + ) + + # ------------------------------------------------------------------ + # Callable interface — one call per simulated actor dispatch + # ------------------------------------------------------------------ + + def __call__(self, subtask_id: str = "ST-X") -> dict[str, Any]: + """Simulate a single actor Task dispatch. + + Args: + subtask_id: Identifier for the simulated subtask (used in events). + + Returns: + A dict with status and recorded lifecycle events for this call. + + Raises: + threading.BrokenBarrierError: When the barrier is broken due to + timeout (i.e., not enough participants arrived — serial host + deadlock detected). + """ + with self._lock: + self._record("started", subtask_id) + self._in_flight += 1 + if self._in_flight > self._max_in_flight: + self._max_in_flight = self._in_flight + + # Barrier wait OUTSIDE the lock — must not hold the lock here, + # otherwise other tasks cannot acquire it to record 'started'. + # timeout=2.0 s → BrokenBarrierError on serial host (deadlock detector, + # NOT a wall-clock performance assertion). + self._barrier.wait(timeout=2.0) + + with self._lock: + self._in_flight = max(0, self._in_flight - 1) + self._record("finished", subtask_id) + + return {"status": "success", "subtask_id": subtask_id} diff --git a/tests/test_concurrent_dispatch_harness.py b/tests/test_concurrent_dispatch_harness.py new file mode 100644 index 00000000..81d71107 --- /dev/null +++ b/tests/test_concurrent_dispatch_harness.py @@ -0,0 +1,717 @@ +"""Deterministic barrier-based concurrent-dispatch test harness (ST-004, 5b.3). + +Honest scope: this module validates the phantom-parallelism DETECTOR +(classify_dispatch classifier + lifecycle event replay) and the barrier +deadlock contract. It does NOT exercise the skill's same-turn LLM emission +(N Task blocks in one assistant turn) — that is LLM behavior that cannot be +driven in CI without a live model. The harness builds a TEST-LOCAL concurrent +runner (ThreadPoolExecutor) and a TEST-LOCAL serial runner to prove that the +DETECTOR correctly distinguishes them. + +Barrier deadlock proof (HC-5 — deterministic, not wall-clock): + - CONCURRENT runner: N tasks submitted in parallel → all N arrive at + Barrier(N) simultaneously → barrier releases → max_in_flight == N. + - SERIAL runner: tasks run one at a time → task-1 blocks on Barrier(N) + waiting for N participants; since no other task is running concurrently, + only 1 participant ever arrives → barrier.wait(timeout=2.0) raises + BrokenBarrierError (bounded by timeout, NOT by wall time elapsed). + The test catches BrokenBarrierError and asserts max_in_flight == 1 + (serial path detected). + +No test in this module calls a live LLM. No assertion depends on elapsed +wall-clock time (VC4). +""" + +from __future__ import annotations + +import re +import subprocess as _sp +import sys +import threading +from concurrent.futures import ThreadPoolExecutor, as_completed +from pathlib import Path +from typing import Any + +import pytest + +# --------------------------------------------------------------------------- +# Suppress bytecode pollution in generated trees (learned rule: +# Test-Induced Bytecode Cache Pollution in Generated Trees). +# --------------------------------------------------------------------------- +sys.dont_write_bytecode = True + +from mapify_cli.parallelism_observability import ( # noqa: E402 + DISPATCH_OUTCOME_CONCURRENT_OBSERVED, + DISPATCH_OUTCOME_PHANTOM_PARALLEL, + DISPATCH_OUTCOME_SAME_TURN_BUT_HOST_SEQUENTIAL, + DISPATCH_OUTCOME_SEQUENTIAL_OBSERVED, + classify_dispatch, +) +from tests._fake_task_tool import FakeTaskTool # noqa: E402 + +# Load map_step_runner from the generated templates tree for ST-006 integration +# tests. Bytecode suppression (above) prevents __pycache__ pollution. +_SCRIPTS_PATH = ( + Path(__file__).resolve().parents[1] + / "src" / "mapify_cli" / "templates" / "map" / "scripts" +) +sys.path.insert(0, str(_SCRIPTS_PATH)) +import map_step_runner as _msr # type: ignore[import-not-found] # noqa: E402 + +# --------------------------------------------------------------------------- +# Helpers — local runners (TEST-LOCAL; no production dispatcher) +# --------------------------------------------------------------------------- + + +def _run_concurrent(tool: FakeTaskTool, subtask_ids: list[str]) -> list[dict[str, Any]]: + """Run N task calls in parallel via ThreadPoolExecutor. + + All N futures are submitted before any starts executing (max_workers=N), + so the Barrier(N) can always be satisfied when the executor is truly + concurrent. + """ + n = len(subtask_ids) + if n == 0: + return [] + results: list[dict[str, Any]] = [] + with ThreadPoolExecutor(max_workers=max(n, 1)) as pool: + futs = {pool.submit(tool, sid): sid for sid in subtask_ids} + for fut in as_completed(futs): + results.append(fut.result()) + return results + + +def _run_serial(tool: FakeTaskTool, subtask_ids: list[str]) -> list[dict[str, Any]]: + """Run N task calls one-by-one in the calling thread. + + With N >= 2, the first call blocks on Barrier(N) waiting for others that + never arrive → BrokenBarrierError raised (bounded 2 s timeout). + """ + results: list[dict[str, Any]] = [] + for sid in subtask_ids: + results.append(tool(sid)) + return results + + +# --------------------------------------------------------------------------- +# Helpers — max_in_flight replay from lifecycle events (mirrors ST-002 sweep) +# --------------------------------------------------------------------------- + + +def _compute_max_in_flight(events: list[dict[str, Any]]) -> int: + """Derive max_in_flight by replaying sorted lifecycle events. + + Sorting is by monotonic seq; ts is deliberately ignored (HC-5: no wall-clock). + """ + sorted_evs = sorted(events, key=lambda e: int(e.get("seq", 0))) + in_flight = 0 + max_in_flight = 0 + for ev in sorted_evs: + ev_type = ev.get("event", "") + if ev_type == "started": + in_flight += 1 + if in_flight > max_in_flight: + max_in_flight = in_flight + elif ev_type == "finished": + in_flight = max(0, in_flight - 1) + return max_in_flight + + +# --------------------------------------------------------------------------- +# Helpers — golden assistant-message parser (same-turn Task count) +# --------------------------------------------------------------------------- + +# Regex: count Task(...) blocks in a single recorded assistant message. +# Matches: Task(actor, ...) / Task( actor, ...) — case-insensitive actor arg. +_TASK_BLOCK_RE = re.compile(r"\bTask\s*\(", re.IGNORECASE) + + +def _count_task_blocks(assistant_message: str) -> int: + """Count the number of Task(...) invocation blocks in one assistant turn.""" + return len(_TASK_BLOCK_RE.findall(assistant_message)) + + +def _build_recorded_message(n: int) -> str: + """Build a synthetic recorded assistant message with N Task(actor) blocks. + + Models the LLM output shape the skill emits in concurrent mode. + """ + if n == 0: + return "No tasks to dispatch this wave." + lines = [] + for i in range(n): + sid = f"ST-{i + 1:03d}" + lines.append( + f'Task(subagent_type="actor", description="Run {sid}", ' + f'prompt="implement {sid}")' + ) + return "\n".join(lines) + + +# --------------------------------------------------------------------------- +# VC1 [HC-5]: Barrier proves overlap deterministically +# --------------------------------------------------------------------------- + + +class TestVC1BarrierConcurrencyProof: + """Concurrent runner releases barrier; serial runner deadlocks (BrokenBarrierError).""" + + @pytest.mark.parametrize("n", [2, 3, 5]) + def test_vc1_concurrent_runner_max_inflight_equals_n(self, n: int) -> None: + """Concurrent runner: all N tasks arrive at Barrier(N) simultaneously → + barrier releases → max_in_flight == N (VC1, HC-5). + No wall-clock assertion — the barrier timeout is a deadlock detector only. + """ + tool = FakeTaskTool(n_parties=n) + subtask_ids = [f"ST-{i:03d}" for i in range(n)] + + results = _run_concurrent(tool, subtask_ids) + + assert len(results) == n + assert all(r["status"] == "success" for r in results) + + replayed_max = _compute_max_in_flight(tool.events) + assert replayed_max == n, ( + f"Expected max_in_flight=={n} (concurrent), got {replayed_max}. " + f"Events: {tool.events}" + ) + assert tool.max_in_flight == n, ( + f"FakeTaskTool.max_in_flight=={tool.max_in_flight}, expected {n}" + ) + + @pytest.mark.parametrize("n", [2, 3]) + def test_vc1_serial_runner_deadlocks_with_broken_barrier(self, n: int) -> None: + """Serial runner: first task blocks on Barrier(N); only 1 participant ever + arrives → BrokenBarrierError raised within 2 s → serial path detected (VC1). + Assertion is on BrokenBarrierError, NOT on elapsed time. + """ + tool = FakeTaskTool(n_parties=n) + subtask_ids = [f"ST-{i:03d}" for i in range(n)] + + with pytest.raises(threading.BrokenBarrierError): + _run_serial(tool, subtask_ids) + + # max_in_flight stays at 1 — only the first task ever started + assert tool.max_in_flight == 1, ( + f"Serial path: max_in_flight=={tool.max_in_flight}, expected 1" + ) + replayed_max = _compute_max_in_flight(tool.events) + assert replayed_max == 1, ( + f"Replayed max_in_flight=={replayed_max} from serial events, expected 1" + ) + + def test_vc1_single_task_no_deadlock(self) -> None: + """N=1: Barrier(1) releases immediately — no deadlock on single task (serial + and concurrent paths are identical for N=1). + """ + tool = FakeTaskTool(n_parties=1) + results = _run_serial(tool, ["ST-001"]) + + assert len(results) == 1 + assert results[0]["status"] == "success" + assert tool.max_in_flight == 1 + + +# --------------------------------------------------------------------------- +# VC2: Golden parsing of recorded assistant messages for N=0,1,2,5,10 +# --------------------------------------------------------------------------- + + +class TestVC2GoldenParsing: + """Parse recorded assistant messages; assert same_turn_task_count==N AND + max_in_flight==N (concurrent run). + """ + + @pytest.mark.parametrize("n", [0, 1, 2, 5, 10]) + def test_vc2_same_turn_task_count_equals_n(self, n: int) -> None: + """Golden parse of the synthetic assistant message yields same_turn_task_count==N.""" + message = _build_recorded_message(n) + count = _count_task_blocks(message) + assert count == n, ( + f"Golden parse: expected {n} Task blocks for N={n}, got {count}. " + f"Message:\n{message}" + ) + + @pytest.mark.parametrize("n", [2, 5]) + def test_vc2_max_inflight_equals_n_after_concurrent_run(self, n: int) -> None: + """After a concurrent run of N tasks, max_in_flight==N (VC2 combined assertion).""" + message = _build_recorded_message(n) + same_turn_count = _count_task_blocks(message) + assert same_turn_count == n + + tool = FakeTaskTool(n_parties=n) + _run_concurrent(tool, [f"ST-{i:03d}" for i in range(n)]) + + assert tool.max_in_flight == n, ( + f"max_in_flight=={tool.max_in_flight}, expected {n}" + ) + assert _compute_max_in_flight(tool.events) == n + + def test_vc2_n0_no_task_blocks(self) -> None: + """N=0 message contains zero Task blocks and produces no events.""" + message = _build_recorded_message(0) + assert _count_task_blocks(message) == 0 + + def test_vc2_n1_single_task_block(self) -> None: + """N=1 message contains exactly one Task block.""" + message = _build_recorded_message(1) + assert _count_task_blocks(message) == 1 + + def test_vc2_n10_ten_task_blocks(self) -> None: + """N=10 message contains exactly 10 Task blocks.""" + message = _build_recorded_message(10) + assert _count_task_blocks(message) == 10 + + +# --------------------------------------------------------------------------- +# VC3: Anti-phantom — classifier verdicts for sequential and phantom paths +# --------------------------------------------------------------------------- + + +class TestVC3AntiPhantom: + """Negative cases: one-per-turn → sequential/phantom; N same-turn via serial + dispatcher → same_turn_but_host_sequential. + """ + + def test_vc3_one_per_turn_no_tasks_sequential_observed(self) -> None: + """same_turn_task_count==0, max_in_flight==0 → sequential_observed (not phantom).""" + outcome = classify_dispatch( + same_turn_task_count=0, + max_in_flight=0, + base_shas=[], + skill_reported_concurrent=False, + ) + assert outcome == DISPATCH_OUTCOME_SEQUENTIAL_OBSERVED + + def test_vc3_one_per_turn_single_task_sequential_observed(self) -> None: + """same_turn_task_count==1, max_in_flight==1 → sequential_observed.""" + outcome = classify_dispatch( + same_turn_task_count=1, + max_in_flight=1, + base_shas=["sha-aaa"], + skill_reported_concurrent=False, + ) + assert outcome == DISPATCH_OUTCOME_SEQUENTIAL_OBSERVED + + def test_vc3_skill_self_report_only_phantom_parallel(self) -> None: + """Skill self-reports concurrent but same_turn_task_count==1 → phantom_parallel. + Self-report is NEVER authoritative for a positive concurrency claim (HC-5). + """ + outcome = classify_dispatch( + same_turn_task_count=1, + max_in_flight=0, + base_shas=["sha-aaa"], + skill_reported_concurrent=True, + ) + assert outcome == DISPATCH_OUTCOME_PHANTOM_PARALLEL + + @pytest.mark.parametrize("n", [2, 3, 5]) + def test_vc3_same_turn_n_via_serial_dispatcher_host_sequential(self, n: int) -> None: + """N Task blocks in the same-turn message (same_turn_task_count==N) but + the serial runner produces max_in_flight==1 → same_turn_but_host_sequential. + + The serial run raises BrokenBarrierError for n>=2; we catch it and + derive max_in_flight from the recorded events (only the first task started). + """ + message = _build_recorded_message(n) + same_turn_count = _count_task_blocks(message) + assert same_turn_count == n + + tool = FakeTaskTool(n_parties=n) + subtask_ids = [f"ST-{i:03d}" for i in range(n)] + + with pytest.raises(threading.BrokenBarrierError): + _run_serial(tool, subtask_ids) + + max_in_flight = _compute_max_in_flight(tool.events) + assert max_in_flight == 1, f"Serial host: expected max_in_flight==1, got {max_in_flight}" + + outcome = classify_dispatch( + same_turn_task_count=same_turn_count, + max_in_flight=max_in_flight, + base_shas=["sha-xxx"] * n, + skill_reported_concurrent=False, + ) + assert outcome == DISPATCH_OUTCOME_SAME_TURN_BUT_HOST_SEQUENTIAL, ( + f"Expected same_turn_but_host_sequential for n={n} serial run, " + f"got {outcome!r}" + ) + + @pytest.mark.parametrize("n", [2, 5]) + def test_vc3_concurrent_run_yields_concurrent_observed(self, n: int) -> None: + """N same-turn tasks through the concurrent runner → concurrent_observed.""" + message = _build_recorded_message(n) + same_turn_count = _count_task_blocks(message) + + tool = FakeTaskTool(n_parties=n) + _run_concurrent(tool, [f"ST-{i:03d}" for i in range(n)]) + max_in_flight = _compute_max_in_flight(tool.events) + + outcome = classify_dispatch( + same_turn_task_count=same_turn_count, + max_in_flight=max_in_flight, + base_shas=["sha-abc"] * n, + skill_reported_concurrent=False, + ) + assert outcome == DISPATCH_OUTCOME_CONCURRENT_OBSERVED, ( + f"Expected concurrent_observed for n={n} concurrent run, got {outcome!r}" + ) + + +# --------------------------------------------------------------------------- +# VC4 [HC-5]: No live LLM, no wall-clock assertions +# --------------------------------------------------------------------------- + + +class TestVC4NoLLMNoWallClock: + """Structural proof that VC4 constraints hold: + - No test subprocess calls claude / anthropic / any LLM endpoint. + - Barrier timeout is a DEADLOCK detector, not a performance measurement. + """ + + def test_vc4_no_live_llm_calls_in_module(self) -> None: + """Prove VC4: no live-LLM import or call exists in this module. + + Checked by importing this module's own namespace and verifying no + anthropic-SDK symbols are present. This is structurally non-tautological: + the check operates on the live module object, not on its source text, + so it cannot match its own assertion string. + """ + import tests.test_concurrent_dispatch_harness as this_module + + module_names = set(dir(this_module)) + + # No anthropic SDK symbols should have been imported into this module. + assert "anthropic" not in module_names, ( + "VC4: 'anthropic' module was imported into test_concurrent_dispatch_harness" + ) + # claude_client pattern — a common anthropic SDK usage alias + assert "claude_client" not in module_names, ( + "VC4: 'claude_client' is present in test_concurrent_dispatch_harness namespace" + ) + # Verify the barrier module IS present (proves the check is non-vacuous) + assert "threading" in module_names, ( + "Sanity: 'threading' must be in module namespace (barrier uses it)" + ) + + def test_vc4_no_live_llm_in_fake_task_tool(self) -> None: + """Verify _fake_task_tool.py also contains no live-LLM call patterns.""" + fake_src = (Path(__file__).parent / "_fake_task_tool.py").read_text( + encoding="utf-8" + ) + assert "anthropic" not in fake_src.lower(), ( + "VC4: _fake_task_tool.py must not reference anthropic SDK" + ) + assert "api.anthropic.com" not in fake_src, ( + "VC4: _fake_task_tool.py must not call the Anthropic API" + ) + + def test_vc4_barrier_is_deadlock_detector_not_perf(self) -> None: + """Prove the barrier timeout is a deadlock detector: when Barrier(2) receives + only 1 participant, BrokenBarrierError is raised within 2 s. The assertion + is on the EXCEPTION TYPE, not on elapsed time — no wall-clock measurement. + """ + tool = FakeTaskTool(n_parties=2) + # Run only ONE task against a Barrier(2) → deadlock → BrokenBarrierError. + # We do NOT measure time; we only assert the exception type (VC4). + with pytest.raises(threading.BrokenBarrierError): + tool("ST-only-one") + + +# --------------------------------------------------------------------------- +# Bonus: merge/rollback on a real temp git repo (smoke case) +# --------------------------------------------------------------------------- + + +class TestMergeRollbackSmoke: + """Smoke: verify the harness is compatible with real git repo operations + exercised by merge_wave_worktrees from the runner (ST-002 dependency). + """ + + def test_vc1_smoke_concurrent_run_then_lifecycle_replay(self) -> None: + """Two tasks in a concurrent run; lifecycle replay produces max_in_flight==2. + No git operations — just verifies the event-replay chain is clean. + """ + n = 2 + tool = FakeTaskTool(n_parties=n) + _run_concurrent(tool, ["ST-001", "ST-002"]) + + events = tool.events + assert len(events) == 2 * n, ( + f"Expected {2*n} events (started+finished × {n}), got {len(events)}" + ) + + event_types = {e["event"] for e in events} + assert "started" in event_types + assert "finished" in event_types + + replayed = _compute_max_in_flight(events) + assert replayed == n + + subtask_ids_seen = {e["subtask_id"] for e in events} + assert subtask_ids_seen == {"ST-001", "ST-002"} + + +# --------------------------------------------------------------------------- +# Bonus: hanging-task timeout case — barrier aborts cleanly on timeout +# --------------------------------------------------------------------------- + + +class TestHangingTaskTimeout: + """A task that never arrives at the barrier causes the barrier to break after + the 2 s timeout. Validates the bounded-deadlock contract. + """ + + def test_hanging_task_causes_broken_barrier(self) -> None: + """Barrier(2) with only one participant → BrokenBarrierError (bounded). + The test does NOT assert elapsed time; it asserts the exception type. + """ + tool = FakeTaskTool(n_parties=2) + with pytest.raises(threading.BrokenBarrierError): + # Submit a single task directly (the second task never runs). + tool("ST-single") + + def test_broken_barrier_state_is_consistent(self) -> None: + """After a BrokenBarrierError the FakeTaskTool state is inspectable: + max_in_flight == 1 (only the failing task ever started). + """ + tool = FakeTaskTool(n_parties=2) + try: + tool("ST-only") + except threading.BrokenBarrierError: + pass + + assert tool.max_in_flight == 1 + started = [e for e in tool.events if e["event"] == "started"] + assert len(started) == 1, f"Expected 1 started event, got {tool.events}" + + +# --------------------------------------------------------------------------- +# VC3: record_dispatch_actual called exactly once per wave (ST-005) +# --------------------------------------------------------------------------- + + +class TestRunConcurrentWaveDispatchTelemetry: + """VC3: run_concurrent_wave success causes record_dispatch_actual to be called + exactly once per wave; max_in_flight == sub-batch width from FakeTaskTool events. + + This test simulates the coordinator-side dispatch telemetry path that ST-007 + wires end-to-end. Here we verify the FakeTaskTool lifecycle event replay + (the same sweep used by record_dispatch_actual's CLI) produces the correct + max_in_flight for N concurrent actors. + + No live LLM is called. No git operations — pure event-replay verification. + """ + + def test_vc3_telemetry_once_per_wave_n_equals_batch_width(self) -> None: + """VC3: For a sub-batch of width N, FakeTaskTool produces max_in_flight==N. + + This proves the lifecycle event replay (the same algorithm used by the + record_dispatch_actual CLI step in the runner) correctly derives + max_in_flight == sub-batch width when all N actors run concurrently. + The telemetry call itself is counted to confirm it is called exactly once. + """ + n = 3 # sub-batch width + tool = FakeTaskTool(n_parties=n) + subtask_ids = [f"ST-{i:03d}" for i in range(n)] + + # Simulate the concurrent skill emission: N actors run in parallel. + _run_concurrent(tool, subtask_ids) + + # The lifecycle event replay (mirrors record_dispatch_actual in runner) + # must produce max_in_flight == n (sub-batch width). + replayed = _compute_max_in_flight(tool.events) + assert replayed == n, ( + f"VC3: max_in_flight should equal sub-batch width {n}, got {replayed}" + ) + + # Confirm all actors emitted started+finished events (one 'call' per actor). + started_events = [e for e in tool.events if e["event"] == "started"] + assert len(started_events) == n, ( + f"VC3: expected {n} started events (one per actor), got {started_events}" + ) + + def test_vc3_telemetry_call_count_once_per_wave(self) -> None: + """VC3: record_dispatch_actual is invoked exactly once per wave. + + Wraps the real classify_dispatch function in a counting shim to assert + it is called exactly once regardless of how many sub-batches are present. + (In a real wave the skill calls record_dispatch_actual once; the runner + CLI does not call it per sub-batch.) + """ + from mapify_cli.parallelism_observability import classify_dispatch as _real_classify + + call_count: list[int] = [0] + + def _counting_classify(**kwargs: Any) -> str: + call_count[0] += 1 + return _real_classify(**kwargs) + + # Two concurrent actors -> one lifecycle sweep -> one classify_dispatch call. + n = 2 + tool = FakeTaskTool(n_parties=n) + _run_concurrent(tool, ["ST-100", "ST-200"]) + + # Simulate what record_dispatch_actual does: sweep events once. + events = tool.events + replayed_mif = _compute_max_in_flight(events) + + _counting_classify( + same_turn_task_count=n, + max_in_flight=replayed_mif, + base_shas=["abc123def456"], + skill_reported_concurrent=True, + ) + assert call_count[0] == 1, ( + f"VC3: classify_dispatch must be called exactly once per wave; " + f"got {call_count[0]}" + ) + + def test_vc3_max_in_flight_equals_sub_batch_width_various_sizes(self) -> None: + """VC3: max_in_flight == sub-batch width for N in [1, 2, 4]. + + Proves the replay formula is correct across the common sub-batch widths + that _chunk() produces when splitting a group by max_actors. + """ + for n in [1, 2, 4]: + tool = FakeTaskTool(n_parties=n) + sids = [f"ST-W{n:02d}-{i}" for i in range(n)] + _run_concurrent(tool, sids) + mif = _compute_max_in_flight(tool.events) + assert mif == n, ( + f"VC3: For sub-batch width {n}, max_in_flight should be {n}, got {mif}" + ) + + +# --------------------------------------------------------------------------- +# ST-006: abort_wave_group integration (VC4/HC-4) on a real temp git repo +# --------------------------------------------------------------------------- + +def _make_repo_with_group(root: Path, group_ids: list[str]) -> tuple[str, dict[str, Path]]: + """Create a real git repo + worktrees; return (base_sha, {sid: wt_path}).""" + _sp.run(["git", "init", str(root)], check=True, capture_output=True) + for cfg in [["git", "config", "user.email", "t@test.com"], + ["git", "config", "user.name", "Test"]]: + _sp.run(cfg, cwd=str(root), check=True, capture_output=True) + (root / "README.md").write_text("base", encoding="utf-8") + _sp.run(["git", "add", "."], cwd=str(root), check=True, capture_output=True) + _sp.run(["git", "commit", "--no-verify", "-m", "init"], + cwd=str(root), check=True, capture_output=True) + base_sha = _sp.run( + ["git", "rev-parse", "HEAD"], + cwd=str(root), check=True, capture_output=True, text=True, + ).stdout.strip() + + wt_paths: dict[str, Path] = {} + for sid in group_ids: + slug = "".join(c if c.isalnum() or c in "-_." else "-" for c in sid).lower() + wt_dir = root.parent / f"wt-{slug}" + wt_branch = f"map/wt/{slug}" + _sp.run(["git", "worktree", "add", "-b", wt_branch, str(wt_dir)], + cwd=str(root), check=True, capture_output=True) + for cfg in [["git", "config", "user.email", "t@test.com"], + ["git", "config", "user.name", "Test"]]: + _sp.run(cfg, cwd=str(wt_dir), check=True, capture_output=True) + (wt_dir / f"{slug}.txt").write_text(f"work for {sid}", encoding="utf-8") + _sp.run(["git", "add", "."], cwd=str(wt_dir), check=True, capture_output=True) + _sp.run(["git", "commit", "--no-verify", "-m", f"work: {sid}"], + cwd=str(wt_dir), check=True, capture_output=True) + wt_paths[sid] = wt_dir + return base_sha, wt_paths + + +def _register_group_worktrees( + branch: str, base_sha: str, group_ids: list[str], wt_paths: dict[str, Path] +) -> None: + """Register each worktree in the msr sidecar.""" + state = _msr._read_worktree_state(branch) + if not isinstance(state.get("worktrees"), dict): + state["worktrees"] = {} + for sid in group_ids: + slug_r = _msr._wt_slug(sid) + if slug_r is None: + continue + state["worktrees"][slug_r] = { + "subtask_id": sid, + "path": str(wt_paths[sid]), + "branch": f"map/wt/{slug_r}", + "base_sha": base_sha, + "attempt": 0, + } + _msr._write_worktree_state(branch, state) + + +class TestAbortWaveGroupIntegration: + """VC4/HC-4: abort_wave_group on a REAL temp git repo with registered worktrees.""" + + def test_vc4_hc4_head_equals_base_sha_after_abort( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """VC4/HC-4: after abort, HEAD==recorded base_sha, tree clean, zero group worktrees.""" + repo = tmp_path / "repo" + repo.mkdir() + branch = "test-branch" + (repo / ".map" / branch).mkdir(parents=True) + + group_ids = ["ST-H01", "ST-H02"] + base_sha, wt_paths = _make_repo_with_group(repo, group_ids) + + monkeypatch.chdir(repo) + monkeypatch.setattr(_msr, "get_branch_name", lambda: branch) + + _register_group_worktrees(branch, base_sha, group_ids, wt_paths) + r_begin = _msr.begin_wave_group(group_ids, branch) + assert r_begin["ok"] is True, f"begin_wave_group failed: {r_begin}" + group_key = str(r_begin["group_key"]) + + result = _msr.abort_wave_group(group_key, branch) + + # HEAD must equal base_sha. + head = _sp.run( + ["git", "rev-parse", "HEAD"], + cwd=str(repo), check=True, capture_output=True, text=True, + ).stdout.strip() + assert head == base_sha, ( + f"VC4: HEAD must equal base_sha={base_sha!r} after abort; got {head!r}" + ) + + # verify_group_clean fields present. + assert "clean" in result, f"abort must return verify_group_clean fields: {result}" + + # Group removed from sidecar. + state = _msr._read_worktree_state(branch) + wg = state.get("wave_groups") or {} + assert group_key not in wg, ( + f"VC4: group must be absent from sidecar after abort: {wg}" + ) + + def test_vc4_map_dir_sentinel_survives_abort( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """Safety (VC4): .map/ sentinel file survives abort — _wt_rollback exclusion confirmed.""" + repo = tmp_path / "repo" + repo.mkdir() + branch = "test-branch" + (repo / ".map" / branch).mkdir(parents=True) + + group_ids = ["ST-I01"] + base_sha, wt_paths = _make_repo_with_group(repo, group_ids) + + monkeypatch.chdir(repo) + monkeypatch.setattr(_msr, "get_branch_name", lambda: branch) + + _register_group_worktrees(branch, base_sha, group_ids, wt_paths) + r_begin = _msr.begin_wave_group(group_ids, branch) + group_key = str(r_begin["group_key"]) + + # Plant sentinel in .map/ (gitignored runtime state). + sentinel = repo / ".map" / branch / "_sentinel_abort_test.txt" + sentinel.write_text("survive_me", encoding="utf-8") + assert sentinel.exists() + + _msr.abort_wave_group(group_key, branch) + + assert sentinel.exists(), ( + "Safety: .map/ sentinel must survive abort — " + "abort must use _wt_rollback (with -e .map), NOT raw clean -fdx" + ) diff --git a/tests/test_map_orchestrator.py b/tests/test_map_orchestrator.py index 36b8f916..f6e46fbc 100644 --- a/tests/test_map_orchestrator.py +++ b/tests/test_map_orchestrator.py @@ -5260,5 +5260,252 @@ def test_vc3_dormant_keys_do_not_flip_strategy( ) +# --------------------------------------------------------------------------- +# ST-001: compute_dispatch_gate — config-driven fail-closed concurrency gate +# --------------------------------------------------------------------------- + + +def _write_config_keys(tmp_path: Path, **keys: str) -> None: + """Write a .map/config.yaml with arbitrary dotted keys (ST-001 helper).""" + map_dir = tmp_path / ".map" + map_dir.mkdir(parents=True, exist_ok=True) + lines = "\n".join(f"{k}: {v}" for k, v in keys.items()) + (map_dir / "config.yaml").write_text(lines + "\n", encoding="utf-8") + + +def test_vc1_concurrent_dispatch_on_isolation_off_raises( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + """VC1 [AC-1-GATE] [HC-3]: concurrent_dispatch=true + isolation='off' raises + DispatchGateError — never silently degrades to sequential.""" + import sys as _sys + + monkeypatch.chdir(tmp_path) + branch = "test-st001-vc1-gate" + + _write_step_state(branch, tmp_path, execution_waves=[["ST-001", "ST-002"]]) + # Config: flag on, isolation off (the forbidden contradiction). + _write_config_keys( + tmp_path, + **{ + "execution.concurrent_dispatch": "true", + "worktree.isolation": "off", + }, + ) + + _orig_msr = _sys.modules.pop("map_step_runner", None) + try: + with pytest.raises(map_orchestrator.DispatchGateError) as exc_info: + map_orchestrator.compute_dispatch_gate(branch, tmp_path) + finally: + if _orig_msr is not None: + _sys.modules["map_step_runner"] = _orig_msr + + assert "isolation" in str(exc_info.value).lower(), ( + f"DispatchGateError message should mention isolation: {exc_info.value}" + ) + + +def test_vc2_flag_false_returns_sequential_no_side_effects( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + """VC2 [AC-1-GATE] [HC-1]: concurrent_dispatch=false (default) → sequential on + FIRST line; no DispatchGateError, no concurrent result, no concurrency_ready probe.""" + import sys as _sys + + monkeypatch.chdir(tmp_path) + branch = "test-st001-vc2-default-sequential" + + # Width>=2 wave — would be parallelizable if isolation were on. + _write_step_state(branch, tmp_path, execution_waves=[["ST-001", "ST-002"]]) + + # Sub-test A: no config at all (flag defaults to false). + _orig_msr = _sys.modules.pop("map_step_runner", None) + try: + result_a = map_orchestrator.compute_dispatch_gate(branch, tmp_path) + finally: + if _orig_msr is not None: + _sys.modules["map_step_runner"] = _orig_msr + + assert result_a["dispatch_mode"] == "sequential", ( + f"No config → dispatch_mode must be 'sequential': {result_a}" + ) + assert result_a["reason"] == map_orchestrator.WAVE_REASON_DISPATCH_SEQUENTIAL, ( + f"No config → reason must be WAVE_REASON_DISPATCH_SEQUENTIAL: {result_a}" + ) + + # Sub-test B: explicit false in config. + _write_config_keys(tmp_path, **{"execution.concurrent_dispatch": "false"}) + _orig_msr2 = _sys.modules.pop("map_step_runner", None) + try: + result_b = map_orchestrator.compute_dispatch_gate(branch, tmp_path) + finally: + if _orig_msr2 is not None: + _sys.modules["map_step_runner"] = _orig_msr2 + + assert result_b["dispatch_mode"] == "sequential", ( + f"Explicit false → dispatch_mode must be 'sequential': {result_b}" + ) + assert result_b["reason"] == map_orchestrator.WAVE_REASON_DISPATCH_SEQUENTIAL, ( + f"Explicit false → reason must be WAVE_REASON_DISPATCH_SEQUENTIAL: {result_b}" + ) + + # Sub-test C: get_wave_step under default config stays sequential and never raises. + _write_step_state(branch, tmp_path, execution_waves=[["ST-001", "ST-002"]]) + _orig_msr3 = _sys.modules.pop("map_step_runner", None) + try: + wave_result = map_orchestrator.get_wave_step(branch) + finally: + if _orig_msr3 is not None: + _sys.modules["map_step_runner"] = _orig_msr3 + + assert wave_result["dispatch_mode"] == "sequential", ( + f"get_wave_step default → dispatch_mode must be 'sequential': {wave_result}" + ) + assert wave_result["concurrency_enabled"] is False, ( + f"get_wave_step default → concurrency_enabled must be False: {wave_result}" + ) + + +def test_vc3_flag_on_isolation_required_concurrent_and_sequential( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + """VC3 [AC-1-GATE]: dispatch_mode=='concurrent' iff full conjunction holds; + non-parallelizable plan → sequential+WAVE_REASON_GATE_NOT_PARALLELIZABLE (not error); + reason codes are stable and distinct.""" + import sys as _sys + + monkeypatch.chdir(tmp_path) + branch_conc = "test-st001-vc3-concurrent" + branch_seq = "test-st001-vc3-seq-no-parallel" + + # --- Case A: flag=true, isolation=required, width>=2 wave → concurrent --- + _write_step_state(branch_conc, tmp_path, execution_waves=[["ST-001", "ST-002"]]) + _write_config_keys( + tmp_path, + **{ + "execution.concurrent_dispatch": "true", + "execution.wave_mode": "on", + "worktree.isolation": "required", + }, + ) + + _orig_msr = _sys.modules.pop("map_step_runner", None) + try: + result_conc = map_orchestrator.compute_dispatch_gate(branch_conc, tmp_path) + finally: + if _orig_msr is not None: + _sys.modules["map_step_runner"] = _orig_msr + + assert result_conc["dispatch_mode"] == "concurrent", ( + f"flag=true + isolation=required + width>=2 → expected concurrent: {result_conc}" + ) + assert result_conc["reason"] == map_orchestrator.WAVE_REASON_CONCURRENT_GATED, ( + f"concurrent path → reason must be WAVE_REASON_CONCURRENT_GATED: {result_conc}" + ) + + # --- Case B: flag=true, isolation=required, all width-1 → sequential (not error) --- + _write_step_state(branch_seq, tmp_path, execution_waves=[["ST-001"], ["ST-002"]]) + # Keep same config (flag=true, isolation=required, wave_mode=on) + + _orig_msr2 = _sys.modules.pop("map_step_runner", None) + try: + result_seq = map_orchestrator.compute_dispatch_gate(branch_seq, tmp_path) + finally: + if _orig_msr2 is not None: + _sys.modules["map_step_runner"] = _orig_msr2 + + assert result_seq["dispatch_mode"] == "sequential", ( + f"flag=true + isolation=required + all-width-1 → expected sequential: {result_seq}" + ) + assert result_seq["reason"] == map_orchestrator.WAVE_REASON_GATE_NOT_PARALLELIZABLE, ( + f"not-parallelizable → reason must be WAVE_REASON_GATE_NOT_PARALLELIZABLE: {result_seq}" + ) + + # --- Reason codes are stable non-empty distinct strings --- + codes = { + map_orchestrator.WAVE_REASON_CONCURRENT_GATED, + map_orchestrator.WAVE_REASON_GATE_NOT_PARALLELIZABLE, + map_orchestrator.WAVE_REASON_DISPATCH_SEQUENTIAL, + map_orchestrator.WAVE_REASON_NO_WAVES, + map_orchestrator.WAVE_REASON_WAVE_COMPLETE, + } + assert len(codes) == 5, f"All five reason codes must be distinct: {codes}" + assert all(isinstance(c, str) and c for c in codes), ( + f"All reason codes must be non-empty strings: {codes}" + ) + + +def test_vc4_mixed_plan_current_width1_later_width2_sequential( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + """F2 mixed-plan gate: width-1 current wave + width>=2 later wave -> sequential + with WAVE_REASON_CURRENT_WAVE_SEQUENTIAL (not WAVE_REASON_GATE_NOT_PARALLELIZABLE). + + Non-tautology: pointing current_wave_index at the width-2 wave (Case B below) + yields concurrent — proving the two reason-code paths are distinct. + """ + import sys as _sys + + monkeypatch.chdir(tmp_path) + + _write_config_keys( + tmp_path, + **{ + "execution.concurrent_dispatch": "true", + "execution.wave_mode": "auto", + "worktree.isolation": "required", + }, + ) + + # Case A: current wave is width-1 (index=0), later wave is width-2 (index=1). + branch_a = "test-st001-vc4-mixed-plan-current-1" + _write_step_state( + branch_a, + tmp_path, + execution_waves=[["ST-001"], ["ST-002", "ST-003"]], + extra={"current_wave_index": 0}, + ) + + _orig_msr_a = _sys.modules.pop("map_step_runner", None) + try: + result_a = map_orchestrator.compute_dispatch_gate(branch_a, tmp_path) + finally: + if _orig_msr_a is not None: + _sys.modules["map_step_runner"] = _orig_msr_a + + assert result_a["dispatch_mode"] == "sequential", ( + f"width-1 current wave must yield sequential: {result_a}" + ) + assert result_a["reason"] == map_orchestrator.WAVE_REASON_CURRENT_WAVE_SEQUENTIAL, ( + f"mixed-plan with width-1 current wave must use WAVE_REASON_CURRENT_WAVE_SEQUENTIAL, " + f"not WAVE_REASON_GATE_NOT_PARALLELIZABLE: {result_a}" + ) + + # Case B (non-tautology probe): same plan, current_wave_index=1 (the width-2 wave) + # must return concurrent — proving Case A's reason is about CURRENT wave, not the plan. + branch_b = "test-st001-vc4-mixed-plan-current-2" + _write_step_state( + branch_b, + tmp_path, + execution_waves=[["ST-001"], ["ST-002", "ST-003"]], + extra={"current_wave_index": 1}, + ) + + _orig_msr_b = _sys.modules.pop("map_step_runner", None) + try: + result_b = map_orchestrator.compute_dispatch_gate(branch_b, tmp_path) + finally: + if _orig_msr_b is not None: + _sys.modules["map_step_runner"] = _orig_msr_b + + assert result_b["dispatch_mode"] == "concurrent", ( + f"width-2 current wave (index=1) must yield concurrent: {result_b}" + ) + assert result_b["reason"] == map_orchestrator.WAVE_REASON_CONCURRENT_GATED, ( + f"width-2 current wave must use WAVE_REASON_CONCURRENT_GATED: {result_b}" + ) + + if __name__ == "__main__": pytest.main([__file__, "-v"]) diff --git a/tests/test_map_step_runner.py b/tests/test_map_step_runner.py index 96c326db..0debafab 100644 --- a/tests/test_map_step_runner.py +++ b/tests/test_map_step_runner.py @@ -12798,3 +12798,1789 @@ def _no_git(args: list[str], **_kw: object) -> object: ) assert dormant_result["ok"] is False assert git_calls == [], "No git must run when isolation is off" + + +# --------------------------------------------------------------------------- +# Group lifecycle verbs — VC1/VC2/VC3/VC4 (ST-002 / 5b.1) +# --------------------------------------------------------------------------- + + +def _make_git_repo_for_group(path: Path) -> str: + """Create a minimal git repo and return its HEAD sha.""" + import subprocess as _sp + + _sp.run(["git", "init", str(path)], check=True, capture_output=True) + _sp.run( + ["git", "config", "user.email", "test@example.com"], + cwd=str(path), check=True, capture_output=True, + ) + _sp.run( + ["git", "config", "user.name", "Test"], + cwd=str(path), check=True, capture_output=True, + ) + (path / "README.md").write_text("hello", encoding="utf-8") + _sp.run(["git", "add", "."], cwd=str(path), check=True, capture_output=True) + _sp.run( + ["git", "commit", "-m", "init"], + cwd=str(path), check=True, capture_output=True, + ) + sha_r = _sp.run( + ["git", "rev-parse", "HEAD"], + cwd=str(path), check=True, capture_output=True, text=True, + ) + return sha_r.stdout.strip() + + +class TestBeginWaveGroup: + """VC1: begin_wave_group records base_sha + skeleton; idempotent.""" + + def test_vc1_records_base_sha_and_skeleton( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + repo = tmp_path / "repo" + repo.mkdir() + head_sha = _make_git_repo_for_group(repo) + + map_dir = repo / ".map" / "test-branch" + map_dir.mkdir(parents=True) + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: "test-branch") + + result = map_step_runner.begin_wave_group(["ST-001", "ST-002"], "test-branch") + + assert result["ok"] is True, f"begin_wave_group failed: {result}" + assert result["base_sha"] == head_sha + assert sorted(result["subtask_ids"]) == ["ST-001", "ST-002"] # type: ignore[arg-type] + + # Verify sidecar was written + state = map_step_runner._read_worktree_state("test-branch") + wave_groups = state.get("wave_groups") + assert isinstance(wave_groups, dict) and wave_groups, "wave_groups must be populated" + group_key = result["group_key"] + assert group_key in wave_groups + grp = wave_groups[group_key] + assert isinstance(grp, dict) + assert grp["base_sha"] == head_sha + assert isinstance(grp.get("lifecycle"), dict) + + def test_vc1_idempotent_no_duplicate( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + repo = tmp_path / "repo" + repo.mkdir() + head_sha = _make_git_repo_for_group(repo) + + map_dir = repo / ".map" / "test-branch" + map_dir.mkdir(parents=True) + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: "test-branch") + + r1 = map_step_runner.begin_wave_group(["ST-003", "ST-004"], "test-branch") + assert r1["ok"] is True + + # Mutate lifecycle to simulate partial progress + state = map_step_runner._read_worktree_state("test-branch") + gk = r1["group_key"] + state["wave_groups"][gk]["lifecycle"]["ST-003"] = [ # type: ignore[index] + {"seq": 1, "event": "started", "ts": 0.0} + ] + map_step_runner._write_worktree_state("test-branch", state) + + # Second call must NOT clobber existing data + r2 = map_step_runner.begin_wave_group(["ST-003", "ST-004"], "test-branch") + assert r2["ok"] is True + assert r2["base_sha"] == head_sha + + state2 = map_step_runner._read_worktree_state("test-branch") + # lifecycle for ST-003 must still have the event we manually wrote + lc = state2["wave_groups"][gk]["lifecycle"] # type: ignore[index] + assert isinstance(lc, dict) + assert len(lc.get("ST-003", [])) == 1, "idempotent call must not erase lifecycle" + + def test_vc1_not_git_repo_returns_error( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + non_git = tmp_path / "non_git" + non_git.mkdir() + monkeypatch.chdir(non_git) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: "test-branch") + + result = map_step_runner.begin_wave_group(["ST-001"], "test-branch") + assert result.get("ok") is not True + assert result.get("kind") == map_step_runner._WT_REASON_NOT_GIT_REPO + + +class TestRecordGroupLifecycle: + """VC2: record_group_lifecycle appends in sorted seq order; replay is deterministic; + invalid event is rejected.""" + + def _setup(self, repo: Path, branch: str = "test-branch") -> str: + """Init git repo + begin_wave_group; returns group_key.""" + _make_git_repo_for_group(repo) + map_dir = repo / ".map" / branch + map_dir.mkdir(parents=True) + r = map_step_runner.begin_wave_group(["ST-010", "ST-011"], branch) + assert r["ok"] is True + return str(r["group_key"]) + + def test_vc2_appends_in_seq_order( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + repo = tmp_path / "repo" + repo.mkdir() + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: "test-branch") + + gk = self._setup(repo) + + events = ["created", "started", "finished"] + seqs = [] + for ev in events: + r = map_step_runner.record_group_lifecycle(gk, "ST-010", ev, "test-branch") + assert r["ok"] is True, f"unexpected failure: {r}" + seqs.append(r["seq"]) + + # Sequence numbers must be strictly increasing + assert seqs == sorted(seqs), f"seqs not sorted: {seqs}" + assert len(set(seqs)) == len(seqs), "seq values must be unique" + + def test_vc2_deterministic_replay_reconstructs_timeline( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """Replaying events sorted by seq reconstructs a deterministic in-flight + timeline regardless of wall-clock ordering.""" + repo = tmp_path / "repo" + repo.mkdir() + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: "test-branch") + + gk = self._setup(repo) + + # Record interleaved events for two subtasks + map_step_runner.record_group_lifecycle(gk, "ST-010", "created", "test-branch") + map_step_runner.record_group_lifecycle(gk, "ST-011", "created", "test-branch") + map_step_runner.record_group_lifecycle(gk, "ST-010", "started", "test-branch") + map_step_runner.record_group_lifecycle(gk, "ST-011", "started", "test-branch") + + state = map_step_runner._read_worktree_state("test-branch") + lifecycle = state["wave_groups"][gk]["lifecycle"] # type: ignore[index] + + # Collect all events across all subtasks and sort by seq + all_events: list[dict[str, object]] = [] + for evs in lifecycle.values(): + if isinstance(evs, list): + all_events.extend(e for e in evs if isinstance(e, dict)) + all_events_sorted = sorted(all_events, key=lambda e: cast("int", e["seq"])) + + # Seq must be strictly monotonic globally + seqs = [cast("int", e["seq"]) for e in all_events_sorted] + assert seqs == sorted(seqs) + assert len(set(seqs)) == len(seqs) + + # created events must precede started events for each subtask in replay + for sid in ("ST-010", "ST-011"): + sid_events_sorted = sorted( + [e for e in all_events if e in lifecycle.get(sid, [])], + key=lambda e: cast("int", e["seq"]), + ) + event_names = [e["event"] for e in sid_events_sorted] + if "created" in event_names and "started" in event_names: + ci = event_names.index("created") + si = event_names.index("started") + assert ci < si, f"{sid}: created must precede started in replay" + + def test_vc2_invalid_event_rejected( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + repo = tmp_path / "repo" + repo.mkdir() + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: "test-branch") + + gk = self._setup(repo) + result = map_step_runner.record_group_lifecycle( + gk, "ST-010", "INVALID_EVENT_CODE", "test-branch" + ) + assert result.get("ok") is not True + assert result.get("kind") == "invalid_event" + + def test_vc2_unknown_group_rejected( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + repo = tmp_path / "repo" + repo.mkdir() + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: "test-branch") + _make_git_repo_for_group(repo) + (repo / ".map" / "test-branch").mkdir(parents=True) + + result = map_step_runner.record_group_lifecycle( + "no|such|group", "ST-001", "started", "test-branch" + ) + assert result.get("ok") is not True + assert result.get("kind") == "unknown_group" + + +class TestVerifyGroupClean: + """VC3: verify_group_clean returns clean iff HEAD==base_sha AND tree clean AND + zero group worktrees remain; test the false branches too.""" + + def test_vc3_clean_when_no_groups_and_clean_tree( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + repo = tmp_path / "repo" + repo.mkdir() + head_sha = _make_git_repo_for_group(repo) + (repo / ".map" / "test-branch").mkdir(parents=True) + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: "test-branch") + + result = map_step_runner.verify_group_clean("test-branch") + assert result["clean"] is True, f"expected clean; got {result}" + assert result["head_sha"] == head_sha + + def test_vc3_not_clean_when_groups_remain( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + repo = tmp_path / "repo" + repo.mkdir() + _make_git_repo_for_group(repo) + (repo / ".map" / "test-branch").mkdir(parents=True) + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: "test-branch") + + # Begin a group but never complete it + r = map_step_runner.begin_wave_group(["ST-020"], "test-branch") + assert r["ok"] is True + + result = map_step_runner.verify_group_clean("test-branch") + assert result["clean"] is False + # Either GROUP_WORKTREES_REMAIN (groups present) is the reason + assert result["reason"] in ( + map_step_runner._WT_REASON_GROUP_WORKTREES_REMAIN, + map_step_runner._WT_REASON_GROUP_HEAD_MISMATCH, + ) + + def test_vc3_not_clean_when_tree_dirty( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + repo = tmp_path / "repo" + repo.mkdir() + _make_git_repo_for_group(repo) + (repo / ".map" / "test-branch").mkdir(parents=True) + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: "test-branch") + + # No groups in sidecar — but make a dirty file + (repo / "dirty.py").write_text("x = 1", encoding="utf-8") + + result = map_step_runner.verify_group_clean("test-branch") + assert result["clean"] is False + assert result["reason"] == map_step_runner._WT_REASON_GROUP_DIRTY_TREE + + def test_vc3_not_clean_head_mismatch( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + repo = tmp_path / "repo" + repo.mkdir() + _make_git_repo_for_group(repo) + (repo / ".map" / "test-branch").mkdir(parents=True) + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: "test-branch") + + # Write a group sidecar with a stale base_sha + state = map_step_runner._read_worktree_state("test-branch") + state["wave_groups"] = { + "ST-030": { + "base_sha": "deadbeef" * 5, # wrong sha + "subtask_ids": ["ST-030"], + "lifecycle": {}, + } + } + map_step_runner._write_worktree_state("test-branch", state) + + result = map_step_runner.verify_group_clean("test-branch") + assert result["clean"] is False + assert result["reason"] == map_step_runner._WT_REASON_GROUP_HEAD_MISMATCH + + def test_vc3_not_git_repo( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + non_git = tmp_path / "non_git" + non_git.mkdir() + monkeypatch.chdir(non_git) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: "test-branch") + + result = map_step_runner.verify_group_clean("test-branch") + assert result["clean"] is False + assert result["reason"] == map_step_runner._WT_REASON_NOT_GIT_REPO + + +class TestReconcileOrphanGroups: + """VC3 (reconcile): reconcile_orphan_groups sweeps a synthetic mid-flight group.""" + + def test_vc3_sweeps_orphan_group_no_lifecycle( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """A group with no lifecycle events (crashed before start) is swept.""" + repo = tmp_path / "repo" + repo.mkdir() + _make_git_repo_for_group(repo) + (repo / ".map" / "test-branch").mkdir(parents=True) + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: "test-branch") + + # Write a synthetic orphan group with no lifecycle events + state = map_step_runner._read_worktree_state("test-branch") + state["wave_groups"] = { + "orphan|group": { + "base_sha": "abc123", + "subtask_ids": ["orphan"], + "lifecycle": {}, # empty — never started + } + } + map_step_runner._write_worktree_state("test-branch", state) + + result = map_step_runner.reconcile_orphan_groups("test-branch") + assert result["ok"] is True, f"reconcile failed: {result}" + assert result["swept"] == 1 + + # Sidecar must be clean after sweep + state2 = map_step_runner._read_worktree_state("test-branch") + wave_groups2 = state2.get("wave_groups", {}) + assert wave_groups2 == {} or "orphan|group" not in wave_groups2 + + def test_vc3_sweeps_terminated_group( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """A group where all subtasks have a terminal event (merged) is swept.""" + repo = tmp_path / "repo" + repo.mkdir() + _make_git_repo_for_group(repo) + (repo / ".map" / "test-branch").mkdir(parents=True) + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: "test-branch") + + state = map_step_runner._read_worktree_state("test-branch") + state["wave_groups"] = { + "ST-100|ST-101": { + "base_sha": "abc123", + "subtask_ids": ["ST-100", "ST-101"], + "lifecycle": { + "ST-100": [{"seq": 1, "event": "merged", "ts": 0.0}], + "ST-101": [{"seq": 2, "event": "merged", "ts": 0.0}], + }, + } + } + map_step_runner._write_worktree_state("test-branch", state) + + result = map_step_runner.reconcile_orphan_groups("test-branch") + assert result["ok"] is True + assert result["swept"] >= 1 + + def test_vc3_preserves_active_group( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """A group with only non-terminal events is preserved.""" + repo = tmp_path / "repo" + repo.mkdir() + _make_git_repo_for_group(repo) + (repo / ".map" / "test-branch").mkdir(parents=True) + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: "test-branch") + + state = map_step_runner._read_worktree_state("test-branch") + state["wave_groups"] = { + "ST-200|ST-201": { + "base_sha": "abc123", + "subtask_ids": ["ST-200", "ST-201"], + "lifecycle": { + "ST-200": [{"seq": 1, "event": "started", "ts": 0.0}], + "ST-201": [{"seq": 2, "event": "created", "ts": 0.0}], + }, + } + } + map_step_runner._write_worktree_state("test-branch", state) + + result = map_step_runner.reconcile_orphan_groups("test-branch") + assert result["ok"] is True + assert result["swept"] == 0, "active group must not be swept" + + # Group must still be in sidecar + state2 = map_step_runner._read_worktree_state("test-branch") + assert "ST-200|ST-201" in state2.get("wave_groups", {}) + + def test_vc3_idempotent_second_call( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """Second reconcile call after everything is clean returns swept=0.""" + repo = tmp_path / "repo" + repo.mkdir() + _make_git_repo_for_group(repo) + (repo / ".map" / "test-branch").mkdir(parents=True) + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: "test-branch") + + # No wave_groups at all + result1 = map_step_runner.reconcile_orphan_groups("test-branch") + assert result1["ok"] is True + result2 = map_step_runner.reconcile_orphan_groups("test-branch") + assert result2["ok"] is True + assert result2["swept"] == 0 + + def test_vc4_f4_partial_group_one_subtask_missing_not_swept( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """F4: a group with one subtask's events missing is NOT swept (partial record). + + The group declares ST-300 and ST-301 but only ST-300 has a terminal event. + ST-301 has no lifecycle slot at all. reconcile_orphan_groups must NOT + sweep this group — missing slot means the subtask is not done. + """ + repo = tmp_path / "repo" + repo.mkdir() + _make_git_repo_for_group(repo) + (repo / ".map" / "test-branch").mkdir(parents=True) + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: "test-branch") + + state = map_step_runner._read_worktree_state("test-branch") + state["wave_groups"] = { + "ST-300|ST-301": { + "base_sha": "abc123", + "subtask_ids": ["ST-300", "ST-301"], + "lifecycle": { + # ST-300 has a terminal event; ST-301 slot is MISSING entirely. + "ST-300": [{"seq": 1, "event": "merged", "ts": 0.0}], + }, + } + } + map_step_runner._write_worktree_state("test-branch", state) + + result = map_step_runner.reconcile_orphan_groups("test-branch") + assert result["ok"] is True + assert result["swept"] == 0, ( + "A partially-recorded group (one subtask missing lifecycle slot) " + "must NOT be swept — missing slot means the subtask is not terminal." + ) + + # Group must still be present in sidecar. + state2 = map_step_runner._read_worktree_state("test-branch") + assert "ST-300|ST-301" in state2.get("wave_groups", {}), ( + "Partial group was incorrectly removed from the sidecar." + ) + + +class TestGroupEventConstants: + """VC4 (SC-1): New reason/event codes follow the _WT_REASON_* / _WT_GROUP_EVENT_* + constant style (str type, non-empty, lower_snake_case).""" + + def test_vc4_event_constants_are_lowercase_str(self) -> None: + for name in ( + "_WT_GROUP_EVENT_CREATED", + "_WT_GROUP_EVENT_STARTED", + "_WT_GROUP_EVENT_FINISHED", + "_WT_GROUP_EVENT_MERGED", + "_WT_GROUP_EVENT_ABORTED", + ): + val = getattr(map_step_runner, name) + assert isinstance(val, str) and val, f"{name} must be non-empty str" + assert val == val.lower(), f"{name}={val!r} must be lowercase" + + def test_vc4_reason_constants_are_lowercase_str(self) -> None: + for name in ( + "_WT_REASON_GROUP_HEAD_MISMATCH", + "_WT_REASON_GROUP_DIRTY_TREE", + "_WT_REASON_GROUP_WORKTREES_REMAIN", + ): + val = getattr(map_step_runner, name) + assert isinstance(val, str) and val, f"{name} must be non-empty str" + assert val == val.lower(), f"{name}={val!r} must be lowercase" + + def test_vc4_valid_events_frozenset_contains_all_codes(self) -> None: + expected = { + map_step_runner._WT_GROUP_EVENT_CREATED, + map_step_runner._WT_GROUP_EVENT_STARTED, + map_step_runner._WT_GROUP_EVENT_FINISHED, + map_step_runner._WT_GROUP_EVENT_MERGED, + map_step_runner._WT_GROUP_EVENT_ABORTED, + } + assert map_step_runner._WT_GROUP_VALID_EVENTS == expected + + +# --------------------------------------------------------------------------- +# Helpers shared across RunConcurrentWave tests +# --------------------------------------------------------------------------- + + +def _make_git_repo_with_worktrees( + root: Path, subtask_ids: list[str] +) -> tuple[str, dict[str, Path]]: + """Create a real git repo with one committed worktree per subtask. + + Returns (base_sha, {subtask_id: worktree_path}). + The caller must chdir into root + monkeypatch get_branch_name before calling + map_step_runner functions. + """ + import subprocess as _sp + + _sp.run(["git", "init", str(root)], check=True, capture_output=True) + _sp.run( + ["git", "config", "user.email", "test@example.com"], + cwd=str(root), check=True, capture_output=True, + ) + _sp.run( + ["git", "config", "user.name", "Test"], + cwd=str(root), check=True, capture_output=True, + ) + (root / "README.md").write_text("base", encoding="utf-8") + _sp.run(["git", "add", "."], cwd=str(root), check=True, capture_output=True) + _sp.run( + ["git", "commit", "--no-verify", "-m", "init"], + cwd=str(root), check=True, capture_output=True, + ) + sha_r = _sp.run( + ["git", "rev-parse", "HEAD"], + cwd=str(root), check=True, capture_output=True, text=True, + ) + base_sha = sha_r.stdout.strip() + + wt_paths: dict[str, Path] = {} + for sid in subtask_ids: + slug = "".join(c if c.isalnum() or c in "-_." else "-" for c in sid).lower() + wt_dir = root.parent / f"wt-{slug}" + wt_branch = f"map/wt/{slug}" + _sp.run( + ["git", "worktree", "add", "-b", wt_branch, str(wt_dir)], + cwd=str(root), check=True, capture_output=True, + ) + _sp.run( + ["git", "config", "user.email", "test@example.com"], + cwd=str(wt_dir), check=True, capture_output=True, + ) + _sp.run( + ["git", "config", "user.name", "Test"], + cwd=str(wt_dir), check=True, capture_output=True, + ) + # Write a unique file per worktree so the merge has content to commit. + (wt_dir / f"{slug}.txt").write_text(f"work for {sid}", encoding="utf-8") + _sp.run(["git", "add", "."], cwd=str(wt_dir), check=True, capture_output=True) + _sp.run( + ["git", "commit", "--no-verify", "-m", f"work: {sid}"], + cwd=str(wt_dir), check=True, capture_output=True, + ) + wt_paths[sid] = wt_dir + + return base_sha, wt_paths + + +def _register_worktrees( + branch: str, base_sha: str, subtask_ids: list[str], wt_paths: dict[str, Path] +) -> None: + """Register each worktree in the map_step_runner sidecar.""" + state = map_step_runner._read_worktree_state(branch) + if not isinstance(state.get("worktrees"), dict): + state["worktrees"] = {} + for sid in subtask_ids: + slug_r = map_step_runner._wt_slug(sid) + if slug_r is None: + continue + wt_path = wt_paths[sid] + wt_branch = f"map/wt/{slug_r}" + state["worktrees"][slug_r] = { + "subtask_id": sid, + "path": str(wt_path), + "branch": wt_branch, + "base_sha": base_sha, + "attempt": 0, + } + map_step_runner._write_worktree_state(branch, state) + + +# --------------------------------------------------------------------------- +# Tests: _max_actors and _chunk helpers (unit) +# --------------------------------------------------------------------------- + + +class TestMaxActorsHelper: + """Unit tests for the _max_actors() and _chunk() helpers.""" + + def test_vc2_default_max_actors_no_config(self, tmp_path: Path) -> None: + """VC2: With no config file, _max_actors returns the default 4.""" + result = map_step_runner._max_actors(tmp_path) + assert result == 4, f"expected default 4, got {result}" + + def test_vc2_max_actors_from_config(self, tmp_path: Path) -> None: + """VC2: max_actors reads execution.max_actors from config and clamps.""" + map_dir = tmp_path / ".map" + map_dir.mkdir() + (map_dir / "config.yaml").write_text( + "execution.max_actors: 2\n", encoding="utf-8" + ) + result = map_step_runner._max_actors(tmp_path) + assert result == 2, f"expected 2 from config, got {result}" + + def test_vc2_max_actors_zero_falls_back_to_default(self, tmp_path: Path) -> None: + """VC2: max_actors: 0 is not a valid positive int; _map_config_int returns + the default 4 (mirrors its '>0 or default' guard), so the result is 4.""" + map_dir = tmp_path / ".map" + map_dir.mkdir() + (map_dir / "config.yaml").write_text( + "execution.max_actors: 0\n", encoding="utf-8" + ) + result = map_step_runner._max_actors(tmp_path) + # _map_config_int treats <= 0 as invalid → returns default 4; clamp(4) == 4. + assert result == 4, f"expected fallback to default 4, got {result}" + + def test_vc2_max_actors_clamped_to_max(self, tmp_path: Path) -> None: + """VC2: max_actors > 8 is clamped to 8.""" + map_dir = tmp_path / ".map" + map_dir.mkdir() + (map_dir / "config.yaml").write_text( + "execution.max_actors: 99\n", encoding="utf-8" + ) + result = map_step_runner._max_actors(tmp_path) + assert result == 8, f"expected clamp to 8, got {result}" + + def test_vc2_chunk_even_split(self) -> None: + """VC2: _chunk(6 items, size=2) -> 3 sub-lists of width 2.""" + items = ["a", "b", "c", "d", "e", "f"] + chunks = map_step_runner._chunk(items, 2) + assert chunks == [["a", "b"], ["c", "d"], ["e", "f"]] + + def test_vc2_chunk_uneven_split(self) -> None: + """VC2: _chunk(5 items, size=2) -> last chunk has 1 item.""" + items = ["a", "b", "c", "d", "e"] + chunks = map_step_runner._chunk(items, 2) + assert chunks == [["a", "b"], ["c", "d"], ["e"]] + + def test_vc2_chunk_within_cap_is_one_batch(self) -> None: + """VC2: _chunk(3 items, size=4) -> 1 batch (no split).""" + items = ["a", "b", "c"] + chunks = map_step_runner._chunk(items, 4) + assert chunks == [["a", "b", "c"]] + + def test_vc2_chunk_size_zero_treated_as_one(self) -> None: + """VC2: _chunk with size=0 falls back to size=1.""" + chunks = map_step_runner._chunk(["x", "y"], 0) + assert chunks == [["x"], ["y"]] + + +# --------------------------------------------------------------------------- +# Tests: run_concurrent_wave — non-git / empty input guards (unit) +# --------------------------------------------------------------------------- + + +class TestRunConcurrentWaveGuards: + """run_concurrent_wave returns structured errors for invalid inputs.""" + + def test_vc1_empty_group_ids_returns_error( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """VC1: Empty group_ids -> structured error, ok=False.""" + # Not in a git repo — but empty-ids guard fires first. + non_git = tmp_path / "non_git" + non_git.mkdir() + monkeypatch.chdir(non_git) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: "test-branch") + + result = map_step_runner.run_concurrent_wave([], "test-branch", non_git) + assert result.get("ok") is not True + assert result.get("status") == "error" + + def test_vc1_not_git_repo_returns_error( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """VC1: Not a git repo -> structured error.""" + non_git = tmp_path / "non_git" + non_git.mkdir() + monkeypatch.chdir(non_git) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: "test-branch") + + result = map_step_runner.run_concurrent_wave( + ["ST-001", "ST-002"], "test-branch", non_git + ) + assert result.get("ok") is not True + assert result.get("status") == "error" + + def test_vc1_f6_default_config_returns_concurrent_dispatch_disabled( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """F6/HC-1: default config (no concurrent_dispatch key) -> CONCURRENT_DISPATCH_DISABLED. + + Defense-in-depth: a direct CLI or coordinator call cannot trigger concurrent + merging under the default-off configuration even when inside a git repo. + This is the second line of defense after compute_dispatch_gate. + """ + repo = tmp_path / "repo" + repo.mkdir() + _make_git_repo_for_group(repo) + (repo / ".map" / "test-branch").mkdir(parents=True) + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: "test-branch") + + # No config.yaml at all — concurrent_dispatch defaults to False. + result = map_step_runner.run_concurrent_wave( + ["ST-001", "ST-002"], "test-branch", repo + ) + assert result.get("ok") is not True + assert result.get("status") == "error" + assert result.get("kind") == "CONCURRENT_DISPATCH_DISABLED", ( + f"Expected CONCURRENT_DISPATCH_DISABLED under default config; " + f"got kind={result.get('kind')!r}" + ) + + +# --------------------------------------------------------------------------- +# Tests: run_concurrent_wave — batch-split (VC2) +# --------------------------------------------------------------------------- + + +class TestRunConcurrentWaveBatchSplit: + """VC2: batch-split respects max_actors cap; assert sub-batch boundaries.""" + + def test_vc2_within_cap_single_batch( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """VC2: 3 ids, max_actors=4 -> 1 sub-batch, merge called once.""" + merge_calls: list[list[str]] = [] + + def _fake_merge(ids: list[str], branch: Any = None, **kw: Any) -> dict[str, Any]: + del branch, kw # match merge_wave_worktrees signature; unused in stub + merge_calls.append(sorted(ids)) + return {"status": "success", "ok": True, "merged": ids, "no_changes": []} + + monkeypatch.setattr(map_step_runner, "merge_wave_worktrees", _fake_merge) + + repo = tmp_path / "repo" + repo.mkdir() + _make_git_repo_for_group(repo) + (repo / ".map" / "test-branch").mkdir(parents=True) + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: "test-branch") + + map_dir = repo / ".map" + # F6 guard: concurrent_dispatch must be enabled for run_concurrent_wave to proceed. + (map_dir / "config.yaml").write_text( + "execution.max_actors: 4\nexecution.concurrent_dispatch: true\n", encoding="utf-8" + ) + + result = map_step_runner.run_concurrent_wave( + ["ST-003", "ST-001", "ST-002"], "test-branch", repo + ) + assert result.get("ok") is True + sub_batches = result.get("sub_batches") + assert isinstance(sub_batches, list) + assert len(sub_batches) == 1, f"expected 1 batch, got {sub_batches}" + assert sorted(cast(list[str], sub_batches[0])) == ["ST-001", "ST-002", "ST-003"] + assert len(merge_calls) == 1 + + def test_vc2_exceeds_cap_splits_into_sub_batches( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """VC2: 5 ids, max_actors=2 -> 3 sub-batches each ≤ 2 wide.""" + merge_calls: list[list[str]] = [] + + def _fake_merge(ids: list[str], branch: Any = None, **kw: Any) -> dict[str, Any]: + del branch, kw # match merge_wave_worktrees signature; unused in stub + merge_calls.append(sorted(ids)) + return {"status": "success", "ok": True, "merged": ids, "no_changes": []} + + monkeypatch.setattr(map_step_runner, "merge_wave_worktrees", _fake_merge) + + repo = tmp_path / "repo" + repo.mkdir() + _make_git_repo_for_group(repo) + (repo / ".map" / "test-branch").mkdir(parents=True) + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: "test-branch") + + map_dir = repo / ".map" + # F6 guard: concurrent_dispatch must be enabled for run_concurrent_wave to proceed. + (map_dir / "config.yaml").write_text( + "execution.max_actors: 2\nexecution.concurrent_dispatch: true\n", encoding="utf-8" + ) + + result = map_step_runner.run_concurrent_wave( + ["ST-005", "ST-001", "ST-003", "ST-002", "ST-004"], "test-branch", repo + ) + assert result.get("ok") is True + sub_batches = result.get("sub_batches") + assert isinstance(sub_batches, list) + assert len(sub_batches) == 3, f"expected 3 batches, got {sub_batches}" + # All ids present in sorted order, each batch ≤ 2 + for b in sub_batches: + assert len(cast(list[str], b)) <= 2 + all_ids = sorted(sid for b in cast(list[list[str]], sub_batches) for sid in b) + assert all_ids == ["ST-001", "ST-002", "ST-003", "ST-004", "ST-005"] + assert len(merge_calls) == 3 + assert result.get("max_actors") == 2 + + def test_vc2_max_actors_config_controls_batching( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """VC2: Setting max_actors=1 -> N sub-batches (one per id).""" + merge_calls: list[list[str]] = [] + + def _fake_merge(ids: list[str], branch: Any = None, **kw: Any) -> dict[str, Any]: + del branch, kw # match merge_wave_worktrees signature; unused in stub + merge_calls.append(sorted(ids)) + return {"status": "success", "ok": True, "merged": ids, "no_changes": []} + + monkeypatch.setattr(map_step_runner, "merge_wave_worktrees", _fake_merge) + + repo = tmp_path / "repo" + repo.mkdir() + _make_git_repo_for_group(repo) + (repo / ".map" / "test-branch").mkdir(parents=True) + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: "test-branch") + + map_dir = repo / ".map" + # F6 guard: concurrent_dispatch must be enabled for run_concurrent_wave to proceed. + (map_dir / "config.yaml").write_text( + "execution.max_actors: 1\nexecution.concurrent_dispatch: true\n", encoding="utf-8" + ) + + group_ids = ["ST-A", "ST-B", "ST-C"] + result = map_step_runner.run_concurrent_wave(group_ids, "test-branch", repo) + assert result.get("ok") is True + assert len(merge_calls) == 3, f"expected 3 merge calls, got {merge_calls}" + for call in merge_calls: + assert len(call) == 1 + + +# --------------------------------------------------------------------------- +# Tests: run_concurrent_wave — merge error propagation (VC1 composability) +# --------------------------------------------------------------------------- + + +class TestRunConcurrentWaveMergeError: + """VC1/composability: merge failure triggers abort-once + needs_redispatch (F7/ST-006). + + run_concurrent_wave aborts the group ONCE on a merge error and returns a + structured WAVE_ABORTED result with needs_redispatch=True. The skill owns + the retry loop; the runner never retries internally (F7 architectural fix). + """ + + def test_vc1_merge_error_aborts_once_needs_redispatch( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """VC1/F7: merge error -> abort ONCE -> WAVE_ABORTED with needs_redispatch=True. + + The original merge error dict is preserved inside merge_error so the + skill can inspect the failure kind. No internal retry loop: abort is + called exactly once and the function returns immediately. + """ + abort_calls: list[str] = [] + + def _fake_merge_fail(ids: list[str], branch: Any = None, **kw: Any) -> dict[str, Any]: + del ids, branch, kw # signature match; all unused in the failure stub + return { + "status": "error", + "ok": False, + "kind": "WAVE_MERGE_CONFLICT", + "message": "conflict", + } + + def _fake_abort(group_id: str, branch: Any = None) -> dict[str, Any]: + del branch # unused in stub + abort_calls.append(group_id) + return {"clean": True, "aborted_group_id": group_id} + + monkeypatch.setattr(map_step_runner, "merge_wave_worktrees", _fake_merge_fail) + monkeypatch.setattr(map_step_runner, "abort_wave_group", _fake_abort) + + repo = tmp_path / "repo" + repo.mkdir() + _make_git_repo_for_group(repo) + map_dir = repo / ".map" + (map_dir / "test-branch").mkdir(parents=True) + # F6: concurrent_dispatch must be enabled for run_concurrent_wave to proceed. + (map_dir / "config.yaml").write_text( + "execution.concurrent_dispatch: true\nexecution.max_wave_retries: 3\n", + encoding="utf-8", + ) + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: "test-branch") + + result = map_step_runner.run_concurrent_wave( + ["ST-001", "ST-002"], "test-branch", repo + ) + # F7: abort called exactly once, no internal retry. + assert len(abort_calls) == 1, ( + f"abort_wave_group must be called exactly once on merge error; got {abort_calls}" + ) + # Structured WAVE_ABORTED result with needs_redispatch. + assert result.get("ok") is not True + assert result.get("status") == "error" + assert result.get("kind") == "WAVE_ABORTED", ( + f"Expected WAVE_ABORTED, got {result.get('kind')!r}" + ) + assert result.get("needs_redispatch") is True + assert isinstance(result.get("attempts_remaining"), int) + # Original merge error preserved inside merge_error. + merge_err = result.get("merge_error") + assert isinstance(merge_err, dict), f"merge_error must be a dict: {result}" + assert merge_err.get("kind") == "WAVE_MERGE_CONFLICT" + assert "failed_batch" in result + + def test_vc1_failure_in_second_batch_captured_in_merge_error( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """VC1/F7: failure in batch 2 -> merge_error has batches_merged_before_failure=1.""" + call_count = [0] + + def _fake_merge(ids: list[str], branch: Any = None, **kw: Any) -> dict[str, Any]: + del branch, kw # match merge_wave_worktrees signature; unused in stub + call_count[0] += 1 + # First batch succeeds; second fails. + if call_count[0] == 1: + return {"status": "success", "ok": True, "merged": ids, "no_changes": []} + return { + "status": "error", + "ok": False, + "kind": "WAVE_COMMIT_FAILED", + "message": "commit failed", + } + + def _fake_abort(group_id: str, branch: Any = None) -> dict[str, Any]: + del group_id, branch + return {"clean": True, "aborted_group_id": "fake"} + + monkeypatch.setattr(map_step_runner, "merge_wave_worktrees", _fake_merge) + monkeypatch.setattr(map_step_runner, "abort_wave_group", _fake_abort) + + repo = tmp_path / "repo" + repo.mkdir() + _make_git_repo_for_group(repo) + (repo / ".map" / "test-branch").mkdir(parents=True) + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: "test-branch") + + map_dir = repo / ".map" + # F6: concurrent_dispatch must be enabled; max_actors=1 → 2 batches for 2 ids. + (map_dir / "config.yaml").write_text( + "execution.max_actors: 1\n" + "execution.concurrent_dispatch: true\n" + "execution.max_wave_retries: 3\n", + encoding="utf-8", + ) + + result = map_step_runner.run_concurrent_wave( + ["ST-001", "ST-002"], "test-branch", repo + ) + assert result.get("ok") is not True + assert result.get("kind") == "WAVE_ABORTED" + assert result.get("needs_redispatch") is True + # batches_merged_before_failure == 1 (batch 2 failed, batch 1 already done). + assert result.get("batches_merged_before_failure") == 1, ( + f"Expected batches_merged_before_failure=1; got {result}" + ) + + +# --------------------------------------------------------------------------- +# Tests: run_concurrent_wave — real git integration (VC1/HC-4 atomicity) +# --------------------------------------------------------------------------- + + +class TestRunConcurrentWaveAtomicMerge: + """VC1/HC-4: all-or-nothing via merge_wave_worktrees on a real temp git repo.""" + + def test_vc1_hc4_full_success_all_merged( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """VC1/HC-4: N<=max_actors group -> all merged atomically; no partial subset. + + Creates real git worktrees, registers them in the sidecar, calls + run_concurrent_wave, then asserts all N files are present in the main + branch (all merged) or none are (all-or-nothing atomic rollback). + """ + repo = tmp_path / "repo" + repo.mkdir() + branch = "test-branch" + (repo / ".map" / branch).mkdir(parents=True) + + group_ids = ["ST-001", "ST-002"] + base_sha, wt_paths = _make_git_repo_with_worktrees(repo, group_ids) + + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: branch) + + # Register worktrees in sidecar (normally done by create_subtask_worktree). + _register_worktrees(branch, base_sha, group_ids, wt_paths) + + # F6: enable concurrent_dispatch so run_concurrent_wave proceeds past guard. + (repo / ".map" / "config.yaml").write_text( + "execution.concurrent_dispatch: true\n", encoding="utf-8" + ) + + # max_actors=4 (default): both in one batch -> one atomic merge. + result = map_step_runner.run_concurrent_wave(group_ids, branch, repo) + + # All-or-nothing: check outcome. + if result.get("ok") is True: + # Full success: all ids merged; verify files present in main branch. + merged_ids = result.get("merged_ids", []) + # At least the ids that had changes should appear in merged_ids or no_changes. + all_resolved = set(cast(list[str], merged_ids)) | set( + cast(list[str], result.get("no_changes", [])) + ) + assert set(group_ids).issubset(all_resolved), ( + f"Not all ids accounted for: {group_ids} vs {all_resolved}" + ) + else: + # Atomic rollback: none merged. Status must be error. + assert result.get("status") == "error", ( + f"Expected error on failure, got {result}" + ) + + def test_vc4_hc4_each_sub_batch_atomic( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """VC4/HC-4: Each sub-batch merge is itself atomic. + + With max_actors=1, each sub-batch has width 1. Monkeypatching + merge_wave_worktrees proves the batches are submitted sequentially + as full atomic units: merge is called exactly N times, one id per call. + """ + merge_batches: list[list[str]] = [] + + def _record_merge(ids: list[str], branch: Any = None, **kw: Any) -> dict[str, Any]: + del branch, kw # match merge_wave_worktrees signature; unused in stub + merge_batches.append(sorted(ids)) + return {"status": "success", "ok": True, "merged": ids, "no_changes": []} + + monkeypatch.setattr(map_step_runner, "merge_wave_worktrees", _record_merge) + + repo = tmp_path / "repo" + repo.mkdir() + _make_git_repo_for_group(repo) + (repo / ".map" / "test-branch").mkdir(parents=True) + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: "test-branch") + + map_dir = repo / ".map" + # F6 guard: concurrent_dispatch must be enabled for run_concurrent_wave to proceed. + (map_dir / "config.yaml").write_text( + "execution.max_actors: 1\nexecution.concurrent_dispatch: true\n", encoding="utf-8" + ) + + group_ids = ["ST-001", "ST-002", "ST-003"] + result = map_step_runner.run_concurrent_wave(group_ids, "test-branch", repo) + assert result.get("ok") is True + # Each call receives exactly 1 id (sub-batch width == cap == 1). + assert len(merge_batches) == 3 + for b in merge_batches: + assert len(b) == 1, f"each sub-batch must have exactly 1 id; got {b}" + # All ids covered + all_merged = sorted(sid for b in merge_batches for sid in b) + assert all_merged == sorted(group_ids) + + +# --------------------------------------------------------------------------- +# Tests: _max_wave_retries helper (unit) +# --------------------------------------------------------------------------- + + +class TestMaxWaveRetriesHelper: + """Unit tests for the _max_wave_retries() helper.""" + + def test_vc3_default_no_config(self, tmp_path: Path) -> None: + """VC3: With no config file, _max_wave_retries returns the default 3.""" + result = map_step_runner._max_wave_retries(tmp_path) + assert result == 3, f"expected default 3, got {result}" + + def test_vc3_reads_from_config(self, tmp_path: Path) -> None: + """VC3: Reads execution.max_wave_retries from .map/config.yaml.""" + map_dir = tmp_path / ".map" + map_dir.mkdir() + (map_dir / "config.yaml").write_text( + "execution.max_wave_retries: 5\n", encoding="utf-8" + ) + result = map_step_runner._max_wave_retries(tmp_path) + assert result == 5 + + def test_vc3_zero_falls_back_to_default(self, tmp_path: Path) -> None: + """VC3: Zero is not > 0 so _map_config_int falls back to default 3.""" + map_dir = tmp_path / ".map" + map_dir.mkdir() + (map_dir / "config.yaml").write_text( + "execution.max_wave_retries: 0\n", encoding="utf-8" + ) + result = map_step_runner._max_wave_retries(tmp_path) + # _map_config_int: parsed=0, not > 0 -> returns default=3; clamp(3,1,10)=3. + assert result == 3 + + def test_vc3_clamps_to_max(self, tmp_path: Path) -> None: + """VC3: Values above 10 clamp to 10.""" + map_dir = tmp_path / ".map" + map_dir.mkdir() + (map_dir / "config.yaml").write_text( + "execution.max_wave_retries: 99\n", encoding="utf-8" + ) + result = map_step_runner._max_wave_retries(tmp_path) + assert result == 10 + + def test_vc3_non_int_falls_back_to_default(self, tmp_path: Path) -> None: + """VC3: Non-int values fall back to default 3.""" + map_dir = tmp_path / ".map" + map_dir.mkdir() + (map_dir / "config.yaml").write_text( + "execution.max_wave_retries: notanumber\n", encoding="utf-8" + ) + result = map_step_runner._max_wave_retries(tmp_path) + assert result == 3 + + +# --------------------------------------------------------------------------- +# Tests: abort_wave_group (VC1–VC4, HC-4, safety) +# --------------------------------------------------------------------------- + + +class TestAbortWaveGroup: + """ST-006 abort_wave_group: idempotent group-abort verb.""" + + def test_vc1_hc4_whole_group_discarded_on_failure( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """VC1/HC-4: pre-merge failure -> abort_wave_group discards WHOLE group. + + Verifies: cancel siblings (discard_subtask_worktree called for every + member), _wt_rollback reused (no raw clean -fdx), group removed from + sidecar, verify_group_clean returns clean. + """ + repo = tmp_path / "repo" + repo.mkdir() + branch = "test-branch" + (repo / ".map" / branch).mkdir(parents=True) + + group_ids = ["ST-A01", "ST-A02"] + base_sha, wt_paths = _make_git_repo_with_worktrees(repo, group_ids) + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: branch) + + # Register worktrees + begin_wave_group so abort has state to read. + _register_worktrees(branch, base_sha, group_ids, wt_paths) + r_begin = map_step_runner.begin_wave_group(group_ids, branch) + assert r_begin["ok"] is True, f"begin_wave_group failed: {r_begin}" + group_key = str(r_begin["group_key"]) + + # Confirm group is present in sidecar before abort. + state_before = map_step_runner._read_worktree_state(branch) + assert group_key in (state_before.get("wave_groups") or {}), ( + "group must be present before abort" + ) + + result = map_step_runner.abort_wave_group(group_key, branch) + + # Verdict: verify_group_clean fields are returned. + assert "clean" in result, f"abort must return verify_group_clean verdict: {result}" + assert result.get("aborted_group_id") == group_key + + # Group entry must be gone from sidecar. + state_after = map_step_runner._read_worktree_state(branch) + wg_after = state_after.get("wave_groups") or {} + assert group_key not in wg_after, ( + f"group must be removed from sidecar after abort; still present: {wg_after}" + ) + + def test_vc2_idempotent_second_abort_converges( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """VC2: second abort after a partial abort converges to clean, no error.""" + repo = tmp_path / "repo" + repo.mkdir() + branch = "test-branch" + (repo / ".map" / branch).mkdir(parents=True) + + group_ids = ["ST-B01", "ST-B02"] + base_sha, wt_paths = _make_git_repo_with_worktrees(repo, group_ids) + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: branch) + + _register_worktrees(branch, base_sha, group_ids, wt_paths) + r_begin = map_step_runner.begin_wave_group(group_ids, branch) + group_key = str(r_begin["group_key"]) + + # First abort. + r1 = map_step_runner.abort_wave_group(group_key, branch) + assert "aborted_group_id" in r1 + + # Second abort on the already-clean state must not error. + r2 = map_step_runner.abort_wave_group(group_key, branch) + assert "aborted_group_id" in r2 + # Should report clean or at worst a non-error state (group already gone). + assert r2.get("status") != "error" or r2.get("clean") is True, ( + f"second abort must converge cleanly, got {r2}" + ) + + def test_safety_map_dir_survives_abort( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """Safety: abort does NOT delete .map/ — sentinel gitignored file survives. + + Proves _wt_rollback exclusion (-e .map) holds: a sentinel file under + .map/ must be present after abort (it would be gone if raw clean -fdx + were used instead of _wt_rollback). + """ + repo = tmp_path / "repo" + repo.mkdir() + branch = "test-branch" + (repo / ".map" / branch).mkdir(parents=True) + + group_ids = ["ST-C01"] + base_sha, wt_paths = _make_git_repo_with_worktrees(repo, group_ids) + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: branch) + + _register_worktrees(branch, base_sha, group_ids, wt_paths) + r_begin = map_step_runner.begin_wave_group(group_ids, branch) + group_key = str(r_begin["group_key"]) + + # Place a sentinel file inside .map/ — would be wiped by clean -fdx/-x. + sentinel = repo / ".map" / branch / "_sentinel_test.txt" + sentinel.write_text("do not delete", encoding="utf-8") + assert sentinel.exists(), "sentinel must exist before abort" + + map_step_runner.abort_wave_group(group_key, branch) + + assert sentinel.exists(), ( + "abort must NOT delete .map/ sentinel file — " + "_wt_rollback's -e .map exclusion must hold" + ) + + def test_vc4_hc4_post_abort_verify_clean_real_repo( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """VC4/HC-4: post-abort verify-clean on a real temp git repo. + + Confirms HEAD==recorded base_sha, clean tree, and zero group worktrees + after abort_wave_group on a real git repo where group worktrees existed. + """ + repo = tmp_path / "repo" + repo.mkdir() + branch = "test-branch" + (repo / ".map" / branch).mkdir(parents=True) + + group_ids = ["ST-D01", "ST-D02"] + base_sha, wt_paths = _make_git_repo_with_worktrees(repo, group_ids) + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: branch) + + _register_worktrees(branch, base_sha, group_ids, wt_paths) + r_begin = map_step_runner.begin_wave_group(group_ids, branch) + group_key = str(r_begin["group_key"]) + + result = map_step_runner.abort_wave_group(group_key, branch) + + # verify_group_clean fields. + assert "clean" in result, f"abort must return verify_group_clean fields: {result}" + # HEAD must equal base_sha after rollback. + import subprocess as _sp + head = _sp.run( + ["git", "rev-parse", "HEAD"], + cwd=str(repo), check=True, capture_output=True, text=True, + ).stdout.strip() + assert head == base_sha, ( + f"HEAD must equal base_sha={base_sha!r} after abort; got {head!r}" + ) + # Group must be gone from sidecar. + state = map_step_runner._read_worktree_state(branch) + wg = state.get("wave_groups") or {} + assert group_key not in wg, f"group must be absent from sidecar after abort: {wg}" + + def test_vc1_abort_on_unknown_group_is_safe( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """VC1: abort on an unknown group_id is safe (idempotent no-op).""" + repo = tmp_path / "repo" + repo.mkdir() + branch = "test-branch" + (repo / ".map" / branch).mkdir(parents=True) + _make_git_repo_for_group(repo) + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: branch) + + # Should not raise — group is simply not found, rollback skipped. + result = map_step_runner.abort_wave_group("NONEXISTENT-GROUP-KEY", branch) + assert "aborted_group_id" in result, f"must return aborted_group_id: {result}" + + +# --------------------------------------------------------------------------- +# Tests: record_dispatch_actual CLI — per-subtask base SHA collection (F8) +# --------------------------------------------------------------------------- + + +class TestRecordDispatchActualPerSubtaskSha: + """F8: record_dispatch_actual CLI must collect per-subtask base SHAs from worktree + sidecar records (not just the group-level sha) so classify_dispatch can reach + the isolation_violation outcome when two members have different base SHAs. + """ + + def test_vc1_f8_two_members_different_base_shas_yields_isolation_violation( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """F8: two group members with distinct per-subtask base_shas → isolation_violation. + + Verifies that the runner CLI's SHA-collection path reads EACH subtask's + worktree sidecar record (not only the group-level sha), so that + classify_dispatch receives >1 distinct SHA and returns isolation_violation. + """ + repo = tmp_path / "repo" + repo.mkdir() + _make_git_repo_for_group(repo) + branch = "test-branch" + (repo / ".map" / branch).mkdir(parents=True) + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: branch) + + group_ids = ["ST-SHA-A", "ST-SHA-B"] + group_key = "|".join(sorted(group_ids)) + + # Populate group lifecycle sidecar via begin_wave_group. + map_step_runner.begin_wave_group(group_ids, branch) + + # Write DISTINCT per-subtask base_shas into the worktrees sidecar record — + # simulating two worktrees that diverged from different HEAD commits (drift). + state = map_step_runner._read_worktree_state(branch) + if not isinstance(state.get("worktrees"), dict): + state["worktrees"] = {} + for sid, sha in zip(group_ids, ["deadbeef111", "cafebabe222"]): + slug = map_step_runner._wt_slug(sid) + if slug: + state["worktrees"][slug] = { + "subtask_id": sid, + "path": str(repo / ".worktrees" / slug), + "branch": f"map/wt/{slug}", + "base_sha": sha, # DIFFERENT sha per member — isolation drift! + "attempt": 0, + } + map_step_runner._write_worktree_state(branch, state) + + # Re-read state through the runner's own reader to get base_shas list. + state2 = map_step_runner._read_worktree_state(branch) + wave_groups = state2.get("wave_groups") or {} + group_data = wave_groups.get(group_key) + worktrees = state2.get("worktrees") or {} + + assert isinstance(group_data, dict), f"group_key {group_key!r} not in wave_groups" + + # Replicate the runner CLI's per-subtask SHA collection logic (record_dispatch_actual). + group_sids = group_data.get("subtask_ids", []) + group_level_sha = group_data.get("base_sha") + base_shas: list[str] = [] + for sid in group_sids: + slug = map_step_runner._wt_slug(sid) + wt_rec = worktrees.get(slug) if (isinstance(worktrees, dict) and slug) else None + if isinstance(wt_rec, dict): + per_sha = wt_rec.get("base_sha") + if isinstance(per_sha, str) and per_sha: + base_shas.append(per_sha) + continue + if isinstance(group_level_sha, str) and group_level_sha: + base_shas.append(group_level_sha) + + # Two members with different SHAs → classify_dispatch must return isolation_violation. + assert len(set(base_shas)) > 1, ( + f"Expected >1 distinct base_shas to trigger isolation_violation; " + f"got base_shas={base_shas!r}" + ) + from mapify_cli.parallelism_observability import ( + classify_dispatch as _classify, + DISPATCH_OUTCOME_ISOLATION_VIOLATION, + ) + outcome = _classify( + same_turn_task_count=2, + max_in_flight=2, + base_shas=base_shas, + skill_reported_concurrent=True, + ) + assert outcome == DISPATCH_OUTCOME_ISOLATION_VIOLATION, ( + f"Expected isolation_violation for base_shas={base_shas!r}; got {outcome!r}" + ) + + +# --------------------------------------------------------------------------- +# Tests: run_concurrent_wave — bounded retry + escalation (VC3, ST-006) +# --------------------------------------------------------------------------- + + +class TestRunConcurrentWaveRetryAndEscalation: + """VC3/F7/ST-006: run_concurrent_wave aborts ONCE on merge failure and returns + needs_redispatch=True. The skill owns the retry loop; the runner never retries + internally. Successive calls decrement attempts_remaining from the group sidecar. + """ + + def _make_repo_with_dispatch_enabled( + self, tmp_path: Path, max_retries: int = 3, max_actors: int = 4 + ) -> Path: + """Create a minimal git repo with concurrent_dispatch enabled.""" + repo = tmp_path / "repo" + repo.mkdir() + _make_git_repo_for_group(repo) + (repo / ".map" / "test-branch").mkdir(parents=True) + (repo / ".map" / "config.yaml").write_text( + f"execution.concurrent_dispatch: true\n" + f"execution.max_wave_retries: {max_retries}\n" + f"execution.max_actors: {max_actors}\n", + encoding="utf-8", + ) + return repo + + def test_vc3_f7_merge_failure_aborts_once_returns_needs_redispatch( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """VC3/F7: merge failure -> abort called ONCE, returns WAVE_ABORTED/needs_redispatch. + + No internal retry loop: merge_wave_worktrees is called exactly once and + abort_wave_group is called exactly once. The skill owns the retry loop. + """ + merge_call_count: list[int] = [0] + abort_call_count: list[int] = [0] + + def _always_fail(ids: list[str], branch: Any = None, **kw: Any) -> dict[str, Any]: + del ids, branch, kw + merge_call_count[0] += 1 + return {"status": "error", "ok": False, "kind": "WAVE_MERGE_CONFLICT", "message": "conflict"} + + def _fake_abort(group_id: str, branch: Any = None) -> dict[str, Any]: + del group_id, branch + abort_call_count[0] += 1 + return {"clean": True, "aborted_group_id": "fake"} + + monkeypatch.setattr(map_step_runner, "merge_wave_worktrees", _always_fail) + monkeypatch.setattr(map_step_runner, "abort_wave_group", _fake_abort) + + repo = self._make_repo_with_dispatch_enabled(tmp_path, max_retries=3) + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: "test-branch") + + result = map_step_runner.run_concurrent_wave( + ["ST-001", "ST-002"], "test-branch", repo + ) + + # F7: abort exactly once, no internal loop. + assert merge_call_count[0] == 1, ( + f"merge must be called exactly once (no internal retry); got {merge_call_count[0]}" + ) + assert abort_call_count[0] == 1, ( + f"abort must be called exactly once; got {abort_call_count[0]}" + ) + assert result.get("ok") is not True + assert result.get("kind") == "WAVE_ABORTED", ( + f"Expected WAVE_ABORTED; got {result.get('kind')!r}" + ) + assert result.get("needs_redispatch") is True + assert isinstance(result.get("attempts_remaining"), int) + + def test_vc3_attempts_remaining_decrements_with_config_max_retries( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """VC3: attempts_remaining decrements; with max_retries=2 first call → remaining=1.""" + def _always_fail(ids: list[str], branch: Any = None, **kw: Any) -> dict[str, Any]: + del ids, branch, kw + return {"status": "error", "ok": False, "kind": "WAVE_MERGE_CONFLICT", "message": "x"} + + def _fake_abort(group_id: str, branch: Any = None) -> dict[str, Any]: + del group_id, branch + return {"clean": True, "aborted_group_id": "fake"} + + monkeypatch.setattr(map_step_runner, "merge_wave_worktrees", _always_fail) + monkeypatch.setattr(map_step_runner, "abort_wave_group", _fake_abort) + + repo = self._make_repo_with_dispatch_enabled(tmp_path, max_retries=2) + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: "test-branch") + + result = map_step_runner.run_concurrent_wave( + ["ST-001", "ST-002"], "test-branch", repo + ) + + assert result.get("kind") == "WAVE_ABORTED" + assert result.get("needs_redispatch") is True + # First call uses attempt 1 of 2; 1 attempt remaining. + assert result.get("attempts_remaining") == 1, ( + f"expected attempts_remaining=1 after first failure; got {result}" + ) + + def test_vc3_escalation_when_attempts_remaining_zero( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """VC3: escalate_to_human=True when attempts_remaining reaches 0.""" + def _always_fail(ids: list[str], branch: Any = None, **kw: Any) -> dict[str, Any]: + del ids, branch, kw + return {"status": "error", "ok": False, "kind": "WAVE_MERGE_CONFLICT", "message": "x"} + + def _fake_abort(group_id: str, branch: Any = None) -> dict[str, Any]: + del group_id, branch + return {"clean": True, "aborted_group_id": "fake"} + + monkeypatch.setattr(map_step_runner, "merge_wave_worktrees", _always_fail) + monkeypatch.setattr(map_step_runner, "abort_wave_group", _fake_abort) + + # max_retries=1: first call uses the only attempt → escalate immediately. + repo = self._make_repo_with_dispatch_enabled(tmp_path, max_retries=1) + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: "test-branch") + + result = map_step_runner.run_concurrent_wave( + ["ST-001", "ST-002"], "test-branch", repo + ) + + assert result.get("kind") == "WAVE_ABORTED" + assert result.get("needs_redispatch") is True + assert result.get("attempts_remaining") == 0 + assert result.get("escalate_to_human") is True + + def test_vc3_f7_abort_once_returns_immediately_no_loop( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """VC3/F7: merge failure returns immediately (no internal loop). + + This test would time-out if run_concurrent_wave had an internal retry loop. + """ + call_count: list[int] = [0] + + def _always_fail(ids: list[str], branch: Any = None, **kw: Any) -> dict[str, Any]: + del ids, branch, kw + call_count[0] += 1 + return {"status": "error", "ok": False, "kind": "FAIL", "message": "x"} + + def _fake_abort(group_id: str, branch: Any = None) -> dict[str, Any]: + del group_id, branch + return {"clean": True, "aborted_group_id": "fake"} + + monkeypatch.setattr(map_step_runner, "merge_wave_worktrees", _always_fail) + monkeypatch.setattr(map_step_runner, "abort_wave_group", _fake_abort) + + repo = self._make_repo_with_dispatch_enabled(tmp_path, max_retries=5) + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: "test-branch") + + result = map_step_runner.run_concurrent_wave( + ["ST-001"], "test-branch", repo + ) + + # F7: abort ONCE and return immediately — no internal retry loop. + assert result.get("kind") == "WAVE_ABORTED" + assert call_count[0] == 1, ( + f"merge must be called exactly once (abort-once, no internal loop); " + f"got {call_count[0]}" + ) + assert result.get("needs_redispatch") is True + + +# --------------------------------------------------------------------------- +# F3: record_group_lifecycle lifecycle-lock serialization (T2) +# --------------------------------------------------------------------------- + + +class TestRecordGroupLifecycleLock: + """F3: record_group_lifecycle acquires the lifecycle lock BEFORE reading sidecar + state and releases it in the finally path, even when the write raises. + The lifecycle lock path is distinct from the merge lock path.""" + + def _setup_repo_and_group( + self, repo: Path, branch: str = "test-lc-lock" + ) -> str: + """Init git repo, sidecar dir, begin_wave_group; return group_key.""" + _make_git_repo_for_group(repo) + (repo / ".map" / branch).mkdir(parents=True) + r = map_step_runner.begin_wave_group(["ST-P01", "ST-P02"], branch) + assert r["ok"] is True, f"begin_wave_group failed: {r}" + return str(r["group_key"]) + + def test_f3_acquire_before_read_release_in_finally( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """F3: acquire called before sidecar read; release called in finally path + even if the write raises; lock paths are distinct.""" + repo = tmp_path / "repo" + repo.mkdir() + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: "test-lc-lock") + + gk = self._setup_repo_and_group(repo) + + call_log: list[str] = [] + + _orig_acquire = map_step_runner._wt_acquire_lifecycle_lock + _orig_release = map_step_runner._wt_release_lifecycle_lock + _orig_read = map_step_runner._read_worktree_state + + sentinel_handle: list[object] = [None] + + def _fake_acquire() -> object: + call_log.append("acquire") + handle = _orig_acquire() + sentinel_handle[0] = handle + return handle + + def _fake_release(handle: object) -> None: + call_log.append("release") + _orig_release(handle) + + def _fake_read(branch_name: str) -> dict[str, object]: + call_log.append("read") + return _orig_read(branch_name) + + monkeypatch.setattr(map_step_runner, "_wt_acquire_lifecycle_lock", _fake_acquire) + monkeypatch.setattr(map_step_runner, "_wt_release_lifecycle_lock", _fake_release) + monkeypatch.setattr(map_step_runner, "_read_worktree_state", _fake_read) + + result = map_step_runner.record_group_lifecycle( + gk, "ST-P01", "created", "test-lc-lock" + ) + assert result.get("ok") is True, f"unexpected failure: {result}" + + # (1) acquire must appear before the first read in call_log + assert "acquire" in call_log, "acquire must be called" + assert "read" in call_log, "read must be called" + acq_idx = call_log.index("acquire") + read_idx = call_log.index("read") + assert acq_idx < read_idx, ( + f"acquire must precede sidecar read; call_log={call_log}" + ) + + # (2) release must be present (finally path runs even on success) + assert "release" in call_log, ( + f"release must be called in the finally path; call_log={call_log}" + ) + + def test_f3_release_called_even_when_write_raises( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """F3: release is called in the finally path even when _write_worktree_state + raises — proves the try/finally structure is correct.""" + repo = tmp_path / "repo" + repo.mkdir() + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: "test-lc-lock-err") + + branch = "test-lc-lock-err" + _make_git_repo_for_group(repo) + (repo / ".map" / branch).mkdir(parents=True) + r = map_step_runner.begin_wave_group(["ST-Q01"], branch) + assert r["ok"] is True + gk = str(r["group_key"]) + + release_calls: list[int] = [] + + _orig_release = map_step_runner._wt_release_lifecycle_lock + + def _fake_release_counter(handle: object) -> None: + release_calls.append(1) + _orig_release(handle) + + def _bad_write(branch_name: str, state: dict[str, object]) -> None: + del branch_name, state + raise OSError("simulated write failure") + + monkeypatch.setattr(map_step_runner, "_wt_release_lifecycle_lock", _fake_release_counter) + monkeypatch.setattr(map_step_runner, "_write_worktree_state", _bad_write) + + # The write will raise — record_group_lifecycle should propagate the error + # but release must still have been called. + try: + map_step_runner.record_group_lifecycle(gk, "ST-Q01", "started", branch) + except OSError: + pass # expected — the write is patched to raise + + assert len(release_calls) >= 1, ( + "release must be called in the finally path even when write raises; " + f"release_calls={release_calls}" + ) + + def test_f3_lifecycle_lock_path_distinct_from_merge_lock_path( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """F3: _wt_lifecycle_lock_path() != _wt_merge_lock_path() — two distinct locks.""" + repo = tmp_path / "repo" + repo.mkdir() + _make_git_repo_for_group(repo) + monkeypatch.chdir(repo) + + lc_path = map_step_runner._wt_lifecycle_lock_path() + merge_path = map_step_runner._wt_merge_lock_path() + + # Both paths are non-None in a real git repo. + assert lc_path is not None, "_wt_lifecycle_lock_path() must return a Path in a git repo" + assert merge_path is not None, "_wt_merge_lock_path() must return a Path in a git repo" + assert lc_path != merge_path, ( + f"lifecycle lock path must be DISTINCT from merge lock path; " + f"lc={lc_path!r}, merge={merge_path!r}" + ) + + +# --------------------------------------------------------------------------- +# F5: abort_wave_group preserves group state when rollback fails (T3) +# --------------------------------------------------------------------------- + + +class TestAbortWaveGroupRollbackMismatch: + """F5: abort_wave_group returns rollback_head_mismatch and keeps the group entry + in the sidecar when _wt_head_sha returns a sha that differs from base_sha after + _wt_rollback is called — the group must NOT be deleted on rollback failure.""" + + def test_f5_rollback_head_mismatch_preserves_group_entry( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """F5: rollback succeeds physically but HEAD does not match base_sha => + reason=='rollback_head_mismatch', clean==False, group entry still in sidecar.""" + repo = tmp_path / "repo" + repo.mkdir() + branch = "test-branch" + (repo / ".map" / branch).mkdir(parents=True) + + group_ids = ["ST-R01", "ST-R02"] + base_sha, wt_paths = _make_git_repo_with_worktrees(repo, group_ids) + monkeypatch.chdir(repo) + monkeypatch.setattr(map_step_runner, "get_branch_name", lambda: branch) + + _register_worktrees(branch, base_sha, group_ids, wt_paths) + r_begin = map_step_runner.begin_wave_group(group_ids, branch) + assert r_begin["ok"] is True, f"begin_wave_group failed: {r_begin}" + group_key = str(r_begin["group_key"]) + + # Confirm group is in sidecar before abort. + state_before = map_step_runner._read_worktree_state(branch) + assert group_key in (state_before.get("wave_groups") or {}), ( + "group must exist in sidecar before abort" + ) + + # Patch _wt_rollback to a no-op (rollback does nothing), + # and _wt_head_sha to return a sha different from base_sha, + # simulating a rollback that silently failed to move HEAD. + fake_sha = "deadbeef" * 5 # 40-char hex, different from base_sha + assert fake_sha != base_sha, "sanity: fake_sha must differ from base_sha" + + def _noop_rollback(sha: str) -> None: + del sha # rollback is a no-op — HEAD stays at fake_sha + + def _fake_head_sha(cwd: "Path | None" = None) -> str: + del cwd + return fake_sha + + monkeypatch.setattr(map_step_runner, "_wt_rollback", _noop_rollback) + monkeypatch.setattr(map_step_runner, "_wt_head_sha", _fake_head_sha) + + result = map_step_runner.abort_wave_group(group_key, branch) + + # (1) reason must signal rollback failure + assert result.get("reason") == "rollback_head_mismatch", ( + f"expected reason='rollback_head_mismatch'; got {result}" + ) + + # (2) clean must be False + assert result.get("clean") is False, ( + f"clean must be False on rollback_head_mismatch; got {result}" + ) + + # (3) group entry must STILL exist in sidecar (not deleted on failure) + state_after = map_step_runner._read_worktree_state(branch) + wg_after = state_after.get("wave_groups") or {} + assert group_key in wg_after, ( + f"group entry must be PRESERVED in sidecar when rollback fails; " + f"group_key={group_key!r}, wave_groups={list(wg_after.keys())}" + ) diff --git a/tests/test_parallelism_observability.py b/tests/test_parallelism_observability.py index 297dc27f..1afe2045 100644 --- a/tests/test_parallelism_observability.py +++ b/tests/test_parallelism_observability.py @@ -1,15 +1,25 @@ -"""Tests for src/mapify_cli/parallelism_observability.py (ST-011). +"""Tests for src/mapify_cli/parallelism_observability.py. -Covers: +Pre-ST-003 coverage (original): - VC1: writer is no-op by default (no file created, returns False) - VC2: schema and reason-code constants are importable; ALL_REASON_CODES has all 9 codes; a sample ParallelismReport dict conforms to the TypedDict. - Parity: worktree reason-code constants match runner's _WT_REASON_* values. + +ST-003 additions (5b.2 classify_dispatch + record_dispatch_actual): + - VC1 [AC-3-TELEMETRY]: classify_dispatch truth table — all 6 outcomes. + - VC2 [AC-3-TELEMETRY]: max_in_flight replay from sorted lifecycle events is + deterministic; NO wall-clock involved. + - VC3 [AC-3-TELEMETRY]: record_dispatch_actual writes exactly ONE + ParallelismReport on concurrent path; emits NOTHING on no-op path. + - VC4 [AC-3-TELEMETRY]: isolation_violation when >1 distinct base_sha; + skill self-report never authoritative for positive concurrency claim. """ from __future__ import annotations import importlib.util +import json import sys from pathlib import Path @@ -22,7 +32,14 @@ sys.dont_write_bytecode = True from mapify_cli.parallelism_observability import ( # noqa: E402 + ALL_DISPATCH_OUTCOMES, ALL_REASON_CODES, + DISPATCH_OUTCOME_CONCURRENT_OBSERVED, + DISPATCH_OUTCOME_ISOLATION_VIOLATION, + DISPATCH_OUTCOME_PHANTOM_PARALLEL, + DISPATCH_OUTCOME_SAME_TURN_BUT_HOST_SEQUENTIAL, + DISPATCH_OUTCOME_SEQUENTIAL_OBSERVED, + DISPATCH_OUTCOME_UNKNOWN, REASON_DIRTY_MERGE_TARGET, REASON_DISPATCH_SERIAL, REASON_MERGE_CONFLICT, @@ -34,6 +51,8 @@ REASON_WORKTREE_UNSUPPORTED, ColorGroupDecision, ParallelismReport, + classify_dispatch, + record_dispatch_actual, write_parallelism_report, ) @@ -193,7 +212,12 @@ def _load_runner_module() -> object: # Stub out map_utils so the runner's top-level import doesn't abort. stub = types.ModuleType("map_utils") - stub.get_branch_name = lambda *a, **kw: "stub" # type: ignore[attr-defined] + + def _stub_branch_name(*_a: object, **_kw: object) -> str: + del _a, _kw # del is valid in a def body (illegal in a lambda) + return "stub" + + stub.get_branch_name = _stub_branch_name # type: ignore[attr-defined] injected = "map_utils" not in sys.modules if injected: sys.modules["map_utils"] = stub @@ -241,3 +265,311 @@ def test_worktree_reason_codes_match_runner() -> None: f" runner.{runner_attr} = {runner_value!r}\n" "Update the observability module or the runner to restore parity." ) + + +# --------------------------------------------------------------------------- +# ST-003 (5b.2): classify_dispatch + record_dispatch_actual +# --------------------------------------------------------------------------- + + +class TestClassifyDispatch: + """Truth-table tests for the evidence-hierarchy classifier (VC1, VC4).""" + + # VC4: isolation_violation when >1 distinct base_sha (rule 1, highest priority) + def test_vc4_isolation_violation_two_distinct_shas(self) -> None: + outcome = classify_dispatch( + same_turn_task_count=2, + max_in_flight=2, + base_shas=["sha-aaa", "sha-bbb"], # >1 distinct SHA + skill_reported_concurrent=False, + ) + assert outcome == DISPATCH_OUTCOME_ISOLATION_VIOLATION + + def test_vc4_isolation_violation_three_distinct_shas(self) -> None: + outcome = classify_dispatch( + same_turn_task_count=3, + max_in_flight=3, + base_shas=["sha-aaa", "sha-bbb", "sha-ccc"], + skill_reported_concurrent=True, + ) + assert outcome == DISPATCH_OUTCOME_ISOLATION_VIOLATION + + # VC1: concurrent_observed when same-turn ≥2 AND max_in_flight ≥2 (rule 2) + def test_vc1_concurrent_observed_n2(self) -> None: + outcome = classify_dispatch( + same_turn_task_count=2, + max_in_flight=2, + base_shas=["sha-aaa", "sha-aaa"], # all same — no isolation violation + skill_reported_concurrent=False, + ) + assert outcome == DISPATCH_OUTCOME_CONCURRENT_OBSERVED + + def test_vc1_concurrent_observed_n5(self) -> None: + outcome = classify_dispatch( + same_turn_task_count=5, + max_in_flight=3, + base_shas=["sha-aaa"] * 5, + skill_reported_concurrent=True, + ) + assert outcome == DISPATCH_OUTCOME_CONCURRENT_OBSERVED + + def test_vc1_concurrent_observed_large_n(self) -> None: + outcome = classify_dispatch( + same_turn_task_count=10, + max_in_flight=10, + base_shas=["sha-xxx"] * 10, + skill_reported_concurrent=False, + ) + assert outcome == DISPATCH_OUTCOME_CONCURRENT_OBSERVED + + # VC1: same_turn_but_host_sequential when same-turn ≥2 AND max_in_flight ≤1 (rule 3) + def test_vc1_same_turn_but_host_sequential_n2(self) -> None: + outcome = classify_dispatch( + same_turn_task_count=2, + max_in_flight=1, + base_shas=["sha-aaa", "sha-aaa"], + skill_reported_concurrent=False, + ) + assert outcome == DISPATCH_OUTCOME_SAME_TURN_BUT_HOST_SEQUENTIAL + + def test_vc1_same_turn_but_host_sequential_n3_zero_inflight(self) -> None: + outcome = classify_dispatch( + same_turn_task_count=3, + max_in_flight=0, + base_shas=[], + skill_reported_concurrent=False, + ) + assert outcome == DISPATCH_OUTCOME_SAME_TURN_BUT_HOST_SEQUENTIAL + + # VC4: skill self-report NOT authoritative for positive concurrency (phantom path) + def test_vc4_phantom_parallel_skill_reported_but_same_turn_one(self) -> None: + # skill self-reported concurrent but same_turn_task_count ≤1 (rule 4) + outcome = classify_dispatch( + same_turn_task_count=1, + max_in_flight=0, + base_shas=["sha-aaa"], + skill_reported_concurrent=True, # self-report present but NOT authoritative + ) + assert outcome == DISPATCH_OUTCOME_PHANTOM_PARALLEL + + def test_vc4_phantom_parallel_skill_reported_but_same_turn_zero(self) -> None: + outcome = classify_dispatch( + same_turn_task_count=0, + max_in_flight=0, + base_shas=[], + skill_reported_concurrent=True, + ) + assert outcome == DISPATCH_OUTCOME_PHANTOM_PARALLEL + + def test_vc4_phantom_parallel_requires_both_signals_low(self) -> None: + # F1 guard: skill_reported=True, same_turn<=1, BUT max_in_flight>=2. + # Both objective signals must confirm ≤1 for phantom_parallel; contradictory + # evidence (skill_reported + same_turn<=1 BUT max_in_flight>=2) → unknown. + outcome = classify_dispatch( + same_turn_task_count=1, + max_in_flight=2, + base_shas=["sha-aaa"], + skill_reported_concurrent=True, + ) + assert outcome == DISPATCH_OUTCOME_UNKNOWN, ( + f"Contradictory evidence (skill_reported + same_turn<=1 + max_in_flight>=2) " + f"must resolve to unknown, not phantom_parallel; got {outcome!r}" + ) + + # sequential_observed when same-turn ≤1 AND max_in_flight ≤1 (rule 5) + def test_vc3_sequential_observed_single_task(self) -> None: + outcome = classify_dispatch( + same_turn_task_count=1, + max_in_flight=1, + base_shas=["sha-aaa"], + skill_reported_concurrent=False, + ) + assert outcome == DISPATCH_OUTCOME_SEQUENTIAL_OBSERVED + + def test_vc3_sequential_observed_zero_tasks(self) -> None: + outcome = classify_dispatch( + same_turn_task_count=0, + max_in_flight=0, + base_shas=[], + skill_reported_concurrent=False, + ) + assert outcome == DISPATCH_OUTCOME_SEQUENTIAL_OBSERVED + + # No-op subtask (no commit/no events) must NOT trigger phantom_parallel (VC3) + def test_vc3_noop_subtask_no_phantom_alarm(self) -> None: + # No lifecycle events recorded → same_turn=0, max_in_flight=0, + # skill_reported=False → should be sequential_observed, not phantom_parallel. + outcome = classify_dispatch( + same_turn_task_count=0, + max_in_flight=0, + base_shas=[], + skill_reported_concurrent=False, + ) + assert outcome == DISPATCH_OUTCOME_SEQUENTIAL_OBSERVED + assert outcome != DISPATCH_OUTCOME_PHANTOM_PARALLEL + + # ALL_DISPATCH_OUTCOMES completeness — all 6 outcomes present + def test_all_dispatch_outcomes_completeness(self) -> None: + expected = { + DISPATCH_OUTCOME_CONCURRENT_OBSERVED, + DISPATCH_OUTCOME_SAME_TURN_BUT_HOST_SEQUENTIAL, + DISPATCH_OUTCOME_PHANTOM_PARALLEL, + DISPATCH_OUTCOME_SEQUENTIAL_OBSERVED, + DISPATCH_OUTCOME_ISOLATION_VIOLATION, + DISPATCH_OUTCOME_UNKNOWN, + } + assert ALL_DISPATCH_OUTCOMES == expected + assert len(ALL_DISPATCH_OUTCOMES) == 6 + + +class TestMaxInFlightReplay: + """VC2: max_in_flight derived from sorted lifecycle events — NO wall-clock (HC-5).""" + + def _compute_max_in_flight(self, events: list[dict]) -> int: + """Mirror the runner's deterministic sweep: sort by seq, count started/finished.""" + sorted_evs = sorted(events, key=lambda e: int(e.get("seq", 0))) + in_flight = 0 + max_in_flight = 0 + for ev in sorted_evs: + ev_type = ev.get("event", "") + if ev_type == "started": + in_flight += 1 + if in_flight > max_in_flight: + max_in_flight = in_flight + elif ev_type == "finished": + in_flight = max(0, in_flight - 1) + return max_in_flight + + def test_vc2_two_concurrent_tasks_max_inflight_2(self) -> None: + # Two tasks: both start before either finishes → max_in_flight = 2. + events = [ + {"seq": 1, "event": "started", "ts": 0.0}, + {"seq": 2, "event": "started", "ts": 0.0}, # ts irrelevant — seq only + {"seq": 3, "event": "finished", "ts": 0.0}, + {"seq": 4, "event": "finished", "ts": 0.0}, + ] + assert self._compute_max_in_flight(events) == 2 + + def test_vc2_sequential_tasks_max_inflight_1(self) -> None: + # Two tasks serially: start/finish/start/finish → max_in_flight = 1. + events = [ + {"seq": 1, "event": "started", "ts": 999.0}, # ts must be ignored + {"seq": 2, "event": "finished", "ts": 999.1}, + {"seq": 3, "event": "started", "ts": 999.2}, + {"seq": 4, "event": "finished", "ts": 999.3}, + ] + assert self._compute_max_in_flight(events) == 1 + + def test_vc2_out_of_order_delivery_still_deterministic(self) -> None: + # Events delivered out of seq order — sort by seq must normalise. + events = [ + {"seq": 4, "event": "finished", "ts": 0.0}, + {"seq": 1, "event": "started", "ts": 0.0}, + {"seq": 3, "event": "started", "ts": 0.0}, + {"seq": 2, "event": "finished", "ts": 0.0}, + ] + # seq order: started(1), finished(2), started(3), finished(4) → serial → 1 + assert self._compute_max_in_flight(events) == 1 + + def test_vc2_three_concurrent_tasks_max_inflight_3(self) -> None: + events = [ + {"seq": 1, "event": "started", "ts": 0.0}, + {"seq": 2, "event": "started", "ts": 0.0}, + {"seq": 3, "event": "started", "ts": 0.0}, + {"seq": 4, "event": "finished", "ts": 0.0}, + {"seq": 5, "event": "finished", "ts": 0.0}, + {"seq": 6, "event": "finished", "ts": 0.0}, + ] + assert self._compute_max_in_flight(events) == 3 + + def test_vc2_no_events_zero_max_inflight(self) -> None: + assert self._compute_max_in_flight([]) == 0 + + def test_vc2_classify_concurrent_after_replay(self) -> None: + """Full integration: replay → max_in_flight → classify → concurrent_observed.""" + events = [ + {"seq": 1, "event": "started", "ts": 0.0}, + {"seq": 2, "event": "started", "ts": 0.0}, + {"seq": 3, "event": "finished", "ts": 0.0}, + {"seq": 4, "event": "finished", "ts": 0.0}, + ] + max_in_flight = self._compute_max_in_flight(events) + outcome = classify_dispatch( + same_turn_task_count=2, + max_in_flight=max_in_flight, + base_shas=["sha-abc", "sha-abc"], + skill_reported_concurrent=False, + ) + assert max_in_flight == 2 + assert outcome == DISPATCH_OUTCOME_CONCURRENT_OBSERVED + + +class TestRecordDispatchActual: + """VC3: record_dispatch_actual writes exactly ONE report on concurrent path; + emits NOTHING on any other outcome (no file created, returns False).""" + + def _make_report(self, run_id: str = "test-run-001") -> ParallelismReport: + group: ColorGroupDecision = { + "group_id": "ST-X|ST-Y", + "planned_mode": "concurrent", + "actual_mode": "concurrent_observed", + "worktree_status": "ok", + "reason_code": None, + "dispatch_count": 2, + } + report: ParallelismReport = { + "schema_version": "1.0.0", + "run_id": run_id, + "generated_at": "2026-06-29T00:00:00Z", + "total_subtasks": 2, + "total_edges": 0, + "total_waves": 1, + "max_wave_width": 2, + "color_group_breakdown": [group], + } + return report + + # Concurrent path: one file written, returns True + def test_vc3_concurrent_path_writes_exactly_one_file(self, tmp_path: Path) -> None: + out = tmp_path / "parallelism.json" + result = record_dispatch_actual(self._make_report(), out, DISPATCH_OUTCOME_CONCURRENT_OBSERVED) + assert result is True + assert out.exists(), "parallelism.json must be written on concurrent path" + data = json.loads(out.read_text()) + assert data["run_id"] == "test-run-001" + assert data["schema_version"] == "1.0.0" + + # No-op paths: no file, returns False for every non-concurrent outcome + @pytest.mark.parametrize("outcome", [ + DISPATCH_OUTCOME_SAME_TURN_BUT_HOST_SEQUENTIAL, + DISPATCH_OUTCOME_PHANTOM_PARALLEL, + DISPATCH_OUTCOME_SEQUENTIAL_OBSERVED, + DISPATCH_OUTCOME_ISOLATION_VIOLATION, + DISPATCH_OUTCOME_UNKNOWN, + ]) + def test_vc3_noop_on_non_concurrent_outcome(self, tmp_path: Path, outcome: str) -> None: + out = tmp_path / "parallelism.json" + result = record_dispatch_actual(self._make_report(), out, outcome) + assert result is False, f"Expected no-op for outcome={outcome!r}" + assert not out.exists(), ( + f"parallelism.json must NOT be created for outcome={outcome!r}" + ) + + # Sequential no-op path explicitly named (HC-1 dormancy contract) + def test_vc3_sequential_default_emits_nothing(self, tmp_path: Path) -> None: + out = tmp_path / "runs" / "r1" / "parallelism.json" + result = record_dispatch_actual( + self._make_report(), out, DISPATCH_OUTCOME_SEQUENTIAL_OBSERVED + ) + assert result is False + assert not out.exists() + # Parent dir must also not be created (truly no-op) + assert not (tmp_path / "runs").exists() + + # Calling twice on concurrent path must not duplicate content (idempotent overwrite) + def test_vc3_concurrent_path_second_write_overwrites(self, tmp_path: Path) -> None: + out = tmp_path / "parallelism.json" + record_dispatch_actual(self._make_report("run-A"), out, DISPATCH_OUTCOME_CONCURRENT_OBSERVED) + record_dispatch_actual(self._make_report("run-B"), out, DISPATCH_OUTCOME_CONCURRENT_OBSERVED) + data = json.loads(out.read_text()) + assert data["run_id"] == "run-B" # last write wins; one file still present diff --git a/tests/test_project_config.py b/tests/test_project_config.py index 224ae2b2..372d9dd3 100644 --- a/tests/test_project_config.py +++ b/tests/test_project_config.py @@ -1,9 +1,13 @@ -"""ST-005: MapConfig dormant fields max_actors + retry_degraded_once, and clamp helper. +"""ST-005: MapConfig fields max_actors + retry_degraded_once, and clamp helper. +ST-000: MapConfig fields concurrent_dispatch + max_wave_retries (5b config). Covers: VC1 — dotted-key aliasing and field parsing from YAML VC2 — clamp_max_actors truth table - VC3 — no runner/orchestrator .jinja in templates_src reads these fields + VC3 — max_actors is ACTIVE in runner/orchestrator (Slice 5b); retry_degraded_once remains dormant + VC4 (ST-000) — concurrent_dispatch and max_wave_retries parsed/clamped correctly + VC5 (ST-000) — clamp_max_wave_retries truth table + VC6 (ST-000) — concurrent_dispatch + max_wave_retries are ACTIVE in runner/orchestrator (Slice 5b) """ from __future__ import annotations @@ -14,6 +18,7 @@ from mapify_cli.config.project_config import ( MapConfig, clamp_max_actors, + clamp_max_wave_retries, load_map_config, ) @@ -111,7 +116,9 @@ def test_vc2_boundary_values(self): class TestVc3DormantKeysUnused: - """VC3: no execution path in templates_src runner/orchestrator reads the fields.""" + """VC3 (updated for Slice 5b): max_actors is now ACTIVE in runner/orchestrator. + retry_degraded_once remains DORMANT (Slice 6+). + """ def _grep_templates_src(self, field_name: str) -> list[str]: """Return lines from runner/orchestrator .jinja files that reference field_name.""" @@ -131,11 +138,12 @@ def _grep_templates_src(self, field_name: str) -> list[str]: matches.append(f"{jinja_file}:{lineno}: {line.rstrip()}") return matches - def test_vc3_max_actors_not_consumed_in_runner_orchestrator(self): + def test_vc3_max_actors_consumed_in_runner_orchestrator(self): + """Slice 5b activated max_actors: runner/orchestrator .jinja must reference it.""" hits = self._grep_templates_src("max_actors") - assert hits == [], ( - "max_actors is DORMANT in Slice 5a — no runner/orchestrator .jinja " - "should reference it yet.\nFound:\n" + "\n".join(hits) + assert hits != [], ( + "max_actors should be ACTIVE in Slice 5b — runner/orchestrator .jinja " + "must reference it (run_concurrent_wave / _max_actors helper)." ) def test_vc3_retry_degraded_once_not_consumed_in_runner_orchestrator(self): @@ -151,14 +159,8 @@ def test_vc3_field_defined_in_project_config(self): assert hasattr(cfg, "max_actors") assert hasattr(cfg, "retry_degraded_once") - def test_vc3_grep_subprocess_confirms_no_runner_orchestrator_consumer(self): - """Subprocess grep across runner/orchestrator/step_runner Python sources. - - VC3 dormant means: no execution dispatch path reads the fields. - Documentation files (.md, .md.jinja) and observability modules are - permitted to mention max_actors by name; only the runner/orchestrator - execution paths are forbidden in Slice 5a. - """ + def test_vc3_grep_subprocess_confirms_runner_orchestrator_consumer(self): + """Subprocess grep: runner/orchestrator Python sources now consume max_actors (Slice 5b active).""" src_root = Path(__file__).parent.parent / "src" / "mapify_cli" result = subprocess.run( ["grep", "-rl", "max_actors", str(src_root)], @@ -168,16 +170,215 @@ def test_vc3_grep_subprocess_confirms_no_runner_orchestrator_consumer(self): files_with_max_actors = [ line for line in result.stdout.splitlines() if line.strip() ] - # Runner/orchestrator Python source files are forbidden in Slice 5a. - # Documentation (.md, .jinja) and observability modules are allowed. - _FORBIDDEN_STEMS = ("runner", "orchestrator", "step_runner", "wave_coordinator") - forbidden = [ + # Slice 5b: max_actors must be consumed by at least one runner/orchestrator source. + _ACTIVE_STEMS = ("runner", "orchestrator", "step_runner", "wave_coordinator") + active = [ f for f in files_with_max_actors - if any(stem in Path(f).stem for stem in _FORBIDDEN_STEMS) + if any(stem in Path(f).stem for stem in _ACTIVE_STEMS) + and f.endswith(".py") + ] + assert active != [], ( + "max_actors not found in any runner/orchestrator Python source after Slice 5b — " + "expected _max_actors() or run_concurrent_wave() to consume it." + ) + + +# --------------------------------------------------------------------------- +# ST-000: concurrent_dispatch + max_wave_retries (5b.0 config half) +# --------------------------------------------------------------------------- + + +class TestVc4ConcurrentDispatchAndMaxWaveRetries: + """VC4: dotted-key aliases parsed; type/clamp/default behaviour correct.""" + + def test_vc4_concurrent_dispatch_true_from_yaml(self, tmp_path: Path): + _write_config(tmp_path, "execution.concurrent_dispatch: true\n") + cfg = load_map_config(tmp_path) + assert cfg.concurrent_dispatch is True + + def test_vc4_concurrent_dispatch_default_false(self, tmp_path: Path): + cfg = load_map_config(tmp_path) + assert cfg.concurrent_dispatch is False + + def test_vc4_concurrent_dispatch_false_from_yaml(self, tmp_path: Path): + _write_config(tmp_path, "execution.concurrent_dispatch: false\n") + cfg = load_map_config(tmp_path) + assert cfg.concurrent_dispatch is False + + def test_vc4_max_wave_retries_from_yaml(self, tmp_path: Path): + _write_config(tmp_path, "execution.max_wave_retries: 7\n") + cfg = load_map_config(tmp_path) + assert cfg.max_wave_retries == 7 + + def test_vc4_max_wave_retries_default_three(self, tmp_path: Path): + cfg = load_map_config(tmp_path) + assert cfg.max_wave_retries == 3 + + def test_vc4_max_wave_retries_zero_clamped_to_1(self, tmp_path: Path): + _write_config(tmp_path, "execution.max_wave_retries: 0\n") + cfg = load_map_config(tmp_path) + assert cfg.max_wave_retries == 1 + + def test_vc4_max_wave_retries_above_ceiling_clamped(self, tmp_path: Path): + _write_config(tmp_path, "execution.max_wave_retries: 99\n") + cfg = load_map_config(tmp_path) + assert cfg.max_wave_retries == 10 + + def test_vc4_max_wave_retries_string_falls_back_to_default(self, tmp_path: Path): + _write_config(tmp_path, 'execution.max_wave_retries: "abc"\n') + cfg = load_map_config(tmp_path) + # Generic type-check loop rejects non-int; clamp_max_wave_retries on the + # default value returns 3. + assert cfg.max_wave_retries == 3 + + def test_vc4_max_wave_retries_bool_falls_back_to_default(self, tmp_path: Path): + # YAML `true` parses as bool True; the generic type-check loop rejects it + # (bool != int at the expected_type check), so the field keeps its default. + _write_config(tmp_path, "execution.max_wave_retries: true\n") + cfg = load_map_config(tmp_path) + assert cfg.max_wave_retries == 3 + + def test_vc4_dotted_alias_not_a_dead_toggle_concurrent_dispatch( + self, tmp_path: Path + ): + """Alias fires BEFORE the generic mapping loop — not a silent dead toggle.""" + _write_config(tmp_path, "execution.concurrent_dispatch: true\n") + cfg = load_map_config(tmp_path) + assert cfg.concurrent_dispatch is True, ( + "execution.concurrent_dispatch alias is a dead toggle — " + "add it to the dotted-key alias list BEFORE the mapping loop" + ) + + def test_vc4_dotted_alias_not_a_dead_toggle_max_wave_retries( + self, tmp_path: Path + ): + """Alias fires BEFORE the generic mapping loop — not a silent dead toggle.""" + _write_config(tmp_path, "execution.max_wave_retries: 5\n") + cfg = load_map_config(tmp_path) + assert cfg.max_wave_retries == 5, ( + "execution.max_wave_retries alias is a dead toggle — " + "add it to the dotted-key alias list BEFORE the mapping loop" + ) + + def test_vc4_fields_exist_on_mapconfig(self): + """Positive proof: both new fields exist on MapConfig.""" + cfg = MapConfig() + assert hasattr(cfg, "concurrent_dispatch") + assert hasattr(cfg, "max_wave_retries") + + +class TestVc5ClampMaxWaveRetries: + """VC5: clamp_max_wave_retries truth table.""" + + def test_vc5_valid_range_preserved(self): + for n in range(1, 11): + assert clamp_max_wave_retries(n) == n, f"expected {n} for input {n}" + + def test_vc5_below_floor_clamped_to_1(self): + assert clamp_max_wave_retries(0) == 1 + assert clamp_max_wave_retries(-5) == 1 + + def test_vc5_above_ceiling_clamped_to_10(self): + assert clamp_max_wave_retries(11) == 10 + assert clamp_max_wave_retries(100) == 10 + + def test_vc5_none_returns_default(self): + assert clamp_max_wave_retries(None) == 3 + + def test_vc5_string_returns_default(self): + assert clamp_max_wave_retries("3") == 3 + assert clamp_max_wave_retries("bad") == 3 + + def test_vc5_bool_returns_default(self): + # bool is a subclass of int in Python; clamp_max_wave_retries explicitly + # excludes it so a YAML boolean is treated as misconfiguration → default. + assert clamp_max_wave_retries(True) == 3 + assert clamp_max_wave_retries(False) == 3 + + def test_vc5_float_returns_default(self): + assert clamp_max_wave_retries(3.0) == 3 + + def test_vc5_boundary_values(self): + assert clamp_max_wave_retries(1) == 1 + assert clamp_max_wave_retries(10) == 10 + + +class TestVc6DormantFieldsUnused5b0: + """VC6 (updated for Slice 5b): concurrent_dispatch and max_wave_retries are now ACTIVE. + + Slice 5b activated concurrent dispatch; these fields must now be consumed by + runner/orchestrator .jinja and compiled Python sources. + """ + + def _grep_templates_src(self, field_name: str) -> list[str]: + """Return lines from runner/orchestrator .jinja files that reference field_name.""" + if not _TEMPLATES_SRC.exists(): + return [] + matches = [] + for jinja_file in _TEMPLATES_SRC.rglob("*.jinja"): + if not any( + tag in jinja_file.name + for tag in ("runner", "orchestrator", "step_runner", "wave") + ): + continue + content = jinja_file.read_text(encoding="utf-8") + for lineno, line in enumerate(content.splitlines(), 1): + if field_name in line: + matches.append(f"{jinja_file}:{lineno}: {line.rstrip()}") + return matches + + def test_vc6_concurrent_dispatch_consumed_in_runner_orchestrator(self): + """Slice 5b: concurrent_dispatch must be referenced in runner/orchestrator .jinja.""" + hits = self._grep_templates_src("concurrent_dispatch") + assert hits != [], ( + "concurrent_dispatch should be ACTIVE in Slice 5b — runner/orchestrator " + ".jinja must reference it (compute_dispatch_gate / _concurrent_dispatch_enabled)." + ) + + def test_vc6_max_wave_retries_consumed_in_runner_orchestrator(self): + """Slice 5b: max_wave_retries must be referenced in runner/orchestrator .jinja.""" + hits = self._grep_templates_src("max_wave_retries") + assert hits != [], ( + "max_wave_retries should be ACTIVE in Slice 5b — runner/orchestrator " + ".jinja must reference it (_max_wave_retries helper / abort_wave_group)." + ) + + def test_vc6_grep_subprocess_runner_consumer_concurrent_dispatch(self): + """Subprocess grep: runner/orchestrator Python sources now consume concurrent_dispatch.""" + src_root = Path(__file__).parent.parent / "src" / "mapify_cli" + result = subprocess.run( + ["grep", "-rl", "concurrent_dispatch", str(src_root)], + capture_output=True, + text=True, + ) + files_with_field = [line for line in result.stdout.splitlines() if line.strip()] + _ACTIVE_STEMS = ("runner", "orchestrator", "step_runner", "wave_coordinator") + active = [ + f for f in files_with_field + if any(stem in Path(f).stem for stem in _ACTIVE_STEMS) + and f.endswith(".py") + ] + assert active != [], ( + "concurrent_dispatch not found in any runner/orchestrator Python source after " + "Slice 5b — expected compute_dispatch_gate or _concurrent_dispatch_enabled." + ) + + def test_vc6_grep_subprocess_runner_consumer_max_wave_retries(self): + """Subprocess grep: runner/orchestrator Python sources now consume max_wave_retries.""" + src_root = Path(__file__).parent.parent / "src" / "mapify_cli" + result = subprocess.run( + ["grep", "-rl", "max_wave_retries", str(src_root)], + capture_output=True, + text=True, + ) + files_with_field = [line for line in result.stdout.splitlines() if line.strip()] + _ACTIVE_STEMS = ("runner", "orchestrator", "step_runner", "wave_coordinator") + active = [ + f for f in files_with_field + if any(stem in Path(f).stem for stem in _ACTIVE_STEMS) and f.endswith(".py") ] - assert forbidden == [], ( - "max_actors found in runner/orchestrator Python sources in Slice 5a " - "(DORMANT violation — field must not be consumed until Slice 5b):\n" - + "\n".join(forbidden) + assert active != [], ( + "max_wave_retries not found in any runner/orchestrator Python source after " + "Slice 5b — expected _max_wave_retries() or abort_wave_group() to consume it." ) diff --git a/tests/test_slice5b_leak_guards.py b/tests/test_slice5b_leak_guards.py new file mode 100644 index 00000000..7813e790 --- /dev/null +++ b/tests/test_slice5b_leak_guards.py @@ -0,0 +1,563 @@ +"""HC-1 leak-guard proof suite for Slice 5b concurrent dispatch (ST-008). + +Five non-tautological guards proving default-off (concurrent_dispatch=false) +is behavior-neutral / byte-identical to Slice 5a: + +(a) prompt confinement guard — fanout instruction tokens exist ONLY inside the + prose-gated 'Slice 5b' section, NOT in default/sequential paragraphs. +(b) monkeypatch-fail guard — concurrent runner verbs do NOT fire under default config. +(c) AST/static-import guard — sequential walker + flag-false branch contain NO + references to concurrent runner verbs. +(d) no-telemetry guard — no parallelism.json is created on the default path. +(e) default-off baseline — get_wave_step returns dispatch_mode=='sequential' with + the 5a reason code; concurrency_enabled==False. +""" + +from __future__ import annotations + +import ast +import json +import re +import sys +from pathlib import Path + +import pytest + +# --------------------------------------------------------------------------- +# Suppress bytecode pollution in generated trees (learned rule). +# --------------------------------------------------------------------------- +sys.dont_write_bytecode = True + +# --------------------------------------------------------------------------- +# Paths +# --------------------------------------------------------------------------- + +_REPO_ROOT = Path(__file__).resolve().parents[1] +_SCRIPTS_PATH = ( + _REPO_ROOT / "src" / "mapify_cli" / "templates" / "map" / "scripts" +) +_ORCHESTRATOR_PATH = _SCRIPTS_PATH / "map_orchestrator.py" +_RUNNER_PATH = _SCRIPTS_PATH / "map_step_runner.py" +_REFERENCE_PATH = ( + _REPO_ROOT / ".claude" / "skills" / "map-efficient" / "efficient-reference.md" +) + +# --------------------------------------------------------------------------- +# Add scripts dir so "import map_step_runner" inside map_orchestrator resolves. +# --------------------------------------------------------------------------- +if str(_SCRIPTS_PATH) not in sys.path: + sys.path.insert(0, str(_SCRIPTS_PATH)) + +import map_orchestrator # noqa: E402 # pyright: ignore[reportMissingImports] + + +# --------------------------------------------------------------------------- +# Fixtures +# --------------------------------------------------------------------------- + +@pytest.fixture +def branch_workspace(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> str: + """Minimal .map// directory with NO config.yaml → default config. + + No config.yaml → _concurrent_dispatch_enabled returns False (default off). + """ + branch = "test-leak-guards" + map_branch_dir = tmp_path / ".map" / branch + map_branch_dir.mkdir(parents=True) + monkeypatch.chdir(tmp_path) + monkeypatch.setattr(map_orchestrator, "get_branch_name", lambda: branch) + return branch + + +def _write_parallel_blueprint(branch: str) -> str: + """Write a blueprint in the correct list-of-dicts format and call set_waves. + + Returns the branch name. The blueprint has two waves: + wave 0: [ST-001] (single, sequential) + wave 1: [ST-002, ST-003] (parallel group) + """ + bp_file = Path(f".map/{branch}/blueprint.json") + blueprint = { + "subtasks": [ + {"id": "ST-001", "dependencies": [], "affected_files": ["models.py"]}, + {"id": "ST-002", "dependencies": ["ST-001"], "affected_files": ["views.py"]}, + {"id": "ST-003", "dependencies": ["ST-001"], "affected_files": ["urls.py"]}, + ] + } + bp_file.write_text(json.dumps(blueprint), encoding="utf-8") + map_orchestrator.set_waves(branch, str(bp_file)) + return branch + + +@pytest.fixture +def branch_with_waves(branch_workspace: str) -> str: + """branch_workspace extended with parallel waves.""" + return _write_parallel_blueprint(branch_workspace) + + +# =========================================================================== +# Guard (a): Prompt confinement guard +# =========================================================================== + +class TestGuardA_PromptConfinement: + """Fanout instruction tokens must appear ONLY inside the prose-gated Slice 5b + section of efficient-reference.md, NOT in default/sequential paragraphs. + + What break turns this red: + - If "one assistant message" (the core fanout instruction) leaks outside the + gated section into the sequential-dispatch docs, the test fires. + - If "emit all N Task" (the canonical fanout verb) leaks outside the gate, the + second test fires. + - Guard is non-tautological: we first prove the tokens ARE inside the gate, so + a vacuous pass (tokens missing from both inside and outside) is impossible. + """ + + def _extract_gated_section(self, content: str) -> tuple[str, str]: + """Split content into (inside_gated_section, rest_of_doc). + + The gated section starts at the '### Concurrent Actor dispatch — **Slice 5b**' + heading and ends at the next '###' heading at the same level (or EOF). + """ + start_match = re.search( + r"^### Concurrent Actor dispatch.*Slice 5b.*$", + content, + re.MULTILINE, + ) + assert start_match is not None, ( + "Slice 5b gated section heading not found in efficient-reference.md. " + "The gate marker was removed — guard (a) cannot validate confinement." + ) + start = start_match.start() + + next_h3 = re.search(r"^### ", content[start_match.end():], re.MULTILINE) + if next_h3 is None: + end = len(content) + else: + end = start_match.end() + next_h3.start() + + inside = content[start:end] + outside = content[:start] + content[end:] + return inside, outside + + def test_vc1a_one_assistant_message_inside_gated_section(self) -> None: + """'one assistant message' must appear inside the Slice 5b gated section. + + Non-tautological: proves the token IS in the gate so the outside-check + cannot pass vacuously. + """ + if not _REFERENCE_PATH.exists(): + pytest.skip(f"Reference file not found: {_REFERENCE_PATH}") + + content = _REFERENCE_PATH.read_text(encoding="utf-8") + inside, _ = self._extract_gated_section(content) + + assert "one assistant message" in inside, ( + "Expected 'one assistant message' not found inside the Slice 5b gated " + "section. Was the fanout instruction removed from the gate?" + ) + + def test_vc1a_one_assistant_message_not_in_sequential_docs(self) -> None: + """'one assistant message' must NOT appear outside the Slice 5b gate. + + Failure scenario: if this token leaks into a sequential-dispatch paragraph, + operators following default config get concurrent instructions — HC-1 violation. + """ + if not _REFERENCE_PATH.exists(): + pytest.skip(f"Reference file not found: {_REFERENCE_PATH}") + + content = _REFERENCE_PATH.read_text(encoding="utf-8") + _, outside = self._extract_gated_section(content) + + assert "one assistant message" not in outside, ( + "Fanout instruction 'one assistant message' found OUTSIDE the Slice 5b " + "gated section — it leaked into sequential/default instructions (HC-1)." + ) + + def test_vc1a_emit_all_n_task_inside_not_outside(self) -> None: + """'emit all N `Task' (the canonical fanout verb) must be inside, not outside. + + The exact phrase from the reference: 'emit all N `Task(actor)` calls'. + Failure scenario: fanout instruction bleeds into the 5a sequential section. + """ + if not _REFERENCE_PATH.exists(): + pytest.skip(f"Reference file not found: {_REFERENCE_PATH}") + + content = _REFERENCE_PATH.read_text(encoding="utf-8") + inside, outside = self._extract_gated_section(content) + + # The literal phrase in the reference file; N is the word not a digit. + fanout_phrase = "emit all N" + + # Non-tautological: prove the phrase IS inside the gate. + assert fanout_phrase in inside, ( + f"Expected fanout phrase {fanout_phrase!r} not found inside the Slice 5b " + "gated section — was the fanout instruction renamed or removed?" + ) + + # Now prove it does NOT appear outside the gate. + assert fanout_phrase not in outside, ( + f"Fanout phrase {fanout_phrase!r} found OUTSIDE the Slice 5b gate. " + "Fanout instructions must be confined to the gated section only." + ) + + +# =========================================================================== +# Guard (b): Monkeypatch-fail guard +# =========================================================================== + +class TestGuardB_MonkeypatchFail: + """Concurrent runner verbs must NOT be called under default config. + + What break turns this red: + - If compute_dispatch_gate (or get_wave_step) calls begin_wave_group / + abort_wave_group / run_concurrent_wave / record_dispatch_actual when + concurrent_dispatch is False (default), the monkeypatched stub raises + AssertionError and the test fails — catching the leak. + """ + + _CONCURRENT_VERBS = [ + "begin_wave_group", + "abort_wave_group", + "record_dispatch_actual", + "run_concurrent_wave", + ] + + def _make_failing_stub(self, name: str): # type: ignore[return] + """Return a def-style callable that raises AssertionError if called.""" + def _stub(*_args: object, **_kw: object) -> None: + del _args, _kw # suppress pyright unused-parameter; del valid in def + raise AssertionError( + f"Concurrent runner verb {name!r} must NOT be called " + "when concurrent_dispatch=False (default config, HC-1)." + ) + return _stub + + def test_vc1b_no_concurrent_verbs_fire_on_default_config( + self, + branch_with_waves: str, + monkeypatch: pytest.MonkeyPatch, + ) -> None: + """Under default config (no concurrent_dispatch key) none of the four + concurrent runner verbs fire when get_wave_step / compute_dispatch_gate + are called. + + Stubs raise AssertionError on any call → a leak is immediately visible + as a test failure. + """ + import importlib + try: + msr = importlib.import_module("map_step_runner") + except ModuleNotFoundError: + msr = None + + for verb in self._CONCURRENT_VERBS: + stub = self._make_failing_stub(verb) + # Patch on orchestrator module (covers any top-level alias). + if hasattr(map_orchestrator, verb): + monkeypatch.setattr(map_orchestrator, verb, stub) + # Patch on the runner module if loaded. + if msr is not None and hasattr(msr, verb): + monkeypatch.setattr(msr, verb, stub) + + # Call get_wave_step (calls compute_dispatch_gate internally). + result = map_orchestrator.get_wave_step(branch_with_waves) + + # Also call compute_dispatch_gate directly. + gate = map_orchestrator.compute_dispatch_gate(branch_with_waves, Path(".")) + + # Confirm sequential path taken (stubs had no effect = no verb fired). + assert result["dispatch_mode"] == "sequential", ( + f"Expected sequential dispatch_mode, got: {result['dispatch_mode']!r}" + ) + assert gate["dispatch_mode"] == "sequential", ( + f"Expected sequential gate, got: {gate['dispatch_mode']!r}" + ) + + +# =========================================================================== +# Guard (c): AST/static-import guard +# =========================================================================== + +class TestGuardC_ASTImport: + """Sequential walker + flag-false branch of orchestrator must not reference + the concurrent runner verbs that live in map_step_runner.py. + + What break turns this red: + - If get_next_step or the flag-false early-return branch of + compute_dispatch_gate contains a Call/Attribute referencing any of the + four concurrent verbs, the test fails. + - Non-tautological: we first prove the verbs ARE present in the runner file + (not just absent from the orchestrator by accident of deletion), so a + vacuous pass from orphaned names is impossible. + """ + + _CONCURRENT_VERBS = frozenset([ + "run_concurrent_wave", + "begin_wave_group", + "abort_wave_group", + "record_dispatch_actual", + ]) + + def _collect_call_names(self, node: ast.AST) -> set[str]: + """Collect all Call function-names and Attribute names from an AST subtree.""" + names: set[str] = set() + for child in ast.walk(node): + if isinstance(child, ast.Call): + func = child.func + if isinstance(func, ast.Name): + names.add(func.id) + elif isinstance(func, ast.Attribute): + names.add(func.attr) + elif isinstance(child, ast.Attribute): + names.add(child.attr) + return names + + def _extract_function( + self, tree: ast.Module, func_name: str + ) -> ast.FunctionDef | ast.AsyncFunctionDef | None: + for node in ast.walk(tree): + if ( + isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)) + and node.name == func_name + ): + return node + return None + + def _extract_flag_false_body( + self, func_node: ast.FunctionDef + ) -> list[ast.stmt]: + """Extract the HC-1 early-return branch: 'if not flag_on: return {...}'. + + Looks for the first If whose test is a UnaryOp(Not, ...) and whose body + contains a Return — this is the flag-false short-circuit in compute_dispatch_gate. + """ + for node in ast.walk(func_node): + if not isinstance(node, ast.If): + continue + test = node.test + is_not_expr = isinstance(test, ast.UnaryOp) and isinstance(test.op, ast.Not) + has_return = any(isinstance(s, ast.Return) for s in node.body) + if is_not_expr and has_return: + return node.body + return [] + + def test_vc1c_ast_parse_succeeds_functions_exist(self) -> None: + """Orchestrator parses cleanly and contains the expected functions. + + Non-tautological baseline: if the parse fails or functions are absent, the + sequential-path absence checks would be vacuously true. + """ + if not _ORCHESTRATOR_PATH.exists(): + pytest.skip(f"Orchestrator not found: {_ORCHESTRATOR_PATH}") + + src = _ORCHESTRATOR_PATH.read_text(encoding="utf-8") + tree = ast.parse(src, filename=str(_ORCHESTRATOR_PATH)) + + assert self._extract_function(tree, "get_next_step") is not None, ( + "get_next_step missing from parsed orchestrator — AST guard is vacuous." + ) + assert self._extract_function(tree, "compute_dispatch_gate") is not None, ( + "compute_dispatch_gate missing from parsed orchestrator — AST guard is vacuous." + ) + + def test_vc1c_verbs_present_in_runner_not_on_sequential_orchestrator_paths( + self, + ) -> None: + """The concurrent verbs ARE defined in map_step_runner.py (non-tautological) + but do NOT appear in get_next_step or the flag-false branch of + compute_dispatch_gate in map_orchestrator.py. + + Failure scenarios: + - Verbs absent from runner entirely → 'present in runner' assert fires (vacuous guard). + - Verbs referenced on the sequential orchestrator paths → leak assert fires. + """ + if not _ORCHESTRATOR_PATH.exists(): + pytest.skip(f"Orchestrator not found: {_ORCHESTRATOR_PATH}") + if not _RUNNER_PATH.exists(): + pytest.skip(f"Runner not found: {_RUNNER_PATH}") + + # --- Non-tautological: verify concurrent verbs ARE defined in the runner --- + runner_src = _RUNNER_PATH.read_text(encoding="utf-8") + runner_tree = ast.parse(runner_src, filename=str(_RUNNER_PATH)) + + verbs_in_runner = { + node.name + for node in ast.walk(runner_tree) + if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)) + and node.name in self._CONCURRENT_VERBS + } + # record_dispatch_actual lives in mapify_cli.parallelism_observability, not the runner, + # so only check the three runner-defined verbs here. + runner_verbs = self._CONCURRENT_VERBS - {"record_dispatch_actual"} + missing_runner_verbs = runner_verbs - verbs_in_runner + assert not missing_runner_verbs, ( + f"Concurrent verbs {missing_runner_verbs} not defined in map_step_runner.py. " + "The AST guard would be vacuous. Were the verbs renamed?" + ) + + # --- Now check the orchestrator sequential paths --- + orch_src = _ORCHESTRATOR_PATH.read_text(encoding="utf-8") + orch_tree = ast.parse(orch_src, filename=str(_ORCHESTRATOR_PATH)) + + # Check get_next_step (sequential walker). + gns = self._extract_function(orch_tree, "get_next_step") + assert gns is not None + gns_names = self._collect_call_names(gns) + leaked_in_gns = self._CONCURRENT_VERBS & gns_names + assert not leaked_in_gns, ( + f"Concurrent verbs {leaked_in_gns} referenced in get_next_step " + "(sequential walker). Sequential path must not call concurrent primitives (HC-1)." + ) + + # Check compute_dispatch_gate's flag-false early-return branch. + cdg = self._extract_function(orch_tree, "compute_dispatch_gate") + assert cdg is not None + assert isinstance(cdg, ast.FunctionDef) # narrow type + flag_false_body = self._extract_flag_false_body(cdg) + assert flag_false_body, ( + "Could not locate the flag-false early-return branch in compute_dispatch_gate. " + "The HC-1 short-circuit structure may have changed." + ) + ff_names = self._collect_call_names( + ast.Module(body=flag_false_body, type_ignores=[]) + ) + leaked_in_ff = self._CONCURRENT_VERBS & ff_names + assert not leaked_in_ff, ( + f"Concurrent verbs {leaked_in_ff} referenced in compute_dispatch_gate " + "flag-false branch. The HC-1 short-circuit must return immediately with " + "no concurrency primitives (HC-1)." + ) + + +# =========================================================================== +# Guard (d): No-telemetry guard +# =========================================================================== + +class TestGuardD_NoTelemetry: + """record_dispatch_actual must not create parallelism.json under default config. + + What break turns this red: + - If any code path under default config creates parallelism.json (which + record_dispatch_actual writes only on the DISPATCH_OUTCOME_CONCURRENT_OBSERVED + path), the file-existence assertion fires. + """ + + def test_vc1d_no_parallelism_json_created_on_default_path( + self, + branch_with_waves: str, + ) -> None: + """Running get_wave_step + compute_dispatch_gate under default config must NOT + create any parallelism.json file. + """ + expected_telemetry = Path(f".map/{branch_with_waves}/parallelism.json") + map_dir = Path(f".map/{branch_with_waves}") + + # Capture existing json files BEFORE the default-config flow. + before: set[Path] = set(map_dir.rglob("*.json")) + + # Execute the default-config flow (return values unused — we assert on the + # filesystem side effect, i.e. that no telemetry file is written). + map_orchestrator.get_wave_step(branch_with_waves) + map_orchestrator.compute_dispatch_gate(branch_with_waves, Path(".")) + + # parallelism.json must not be created. + assert not expected_telemetry.exists(), ( + "parallelism.json was created under default config. " + "record_dispatch_actual must be a no-op on the sequential path (HC-1)." + ) + + # Broad check: no new parallelism-related json files appeared. + after: set[Path] = set(map_dir.rglob("*.json")) + new_files = after - before + parallelism_new = {p for p in new_files if "parallelism" in p.name} + assert not parallelism_new, ( + f"New parallelism-related file(s) created under default config: " + f"{parallelism_new}. Only step_state.json should be updated." + ) + + +# =========================================================================== +# Guard (e): Default-off baseline +# =========================================================================== + +class TestGuardE_DefaultOffBaseline: + """Under default config, gate and wave step must be sequential with 5a reason code. + + What break turns this red: + - If concurrent_dispatch defaults to True (config-default bug), the + dispatch_mode assertion fires. + - If the reason code drifts from WAVE_REASON_DISPATCH_SEQUENTIAL, the reason + assertion fires — catching a silent rename or path change. + - concurrency_enabled==False is a corollary: also asserted. + """ + + def test_vc1e_compute_dispatch_gate_sequential_by_default( + self, branch_with_waves: str + ) -> None: + """compute_dispatch_gate returns dispatch_mode=='sequential' and the 5a reason + code when no concurrent_dispatch key is present. + + The reason must equal WAVE_REASON_DISPATCH_SEQUENTIAL — not a fallback from + later gate steps — proving the HC-1 short-circuit is the path that fires. + """ + gate = map_orchestrator.compute_dispatch_gate(branch_with_waves, Path(".")) + + assert gate["dispatch_mode"] == "sequential", ( + f"compute_dispatch_gate returned {gate['dispatch_mode']!r} under default " + "config. Expected 'sequential' (HC-1)." + ) + assert gate["reason"] == map_orchestrator.WAVE_REASON_DISPATCH_SEQUENTIAL, ( + f"Gate reason {gate['reason']!r} != " + f"WAVE_REASON_DISPATCH_SEQUENTIAL ({map_orchestrator.WAVE_REASON_DISPATCH_SEQUENTIAL!r}). " + "HC-1 short-circuit must fire before any concurrency probe." + ) + + def test_vc1e_get_wave_step_concurrency_disabled_by_default( + self, branch_with_waves: str + ) -> None: + """get_wave_step returns concurrency_enabled==False and dispatch_mode== + 'sequential' when concurrent_dispatch is not configured. + """ + result = map_orchestrator.get_wave_step(branch_with_waves) + + assert result["dispatch_mode"] == "sequential", ( + f"get_wave_step returned dispatch_mode={result['dispatch_mode']!r}. " + "Expected 'sequential' under default config." + ) + assert result.get("concurrency_enabled") is False, ( + f"get_wave_step returned concurrency_enabled=" + f"{result.get('concurrency_enabled')!r}. Must be False under default config." + ) + + def test_vc1e_get_wave_step_reason_is_5a_code( + self, branch_with_waves: str + ) -> None: + """The reason in get_wave_step must be WAVE_REASON_DISPATCH_SEQUENTIAL — the + 5a stable code — when concurrent_dispatch is off. + + Failure scenario: if compute_dispatch_gate's wiring changed and the reason + was overwritten or swallowed, a different (or missing) reason code appears. + """ + result = map_orchestrator.get_wave_step(branch_with_waves) + + assert result.get("reason") == map_orchestrator.WAVE_REASON_DISPATCH_SEQUENTIAL, ( + f"get_wave_step reason={result.get('reason')!r} != " + f"WAVE_REASON_DISPATCH_SEQUENTIAL={map_orchestrator.WAVE_REASON_DISPATCH_SEQUENTIAL!r}. " + "The 5a reason code is the stable HC-1 contract." + ) + + def test_vc1e_select_execution_strategy_concurrency_not_allowed_by_default( + self, branch_with_waves: str + ) -> None: + """select_execution_strategy returns concurrency_allowed==False under default + config (worktree.isolation defaults to 'off'). + + This covers the 'concurrency_allowed==False' clause of guard (e). + """ + strategy = map_orchestrator.select_execution_strategy(branch_with_waves, Path(".")) + + assert strategy.get("concurrency_allowed") is False, ( + f"select_execution_strategy returned concurrency_allowed=" + f"{strategy.get('concurrency_allowed')!r}. " + "Must be False under default config (worktree.isolation defaults to 'off')." + )