Summary
When the setup agent returns a JSON payload that parses successfully but is missing required keys (most commonly master_path), WorkspaceRecipe.from_dict raises a bare KeyError that takes down the whole pipeline.
Repro
Run the pipeline against a project with many git submodules (e.g. damn-vulnerable-defi) with a tight setup budget so the agent runs out of iterations before producing a fully-formed recipe:
KAI_SETUP_ITERS=15 KAI_LOG_STRUCTURED=1 \
uv run python -m kai.main pipeline \
--repo-path /tmp/damn-vulnerable-defi \
--verbose --log-structured --skip-fixer --no-iterative
Observed
ERROR __main__: Pipeline crashed: 'master_path' — saving partial results
Traceback (most recent call last):
File "src/kai/main.py", line 1088, in main
result = run_pipeline(…)
File "src/kai/main.py", line 628, in run_pipeline
recipe = WorkspaceRecipe.from_dict(recipe_data)
File "src/kai/workspace/recipe.py", line 46, in from_dict
master_path=data["master_path"],
~~~~^^^^^^^^^^^^^^^^
KeyError: 'master_path'
The crash log claims "saving partial results" but the resulting output/state/<run_id>/run.json is a near-empty RunRecord with status: "failed" and no usage summary.
Expected
WorkspaceRecipe.from_dict should validate inputs and raise a typed error (e.g. InvalidRecipeError("missing key master_path", recipe_data=…)). The retry path in src/kai/main.py:571-628 can then format a useful diagnostic and decide whether to retry / fall back.
Suggested fix
Make WorkspaceRecipe a pydantic model and use model_validate — pydantic surfaces every missing field with a structured error that's easy to log.
Files
src/kai/workspace/recipe.py:46
src/kai/main.py:571-628
Severity
High — any malformed recipe takes the whole run down with a non-recoverable traceback. Surfaced during Phase 0 smoke for the benchmark harness (evaluation/).
Summary
When the setup agent returns a JSON payload that parses successfully but is missing required keys (most commonly
master_path),WorkspaceRecipe.from_dictraises a bareKeyErrorthat takes down the whole pipeline.Repro
Run the pipeline against a project with many git submodules (e.g. damn-vulnerable-defi) with a tight setup budget so the agent runs out of iterations before producing a fully-formed recipe:
Observed
The crash log claims "saving partial results" but the resulting
output/state/<run_id>/run.jsonis a near-emptyRunRecordwithstatus: "failed"and no usage summary.Expected
WorkspaceRecipe.from_dictshould validate inputs and raise a typed error (e.g.InvalidRecipeError("missing key master_path", recipe_data=…)). The retry path insrc/kai/main.py:571-628can then format a useful diagnostic and decide whether to retry / fall back.Suggested fix
Make
WorkspaceRecipea pydantic model and usemodel_validate— pydantic surfaces every missing field with a structured error that's easy to log.Files
src/kai/workspace/recipe.py:46src/kai/main.py:571-628Severity
High — any malformed recipe takes the whole run down with a non-recoverable traceback. Surfaced during Phase 0 smoke for the benchmark harness (
evaluation/).