Skip to content

Pipeline crashes with bare KeyError when setup recipe is missing required fields #94

@eren23

Description

@eren23

Summary

When the setup agent returns a JSON payload that parses successfully but is missing required keys (most commonly master_path), WorkspaceRecipe.from_dict raises a bare KeyError that takes down the whole pipeline.

Repro

Run the pipeline against a project with many git submodules (e.g. damn-vulnerable-defi) with a tight setup budget so the agent runs out of iterations before producing a fully-formed recipe:

KAI_SETUP_ITERS=15 KAI_LOG_STRUCTURED=1 \
uv run python -m kai.main pipeline \
  --repo-path /tmp/damn-vulnerable-defi \
  --verbose --log-structured --skip-fixer --no-iterative

Observed

ERROR __main__: Pipeline crashed: 'master_path' — saving partial results
Traceback (most recent call last):
  File "src/kai/main.py", line 1088, in main
    result = run_pipeline(…)
  File "src/kai/main.py", line 628, in run_pipeline
    recipe = WorkspaceRecipe.from_dict(recipe_data)
  File "src/kai/workspace/recipe.py", line 46, in from_dict
    master_path=data["master_path"],
                ~~~~^^^^^^^^^^^^^^^^
KeyError: 'master_path'

The crash log claims "saving partial results" but the resulting output/state/<run_id>/run.json is a near-empty RunRecord with status: "failed" and no usage summary.

Expected

WorkspaceRecipe.from_dict should validate inputs and raise a typed error (e.g. InvalidRecipeError("missing key master_path", recipe_data=…)). The retry path in src/kai/main.py:571-628 can then format a useful diagnostic and decide whether to retry / fall back.

Suggested fix

Make WorkspaceRecipe a pydantic model and use model_validate — pydantic surfaces every missing field with a structured error that's easy to log.

Files

  • src/kai/workspace/recipe.py:46
  • src/kai/main.py:571-628

Severity

High — any malformed recipe takes the whole run down with a non-recoverable traceback. Surfaced during Phase 0 smoke for the benchmark harness (evaluation/).

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions