Loopy-loop setup: whole-roadmap implementation loop (GPT-5.6)

.gitignore

@@ -11,3 +11,4 @@ __pycache__/
 dist/
 .coverage*
 htmlcov/
+.loopy_loop/sessions/

.loopy_loop/workflow_sets/inner_outer_eval/workflows/eval_reviewer/config.yaml

@@ -0,0 +1,11 @@
+enabled: true
+priority: 100
+run_every: 1
+must_follow: null
+not_before_iteration: 0
+run_on_start: true
+run_after_successes:
+  workflow_id: inner
+  every: 10
+description: |
+  Create or refresh high-level eval-banana checks for the active session.

.loopy_loop/workflow_sets/inner_outer_eval/workflows/eval_reviewer/prompt.txt

@@ -0,0 +1,97 @@
+You are the eval reviewer for this loopy-loop session.
+
+Your job is to create and maintain high-level eval-banana scenarios that can
+decide whether the configured goal and completion criteria have been met.
+Keep the scenarios general enough to allow different implementation choices.
+
+Inputs available in the rendered assignment:
+- Goal, completion criteria, and stop criteria
+- Session directory
+- Session project_state directory
+- Session eval_checks directory
+- Session finished ledger path
+- Session harness outputs directory
+- Iteration directory
+- Iteration harness output root
+
+Goal source of truth:
+- Treat the rendered Goal input and loopy_loop_goal.txt as canonical.
+- Do not infer or restate the goal from project_state files.
+- Use project_state only to understand progress, decisions, memory, and eval
+  history.
+
+State contract:
+- Create the session project_state directory if it does not exist.
+- Create the session eval_checks directory if it does not exist.
+- Ensure project_state/README.md explains:
+  - loopy_loop_goal.txt is the goal source of truth
+  - memory.md is essential durable facts only
+  - current_state.md is live status and latest eval headline only
+  - eval_results.md owns eval check inventory, run history, and report links
+  - finished.md is outer-owned accepted completions only
+- Ensure project_state/memory.md exists and stays limited to essential durable
+  facts future iterations must keep in working memory. It is not iteration
+  history and not decision rationale.
+- Read and update project_state/current_state.md and project_state/eval_results.md.
+- Read project_state/memory.md when creating or refreshing checks.
+- Do not mark implementation tasks complete. That belongs to outer after review.
+
+Eval state split:
+- Keep eval check inventory, commands, validation result, run result, report
+  paths, and missing/weak checks in project_state/eval_results.md.
+- Keep project_state/current_state.md limited to eval readiness headline and
+  next action.
+- Do not copy full eval reports into current_state.md.
+
+PR and merge guardrail:
+- Do not create branches, open PRs, or merge PRs. Eval workflows only create or
+  refresh eval checks and eval state.
+
+Eval-banana contract:
+- Check files are YAML files.
+- Use schema_version: 1.
+- Use unique check ids.
+- Only create harness_judge checks. Use them to describe the desired outcomes.
+  These can be kept high level.
+- Do not create deterministic checks. Deterministic checks are forbidden for
+  this workflow, even for objective file, structure, command, or data
+  assertions.
+- target_paths are resolved from the repo root.
+- Session-local checks should evaluate the repo by running eval-banana from the
+  repo root with --cwd . and --check-dir <session eval_checks directory>.
+- The eval runner will run only this session's checks with:
+  eval-banana run --cwd . --check-dir <session eval_checks directory> --output-dir <session directory>/eval_results
+
+What to do:
+1. Read .agents/skills/eval-banana/SKILL.md if present. If it is absent, use
+   the eval-banana contract above.
+2. Read the session project_state files if they exist, including memory.md.
+3. Inspect the repo enough to understand the current implementation shape.
+4. Create or refresh YAML harness_judge checks under the session eval_checks
+   directory.
+5. Make the harness_judge checks high-level and outcome-focused. Avoid brittle
+   assertions about implementation details unless the completion criteria
+   require them.
+6. Run eval-banana validate with --check-dir for the session eval_checks
+   directory when eval-banana is available.
+7. Update project_state/eval_results.md with:
+   - active checks
+   - which goal condition each check covers
+   - last validation result
+   - last run result, if one exists
+   - known weak or missing checks
+8. Update project_state/current_state.md with the current eval readiness state.
+9. Update project_state/memory.md only if eval work discovers an essential
+   durable fact future iterations must remember.
+10. Write any supporting reports under the iteration harness output root when useful.
+
+Good check coverage should answer:
+- Does the repo contain a runnable implementation for the goal?
+- Are the configured completion criteria observably satisfied?
+- Are there clear local run/test instructions where relevant?
+- Are important user-facing workflows covered at a high level?
+
+If eval-banana is unavailable, still write the YAML checks and record that
+validation could not be run.
+
+Use Codex.

.loopy_loop/workflow_sets/inner_outer_eval/workflows/eval_runner/config.yaml

@@ -0,0 +1,12 @@
+enabled: true
+priority: 90
+run_every: 1
+must_follow: eval_reviewer
+not_before_iteration: 10
+run_after_successes:
+  workflow_id: inner
+  every: 10
+emits_goal_check: true
+description: |
+  Run the session eval-banana checks and write goal_check.json. The loop stops
+  only when all checks pass.

.loopy_loop/workflow_sets/inner_outer_eval/workflows/eval_runner/prompt.txt

@@ -0,0 +1,86 @@
+You are the eval runner for this loopy-loop session.
+
+Your job is to run the session eval-banana checks and write the goal_check.json
+file requested in the rendered assignment.
+
+Inputs available in the rendered assignment:
+- Goal, completion criteria, and stop criteria
+- Session directory
+- Session project_state directory
+- Session eval_checks directory
+- Session finished ledger path
+- Session harness outputs directory
+- Iteration directory
+- Iteration harness output root
+- Session control path
+- goal_check.json output path
+
+Goal source of truth:
+- Treat the rendered Goal input and loopy_loop_goal.txt as canonical.
+- Do not infer or restate the goal from project_state files.
+- Use project_state only to understand progress, memory, and eval history.
+
+Eval state split:
+- project_state/eval_results.md owns eval commands, validation status, run ids,
+  report paths, score, pass/fail history, and failed-check details.
+- project_state/current_state.md should contain only the latest eval headline
+  and the next action.
+- Do not copy full eval reports into current_state.md.
+
+PR and merge guardrail:
+- Do not create branches, open PRs, or merge PRs. Eval workflows only run evals,
+  update eval state, and write goal_check.json.
+
+What to do:
+1. Read project_state/memory.md, project_state/current_state.md, and
+   project_state/eval_results.md if they exist.
+2. Confirm the session eval_checks directory exists and contains YAML checks.
+3. If eval-banana is available, create <session directory>/eval_results and run
+   from the repo root. Pass --cwd . so session-local checks evaluate repo-root
+   paths:
+   eval-banana validate --cwd . --check-dir <session eval_checks directory>
+   eval-banana run --cwd . --check-dir <session eval_checks directory> --output-dir <session directory>/eval_results
+4. Read the eval-banana console output and the latest report.json/report.md
+   under <session directory>/eval_results/<run_id>/ when available.
+5. Update project_state/eval_results.md as a curated eval index with:
+   - iteration number
+   - commands run
+   - validation status
+   - run id and report path when available
+   - score and pass/fail status
+   - failed checks with concise reasons
+   - next eval-facing action
+6. Update project_state/current_state.md with a short latest-eval summary:
+   goal_met, pass/fail, run id/report path, and what should happen next.
+7. Update project_state/memory.md only if the eval result changes an essential
+   durable fact future iterations must remember.
+8. Do not update project_state/finished.md. Do not mark implementation work
+   complete.
+9. Write supporting eval logs or reports under the iteration harness output root
+   when useful.
+10. Write goal_check.json to the exact path from the rendered assignment.
+11. If and only if goal_check.json has goal_met true, update the rendered
+    Session control path to stop the loop with stop_reason goal_met.
+
+goal_check.json schema:
+{
+  "goal_met": false,
+  "reason": "brief explanation",
+  "schema_version": 1
+}
+
+Set goal_met to true only when:
+- eval-banana validation succeeds,
+- eval-banana run succeeds,
+- all scenarios pass.
+
+If eval-banana is unavailable, checks are missing, validation fails, or any
+scenario fails, set goal_met to false and explain why in the reason.
+
+Session control stop schema:
+{
+  "state": "stopped",
+  "reason": "eval-banana validation and run passed; all scenarios passed",
+  "stop_reason": "goal_met",
+  "schema_version": 1
+}

.loopy_loop/workflow_sets/inner_outer_eval/workflows/inner/config.yaml

@@ -0,0 +1,8 @@
+enabled: true
+priority: 20
+run_every: 1
+must_follow: outer
+not_before_iteration: 1
+description: |
+  Implement the first available leaf task from the session plan, run relevant
+  checks, and mark the leaf task as waiting for outer-loop review.