Phase1: Feat: add unified vera generate command

AGENTS.md

@@ -29,7 +29,7 @@ cp .env.example .env       # Add API keys (ANTHROPIC_API_KEY, OPENAI_API_KEY, et
 
 | Area | Key paths | When to edit |
 |------|-----------|--------------|
-| **CLI** | `vera.py` | Subcommands: generate, judge, score, pool, pipeline |
+| **CLI** | `vera.py`, `vera_cli/`, `utils/config_schema.py` | Thin entry point, command adapters, and shared command/config wiring |
 | **Generation** | `generate_conversations/` | Conversation simulation, turns, personas |
 | **Judging** | `judge/` | Rubric scoring, TSV output, question navigation |
 | **LLM providers** | `llm_clients/`, `llm_clients/llm_factory.py` | New models, custom HTTP/API providers |
@@ -39,7 +39,11 @@ cp .env.example .env       # Add API keys (ANTHROPIC_API_KEY, OPENAI_API_KEY, et
 | **Config** | `utils/model_config_loader.py`, `llm_clients/config.py` | Model name resolution, API keys |
 | **Shared utils** | `utils/` | Naming, logging, conversation layout |
 
-**Entry point (target):** `vera.py` subcommands only. Legacy scripts (`generate.py`, `judge.py`, `run_pipeline.py`) are deleted entirely at the end of Phase 1 of the migration (not the full 6-phase migration) — see [docs/architecture.md](docs/architecture.md#migration-from-current-layout).
+**Entry points:** `vera.py generate` is the first unified CLI feature. `generate.py`
+remains temporarily as a compatibility adapter, while `judge.py` and
+`run_pipeline.py` remain the current entry points for features not migrated yet.
+Each legacy script is removed after its replacement feature is available; see
+[docs/architecture.md](docs/architecture.md#migration-from-current-layout).
 
 **Temporary experiments:** `tmp_tests/` (not committed). **Permanent tests:** `tests/`.
 
@@ -87,25 +91,16 @@ uv run pytest tests/integration/
 
 ## Key Commands
 
-Target CLI (`vera.py` — not implemented yet; use legacy commands below until Phase 1 of the migration completes):
+Unified generation CLI:
 
 ```bash
-# End-to-end pipeline (target)
-uv run python vera.py pipeline \
-  --user-agent claude-sonnet-4-5-20250929 \
-  --provider-agent gpt-4o \
-  --runs 1 \
-  --turns 10 \
-  --judge-model claude-sonnet-4-5-20250929 \
-  --max-personas 5
-
-# Generate / judge / score (target)
-uv run python vera.py generate -u claude-sonnet-4-5-20250929 -p gpt-4o -t 6 -r 1
-uv run python vera.py judge -f output/{YOUR_P_RUN}/ -j claude-sonnet-4-5-20250929
-uv run python vera.py score -r output/{YOUR_P_RUN}/evaluations/{YOUR_J_RUN}/results.csv
+uv run python vera.py generate \
+  -c gpt-4o \
+  -u claude-sonnet-4-5-20250929:1 \
+  --target SI
 ```
 
-Legacy (current implementation):
+Legacy commands for pipeline, judging, and compatibility:
 
 ```bash
 uv run python run_pipeline.py \

README.md

@@ -15,6 +15,7 @@ We value every interaction that follows the [Code of Conduct](https://www.contri
 - [Connecting your own LLM, Agent, or API](#connecting-your-own-llm-or-api)
 - [Recommended settings](#recommended-settings)
 - [Reliable VERA-MH score (automated)](#reliable-vera-mh-score-automated)
+- [Unified CLI: generation](#unified-cli-generation)
 - [Running VERA-MH step by step](#running-vera-mh-step-by-step)
 - [Using Extra Parameters](#using-extra-parameters)
 - [Data Files](#data-files)
@@ -32,7 +33,7 @@ We value every interaction that follows the [Code of Conduct](https://www.contri
 
 # Getting started
 
-This page covers [Environment setup](#environment-setup), optional [custom provider wiring](#connecting-your-own-llm-or-api), [Recommended settings](#recommended-settings) for comparable scores, the [automated pooled pipeline](#reliable-vera-mh-score-automated), and [Running VERA-MH step by step](#running-vera-mh-step-by-step) (`run_pipeline.py`, `generate.py`, `judge.py`, scoring, comparison, and improvement reports).
+This page covers [Environment setup](#environment-setup), optional [custom provider wiring](#connecting-your-own-llm-or-api), [Recommended settings](#recommended-settings) for comparable scores, the [automated pooled pipeline](#reliable-vera-mh-score-automated), the [unified generation CLI](#unified-cli-generation), and [Running VERA-MH step by step](#running-vera-mh-step-by-step) (`run_pipeline.py`, `generate.py`, `judge.py`, scoring, comparison, and improvement reports).
 
 ## Environment setup
 
@@ -130,6 +131,44 @@ uv run python scripts/pool_vera_scores.py -o <pool_parent_dir> \
 
 Use `uv run python scripts/pool_vera_scores.py --help` for options (including `--extract-from-log` for parsing a saved `run_pipeline.py` log).
 
+## Unified CLI: generation
+
+`vera.py generate` is the first command available through the unified CLI. A
+target selects the complete reusable evaluation bundle; generation consumes its
+personas and persona prompt:
+
+```bash
+uv run python vera.py generate \
+  -c gpt-4o \
+  -u claude-sonnet-4-5-20250929:1 \
+  --target SI
+```
+
+Use `--personas SI` when you want only SI's persona component explicitly:
+
+```bash
+uv run python vera.py generate \
+  -c gpt-4o \
+  -u claude-sonnet-4-5-20250929:1 \
+  --personas SI
+```
+
+Both forms resolve `data/SI/manifest.json` before dispatch. CLI defaults are
+defined at the flag boundary (`--turns 30`, `--output output`, unlimited
+concurrency, persona first). Config-driven runs provide every generation
+behavior field explicitly:
+
+```bash
+uv run python vera.py generate --config run.json
+```
+
+Run-defining flags and `--config` cannot be mixed. `--sample`, `--debug`, and
+`--print` may accompany config input. Executed runs record `sample` and `debug`
+as invocation metadata in their resolved config; `--print` creates no run. The
+legacy `generate.py` remains temporarily as a compatibility adapter; judging
+and pipeline execution continue to use `judge.py` and `run_pipeline.py` until
+their unified commands are added.
+
 ## Running VERA-MH step by step
 
 1. **(Optional) Custom provider** — If your product is not supported, implement and register a client (see [Connecting your own LLM, Agent, or API](#connecting-your-own-llm-or-api); full detail in [docs/evaluating.md](docs/evaluating.md)).

data/SI/manifest.json

@@ -0,0 +1,7 @@
+{
+  "rubric_file": "rubric.tsv",
+  "rubric_prompt_beginning_file": "rubric_prompt_beginning.txt",
+  "question_prompt_file": "question_prompt.txt",
+  "personas": ["personas.tsv"],
+  "persona_context_template_file": "persona_context_template.txt"
+}

docs/architecture.md

@@ -53,12 +53,12 @@ Deep dives: [judge.md](./judge.md) (question flow and rubric navigation), [struc
 
 ```text
 CLI layer
-├── vera.py — sole executable; loads arguments and dispatches
+├── vera.py — sole executable; builds the root parser and dispatches
 └── vera_cli/
-    ├── arguments.py — top-level parser
-    ├── *_arguments.py — per-command flags and CLI defaults
-    ├── *_config.py — per-command canonical resolution
-    └── *_command.py — thin command adapters
+    ├── <command>.py — one module per command: flags, CLI defaults,
+    │                  canonical resolution, and the domain call
+    ├── config.py — shared config input, input exclusivity, path resolution
+    └── targets.py — target discovery and manifest validation
     ↓ calls
 Domain packages (generate/, judge/, score/)
     ↓ register handlers with
@@ -105,15 +105,20 @@ command, and renders CLI errors. Full flag/config reference:
 The CLI layer has three responsibilities:
 
 - `vera.py` is the thin executable and contains no command-specific business
-  logic.
-- `vera_cli/arguments.py` builds the top-level parser. Small per-command
-  `*_arguments.py` modules define that command's flags and CLI defaults.
-- Shared config input and target-manifest helpers stay in focused modules;
-  per-command `*_config.py` modules enforce input exclusivity and produce the
-  complete canonical configuration before print, persistence, or dispatch.
-- Per-command `*_command.py` modules contain only orchestration adapters. They
-  receive resolved values and call importable Python functions directly; they
-  never invoke another CLI parser or subprocess.
+  logic. It builds the root parser, registers each command, and renders CLI
+  errors.
+- One `vera_cli/<command>.py` module owns everything specific to that command:
+  its flags, its CLI defaults, its canonical resolution, and its call into the
+  domain. A command is reachable only once registered in `vera.py`.
+- Resolution completes before print, persistence, or dispatch: input
+  exclusivity is enforced, names become verified absolute paths, and the result
+  is a canonical configuration.
+- The domain call is an orchestration adapter only. It receives resolved values
+  and calls importable Python functions directly; it never invokes another CLI
+  parser or subprocess.
+- Shared input and target-manifest helpers stay in focused modules
+  (`vera_cli/config.py`, `vera_cli/targets.py`). See
+  [../vera_cli/README.md](../vera_cli/README.md) for the command contract.
 
 `utils/config_schema.py` owns schema validation and canonical serialization. It
 does not parse CLI arguments, read config or manifest files, resolve paths, or
@@ -127,14 +132,22 @@ rather than to a script entry point.
 Legacy root scripts may remain temporarily while their replacement feature is
 migrated. During that transition, `vera_cli` may import the reusable function
 from a root script, but never its argument parser. For generation, the temporary
-flow is `vera_cli.generate_command` → `generate.main`. Removing `generate.py`
-and moving that function plus the existing `generate_conversations/` code into
-the permanent `generate/` package is one atomic later change, so a root
-`generate.py` module and a top-level `generate/` package never coexist.
+flow is `vera_cli.generate` → `generate.run_for_user_models` → `generate.main`.
+Removing `generate.py` and moving those functions plus the existing
+`generate_conversations/` code into the permanent `generate/` package is one
+atomic later change, so a root `generate.py` module and a top-level `generate/`
+package never coexist.
+
+`run_for_user_models` and its `_legacy_model_config` helper are explicit
+stopgaps. They put the expansion of a run's user models, and the flattening of
+`ModelSpec` into the legacy dict signature, on the domain side of the boundary
+rather than in the CLI. Both are deleted when the generation domain accepts
+`ModelSpec` directly, which is also what lets `generate` and `judge` describe
+models identically.
 
 | Subcommand | Delegates to | Purpose |
 |------------|--------------|---------|
-| `vera generate` | generation application function (temporarily `generate.main`) | Simulate conversations → `c_<chatbot>/<run>/conversations/` |
+| `vera generate` | generation application function (temporarily `generate.run_for_user_models`) | Simulate conversations → `c_<chatbot>/<run>/conversations/` |
 | `vera judge` | `judge.runner` | Evaluate transcripts → `evaluations/<rubric>/j_*` |
 | `vera score` | `score.score` | Aggregate `results.csv` → scores and visualizations |
 | `vera pool` | `score.pool` | Concatenate multiple evaluation folders into one pooled result |
@@ -276,7 +289,7 @@ orthogonal.
 
 | Package / path | Owns | Key modules |
 |----------------|------|-------------|
-| `vera_cli/` | CLI flags/defaults, input resolution, thin command adapters | `arguments.py`, `config.py`, `targets.py`, per-command modules |
+| `vera_cli/` | CLI flags/defaults, input resolution, thin command adapters | `<command>.py` per command, `config.py`, `targets.py` |
 | `generate/` | Simulation, turns, batch runner (pure core; handler owns I/O) | `conversation_simulator.py`, `runner.py` |
 | `judge/` | Rubric navigation, LLM judge, improvement reporting (pure core; handler owns I/O) | `question_navigator.py`, `llm_judge.py`, `scripts/summarize_results.py` |
 | `score/` | Aggregation, visualization, pooling — split out of `judge/` | `score.py`, `score_viz.py`, `pool.py` |

docs/rubric.md

@@ -34,15 +34,24 @@ Risk-level scoring additionally assumes personas use the levels `None`, `Low`,
 A complete bundle used for both generation and judging has this shape:
 
 ```text
-data/NEW_RUBRIC/
-├── rubric_manifest.json
+data/NEW_TARGET/
+├── manifest.json
 ├── rubric.tsv
 ├── rubric_prompt_beginning.txt
 ├── question_prompt.txt
 ├── personas.tsv
 └── persona_context_template.txt
 ```
 
+`manifest.json` is the target manifest, and the only manifest the unified CLI
+reads. It carries no compatibility fallback: the new CLI does not look for
+`rubric_manifest.json`.
+
+The legacy scripts listed under
+[Legacy entry points](#legacy-entry-points-being-removed) still read
+`rubric_manifest.json`. Only add a copy under that name if you need to run those
+scripts against this bundle before they are removed.
+
 `personas.tsv` and `persona_context_template.txt` are needed for conversation
 generation. Judging existing conversations only requires the rubric and judge
 prompt files.
@@ -100,13 +109,29 @@ For more detail about ordinary Yes/No navigation, see
 
 ## Running a compatible rubric
 
+Generate conversations with the unified CLI by naming the target directory:
+
+```bash
+uv run python vera.py generate \
+  -c <chatbot-model> \
+  -u <user-model>:1 \
+  --target NEW_TARGET
+```
+
+### Legacy entry points (being removed)
+
+Judging and pipeline runs have no `vera` command yet, so they still go through
+the legacy root scripts. These scripts are being deleted one at a time as each
+`vera` command replaces them, and they are the only reason a bundle needs a
+`rubric_manifest.json` copy. Nothing below reflects the unified CLI's behavior.
+
 Judge existing conversations with the new bundle:
 
 ```bash
 uv run python judge.py \
   --folder output/my-run \
   --judge-model <model> \
-  --rubrics data/NEW_RUBRIC/rubric_manifest.json
+  --rubrics data/NEW_TARGET/rubric_manifest.json
 ```
 
 For a complete generation, judging, and scoring run, select the bundle for both
@@ -117,9 +142,9 @@ uv run python run_pipeline.py \
   --user-agent <model> \
   --provider-agent <model> \
   --judge-model <model> \
-  --rubrics data/NEW_RUBRIC/rubric_manifest.json \
-  --rubric-manifest data/NEW_RUBRIC/rubric_manifest.json \
-  --personas-tsv data/NEW_RUBRIC/personas.tsv
+  --rubrics data/NEW_TARGET/rubric_manifest.json \
+  --rubric-manifest data/NEW_TARGET/rubric_manifest.json \
+  --personas-tsv data/NEW_TARGET/personas.tsv
 ```
 
 `--rubrics` and `--rubric-manifest` are independent: the former selects the
@@ -133,7 +158,7 @@ IDs are joined to the correct question text:
 ```bash
 uv run python scripts/summarize_results.py \
   --results output/my-run/evaluations/my-evaluation/results.csv \
-  --rubric data/NEW_RUBRIC/rubric.tsv
+  --rubric data/NEW_TARGET/rubric.tsv
 ```
 
 ## Current limitation

docs/vera-cli-use-cases.md

@@ -15,7 +15,18 @@ Three entities, each with a single-letter prefix used throughout the CLI, config
 - **`c` — chatbot**: the provider/agent LLM under test (previously called "provider" or "agent" inconsistently — `chatbot` is now the standard term).
 - **`j` — judge**: the LLM evaluating a transcript against a rubric.
 
-**These letters are `vera.py`-only and are not the same flags as today's scripts.** `generate.py`/`judge.py` already use `-c`/`-r` for unrelated things (`-c` is `--max-concurrent` in `generate.py` and `--conversation` in `judge.py`; `-r` is `--runs` in `generate.py` and `--rubrics` in `judge.py`). `vera.py` intentionally repurposes them for the `u`/`c`/`j` vocabulary above. There is no coexistence window: Phase 1 of the migration (see [architecture.md#migration-from-current-layout](./architecture.md#migration-from-current-layout)) deletes `generate.py`/`judge.py`/`run_pipeline.py` entirely in the same change that ships `vera.py`, so the old and new meanings of `-c`/`-r` never need to be told apart at runtime.
+**These letters are `vera.py`-only and are not the same flags as the legacy
+scripts.** `generate.py`, `judge.py`, and `run_pipeline.py` are legacy entry
+points in the process of being removed: each is deleted once its replacement
+`vera.py` command ships, and they exist only to keep unmigrated workflows
+running in the meantime. Do not build on them.
+
+They already use `-c`/`-r` for unrelated things (`-c` is `--max-concurrent` in
+`generate.py` and `--conversation` in `judge.py`; `-r` is `--runs` in
+`generate.py` and `--rubrics` in `judge.py`). `vera.py` intentionally repurposes
+them for the `u`/`c`/`j` vocabulary above. The two meanings never need to be
+told apart at runtime, because you invoke either `vera.py` or a legacy script
+explicitly.
 
 A **target** is separate from the u/c/j entities: it is a reusable evaluation
 bundle containing a rubric, personas, and all generation/judging prompts. A
@@ -92,7 +103,7 @@ a supported first-class path, not only a compatibility fallback; it selects the
 persona files and persona prompt from that target's manifest.
 
 Generation behavior is also controlled at this input boundary. CLI invocations
-default to `--turns 3`, `--output output`, unlimited concurrency, no total-word
+default to `--turns 30`, `--output output`, unlimited concurrency, no total-word
 cap, persona-first ordering, and one unnamed session. The explicit persona
 component includes the files and context template resolved from the target named
 by `--personas`; `--target` resolves the same fields while also selecting the
@@ -192,7 +203,7 @@ Top-level `generation` and `judging` blocks are **completely orthogonal** — mo
       {"name": "gpt-5", "repeats": 2}
     ],
     "personas": ["data/personas_a.json", "data/personas_b.json"],
-    "turns": 3,
+    "turns": 30,
     "output": "output",
     "max_concurrent": null,
     "max_total_words": null,