feat(cli): add unified vera.py CLI orchestrator wired to domain services

AGENTS.md

@@ -29,6 +29,7 @@ cp .env.example .env       # Add API keys (ANTHROPIC_API_KEY, OPENAI_API_KEY, et
 
 | Area | Key paths | When to edit |
 |------|-----------|--------------|
+| **Unified CLI** | `vera.py`, `utils/config_schema.py` | Command/config wiring shared across generate, judge, score, pool, pipeline |
 | **Generation** | `generate.py`, `generate_conversations/` | Conversation simulation, turns, personas |
 | **Judging** | `judge.py`, `judge/` | Rubric scoring, TSV output, question navigation |
 | **LLM providers** | `llm_clients/`, `llm_clients/llm_factory.py` | New models, custom HTTP/API providers |
@@ -38,7 +39,7 @@ cp .env.example .env       # Add API keys (ANTHROPIC_API_KEY, OPENAI_API_KEY, et
 | **Config** | `utils/model_config_loader.py`, `llm_clients/config.py` | Model name resolution, API keys |
 | **Shared utils** | `utils/` | Naming, logging, conversation layout |
 
-**Entry points:** `generate.py` (simulate), `judge.py` (evaluate), `run_pipeline.py` (full workflow), `judge/score.py` (scoring/visualization).
+**Entry points:** `vera.py` (unified CLI: generate/judge/score/pool/pipeline, see [README's Unified CLI section](./README.md#unified-cli)); `generate.py`, `judge.py`, `run_pipeline.py`, `judge/score.py` remain as legacy per-step scripts and compatibility adapters (see [docs/design/vera-cli-runtime-wiring.md](./docs/design/vera-cli-runtime-wiring.md)).
 
 **Temporary experiments:** `tmp_tests/` (not committed). **Permanent tests:** `tests/`.
 
@@ -76,6 +77,8 @@ uv run pytest tests/integration/
 
 ## Key Commands
 
+`vera.py` is the unified CLI (generate/judge/score/pool/pipeline); see [README's Unified CLI section](./README.md#unified-cli). The commands below use the legacy per-step scripts, still valid as compatibility adapters.
+
 ```bash
 # End-to-end pipeline (preferred for full workflows)
 uv run python run_pipeline.py \
@@ -167,6 +170,7 @@ One canonical home per concern — cross-link, don't copy paragraphs.
 - **Judge behavior:** [docs/judge.md](./docs/judge.md)
 - **Structured output:** [docs/structured-output.md](./docs/structured-output.md)
 - **Pre-commit hooks:** [docs/pre-commit-hooks.md](./docs/pre-commit-hooks.md)
+- **Unified CLI wiring:** [docs/design/vera-cli-runtime-wiring.md](./docs/design/vera-cli-runtime-wiring.md)
 - **Claude Code commands:** [CLAUDE.md](./CLAUDE.md), [.claude/commands/](./.claude/commands/)
 
 ## Docker

README.md

@@ -12,6 +12,7 @@ We value every interaction that follows the [Code of Conduct](https://www.contri
 
 - [Getting Started](#getting-started)
 - [Environment setup](#environment-setup)
+- [Unified CLI](#unified-cli)
 - [Connecting your own LLM, Agent, or API](#connecting-your-own-llm-or-api)
 - [Recommended settings](#recommended-settings)
 - [Reliable VERA-MH score (automated)](#reliable-vera-mh-score-automated)
@@ -32,7 +33,7 @@ We value every interaction that follows the [Code of Conduct](https://www.contri
 
 # Getting started
 
-This page covers [Environment setup](#environment-setup), optional [custom provider wiring](#connecting-your-own-llm-or-api), [Recommended settings](#recommended-settings) for comparable scores, the [automated pooled pipeline](#reliable-vera-mh-score-automated), and [Running VERA-MH step by step](#running-vera-mh-step-by-step) (`run_pipeline.py`, `generate.py`, `judge.py`, scoring, comparison, and improvement reports).
+This page covers [Environment setup](#environment-setup), the [unified CLI](#unified-cli), optional [custom provider wiring](#connecting-your-own-llm-or-api), [Recommended settings](#recommended-settings) for comparable scores, the [automated pooled pipeline](#reliable-vera-mh-score-automated), and [Running VERA-MH step by step](#running-vera-mh-step-by-step) (`run_pipeline.py`, `generate.py`, `judge.py`, scoring, comparison, and improvement reports).
 
 ## Environment setup
 
@@ -58,6 +59,54 @@ This page covers [Environment setup](#environment-setup), optional [custom provi
    pre-commit install
    ```
 
+## Unified CLI
+
+`vera.py` provides one command surface for generation, judging, scoring, pooling,
+and the end-to-end pipeline. It calls parser-independent domain functions; the
+legacy `generate.py`, `judge.py`, and scoring CLIs are not runtime dependencies:
+
+```bash
+uv run python vera.py generate \
+  -c gpt-4o \
+  -u claude-sonnet-4-5-20250929 \
+  --personas data/SI/personas.tsv
+
+uv run python vera.py judge \
+  -j gpt-5.4 \
+  --rubric data/SI/rubric_manifest.json \
+  --conversations output/<generation-run>
+
+uv run python vera.py pipeline \
+  -c gpt-4o \
+  -u claude-sonnet-4-5-20250929 \
+  -j gpt-5.4 \
+  --target SI
+```
+
+Run-defining CLI flags and JSON config are strictly either/or. `--sample` is the
+sole debug-only flag that may accompany `--config`. For standalone judging, put
+conversation paths in `judging.conversations` when using config:
+
+```json
+{
+  "judging": {
+    "models": [{"name": "gpt-5.4", "repeats": 1}],
+    "rubrics": [{"name": "SI"}],
+    "conversations": ["output/example-run"]
+  }
+}
+```
+
+Then run `uv run python vera.py judge --config run.json`. Relative paths inside
+config resolve from the repository root. `--sample N` is a debug-only cap and is
+never serialized into the resolved run config. `vera resume` is reserved but fails
+explicitly until the checksum/state recovery contract is implemented.
+
+Generation always consumes persona files. `--target` is shorthand that resolves a
+rubric manifest into both its persona files and judging rubric. A target manifest
+without `personas` remains valid for judge-only use, but cannot be used to generate;
+VERA fails explicitly rather than silently selecting default personas.
+
 ## Connecting your own LLM, Agent, or API
 
 Use this when the **provider** you want to evaluate (the mental-health chatbot under test) is **not** already available as a built-in model name in `generate.py`—for example a private HTTP API, an internal gateway, or a new cloud provider.

docs/design/vera-cli-runtime-wiring.md

@@ -0,0 +1,42 @@
+# VERA CLI runtime wiring
+
+## Context
+
+`vera.py` already defined the Phase 1 command and config shape from the architecture
+contract on `feat/VERA_2.0`, but its handlers stopped after parsing. Wiring those
+handlers exposed one ambiguity: standalone judging needs conversation paths, while
+AD-17 requires an invocation to use either JSON config or run-defining command-line
+flags, never both.
+
+## Decision
+
+- `JudgingConfig` owns `conversations`, mirroring `--conversations`.
+- Config-sourced paths resolve from the repository root, as required by AD-28.
+- `--sample` remains an invocation-only debug cap and is removed from serialized
+  `RunConfig`. Per AD-17, it is the sole flag allowed alongside `--config`;
+  `--debug` and `--print` remain CLI-only.
+- `vera.py` delegates to parser-independent application functions:
+  `generate_conversations.run_generation`, `judge.run_judging`, and
+  `judge.score.score_results_file`. It never imports or dynamically loads the
+  legacy CLI entry points.
+- The generation application function accepts resolved persona files, never a
+  rubric manifest. `--target` is an orchestration convenience: `vera.py` reads
+  its manifest, validates that it defines personas, and passes those paths to
+  generation while passing the manifest itself only to judging.
+- A manifest's `personas` field remains optional for judge-only use. It is
+  required contextually when that manifest is selected through `--target` for
+  generation; a missing list fails explicitly and never falls back to SI data.
+- The legacy scripts are compatibility adapters while callers migrate. Their
+  parsers are not architectural dependencies and can be deleted separately.
+  The legacy `generate.py --rubric-manifest` adapter performs its own manifest
+  translation before calling the generation application function.
+- Pooling already exposes `pool_evaluation_directories`; `vera.py` calls that
+  function rather than its script entry point.
+- `vera resume` fails explicitly until the architecture's checksum, state ownership,
+  and partial-write recovery contract is implemented.
+
+## Compatibility
+
+Existing config files without `judging.conversations` remain valid for `pipeline`,
+where conversations come from the generation stage. A standalone `judge` config must
+now include that field instead of combining `--config` with `--conversations`.

generate.py

@@ -2,19 +2,12 @@
 
 import argparse
 import asyncio
-import os
 import sys
-from datetime import datetime
 from typing import Any, Dict, List, Optional
 
-from generate_conversations import ConversationRunner
+from generate_conversations import run_generation
 from llm_clients.llm_interface import DEFAULT_START_PROMPT
 from utils.debug import set_debug
-from utils.naming import (
-    build_generation_run_folder_name,
-    model_token_for_run_folder,
-    parse_generation_run_folder_name,
-)
 from utils.rubric_manifest import (
     load_manifest_persona_context_template,
     load_manifest_personas,
@@ -40,165 +33,45 @@ async def main(
     session_types: Optional[List[str]] = None,
     resume: bool = False,
     rubric_manifest: Optional[str] = None,
+    persona_files: Optional[List[str]] = None,
 ) -> tuple[List[Dict[str, Any]], str]:
-    """
-    Generate conversations and return results.
-
-    Args:
-        # TODO: should the extra config be separated?
-        persona_model_config: Configuration dictionary for the persona model
-        agent_model_config: Configuration dictionary for the agent model
-        persona_extra_run_params: Extra parameters for the persona model
-        agent_extra_run_params: Extra parameters for the agent model
-        max_turns: Maximum turns per conversation
-        runs_per_prompt: Number of runs per prompt
-        persona_names: List of persona names to use. If None, uses all personas.
-        verbose: Whether to print status messages
-        output_folder: Parent directory for new runs (default ``output/``), or the
-            existing ``p_*`` run folder when ``resume`` is True.
-        max_total_words: Optional maximum total words across all responses
-        max_concurrent: Maximum number of concurrent conversations. If None, runs all
-            conversations concurrently.
-        max_personas: Optional maximum number of personas to load from CSV. If None,
-            loads all personas.
-        persona_speaks_first: If True (default), persona speaks first; else provider
-            speaks first. max_turns is adjusted so the provider always speaks last.
-        rubric_manifest: Optional path to a rubric bundle manifest (see
-            docs/architecture.md#rubric-bundle-manifest). When set, personas load
-            from the manifest's ``personas`` list instead of the default
-            ``data/SI/personas.tsv`` -- Phase 0's generation-side counterpart to
-            ``judge.py --rubrics``, so a manifest attaches personas and rubric
-            together. Only the first entry is used if the manifest lists more
-            than one.
-
-    Returns:
-        List of conversation results
-
-    Raises:
-        ValueError: Configuration error
-        Exception: Other errors
-    """
-    if verbose:
-        print("🔄 Generating conversations with the following parameters:")
-        print(f"  - Persona model: {persona_model_config}")
-        print(f"  - Agent model: {agent_model_config}")
-        print(f"  - Persona extra run params: {persona_extra_run_params}")
-        print(f"  - Agent extra run params: {agent_extra_run_params}")
-        print(f"  - Max turns: {max_turns}")
-        print(f"  - Runs per prompt: {runs_per_prompt}")
-        print(f"  - Persona names: {persona_names}")
-        print(f"  - Output folder: {output_folder}")
-        print(f"  - Run ID: {run_id}")
-        print(f"  - Max concurrent: {max_concurrent}")
-        print(f"  - Max total words: {max_total_words}")
-        print(f"  - Max personas: {max_personas}")
-        print(f"  - Persona speaks first: {persona_speaks_first}")
-        print(f"  - Resume: {resume}")
-
-    # Generate default folder name if not provided
-    if output_folder is None:
-        output_folder = "output"
-
-    persona_prompt_path = "data/SI/personas.tsv"
+    """Compatibility wrapper around the parser-independent generation service."""
+    if rubric_manifest and persona_files:
+        raise ValueError("rubric_manifest and persona_files are mutually exclusive")
+
+    resolved_persona_files = persona_files or ["data/SI/personas.tsv"]
     persona_context_template_path = "data/SI/persona_context_template.txt"
     if rubric_manifest:
-        manifest_personas = await load_manifest_personas(rubric_manifest)
-        if not manifest_personas:
+        resolved_persona_files = await load_manifest_personas(rubric_manifest)
+        if not resolved_persona_files:
             raise ValueError(
                 f"Rubric bundle manifest {rubric_manifest} has no personas listed"
             )
-        if len(manifest_personas) > 1:
-            print(
-                f"Warning: manifest lists multiple persona files "
-                f"({manifest_personas}); multi-persona-file support is not yet "
-                f"implemented, using only the first: {manifest_personas[0]}",
-                file=sys.stderr,
-            )
-        persona_prompt_path = manifest_personas[0]
         persona_context_template_path = await load_manifest_persona_context_template(
             rubric_manifest
         )
 
-    if resume:
-        if not os.path.isdir(output_folder):
-            raise ValueError(
-                "Resume mode requires --output to point to an existing run folder."
-            )
-        run_folder_name = os.path.basename(os.path.normpath(output_folder))
-        run_meta = parse_generation_run_folder_name(run_folder_name)
-        expected_persona = model_token_for_run_folder(persona_model_config["model"])
-        expected_agent = model_token_for_run_folder(agent_model_config["model"])
-
-        if run_meta["persona"] != expected_persona:
-            raise ValueError(
-                "Resume folder persona model does not match current --user-agent. "
-                f"Expected p_{expected_persona}, got p_{run_meta['persona']}."
-            )
-        if run_meta["agent"] != expected_agent:
-            raise ValueError(
-                "Resume folder provider model does not match current --provider-agent. "
-                f"Expected a_{expected_agent}, got a_{run_meta['agent']}."
-            )
-        if run_meta["turns"] != max_turns:
-            raise ValueError(
-                "Resume folder max turns does not match current --turns. "
-                f"Expected t{max_turns}, got t{run_meta['turns']}."
-            )
-        if run_meta["runs"] != runs_per_prompt:
-            raise ValueError(
-                "Resume folder runs-per-prompt does not match current --runs. "
-                f"Expected r{runs_per_prompt}, got r{run_meta['runs']}."
-            )
-        if run_id is None:
-            run_id = run_folder_name
-        elif run_id != run_folder_name:
-            raise ValueError(
-                "Resume mode requires --run-id to match the run folder name when set."
-            )
-    elif run_id is None:
-        timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
-        run_id = build_generation_run_folder_name(
-            persona_model_config["model"],
-            agent_model_config["model"],
-            max_turns,
-            runs_per_prompt,
-            timestamp,
-        )
-        output_folder = f"{output_folder}/{run_id}"
-        # TODO: do we want to give a message if the folder already exists?
-        os.makedirs(output_folder, exist_ok=True)
-
-    # Configuration
-    runner = ConversationRunner(
+    return await run_generation(
         persona_model_config=persona_model_config,
         agent_model_config=agent_model_config,
+        persona_files=resolved_persona_files,
+        persona_extra_run_params=persona_extra_run_params,
+        agent_extra_run_params=agent_extra_run_params,
         max_turns=max_turns,
         runs_per_prompt=runs_per_prompt,
-        folder_name=output_folder,
+        persona_names=persona_names,
+        verbose=verbose,
+        output_folder=output_folder,
         run_id=run_id,
         max_concurrent=max_concurrent,
         max_total_words=max_total_words,
         max_personas=max_personas,
         persona_speaks_first=persona_speaks_first,
         session_types=session_types,
         resume=resume,
-        persona_prompt_path=persona_prompt_path,
         persona_context_template_path=persona_context_template_path,
     )
 
-    # Run conversations
-    results = await runner.run_conversations(persona_names=persona_names)
-
-    if verbose:
-        skipped_n = sum(1 for r in results if r.get("skipped"))
-        ok_n = len(results) - skipped_n
-        msg = f"✅ Generated {ok_n} conversations → {output_folder}/"
-        if skipped_n:
-            msg += f" ({skipped_n} skipped)"
-        print(msg)
-
-    return results, output_folder
-
 
 if __name__ == "__main__":
     parser = argparse.ArgumentParser(description="Generate LLM conversations")

generate_conversations/__init__.py

@@ -1,5 +1,6 @@
 """Generate Conversations Package - LLM Conversation Simulation"""
 
 from .runner import ConversationRunner
+from .service import run_generation
 
-__all__ = ["ConversationRunner"]
+__all__ = ["ConversationRunner", "run_generation"]