promote: CLI hygiene, store-aware doctor, preservation guardrails + doc freshness

.claude/CLAUDE.md

@@ -114,7 +114,7 @@ falsifyai/                    ← repo root
 └── examples/
 ```
 
-All subpackages have empty `__init__.py` files only — no implementation yet.
+All subpackages shown above are implemented — the pipeline runs end-to-end (spec → materialize → execute → judge → resolve → save → CLI). See [CHANGELOG.md](../CHANGELOG.md) for what landed when.
 
 ## Design anchors (when implementing, do not reinvent)
 

AGENTS.md

@@ -114,7 +114,7 @@ falsifyai/                    ← repo root
 └── examples/
 ```
 
-All subpackages have empty `__init__.py` files only — no implementation yet.
+All subpackages shown above are implemented — the pipeline runs end-to-end (spec → materialize → execute → judge → resolve → save → CLI). See [CHANGELOG.md](CHANGELOG.md) for what landed when.
 
 ## Design anchors (when implementing, do not reinvent)
 

CHANGELOG.md

@@ -6,6 +6,22 @@ follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ## [Unreleased]
 
+### Documentation
+
+- **Architecture-doc freshness + preservation guardrails.**
+  [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) now reflects the full
+  11-command CLI surface (the read-only consumers `inspect`, `history`,
+  `timeline`, `matrix`, `verify`, `export` alongside `run` / `replay` / `diff`
+  and the `doctor` diagnostic) and the current perturbation (`unicode_chars`,
+  `paraphrase`) and invariant (`schema_match`) families. Two anti-entropy guards
+  now back the doc's claims with executable invariants: a single parametrized
+  harness asserts every read-only consumer closes its `ReplayStore` (on normal
+  return *and* on post-construction read failure), and the CLI import-hygiene
+  guard now also forbids read-only commands from importing
+  `falsifyai.verdict.resolver` — mechanizing the long-standing "consumers never
+  re-resolve" guarantee and closing a gap where `replay` had no such check. No
+  runtime behavior changes.
+
 ## [0.6.4] — 2026-06-05
 
 Patch release. Closes a self-falsification in the *evidence-generation* layer

README.md

@@ -206,7 +206,7 @@ See [`docs/case-studies/`](docs/case-studies/) for the index and the framing con
 
 ## CLI reference
 
-Ten subcommands, one workflow:
+Eleven subcommands, one workflow:
 
 ```bash
 falsifyai run <spec.yaml> [--store-path PATH] [--nli]
@@ -221,6 +221,7 @@ falsifyai minimize <spec.yaml> [--case CASE_ID] [--family typo_noise|unicode] [-
 falsifyai verify <session_id> [--store-path PATH]
 falsifyai verify --all         [--store-path PATH]
 falsifyai export <session_id> --bundle <output>.fai.zip [--spec-path PATH] [--allow-corrupted] [--overwrite] [--exported-at ISO8601] [--store-path PATH]
+falsifyai doctor [--store-path PATH]                            # read-only environment diagnostics; run this first if an install looks broken
 ```
 
 `history` shows raw newest-first rows and refuses to aggregate; `timeline` is its inference counterpart (chronological trend + regression detection). `matrix` generalizes the pairwise `diff` to N model runs. `minimize` searches for the *smallest* perturbation that breaks a case — the minimal falsifier.
@@ -264,7 +265,7 @@ Ship the *evidence* with your PR, not just the pass/fail signal:
 - **Not a prompt optimization suite.** No prompt tuning, no automated A/B over wordings. The spec is authored deliberately.
 - **Not a telemetry platform.** No streaming, no production dashboards, no time-series. The artifact is per-run preserved evidence.
 - **Not a generalized observability product.** The CLI compresses; the artifact preserves. The headline tells you whether to look; the artifact tells you what to look at.
-- **Not a workflow orchestrator.** Ten subcommands are the entire surface.
+- **Not a workflow orchestrator.** Eleven subcommands are the entire surface.
 - **Not an AI governance suite.** Governance platforms consume reliability evidence; FalsifyAI produces it.
 
 These exclusions keep the surface compressible. Adding any of them corrupts the discipline.
@@ -383,6 +384,13 @@ uv sync --extra dev
 uv run pytest
 ```
 
+If anything looks off — a confusing install, a missing optional extra, an
+unwritable store path — run the read-only diagnostic before filing an issue:
+
+```bash
+uv run falsifyai doctor
+```
+
 ---
 
 ## License

docs/ARCHITECTURE.md

@@ -109,14 +109,20 @@ piece of behavior lives in exactly one of them.
 | **Evidence interpretation** | Judges observations and compresses them into a verdict (which is *a claim about the evidence*). | `falsifyai.invariants`, `falsifyai.verdict`, `falsifyai.falsifiability`, `falsifyai.cli.render` |
 | **Evidence preservation** — *the durable product* | Persists the full evidence trail so it outlives the run. The replay artifact is the system's central object; the other two layers exist to produce and feed it. | `falsifyai.replay` (artifact, store, serializer) |
 
-The CLI subcommands (`falsifyai run` / `replay` / `diff`) are **consumers
-of these layers**, not a fourth layer:
+The CLI subcommands are **consumers of these layers**, not a fourth layer. The
+surface is 11 commands: the producer `run` (and `minimize`), the read-only
+consumers `replay`, `diff`, `inspect`, `history`, `timeline`, `matrix`,
+`verify`, and `export`, plus the `doctor` environment diagnostic.
 
 - `run` orchestrates generation → interpretation → preservation.
 - `replay` reads preservation and feeds it back through interpretation
   for rendering (never re-resolves).
 - `diff` reads two preserved artifacts and compares their interpretation
   outputs.
+- the other read-only consumers (`inspect`, `history`, `timeline`, `matrix`,
+  `verify`, `export`) follow `replay`'s shape: they read preserved verdicts and
+  never re-enter the resolver. `doctor` touches neither artifacts nor the
+  resolver — it probes the environment and store backend.
 
 ### The line to hold
 
@@ -437,8 +443,8 @@ know which question is being asked.
 
 These are not the same epistemic problem. The MVP `falsifyai run` answers
 *"Is this stable?"* (falsification-first). `falsifyai diff` answers *"Did
-migration regress?"* (comparative). Future commands (`history`,
-`inspect`) will address the others.
+migration regress?"* (comparative). The shipped consumers (`history`,
+`inspect`, `timeline`, `matrix`) address the others.
 
 **Architectural implication:** don't prematurely encode this as a CLI
 surface split (e.g., `run` vs `measure-fragility`). The strategy may be
@@ -456,13 +462,13 @@ A one-line orientation for each subpackage:
 | Subpackage | Role |
 |---|---|
 | `falsifyai.spec` | Pydantic models + YAML loader + `materialize()` |
-| `falsifyai.perturbation` | `Perturbation` Protocol + `typo_noise` + `casing_variant` + registry |
+| `falsifyai.perturbation` | `Perturbation` Protocol + `typo_noise` + `casing_variant` + `unicode_chars` + `paraphrase` (bidirectional-NLI validity) + registry |
 | `falsifyai.execution` | `ModelAdapter` Protocol + `LiteLLMAdapter` + `ExecutionEngine` + `InMemoryCache` |
-| `falsifyai.invariants` | `Invariant` Protocol + `contains` + `semantic_equivalence` + `EmbeddingBackend` |
+| `falsifyai.invariants` | `Invariant` Protocol + `contains` + `semantic_equivalence` + `schema_match` + `EmbeddingBackend` + plugin entry-point group |
 | `falsifyai.verdict` | `Verdict` enum + `resolver` (priority chain) + `stratify` + `consistency` |
 | `falsifyai.falsifiability` | Per-case + suite-level falsifiability scoring |
-| `falsifyai.replay` | `ReplayStore` Protocol + `SQLiteStore` + `InMemoryStore` + artifact + serializer |
-| `falsifyai.cli` | `main` (argparse) + `run` + `replay` + `diff` + `render` + `errors` |
+| `falsifyai.replay` | `ReplayStore` Protocol + `SQLiteStore` + `InMemoryStore` + pluggable store backends + artifact + serializer |
+| `falsifyai.cli` | `main` (argparse) + `run` / `minimize` + read-only consumers (`replay`, `diff`, `inspect`, `history`, `timeline`, `matrix`, `verify`, `export`) + `doctor` + `render` + `errors` |
 
 A new contributor should be able to find any feature in <30 seconds using
 this table.

falsifyai/cli/doctor.py

@@ -2,8 +2,9 @@
 
 Reports the runtime facts a user needs to explain a confusing install: the
 Python and package versions, whether the core runtime dependencies import,
-which optional extras (``[semantic]``, ``[nli]``) are available, and whether a
-replay store can actually be written.
+which optional extras (``[semantic]``, ``[nli]``) are available, which store
+backend the configured ``--store-path`` selects (and whether that backend is
+registered), and whether a replay store can actually be written.
 
 It *diagnoses only*. It never installs anything, writes config, or mutates user
 data -- the store-writability probe lives in a tempfile and is deleted, and
@@ -95,8 +96,45 @@ def _extra_check(label: str, modules: list[str], extra: str) -> Check:
     return Check(label, "not installed", _INFO, f'pip install "falsifyai[{extra}]"')
 
 
-def _store_check(store_path: str) -> Check:
-    """Prove a replay store can be written -- via a throwaway tempfile DB, never
+def _store_checks(store_path: str) -> list[Check]:
+    """Diagnose the store the configured ``--store-path`` actually selects.
+
+    Two facts. (1) *Backend resolution*: is a store registered for the selected
+    URI scheme? This is the same resolution ``build_store`` performs, so an
+    unknown scheme (a missing store plugin) fails here in diagnostics instead of
+    surfacing as a crash at ``run`` time. (2) *Writability*: for the built-in
+    SQLite store, prove a store can actually be written. A plugin store
+    (``postgres://`` ...) is reported as registered but *not* write-probed --
+    constructing one may open a network connection or allocate resources, which
+    would break doctor's diagnose-only contract.
+    """
+    from falsifyai.replay.registry import discover_stores, store_scheme
+
+    scheme = store_scheme(store_path)
+    backends = sorted(discover_stores())  # reads entry points only; no construction
+
+    if scheme not in backends:
+        return [
+            Check(
+                "store backend",
+                f"{scheme} (no backend registered)",
+                _FAIL,
+                f"available: {', '.join(backends) or 'none'} -- install a store plugin "
+                f"for {scheme!r}",
+            )
+        ]
+
+    backend = Check("store backend", f"{scheme} (available: {', '.join(backends)})", _OK)
+    if scheme == "sqlite":
+        return [backend, _sqlite_write_check(store_path)]
+    if scheme == "memory":
+        return [backend, Check("store write", ":memory: (ephemeral, no disk)", _OK)]
+    # A registered plugin backend: don't construct it (possible side effects).
+    return [backend, Check("store write", f"{store_path} (plugin store; not probed)", _INFO)]
+
+
+def _sqlite_write_check(store_path: str) -> Check:
+    """Prove a SQLite store can be written -- via a throwaway tempfile DB, never
     the user's real store -- and that the configured store dir is writable."""
     from falsifyai.replay.sqlite_store import SQLiteStore
 
@@ -129,7 +167,7 @@ def collect_checks(store_path: str) -> list[Check]:
         _core_deps_check(),
         _extra_check("[semantic] extra", ["sentence_transformers"], "semantic"),
         _extra_check("[nli] extra", ["transformers", "torch"], "nli"),
-        _store_check(store_path),
+        *_store_checks(store_path),
     ]
 
 

falsifyai/cli/main.py

@@ -28,17 +28,6 @@
 import sys
 from collections.abc import Sequence
 
-from falsifyai.cli import diff as diff_cmd
-from falsifyai.cli import doctor as doctor_cmd
-from falsifyai.cli import export as export_cmd
-from falsifyai.cli import history as history_cmd
-from falsifyai.cli import inspect as inspect_cmd
-from falsifyai.cli import matrix as matrix_cmd
-from falsifyai.cli import minimize as minimize_cmd
-from falsifyai.cli import replay as replay_cmd
-from falsifyai.cli import run as run_cmd
-from falsifyai.cli import timeline as timeline_cmd
-from falsifyai.cli import verify as verify_cmd
 from falsifyai.cli.errors import CLIError
 
 
@@ -322,28 +311,56 @@ def main(argv: Sequence[str] | None = None) -> int:
         parser.print_help()
         return 0
 
+    # Command modules are imported lazily inside each branch, not at module top
+    # level: ``run`` / ``minimize`` pull in the model-execution stack (litellm),
+    # which is dead weight — and emits import-time warnings — for the read-only
+    # commands (``doctor``, ``verify``, ``replay``, ...). Dispatching one must not
+    # load the model stack. The guard in tests/meta/test_cli_import_hygiene.py
+    # fails if a command import leaks back to module top level.
     try:
         if args.command == "run":
+            from falsifyai.cli import run as run_cmd
+
             return run_cmd.cmd_run(args)
         if args.command == "replay":
+            from falsifyai.cli import replay as replay_cmd
+
             return replay_cmd.cmd_replay(args)
         if args.command == "inspect":
+            from falsifyai.cli import inspect as inspect_cmd
+
             return inspect_cmd.cmd_inspect(args)
         if args.command == "diff":
+            from falsifyai.cli import diff as diff_cmd
+
             return diff_cmd.cmd_diff(args)
         if args.command == "history":
+            from falsifyai.cli import history as history_cmd
+
             return history_cmd.cmd_history(args)
         if args.command == "timeline":
+            from falsifyai.cli import timeline as timeline_cmd
+
             return timeline_cmd.cmd_timeline(args)
         if args.command == "matrix":
+            from falsifyai.cli import matrix as matrix_cmd
+
             return matrix_cmd.cmd_matrix(args)
         if args.command == "minimize":
+            from falsifyai.cli import minimize as minimize_cmd
+
             return minimize_cmd.cmd_minimize(args)
         if args.command == "verify":
+            from falsifyai.cli import verify as verify_cmd
+
             return verify_cmd.cmd_verify(args)
         if args.command == "export":
+            from falsifyai.cli import export as export_cmd
+
             return export_cmd.cmd_export(args)
         if args.command == "doctor":
+            from falsifyai.cli import doctor as doctor_cmd
+
             return doctor_cmd.cmd_doctor(args)
     except CLIError as exc:
         print(f"falsifyai: error: {exc}", file=sys.stderr)

falsifyai/execution/__init__.py

@@ -1,12 +1,16 @@
 """Execution layer — adapter, cache, and engine."""
 
+from typing import TYPE_CHECKING
+
 from falsifyai.execution.adapter import ModelAdapter
 from falsifyai.execution.cache import ExecutionCache, InMemoryCache
 from falsifyai.execution.engine import ExecutionEngine
 from falsifyai.execution.errors import ExecutionError
-from falsifyai.execution.litellm_adapter import LiteLLMAdapter
 from falsifyai.execution.models import Execution, ModelRequest
 
+if TYPE_CHECKING:
+    from falsifyai.execution.litellm_adapter import LiteLLMAdapter
+
 __all__ = [
     "Execution",
     "ExecutionCache",
@@ -17,3 +21,19 @@
     "ModelAdapter",
     "ModelRequest",
 ]
+
+
+def __getattr__(name: str) -> object:
+    # ``LiteLLMAdapter`` is resolved lazily (PEP 562): importing it pulls in
+    # ``litellm`` — heavy, and noisy with import-time warnings. Touching any
+    # execution *submodule* (e.g. ``execution.models`` via a replay artifact)
+    # runs this package ``__init__``; eagerly importing the adapter here would
+    # drag litellm into every read-only CLI command. Deferring it keeps
+    # ``from falsifyai.execution import LiteLLMAdapter`` working while only
+    # paying the cost when the adapter is actually used. Guarded by
+    # tests/meta/test_cli_import_hygiene.py.
+    if name == "LiteLLMAdapter":
+        from falsifyai.execution.litellm_adapter import LiteLLMAdapter
+
+        return LiteLLMAdapter
+    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")

tests/integration/test_run_end_to_end.py

@@ -99,3 +99,50 @@ def test_artifact_roundtrip_via_load_session(tmp_path, monkeypatch) -> None:
 
     # Semantic equality across save -> load.
     assert loaded == sessions
+
+
+# ---------------------------------------------------------------------------
+# Store lifecycle: run is a *producer*; it owns the store it writes to and must
+# close it whether the save succeeds or raises. (Read-only consumers are covered
+# by tests/unit/test_cli_store_lifecycle.py; run is kept here, with the model
+# stack already mocked, so the producer/consumer boundary stays clean.)
+# ---------------------------------------------------------------------------
+
+
+class _TrackingStore:
+    """Minimal producer-side store double: records save_session + close."""
+
+    def __init__(self, *, fail_save: bool = False) -> None:
+        self.saved: list = []
+        self.closed = False
+        self._fail_save = fail_save
+
+    def save_session(self, artifact) -> None:
+        if self._fail_save:
+            raise RuntimeError("forced save failure")
+        self.saved.append(artifact)
+
+    def close(self) -> None:
+        self.closed = True
+
+
+def test_run_closes_store_after_save(monkeypatch, capsys) -> None:
+    _patch_adapter(monkeypatch, "Paris is the capital of France.")
+    store = _TrackingStore()
+    monkeypatch.setattr(cli_run, "build_store", lambda _p: store)
+
+    cli_run.cmd_run(_args(_SMOKE_SPEC, ":memory:"))
+
+    assert len(store.saved) == 1
+    assert store.closed is True
+
+
+def test_run_closes_store_when_save_fails(monkeypatch, capsys) -> None:
+    _patch_adapter(monkeypatch, "Paris is the capital of France.")
+    store = _TrackingStore(fail_save=True)
+    monkeypatch.setattr(cli_run, "build_store", lambda _p: store)
+
+    with pytest.raises(RuntimeError):
+        cli_run.cmd_run(_args(_SMOKE_SPEC, ":memory:"))
+
+    assert store.closed is True