Dogfood GR00T visual evidence artifacts

CONTEXT.md

@@ -12,6 +12,26 @@ _Avoid_: Model Judge, external VLM judge, standalone VLM evaluator
 A compact set of visual evidence, metrics, baseline comparisons, and decision metadata prepared for agent visual review or human escalation.
 _Avoid_: Screenshot folder, dashboard dump, artifact pile
 
+**Evidence Producer**:
+A downstream-project-owned runner that executes robot, simulator, or hardware-specific behavior and emits roboharness evidence artifacts. Roboharness may define artifact schemas and assembly rules, but it does not own the downstream robot runtime, safety startup, controller transport, or task planner.
+_Avoid_: Roboharness runtime, hidden simulator backend, adapter that owns project control
+
+**Semantic Snapshot**:
+A named semantic phase sample containing the project state and metrics needed for deterministic replay, checking, or review. It is not required to come from a `SimulatorBackend.step()` loop.
+_Avoid_: Raw screenshot, simulator checkpoint only, full episode recording
+
+**Semantic Snapshot Bundle**:
+An ordered, replayable set of **Semantic Snapshot** records plus run metadata. It is the evidence-producer handoff into renderers, metric checks, log diagnostics, proof-pack assembly, or visual review preparation.
+_Avoid_: Screenshot folder, ad hoc artifact directory, renderer-specific dump
+
+**Renderer Report**:
+The structured output from rendering a **Semantic Snapshot Bundle** through one renderer. It records capture/motion trustworthiness, per-snapshot image evidence, renderer metadata, and flags; it does not decide final approval by itself.
+_Avoid_: Visual verdict, screenshot list, renderer log only
+
+**Autonomous Evidence Report**:
+The machine-readable report that aggregates run identity, plan/runtime metadata, summary metrics, snapshot metrics, renderer reports, verdict reasons, and failure taxonomy before proof-pack approval or human escalation.
+_Avoid_: HTML report, visual review record, baseline blessing
+
 **Visual Reviewer Invocation**:
 An isolated agent visual review call that reads a proof pack and returns a structured verdict. It may run through a subagent, CLI trigger, CI job, or MCP tool, but its context is bounded to the evidence and review contract.
 _Avoid_: Separate VLM, same-session self-review, screenshot glance

docs/adr/0002-post-hoc-semantic-snapshot-evidence.md

@@ -0,0 +1,56 @@
+# Post-hoc semantic snapshot evidence is first-class
+
+Status: accepted
+
+Roboharness will support post-hoc semantic snapshot evidence as a first-class
+artifact path alongside the existing `SimulatorBackend.step()` harness loop. A
+downstream project may run its own robot runtime, simulator launch topology,
+hardware-live surface, safety checks, and controller transport, then hand
+Roboharness an ordered semantic snapshot bundle. Roboharness owns the reusable
+artifact language around that handoff: semantic snapshot bundles, renderer
+reports, autonomous evidence reports, proof-pack assembly inputs, baseline
+comparison surfaces, and bounded visual review preparation.
+
+The `SimulatorBackend` protocol remains the simple package-first path for
+step-oriented simulators. It is not the only way to use Roboharness. This keeps
+hardware-style and deploy-live flows from pretending to be action-sequence
+loops just to fit the original core API.
+
+Project-specific execution stays outside Roboharness. The downstream project
+owns task planning, runtime startup, robot model loading, safety evidence,
+control backend selection, renderer implementation, and any domain-specific
+semantic checks. Roboharness may validate and aggregate the artifacts those
+systems emit, but it must not absorb the project runtime.
+
+The first dogfood target is GR00T whole-body control. That repo already
+demonstrates a useful flow: run a representative manipulation surface, record
+semantic phase snapshots, replay those snapshots through Meshcat or MuJoCo,
+then aggregate snapshot metrics, renderer trustworthiness, verdict reasons, and
+failure taxonomy into an autonomous report. If Roboharness cannot express that
+flow naturally, the Roboharness abstraction is incomplete.
+
+Consequences:
+
+- Add a public `roboharness.evidence` package for artifact models and JSON
+  helpers.
+- Keep `roboharness.core.Harness` and `SimulatorBackend` intact as one ingestion
+  path rather than broadening them into downstream runtime orchestration.
+- Keep `roboharness.approval` focused on paired evidence, visual review records,
+  and approval summaries.
+- Let project `HarnessContract` workflows refer to evidence expectations over
+  time, but do not turn the contract into a renderer or runtime configuration
+  system.
+- Validate GR00T-style snapshot bundles and renderer reports through fixtures
+  before migrating the GR00T runner itself.
+
+Rejected alternatives:
+
+- A GR00T-only adapter layer in the downstream repo. This would prove only that
+  another wrapper can be written, not that Roboharness has the right reusable
+  evidence abstraction.
+- Forcing deploy-live and hardware-live evidence through `SimulatorBackend`.
+  That would couple Roboharness to action-loop assumptions that are false for
+  many real robot validation surfaces.
+- Moving robot runtime/session ownership into Roboharness. That would expand
+  scope beyond the evidence and approval layer and make each downstream robot
+  integration harder to maintain.

docs/plans/gr00t-dogfood-semantic-snapshot-evidence.md

@@ -0,0 +1,106 @@
+# GR00T Dogfood: Semantic Snapshot Evidence
+
+Status: active
+
+## Context
+
+GR00T whole-body control built a mature visual harness around semantic phase
+snapshots, deterministic replay rendering, renderer trustworthiness reports,
+and autonomous visual reports. That success should move the reusable evidence
+language back into Roboharness instead of leaving each downstream repository to
+rebuild the same layer.
+
+This is dogfooding in the strict sense: GR00T should be able to use
+Roboharness for the visual evidence lifecycle without Roboharness taking over
+GR00T's robot runtime, task planner, control backends, safety startup, or
+renderer implementation.
+
+## Accepted Direction
+
+- Roboharness owns public artifact concepts for semantic snapshot bundles,
+  renderer reports, and autonomous evidence reports.
+- GR00T visual harness dogfooding treats `roboharness.evidence` as a hard
+  artifact-layer dependency, not an optional fallback. During local development,
+  GR00T installs the sibling Roboharness checkout into its active `.venv`;
+  after the API supports the repo's visual harness, GR00T can switch to a `uv`
+  git-main dependency.
+- `SimulatorBackend.step()` remains one useful ingestion path, not the only
+  Roboharness model.
+- Downstream projects own evidence producers: runtime sessions, robot-specific
+  execution, semantic snapshot recording, and renderer implementations.
+- Roboharness owns reusable artifact validation, JSON round-tripping,
+  proof-pack assembly inputs, baseline/approval surfaces, and bounded visual
+  review preparation.
+- GR00T compatibility is a first acceptance gate, but the first Roboharness
+  slice uses small fixtures rather than importing or depending on the GR00T
+  repository.
+
+## Non-Goals
+
+- Do not migrate the GR00T runner in this slice.
+- Do not add a new simulator backend.
+- Do not change MuJoCo wedge approval semantics.
+- Do not make `HarnessContract` a runtime or renderer configuration system.
+- Do not bless new baselines or alter human escalation policy.
+
+## First Slice
+
+1. Add a public `roboharness.evidence` package with typed artifact models and
+   JSON helpers.
+2. Support GR00T-style `snapshot_bundle.json` and renderer report shapes through
+   round-trip tests.
+3. Export the new artifact API from package boundaries without importing
+   optional simulator dependencies.
+4. Record the architecture direction in `CONTEXT.md` and ADR 0002.
+
+## Acceptance Gates
+
+- A core-package import still works without optional simulator extras.
+- Unit tests prove semantic snapshot bundles, renderer reports, and autonomous
+  evidence reports can be parsed and re-emitted without losing unknown
+  downstream fields.
+- Focused lint passes for touched Python files.
+
+## Intuitive Flow Precheck
+
+- Scope: keep this slice to public artifact models, JSON helpers, exports, and
+  fixture tests. Do not migrate downstream runners.
+- Risks: preserve unknown downstream fields so GR00T can adopt the API without
+  losing project-specific metrics, runtime, plan, or renderer metadata.
+- Tests: add focused unit tests for GR00T-style snapshot bundles, renderer
+  reports, autonomous evidence reports, and core package import without
+  optional simulator extras.
+- DX: expose the API from `roboharness.evidence` and top-level package exports
+  so downstream contracts can import it without depending on examples.
+- Execution: implement in one bounded slice, then run focused tests and lint on
+  touched Python files.
+
+## Later Slices
+
+- [x] Add a GR00T dogfood gate that installs Roboharness explicitly and fails
+  when `roboharness.evidence` is missing or stale.
+- [x] Generate visual review manifests from evidence bundles and contract
+  dimensions.
+- [x] Add proof-pack assembly helpers that consume autonomous evidence reports.
+- [x] Teach a GR00T project harness skill to use the new Roboharness evidence
+  API.
+- Migrate GR00T visual harness code only after fixture compatibility and public
+  boundaries are stable.
+
+## Follow-Up Slice: Proof Pack And Static Review
+
+Implemented after the first artifact-model slice:
+
+- `roboharness.evidence` now exposes case-level proof-pack assembly for
+  downstream visual harness case directories.
+- The first proof-pack assembler consumes `autonomous_report.json`,
+  `snapshot_bundle.json`, and renderer `report.json` files without importing
+  GR00T or owning downstream runtime execution.
+- Static visual-review manifest generation is current-only and selects
+  case-local keyframes from the proof pack. It sets
+  `allow_automatic_visual_pass=false`, so current-only review can veto or
+  escalate but cannot bless an automatic visual pass.
+- The `roboharness proof-pack` CLI writes `proof_pack.json` and optionally a
+  validated `visual_review_manifest.json`.
+- GR00T owns the project-specific dogfood gate and generated project harness
+  contract under its `skills/visual-harness/roboharness/` directory.

src/roboharness/__init__.py

@@ -43,6 +43,26 @@
     evaluate_policy,
 )
 from roboharness.evaluate.result import EvaluationResult, Operator, Severity, Verdict
+from roboharness.evidence import (
+    AutonomousEvidenceReport,
+    CaseProofPack,
+    ProofPackArtifact,
+    ProofPackImageRef,
+    RenderedImage,
+    RendererReport,
+    RendererSnapshot,
+    SemanticSnapshot,
+    SemanticSnapshotBundle,
+    SuiteProofPack,
+    SuiteProofPackCase,
+    VisualReviewQueue,
+    VisualReviewQueueItem,
+    build_case_proof_pack,
+    build_paired_visual_review_manifest,
+    build_static_visual_review_manifest,
+    build_suite_proof_pack,
+    build_visual_review_queue,
+)
 from roboharness.runner import BatchResult, ParallelTrialRunner, TrialSpec
 from roboharness.storage.history import EvaluationHistory, EvaluationRecord, TrendResult
 
@@ -55,8 +75,10 @@
     "REACH_PROTOCOL",
     "ApprovalPolicy",
     "AssertionEngine",
+    "AutonomousEvidenceReport",
     "BatchResult",
     "CaptureResult",
+    "CaseProofPack",
     "Checkpoint",
     "CheckpointStore",
     "ComponentAssumption",
@@ -79,15 +101,31 @@
     "MetricGate",
     "Operator",
     "ParallelTrialRunner",
+    "ProofPackArtifact",
+    "ProofPackImageRef",
+    "RenderedImage",
+    "RendererReport",
+    "RendererSnapshot",
     "SemanticPhase",
+    "SemanticSnapshot",
+    "SemanticSnapshotBundle",
     "Severity",
+    "SuiteProofPack",
+    "SuiteProofPackCase",
     "TaskPhase",
     "TaskProtocol",
     "TrendResult",
     "TrialSpec",
     "ValidationCommand",
     "Verdict",
     "VisualReviewDimension",
+    "VisualReviewQueue",
+    "VisualReviewQueueItem",
+    "build_case_proof_pack",
+    "build_paired_visual_review_manifest",
+    "build_static_visual_review_manifest",
+    "build_suite_proof_pack",
+    "build_visual_review_queue",
     "check_eval_threshold",
     "default_registry",
     "evaluate_policy",

src/roboharness/approval/__init__.py

@@ -13,20 +13,24 @@
 from roboharness.approval.visual_review import (
     MANIFEST_SCHEMA_VERSION,
     RECORD_SCHEMA_VERSION,
+    VISUAL_REVIEW_SUMMARY_SCHEMA_VERSION,
     VisualReviewPackage,
     VisualReviewResult,
     VisualReviewValidationError,
     build_visual_review_prompt,
     build_visual_review_schema,
+    build_visual_review_summary,
     ingest_visual_review_record,
     validate_visual_review_manifest,
     validate_visual_review_record,
     write_visual_review_package,
+    write_visual_review_summary,
 )
 
 __all__ = [
     "MANIFEST_SCHEMA_VERSION",
     "RECORD_SCHEMA_VERSION",
+    "VISUAL_REVIEW_SUMMARY_SCHEMA_VERSION",
     "EvidencePair",
     "EvidenceStatus",
     "EvidenceTarget",
@@ -36,6 +40,7 @@
     "VisualReviewValidationError",
     "build_visual_review_prompt",
     "build_visual_review_schema",
+    "build_visual_review_summary",
     "ingest_visual_review_record",
     "render_lightbox_shell",
     "render_zoomable_image",
@@ -44,4 +49,5 @@
     "validate_visual_review_manifest",
     "validate_visual_review_record",
     "write_visual_review_package",
+    "write_visual_review_summary",
 ]

src/roboharness/approval/visual_review.py

@@ -12,6 +12,7 @@
 
 MANIFEST_SCHEMA_VERSION = "roboharness_visual_review_manifest/v1"
 RECORD_SCHEMA_VERSION = "roboharness_visual_review/v1"
+VISUAL_REVIEW_SUMMARY_SCHEMA_VERSION = "roboharness_visual_review_summary/v1"
 
 SUPPORTED_DIMENSIONS = frozenset(
     {
@@ -98,6 +99,56 @@ def __init__(self, errors: Sequence[str]):
         super().__init__("; ".join(self.errors))
 
 
+def build_visual_review_summary(
+    manifest: Mapping[str, Any],
+    record: Mapping[str, Any],
+    *,
+    manifest_path: str = "visual_review_manifest.json",
+    record_path: str = "visual_review.json",
+) -> dict[str, Any]:
+    """Build a persisted summary from a bounded visual review record."""
+
+    result = ingest_visual_review_record(
+        manifest,
+        record,
+        manifest_path=manifest_path,
+        record_path=record_path,
+    )
+    case_id = manifest.get("case_id") if isinstance(manifest.get("case_id"), str) else ""
+    if not case_id and isinstance(record.get("case_id"), str):
+        case_id = str(record["case_id"])
+    return {
+        "schema_version": VISUAL_REVIEW_SUMMARY_SCHEMA_VERSION,
+        "case_id": case_id,
+        "is_valid": result.is_valid,
+        "effective_visual_verdict": result.effective_visual_verdict,
+        "summary": dict(result.summary),
+    }
+
+
+def write_visual_review_summary(
+    manifest: Mapping[str, Any],
+    record: Mapping[str, Any],
+    path: str | Path,
+    *,
+    manifest_path: str = "visual_review_manifest.json",
+    record_path: str = "visual_review.json",
+) -> Path:
+    """Write a persisted summary from a bounded visual review record."""
+
+    output_path = Path(path)
+    save_json(
+        build_visual_review_summary(
+            manifest,
+            record,
+            manifest_path=manifest_path,
+            record_path=record_path,
+        ),
+        output_path,
+    )
+    return output_path
+
+
 def build_visual_review_schema() -> dict[str, Any]:
     """Return the JSON schema given to the visual reviewer."""
     return {