feat: add seconds-based task horizons

CHANGELOG.md

@@ -21,6 +21,12 @@ All notable changes to this project are documented here. The format is based on
 
 ### Changed
 
+- **Task horizon binding now follows compatibility checking.** An
+  embodiment's optional `bind_task()` hook receives the resolved step envelope
+  only after the policy/embodiment/task triple is known to be compatible. This
+  prevents adapters from acting on a seconds-derived budget built from an
+  invalid control rate (#160).
+
 - **Docs site migrated from MkDocs Material to Docusaurus.** The site at
   inspectrobots.org now builds from `website/` (Docusaurus 3) while the
   Markdown source stays in `docs/`; every existing URL, `llms.txt`, and
@@ -39,6 +45,13 @@ All notable changes to this project are documented here. The format is based on
 
 ### Added
 
+- **Seconds-based benchmark horizons:** `Task(max_seconds=...)` gives every
+  compatible embodiment the same physical-time budget. `eval()` resolves it
+  with `ceil(max_seconds * embodiment.info.control_hz)`, rejects missing or
+  invalid control rates before `bind_task()` or rollout, and records both the
+  declared seconds and resolved steps in eval logs, CLI summaries, inspection,
+  and HTML reports (#160).
+
 - **Policy lifecycle hook: `on_trial_end`** — policies can now hook into
   the end of a trial to persist state or artifacts. The orchestrator calls
   `policy.on_trial_end(record, log_dir, run_id)` and any metadata the policy

docs/guide/cli.md

@@ -210,6 +210,10 @@ inspect-robots eval-set 'kitchenbench/*' --policy xpolicylab -P url=ws://host:19
 inspect-robots eval-set cubepick-reach my-other-task --policy scripted --embodiment cubepick
 ```
 
+For a task declared with `max_seconds`, its summary row includes both the
+physical-time budget and the integer step limit resolved from the selected
+embodiment, for example `[120s -> 1200 steps at 10 Hz]`.
+
 Multiple patterns may match the same task; it still runs once. A pattern that
 matches nothing is an error listing every registered task. `--policy` and
 `--embodiment` (and `-P`/`-E`, `--sim`, `--epochs`, `--fail-on-error`,
@@ -253,6 +257,7 @@ policy:      scripted
 embodiment:  cubepick
 run status:  completed
 outcome:     5 succeeded
+horizon:     120s -> 1200 steps at 10 Hz
 scenes:      5   trials: 5
 metrics:
   success_at_end: 1
@@ -261,6 +266,10 @@ scenes:
   ...
 ```
 
+The `horizon` line appears for seconds-based tasks; step-only logs retain the
+existing output. The HTML viewer likewise separates declared seconds from the
+resolved step limit.
+
 `completed` is the display form of the log's `success` status value; the
 on-disk field and Python API keep `success`.
 

docs/guide/concepts.md

@@ -22,8 +22,9 @@ without inheriting anything. Convenience base classes (`PolicyBase`,
 ## Tasks and scenes
 
 A [`Task`](/api/#inspect_robots.task.Task) is an embodiment-agnostic benchmark: a dataset
-of [`Scene`](/api/#inspect_robots.scene.Scene)s plus scorer(s), a step horizon, and an epoch
-count. A `Scene` is the robotics analog of Inspect AI's `Sample`, one initial
+of [`Scene`](/api/#inspect_robots.scene.Scene)s plus scorer(s), a step- or seconds-based
+horizon, and an epoch count. A seconds horizon is resolved to integer steps from
+the paired embodiment's declared control rate. A `Scene` is the robotics analog of Inspect AI's `Sample`, one initial
 condition: an instruction, an optional success [`Target`](/api/#inspect_robots.scene.Target),
 and a seed.
 
@@ -33,7 +34,8 @@ Before any rollout, [`check_compatibility`](/api/#inspect_robots.compat.check_co
 `(policy, embodiment)` pair: action dimensions and [`ActionSemantics`](/api/#inspect_robots.spaces.ActionSemantics)
 (control mode, rotation representation, gripper, frame), the observation
 cameras/state keys the policy requires (resolving a name remap), the control rate,
-and whether each scene is realizable on the embodiment. Hard mismatches fail fast
+whether a seconds-based task has a usable control rate, and whether each scene
+is realizable on the embodiment. Hard mismatches fail fast
 with a [`CompatibilityError`](/api/#inspect_robots.errors.CompatibilityError).
 
 ## The rollout

docs/guide/writing-a-benchmark.md

@@ -63,6 +63,31 @@ task = Task(
 )
 ```
 
+## Horizons
+
+A task declares exactly one rollout horizon. Use `max_steps` when the protocol
+is inherently discrete, as in the examples above. Use `max_seconds` when every
+embodiment should receive the same physical-time budget:
+
+```python
+task = Task(
+    name="two-minute-reach",
+    scenes=[...],
+    scorer=success_at_end(),
+    max_seconds=120.0,
+)
+```
+
+At evaluation time, Inspect Robots resolves the budget as
+`ceil(max_seconds * embodiment.info.control_hz)`. A 120-second task therefore
+runs for 1,200 steps at 10 Hz and 1,800 steps at 15 Hz. The eval log records
+both the declared seconds and the resolved integer step limit.
+
+A seconds-based task is incompatible with an embodiment whose `control_hz` is
+missing, non-finite, zero, or negative. Event-driven embodiments should use
+`max_steps`. Resolution changes the step budget only: `rollout()` does not add
+wall-clock pacing, so real-time cadence remains the embodiment's responsibility.
+
 ## Registering for discovery
 
 Wrap a task factory with [`task`](/api/#inspect_robots.registry.task) so it resolves by name in

plans/0026-seconds-based-horizons.md

@@ -0,0 +1,79 @@
+# 0026: Seconds-based task horizons
+
+Issue: [#160](https://github.com/robocurve/inspect-robots/issues/160)
+Status: proposed (implementation follows the issue sketch and resolves its open questions)
+
+## 1. Problem
+
+`Task.max_steps` fixes the rollout budget before a task is paired with an
+embodiment, but a step has a physical duration only after the embodiment's
+`control_hz` is known. The same 600-step task therefore receives 60 seconds on
+a 10 Hz embodiment and 40 seconds on a 15 Hz embodiment. Benchmarks that define
+physical-time protocols cannot currently express their horizon without
+hard-coding one embodiment's rate.
+
+## 2. Design
+
+`Task` accepts exactly one horizon:
+
+- `max_steps: int | None`, preserving the existing step-based contract; or
+- keyword-only `max_seconds: float | None`, a finite positive physical-time
+  budget resolved by `eval()`.
+
+After policy binding, `eval()` runs compatibility checking. A seconds-based
+task is incompatible with an embodiment whose `control_hz` is missing,
+non-finite, or non-positive. Once compatibility passes, the task resolves a
+`TaskEnvelope` with:
+
+```python
+max_steps = ceil(max_seconds * embodiment.info.control_hz)
+```
+
+The resolved envelope is passed to `bind_task()` and the rollout. This changes
+the hook order from bind-then-compatibility to compatibility-then-bind. The
+ordering is required because an adapter must never receive a seconds-derived
+step budget based on an invalid rate.
+
+`EvalSpec` records both values:
+
+- `max_seconds`: the declared physical-time budget, or `None` for a
+  step-based task;
+- `max_steps`: the resolved integer budget used by the rollout.
+
+The schema version remains 1 because `max_seconds` is additive and defaults to
+`None` when newer code reads an older log.
+
+## 3. Open-question resolutions
+
+- **Missing or zero rate:** reject at compatibility-check time. Event-driven
+  embodiments must use `max_steps` because physical seconds cannot be resolved
+  from their declared contract.
+- **Scorers:** stay steps-only in this change. They already receive the recorded
+  trajectory, and adding task configuration to the scoring interface is a
+  separate API decision.
+- **`eval-set`:** show the declared seconds and resolved steps on each task row.
+  Saved-log inspection and the HTML report show the same pair.
+
+## 4. Non-goals
+
+- No wall-clock pacing in `rollout()`; self-paced embodiments still own their
+  cadence.
+- No CLI `--max-seconds` override for ad-hoc runs. This change lets benchmark
+  authors declare physical-time protocols; the existing ad-hoc `--max-steps`
+  control is unchanged.
+- No scorer API or metric changes.
+- No dependency changes.
+
+## 5. Validation
+
+- Task construction rejects missing, duplicate, non-positive, and non-finite
+  horizon declarations.
+- Compatibility reports invalid embodiment rates before `bind_task()` or
+  rollout.
+- Resolution uses `ceil`, including non-integral products, and the exact
+  resolved value reaches `TaskEnvelope`, `rollout()`, and `EvalLog`.
+- Older schema-v1 logs without `max_seconds` still read with `None`.
+- CLI run/inspect/eval-set output and HTML reports surface the declared and
+  resolved horizons without changing old step-only output.
+- Ruff, formatting, strict mypy, 100% line and branch coverage, and the strict
+  Docusaurus build remain green.

src/inspect_robots/CLAUDE.md

@@ -10,9 +10,9 @@ interfaces. The package is `mypy --strict` clean and ships `py.typed`.
 | `types.py` | `Observation`, `Action`, `ActionChunk`, `StepResult` (frozen, NumPy-native) |
 | `spaces.py` | `Box`, `ObservationSpace`, `ActionSemantics`, `StateSpec` + canonical state vocab |
 | `policy.py` | `Policy` Protocol + `PolicyBase` ABC, `PolicyInfo`, `PolicyConfig`; optional duck-typed `bind(embodiment_info)` hook for embodiment-adaptive policies plus `transcript()` and `transcript_delta()` hooks for complete and live per-trial audit records |
-| `embodiment.py` | `Embodiment` Protocol + `EmbodimentBase` ABC, `EmbodimentInfo`, capability flags; optional duck-typed `bind_task(envelope)` hook for horizon-aware adapters (called by `eval()` before compat; optional input — never fires on direct `rollout()`, keep a fallback) |
+| `embodiment.py` | `Embodiment` Protocol + `EmbodimentBase` ABC, `EmbodimentInfo`, capability flags; optional duck-typed `bind_task(envelope)` hook for horizon-aware adapters (called by `eval()` after compat with a resolved step envelope; optional input — never fires on direct `rollout()`, keep a fallback) |
 | `scene.py` | `Scene` (the Inspect `Sample` analog), `Target`, `ListSceneDataset` |
-| `task.py` | `Task` (scenes + scorer + horizon), `Epochs`, `TaskEnvelope` (`Task.envelope` — the adapter-safe identity+limits view passed to `bind_task` hooks) |
+| `task.py` | `Task` (scenes + scorer + exactly one `max_steps`/`max_seconds` horizon), `Epochs`, `TaskEnvelope` (`resolve_envelope(control_hz)` — the adapter-safe identity+resolved-step limits view passed to `bind_task` hooks) |
 | `scorer.py` | `Score`/`Scorer`, epoch reducers, builtin scorers (incl. operator/VLM) |
 | `controller.py` | `Controller` middleware: `DefaultController` (open-loop chunking), `SmoothingController` |
 | `approver.py` | `Approver` safety gate: `AutoApprover`, `ClampApprover`, `DeltaLimitApprover` (semantics-aware no-wild-swings limit), `ChainApprover` |

src/inspect_robots/_html.py

@@ -556,7 +556,11 @@ def render_html(
     )
     if log.eval.seed is not None:
         definitions.append(_definition("seed", log.eval.seed))
-    if log.eval.max_steps is not None:
+    if log.eval.max_seconds is not None:
+        definitions.append(_definition("max seconds", log.eval.max_seconds))
+        if log.eval.max_steps is not None:
+            definitions.append(_definition("resolved max steps", log.eval.max_steps))
+    elif log.eval.max_steps is not None:
         definitions.append(_definition("max steps", log.eval.max_steps))
     if log.stats.mean_inference_latency_s is not None:
         definitions.append(

src/inspect_robots/cli.py

@@ -565,6 +565,43 @@ def _step_limit_count(log: EvalLog) -> int:
     )
 
 
+def _seconds_horizon(log: EvalLog) -> tuple[float, int, float | None] | None:
+    """Return a validated declared/resolved seconds horizon from a saved log."""
+    max_seconds = log.eval.max_seconds
+    max_steps = log.eval.max_steps
+    if not (
+        isinstance(max_seconds, (int, float))
+        and not isinstance(max_seconds, bool)
+        and math.isfinite(max_seconds)
+        and max_seconds > 0
+        and isinstance(max_steps, int)
+        and not isinstance(max_steps, bool)
+        and max_steps > 0
+    ):
+        return None
+
+    rate = log.eval.embodiment_info.get("control_hz")
+    valid_rate = (
+        float(rate)
+        if isinstance(rate, (int, float))
+        and not isinstance(rate, bool)
+        and math.isfinite(rate)
+        and rate > 0
+        else None
+    )
+    return float(max_seconds), max_steps, valid_rate
+
+
+def _seconds_horizon_text(log: EvalLog) -> str | None:
+    """Format a seconds-based horizon for compact CLI metadata surfaces."""
+    horizon = _seconds_horizon(log)
+    if horizon is None:
+        return None
+    max_seconds, max_steps, rate = horizon
+    text = f"{max_seconds:g}s -> {max_steps} steps"
+    return f"{text} at {rate:g} Hz" if rate is not None else text
+
+
 def _outcome_line(log: EvalLog) -> tuple[str, bool] | None:
     """Return an outcome digest and unmapped flag, or ``None`` with no reasons."""
     reasons: list[object] = [
@@ -599,17 +636,31 @@ def _print_step_limit_notice(log: EvalLog, is_adhoc: bool) -> None:
 
     note = f"note: {count}/{log.results.total_trials} trials hit the step limit before terminating"
     max_steps = log.eval.max_steps
+    seconds_horizon = _seconds_horizon(log)
+    if seconds_horizon is not None:
+        max_seconds, resolved_steps, rate = seconds_horizon
+        parenthetical = f"max_seconds={max_seconds:g}, resolved max_steps={resolved_steps}"
+        if rate is not None:
+            parenthetical += f" at {rate:g} Hz"
+        note += f" ({parenthetical})"
     # Guards below reject bool/str values a hand-edited log can smuggle past
     # from_dict (bool is an int subclass, so isinstance alone lets True in).
-    if isinstance(max_steps, int) and not isinstance(max_steps, bool):
+    elif isinstance(max_steps, int) and not isinstance(max_steps, bool):
         parenthetical = f"max_steps={max_steps}"
         rate = log.eval.embodiment_info.get("control_hz")
-        if isinstance(rate, (int, float)) and not isinstance(rate, bool) and rate > 0:
+        if (
+            isinstance(rate, (int, float))
+            and not isinstance(rate, bool)
+            and math.isfinite(rate)
+            and rate > 0
+        ):
             parenthetical += f", ~{max_steps / rate:g}s at {rate:g} Hz"
         note += f" ({parenthetical})"
     print(_styled(note, _YELLOW))
     if is_adhoc:
         hint = "hint: raise it with --max-steps N or: inspect-robots config set max_steps N"
+    elif seconds_horizon is not None:
+        hint = f"hint: task {log.eval.task!r} defines its own max_seconds"
     else:
         hint = f"hint: task {log.eval.task!r} defines its own max_steps"
     print(_styled(hint, _DIM))
@@ -984,6 +1035,9 @@ def _print_eval_set_summary(success: bool, logs: Sequence[EvalLog], log_dir: str
         )
         detail = metrics or (log.error or "")
         row = f"  [{_styled(_display_status(log.status), _GREEN if ok else _RED)}] {log.eval.task}"
+        horizon = _seconds_horizon_text(log)
+        if horizon is not None:
+            row += f" [{horizon}]"
         print(f"{row}  {detail}" if detail else row)
     print(f"{_styled('log dir:', _CYAN)} {_styled(log_dir, _DIM)}")
     if not success:
@@ -1105,6 +1159,9 @@ def _cmd_inspect(path: str, *, transcript: bool = False) -> int:
             print(line)
     print(f"created:     {log.eval.created}")
     print(f"git:         {log.eval.git_commit}")
+    horizon = _seconds_horizon_text(log)
+    if horizon is not None:
+        print(f"horizon:     {horizon}")
     trials = f"trials: {log.results.total_trials}"
     if log.results.errored_trials:
         trials += f" ({log.results.errored_trials} errored)"

src/inspect_robots/compat.py

@@ -11,6 +11,7 @@
 
 from __future__ import annotations
 
+import math
 from dataclasses import dataclass, field
 
 from inspect_robots.embodiment import Embodiment
@@ -169,11 +170,46 @@ def check_compatibility(
         )
 
     if task is not None:
+        _check_task_horizon(task, embodiment, issues)
         _check_scenes_realizable(task, embodiment, issues)
 
     return report
 
 
+def _check_task_horizon(task: Task, embodiment: Embodiment, issues: list[CompatIssue]) -> None:
+    """Require a usable embodiment rate when a task declares physical seconds."""
+    if task.max_seconds is None:
+        return
+
+    control_hz = embodiment.info.control_hz
+    if (
+        control_hz is None
+        or isinstance(control_hz, bool)
+        or not math.isfinite(control_hz)
+        or control_hz <= 0
+    ):
+        issues.append(
+            CompatIssue(
+                "error",
+                "task_horizon_control_rate",
+                f"task {task.name!r} declares max_seconds={task.max_seconds:g}, but "
+                f"embodiment {embodiment.info.name!r} has control_hz={control_hz!r}; "
+                "seconds-based tasks require a finite positive embodiment rate",
+            )
+        )
+        return
+
+    if not math.isfinite(task.max_seconds * control_hz):
+        issues.append(
+            CompatIssue(
+                "error",
+                "task_horizon_control_rate",
+                f"task {task.name!r} max_seconds={task.max_seconds:g} at embodiment "
+                f"control_hz={control_hz:g} does not yield a finite step budget",
+            )
+        )
+
+
 def _check_scenes_realizable(task: Task, embodiment: Embodiment, issues: list[CompatIssue]) -> None:
     sup_setups = embodiment.info.supported_setups
     sup_targets = embodiment.info.supported_target_kinds

src/inspect_robots/embodiment.py

@@ -70,15 +70,16 @@ class Embodiment(Protocol):
 
     Embodiments may additionally define an **optional** ``bind_task(envelope)``
     hook (not part of this Protocol, so existing embodiments stay conformant):
-    ``eval()`` calls it with the task's
+    after compatibility checking, ``eval()`` calls it with the task's resolved
     [`TaskEnvelope`][inspect_robots.task.TaskEnvelope] before the
-    compatibility check, letting adapters learn the rollout horizon — e.g. an
-    operator countdown showing elapsed/total. The hook is optional *input*,
-    not a guarantee: it never fires on direct ``rollout()`` calls or on older
-    cores, so adapters must keep a graceful fallback. It fires once per
-    ``eval()``, which can be several times over an embodiment's lifetime;
-    each call replaces the previous envelope. ``EmbodimentBase`` ships a
-    no-op default.
+    first reset, letting adapters learn the rollout horizon — e.g. an operator
+    countdown showing elapsed/total. A seconds-based task is converted to
+    integer steps using ``info.control_hz`` before the hook fires. The hook is
+    optional *input*, not a guarantee: it never fires on direct ``rollout()``
+    calls or on older cores, so adapters must keep a graceful fallback. It
+    fires once per ``eval()``, which can be several times over an embodiment's
+    lifetime; each call replaces the previous envelope. ``EmbodimentBase``
+    ships a no-op default.
     """
 
     info: EmbodimentInfo