Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,3 +95,22 @@ actual registered task names.
rewrites GitHub-only alert syntax (`> [!NOTE]` etc.) in README.md into bold
blockquotes (`> **Note:**`) that PyPI renders; keep using alert syntax in the
README itself. Config lives at the bottom of pyproject.toml.

## Writing style (public-facing text)

READMEs, docs pages, repo/collection descriptions, and HF model cards must
avoid AI-writing tells. This repo hosts the canonical rule: the full version
with the gating checklist lives in
[docs/model-cards.md, "Writing style"](docs/model-cards.md);
short version:

- No em dashes in prose. Use periods, colons, commas, or parentheses (`—` is
fine as an empty table cell and inside code blocks).
- Bold only for definition-list lead-ins (`**term:**`) and at most one critical
imperative per safety bullet. Never mid-sentence for emphasis.
- No decorative emoji (functional ✅/⚠️ marks and 🤗 for Hugging Face are fine),
no slogans or chiasmus, no "not just X, but Y".
- Headers use colons, never em dashes or italics.

Style-only edits must never touch YAML frontmatter, code blocks, numbers,
links, or safety qualifiers.
6 changes: 3 additions & 3 deletions docs/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ repository that:

1. **depends on [Inspect Robots](https://github.com/robocurve/inspect-robots)**,
2. **defines one or more Inspect Robots `Task`s**, and
3. **registers them via entry points** `[project.entry-points."inspect_robots.tasks"]`
3. **registers them via entry points**: `[project.entry-points."inspect_robots.tasks"]`
(and, if it ships a simulator/embodiment or policy,
`inspect_robots.embodiments` / `inspect_robots.policies`).

Expand Down Expand Up @@ -33,10 +33,10 @@ Benchmark(
)
```

The homepage card grid is **generated from this catalog at build time**, so adding
The homepage card grid is generated from this catalog at build time, so adding
an entry updates the site automatically. A test validates every entry (unique
name, well-formed `https://github.com/...` repo URL, ≥1 task key), and CI requires
100% coverage — keep `task_keys` in sync with the benchmark's actual registered
100% coverage. Keep `task_keys` in sync with the benchmark's actual registered
task names.

See the [API reference](api.md) for the `Benchmark` fields.
2 changes: 1 addition & 1 deletion docs/model-cards.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ Cards are read as credibility signals: prose patterns that readers flag as AI-ge
undermine the numbers next to them. Avoid the tells.

- No em dashes in prose; use periods, colons, commas, or parentheses. `—` is allowed only
as an empty table cell.
as an empty table cell or inside a code block.
- Bold only for definition-list lead-ins (`**term:**`) and at most one critical imperative
per safety bullet; never mid-sentence for emphasis.
- No decorative emoji (🤗 for Hugging Face is fine), no slogans or chiasmus, no
Expand Down
2 changes: 1 addition & 1 deletion mkdocs.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
site_name: WorldEvals
site_description: >-
The Inspect Evals for robotics a curated catalog of physical-AI benchmarks
The Inspect Evals for robotics: a curated catalog of physical-AI benchmarks
built on Inspect Robots.
site_url: https://worldevals.org/
repo_url: https://github.com/robocurve/worldevals
Expand Down
12 changes: 6 additions & 6 deletions scripts/gen_catalog.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
"""Generate the WorldEvals homepage from the catalog at build time.

Run automatically by the ``mkdocs-gen-files`` plugin during ``mkdocs build`` it
Run automatically by the ``mkdocs-gen-files`` plugin during ``mkdocs build``: it
imports :data:`worldevals.catalog.CATALOG` and renders it as a Material grid-card
listing, so the site is always in sync with ``catalog.py`` (the single source of
truth), mirroring how Inspect Evals generates its listing from per-eval metadata.
Expand Down Expand Up @@ -55,20 +55,20 @@ def render_home() -> str:
# WorldEvals

<p style="font-size: 1.25rem; font-weight: 500; margin-bottom: 0.25rem;">
The <strong>Inspect Evals</strong> for robotics.
The Inspect Evals for robotics.
</p>

A curated catalog of physical-AI / VLA benchmarks built on
[Inspect Robots](https://github.com/robocurve/inspect-robots). Each benchmark lives in **its
own repository** (so it owns its release cadence, dependencies, and hardware
[Inspect Robots](https://github.com/robocurve/inspect-robots). Each benchmark lives in its
own repository (so it owns its release cadence, dependencies, and hardware
notes); WorldEvals is the index that ties them together.

[:octicons-mark-github-16: GitHub](https://github.com/robocurve/worldevals){{ .md-button }}
[Add a benchmark](contributing.md){{ .md-button }}

!!! tip "Two views of the collection"
`inspect-robots list` tells you what Inspect Robots tasks are **installed**.
`worldevals list` tells you what benchmarks **exist** and how to get them.
`inspect-robots list` tells you what Inspect Robots tasks are installed.
`worldevals list` tells you what benchmarks exist and how to get them.

## Benchmarks

Expand Down
Loading