Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 38 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -301,12 +301,48 @@
uv run --no-sync python -m pytest plugins/inspect-robots-agent/tests -q
--cov=inspect_robots_agent --cov-report=term-missing --cov-fail-under=0

# The CaP-X code-as-policy suite needs no GPU, sockets, or CaP-X checkout:
# httpx.MockTransport scripts the LLM plus all three model-server protocols,
# and an in-test joint-space embodiment exercises the complete rollout.
plugin-capx:
name: plugin · capx
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Install uv
uses: astral-sh/setup-uv@v5
with:
enable-cache: true
cache-dependency-glob: |
**/pyproject.toml
**/uv.lock
- name: Sync workspace (core + plugins, dev extras)
run: uv sync --locked --python 3.11 --all-packages --extra dev
- name: Ruff (plugin, lint)
run: uv run --no-sync ruff check plugins/inspect-robots-capx
- name: Ruff (plugin, format check)
run: uv run --no-sync ruff format --check plugins/inspect-robots-capx
- name: Mypy (plugin, strict)
run: >
uv run --no-sync mypy
--config-file plugins/inspect-robots-capx/pyproject.toml
plugins/inspect-robots-capx/src/inspect_robots_capx
plugins/inspect-robots-capx/tests
- name: Pytest (plugin, with coverage)
# Report-only, like the other plugin jobs. The root 100% gate remains
# scoped to the numpy-only inspect_robots core package.
run: >
uv run --no-sync python -m pytest plugins/inspect-robots-capx/tests -q
--cov=inspect_robots_capx --cov-report=term-missing --cov-fail-under=0

# Single required status check for branch protection: green iff every job
# above is green. Add new jobs to 'needs' here or they won't gate merges.
ci-ok:

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}
name: ci-ok
if: ${{ always() }}
needs: [quality, test, test-rerun, test-extra, core-only-import, plugin-isaacsim, plugin-xpolicylab, plugin-ros, plugin-agent]
needs: [quality, test, test-rerun, test-extra, core-only-import, plugin-isaacsim, plugin-xpolicylab, plugin-ros, plugin-agent, plugin-capx]
runs-on: ubuntu-latest
steps:
- name: Fail unless every needed job succeeded
Expand All @@ -319,7 +355,7 @@
alert-red-main:
name: alert on red main
if: ${{ failure() && github.event_name == 'push' }}
needs: [quality, test, test-rerun, test-extra, core-only-import, plugin-isaacsim, plugin-xpolicylab, plugin-ros, plugin-agent]
needs: [quality, test, test-rerun, test-extra, core-only-import, plugin-isaacsim, plugin-xpolicylab, plugin-ros, plugin-agent, plugin-capx]
runs-on: ubuntu-latest
permissions:
issues: write
Expand Down
23 changes: 23 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -175,3 +175,26 @@ jobs:
# The plugin is versioned independently; a core release that doesn't
# bump it must not fail on the already-uploaded file.
skip-existing: true

publish-capx:
name: publish inspect-robots-capx
needs: [cut]
if: ${{ !cancelled() && (github.event_name == 'release' || needs.cut.result == 'success') }}
runs-on: ubuntu-latest
environment: pypi-capx
permissions:
id-token: write
steps:
- uses: actions/checkout@v4
with:
ref: ${{ needs.cut.outputs.tag || github.ref }}
- name: Install uv
uses: astral-sh/setup-uv@v5
- name: Build sdist + wheel
run: uv build --package inspect-robots-capx
- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
# The plugin is versioned independently; a core release that doesn't
# bump it must not fail on the already-uploaded file.
skip-existing: true
4 changes: 3 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,9 @@ rollout, scores it, and writes an immutable `EvalLog`. Mirrors Inspect AI's
(policy adapter speaking the XPolicyLab websocket protocol — 40+ served VLAs,
no xpolicylab dep), and
`plugins/inspect-robots-agent/` (LLMs as policies via the OpenAI-compatible
wire format — httpx only, no provider SDKs; registered as `agent`).
wire format — httpx only, no provider SDKs; registered as `agent`), and
`plugins/inspect-robots-capx/` (CaP-X code-as-policy over perception and IK
HTTP servers; registered as `capx`).

## Working here

Expand Down
25 changes: 25 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,27 @@ Read the recorded agent conversation with
`inspect-robots view LOG.json`, including the camera frames the model saw (for
`--store-frames` runs).

### Generate robot policy code with CaP-X:

The [inspect-robots-capx](plugins/inspect-robots-capx/) plugin evaluates a
code-as-policy agent in the same policy slot. The LLM writes Python against
SAM3 segmentation, Contact-GraspNet planning, Pyroki IK, and speed-limited
joint-motion helpers. CaP-X model servers run separately, while the persistent
code namespace and action queue run inside the evaluator.

```bash
uv pip install inspect-robots-capx

inspect-robots "place the fork on the plate" --policy capx \
--embodiment <joint-space-embodiment> \
-P model=anthropic/claude-fable-5 -P sam3_url=http://gpu-box:8114
```

The v1 adapter requires one `joint_pos` arm with a declared gripper, full
joint-state proprioception, a control rate, and a camera. See the
[plugin README](plugins/inspect-robots-capx/) for model-server bringup, depth
metadata, gripper polarity, and the model-code trust boundary.

### Run in simulation

The same instruction runs on your configured simulator instead of the
Expand Down Expand Up @@ -264,6 +285,10 @@ adapter shipped from this repo as separate packages:
embodiment through tool calls, as a first-class policy. The same
`--policy agent` runs ad-hoc instructions and scores on registered tasks
next to fine-tuned VLAs.
- **[inspect-robots-capx](plugins/inspect-robots-capx/)**: evaluate CaP-X-style
code-as-policy agents against a joint-space embodiment. Model-generated
Python calls separately served SAM3, Contact-GraspNet, and Pyroki helpers,
then queues approver-checked joint targets behind `--policy capx`.

```bash
# Isaac Lab world + a π0 checkpoint served by XPolicyLab, evaluated end to end:
Expand Down
Loading
Loading