Skip to content

PR 09: Add sample-centered CLI inspection#128

Closed
cm2435 wants to merge 3 commits into
codex/episode-sample-pr08-read-apisfrom
codex/episode-sample-pr09-cli-inspection
Closed

PR 09: Add sample-centered CLI inspection#128
cm2435 wants to merge 3 commits into
codex/episode-sample-pr08-read-apisfrom
codex/episode-sample-pr09-cli-inspection

Conversation

@cm2435

@cm2435 cm2435 commented May 26, 2026

Copy link
Copy Markdown
Contributor

Review Guide

Stack position: PR 09, base codex/episode-sample-pr08-read-apis, head codex/episode-sample-pr09-cli-inspection.

Implements RFC Docs

  • 04-ecosystem/01-cli.md: CLI inspection, not CLI submission.
  • 04-cli-builtins-examples-ecosystem.md: ecosystem entrypoint changes.
  • 09-implementation-plan/10-pr-09-cli-inspection.md.
  • Part of 09-implementation-plan/14-remediation-plan.md: CLI concept-fragment cleanup.

Why This PR Exists

The CLI should inspect experiments and samples through read models. It should not become a second job-submission API. This PR adds read-only experiment/sample inspection commands and removes old mutating/runtime commands from this surface.

What Actually Changes

  • Adds experiment CLI domain files under ergon_cli/domains/experiments:
    • experiment list
    • experiment show
    • experiment samples
    • experiment sampler-invocations
  • Adds sample CLI domain files under ergon_cli/domains/samples:
    • sample show
    • sample events
    • sample graph
  • Removes legacy/mutating CLI surfaces including sample cancel, sample list --definition-id, experiment tags, and experiment by-tag.
  • Adds guards/tests for no generic experiment submit command and CLI state rendering.
  • Updates experiment/sample CLI tests to use read-model fakes.

Gotchas / Review Risks

  • The examples CLI path is not materially changed here; this is inspection-focused.
  • CLI services import core read services directly. That may be acceptable for now, but the remediation plan calls out a cleaner HTTP/read-client seam later.
  • ergon benchmark public naming is not addressed in this PR.
  • The definition identity naming guard is adjusted so legitimate experiment_id fields do not fail it.

Cleaned Later

  • PR 10 moves the dashboard to the same read APIs.
  • PR 11 removes definition runtime/public launch compatibility.
  • Future cleanup should tighten CLI boundary shape and benchmark naming.

Validation Notes

Key tests are test_no_generic_experiment_submit.py, test_cli_state_rendering.py, test_experiment_cli.py, and test_sample_cli.py. Agent review saw Python lint/type, unit/state, and integration passing, with frontend/smoke failures elsewhere in the stack.

@github-actions

Copy link
Copy Markdown

E2E smoke — researchrubrics

No PNG screenshots were uploaded for this leg. See screenshots/pr-128 for the uploaded placeholder.

@github-actions

Copy link
Copy Markdown

E2E smoke — minif2f

No PNG screenshots were uploaded for this leg. See screenshots/pr-128 for the uploaded placeholder.

@github-actions

Copy link
Copy Markdown

E2E smoke — swebench-verified

No PNG screenshots were uploaded for this leg. See screenshots/pr-128 for the uploaded placeholder.

@cm2435
cm2435 force-pushed the codex/episode-sample-pr08-read-apis branch from 36d4fe1 to 78097f1 Compare May 26, 2026 23:16
@cm2435
cm2435 force-pushed the codex/episode-sample-pr09-cli-inspection branch from cb0d9a0 to 7c5d180 Compare May 26, 2026 23:18
@github-actions

Copy link
Copy Markdown

E2E smoke — researchrubrics

Screenshots pushed to screenshots/pr-128.

researchrubrics/542b7ddc-56dd-43a6-957c-fbf5a5a1c0ab-activity-stack.png
researchrubrics/542b7ddc-56dd-43a6-957c-fbf5a5a1c0ab-sad.png
researchrubrics/542b7ddc-56dd-43a6-957c-fbf5a5a1c0ab-visual-debugger-full.png
researchrubrics/66b6acc0-8743-47f6-bd1a-1c675c9eb4b9-activity-stack.png
researchrubrics/66b6acc0-8743-47f6-bd1a-1c675c9eb4b9-happy.png
researchrubrics/66b6acc0-8743-47f6-bd1a-1c675c9eb4b9-visual-debugger-full.png
researchrubrics/de40211c-878e-4275-8583-47cd9dea9d21-activity-stack.png
researchrubrics/de40211c-878e-4275-8583-47cd9dea9d21-happy.png
researchrubrics/de40211c-878e-4275-8583-47cd9dea9d21-visual-debugger-full.png

@github-actions

Copy link
Copy Markdown

E2E smoke — swebench-verified

Screenshots pushed to screenshots/pr-128.

swebench-verified/0c583fc4-d5e0-474a-98f1-d58378b9736f-activity-stack.png
swebench-verified/0c583fc4-d5e0-474a-98f1-d58378b9736f-happy.png
swebench-verified/0c583fc4-d5e0-474a-98f1-d58378b9736f-visual-debugger-full.png
swebench-verified/1d86f6f4-eeea-4a6d-adcd-fa78dc1303a8-activity-stack.png
swebench-verified/1d86f6f4-eeea-4a6d-adcd-fa78dc1303a8-happy.png
swebench-verified/1d86f6f4-eeea-4a6d-adcd-fa78dc1303a8-visual-debugger-full.png
swebench-verified/3ad87796-1ada-46ab-9d01-4a4126f36c2b-activity-stack.png
swebench-verified/3ad87796-1ada-46ab-9d01-4a4126f36c2b-sad.png
swebench-verified/3ad87796-1ada-46ab-9d01-4a4126f36c2b-visual-debugger-full.png

@github-actions

Copy link
Copy Markdown

E2E smoke — minif2f

Screenshots pushed to screenshots/pr-128.

minif2f/34288200-b7b3-4b52-884f-de215b327e6b-activity-stack.png
minif2f/34288200-b7b3-4b52-884f-de215b327e6b-sad.png
minif2f/34288200-b7b3-4b52-884f-de215b327e6b-visual-debugger-full.png
minif2f/7165b4a4-b405-4aa0-abd7-48d6bbb508f9-activity-stack.png
minif2f/7165b4a4-b405-4aa0-abd7-48d6bbb508f9-happy.png
minif2f/7165b4a4-b405-4aa0-abd7-48d6bbb508f9-visual-debugger-full.png
minif2f/9139331d-d50b-41e8-87d4-72e889fee042-activity-stack.png
minif2f/9139331d-d50b-41e8-87d4-72e889fee042-happy.png
minif2f/9139331d-d50b-41e8-87d4-72e889fee042-visual-debugger-full.png

@cm2435
cm2435 force-pushed the codex/episode-sample-pr09-cli-inspection branch from 7c5d180 to 492df28 Compare May 27, 2026 14:00
@cm2435
cm2435 force-pushed the codex/episode-sample-pr08-read-apis branch from 78097f1 to 1aa7fd9 Compare May 27, 2026 14:00
@github-actions

Copy link
Copy Markdown

E2E smoke — swebench-verified

No PNG screenshots were uploaded for this leg. See screenshots/pr-128 for the uploaded placeholder.

@github-actions

Copy link
Copy Markdown

E2E smoke — researchrubrics

No PNG screenshots were uploaded for this leg. See screenshots/pr-128 for the uploaded placeholder.

@github-actions

Copy link
Copy Markdown

E2E smoke — minif2f

No PNG screenshots were uploaded for this leg. See screenshots/pr-128 for the uploaded placeholder.

1 similar comment
@github-actions

Copy link
Copy Markdown

E2E smoke — minif2f

No PNG screenshots were uploaded for this leg. See screenshots/pr-128 for the uploaded placeholder.

@github-actions

Copy link
Copy Markdown

E2E smoke — swebench-verified

No PNG screenshots were uploaded for this leg. See screenshots/pr-128 for the uploaded placeholder.

@github-actions

Copy link
Copy Markdown

E2E smoke — researchrubrics

No PNG screenshots were uploaded for this leg. See screenshots/pr-128 for the uploaded placeholder.

@cm2435
cm2435 force-pushed the codex/episode-sample-pr09-cli-inspection branch from 492df28 to 1eb807b Compare May 27, 2026 14:19
@github-actions

Copy link
Copy Markdown

E2E smoke — researchrubrics

No PNG screenshots were uploaded for this leg. See screenshots/pr-128 for the uploaded placeholder.

@github-actions

Copy link
Copy Markdown

E2E smoke — minif2f

No PNG screenshots were uploaded for this leg. See screenshots/pr-128 for the uploaded placeholder.

@github-actions

Copy link
Copy Markdown

E2E smoke — swebench-verified

No PNG screenshots were uploaded for this leg. See screenshots/pr-128 for the uploaded placeholder.

@cm2435
cm2435 force-pushed the codex/episode-sample-pr09-cli-inspection branch from 1eb807b to 45490c2 Compare May 27, 2026 14:24
@github-actions

Copy link
Copy Markdown

E2E smoke — swebench-verified

No PNG screenshots were uploaded for this leg. See screenshots/pr-128 for the uploaded placeholder.

@github-actions

Copy link
Copy Markdown

E2E smoke — minif2f

No PNG screenshots were uploaded for this leg. See screenshots/pr-128 for the uploaded placeholder.

@github-actions

Copy link
Copy Markdown

E2E smoke — researchrubrics

No PNG screenshots were uploaded for this leg. See screenshots/pr-128 for the uploaded placeholder.

@github-actions

Copy link
Copy Markdown

E2E smoke — minif2f

No PNG screenshots were uploaded for this leg. See screenshots/pr-128 for the uploaded placeholder.

@github-actions

Copy link
Copy Markdown

E2E smoke — swebench-verified

No PNG screenshots were uploaded for this leg. See screenshots/pr-128 for the uploaded placeholder.

@github-actions

Copy link
Copy Markdown

E2E smoke — researchrubrics

No PNG screenshots were uploaded for this leg. See screenshots/pr-128 for the uploaded placeholder.

@github-actions

Copy link
Copy Markdown

E2E smoke — swebench-verified

No PNG screenshots were uploaded for this leg. See screenshots/pr-128 for the uploaded placeholder.

@github-actions

Copy link
Copy Markdown

E2E smoke — researchrubrics

No PNG screenshots were uploaded for this leg. See screenshots/pr-128 for the uploaded placeholder.

@github-actions

Copy link
Copy Markdown

E2E smoke — minif2f

No PNG screenshots were uploaded for this leg. See screenshots/pr-128 for the uploaded placeholder.

@cm2435
cm2435 force-pushed the codex/episode-sample-pr09-cli-inspection branch from 45490c2 to a2a8ddf Compare May 27, 2026 14:44
@cm2435
cm2435 force-pushed the codex/episode-sample-pr08-read-apis branch from c0584b5 to 76c0d70 Compare May 27, 2026 14:44
@github-actions

Copy link
Copy Markdown

E2E smoke — minif2f

No PNG screenshots were uploaded for this leg. See screenshots/pr-128 for the uploaded placeholder.

@github-actions

Copy link
Copy Markdown

E2E smoke — swebench-verified

No PNG screenshots were uploaded for this leg. See screenshots/pr-128 for the uploaded placeholder.

@github-actions

Copy link
Copy Markdown

E2E smoke — researchrubrics

No PNG screenshots were uploaded for this leg. See screenshots/pr-128 for the uploaded placeholder.

@github-actions

Copy link
Copy Markdown

E2E smoke — swebench-verified

No PNG screenshots were uploaded for this leg. See screenshots/pr-128 for the uploaded placeholder.

@github-actions

Copy link
Copy Markdown

E2E smoke — minif2f

No PNG screenshots were uploaded for this leg. See screenshots/pr-128 for the uploaded placeholder.

@github-actions

Copy link
Copy Markdown

E2E smoke — researchrubrics

No PNG screenshots were uploaded for this leg. See screenshots/pr-128 for the uploaded placeholder.

@cm2435

cm2435 commented Jun 7, 2026

Copy link
Copy Markdown
Contributor Author

Closed after landing the cumulative episode/sample stack snapshot through PR132 directly onto dev in merge commit 1c22130. This PR's changes are included in dev.

@cm2435 cm2435 closed this Jun 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant