Skip to content

feat(sdk): resume an existing run after a crash - #61

Merged
prashar32 merged 1 commit into
mainfrom
feat/sdk-resume-run
Jun 5, 2026
Merged

feat(sdk): resume an existing run after a crash#61
prashar32 merged 1 commit into
mainfrom
feat/sdk-resume-run

Conversation

@prashar32

Copy link
Copy Markdown
Owner

Why

The crash-resume moat is fully supported server-side — on restart the daemon reloads non-terminal runs with their already-spent budget, so a SIGKILL’d run keeps enforcing without re-spending. But the Python SDK could only create a run (governed_run); it had no way to attach to an existing one, so the resume story wasn’t reachable from Python without dropping to the low-level client.

What

Runtime.resume_run(run_id) — a context manager that attaches to an existing run by id. Unlike governed_run it neither creates a new run nor cancels on error: the run keeps its server-side budget and spent usage, so a resumed agent fetches its last checkpoint and continues where it left off, and can’t overspend by restarting.

with rt.resume_run(run_id) as run:
    cp = run.latest_checkpoint()
    start = cp["payload"]["cursor"] if cp else 0
    for i in range(start, total):       # skip the steps you already paid for
        run.step()                      # counts against the SAME budget
        run.checkpoint("step", {"cursor": i + 1})

Verified the server side first (the Day-8 spike)

Before writing the SDK, I proved the protocol against a real kill -9: a run advanced 5 of 8 loop steps → kill -9 the daemon → restart (resumed runs from store count=1, loops still 5, checkpoint intact) → continued: steps 6/7/8 → 200, step 9 → 402. Exactly 3 more allowed (8 total across the SIGKILL), not 8 more — no re-spend.

Tests + docs

  • test_resume_run_attaches_without_creating — attaches by id, sets current_run, reads the pre-crash checkpoint, keeps stepping the same run. The stub daemon now serves GET /v1/runs/{id}.
  • SDK README gains a Resume after a crash section; CHANGELOG updated.
  • Full SDK suite: 16 pass (2 framework-integration tests skip without langchain/openai-agents).

The SDK could open a governed run (governed_run) but had no way to ATTACH to an
existing one — so the crash-resume story, which the daemon fully supports
server-side (it reloads non-terminal runs on restart with their already-spent
budget), wasn't reachable from Python without dropping to the low-level client.

Add Runtime.resume_run(run_id): a context manager that attaches to an existing run
by id. Unlike governed_run it neither creates a new run nor cancels on error — the
run keeps its server-side budget and spent usage, so a resumed agent can fetch its
last checkpoint and continue where it left off, and can't overspend by restarting.

Verified the server side end-to-end first: a run advanced 5 of 8 loop steps, then
a SIGKILL of the daemon; on restart it reloaded the run (loops=5 + the checkpoint)
and continued — allowing exactly 3 more steps before halting, proving no re-spend.

Tests: a resume test (attach by id, current_run set, read the checkpoint, keep
stepping the same run) plus the stub daemon now serves GET /v1/runs/{id}.
Documented in the SDK README and CHANGELOG.
@prashar32
prashar32 merged commit dfc0ab3 into main Jun 5, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant