P1 subset: close the store-open races and stop the docs lying about the shell

[macanderson]

What this is

The ask was "fix all P1 issues in one PR." That is not possible as stated, and the reason is structural rather than a matter of effort — so this PR lands the part that is landable and puts the rest in front of you as decisions.

I classified all 105 open checkbox rows across the six P1 issues against the code on main:

Issue	Open rows	Unblocked	Decision-gated	Already done
#618 public API / protocol break	29	4	23	2
#617 on-disk state	6	2	3	1
#616 budgets & caps	26	2	23	1
#615 trust boundary	13	5	7	1
#612 async boundary	31	4	23	4
#306 CI merge queue	—	repo-settings change, owner-only
Total	105	17	79	9

79 of 105 rows cannot be closed by anyone but you, and the issues say so themselves — each opens with "What decision this asks for." #618's two answers (take one coordinated protocol break vs. declare the types frozen) lead to opposite diffs and it explicitly notes freezing is "equally valid and much cheaper." #616's rows are blocked on numbers nobody has written down. #617 needs a ruling on two destructive operations. Guessing any of these wrong is expensive, so I didn't.

This PR is the 17 unblocked rows minus the cloud/serve surface you asked me to skip — and it grew, because closing one of them surfaced three further real bugs on the same path.

#617 items 8 + 10 — four races on the workspace-open path

Item 8 was "the version read is outside the transaction." Fixing it and adding the test the issue asked for by name (four threads, one fresh path) turned up three more, all real:

1. apply_migration re-applied a step another process had already stamped. Now IMMEDIATE, re-reading user_version under the write lock. Losing the race is now the no-op it should always have been — which is also what makes a future non-replay-safe step (ALTER TABLE … ADD COLUMN) safe to append.
2. Store::open failed outright with UNIQUE constraint failed: enterprise_export_migration.singleton — an existence check followed by an unguarded INSERT. Now ON CONFLICT DO NOTHING, which is what its own sibling branch already did.
3. PRAGMA journal_mode=WAL ran before busy_timeout was set, and SQLite skips the busy handler on a WAL upgrade — so the one statement needing an exclusive lock was the one statement with no retry budget. Now behind a bounded retry that mirrors stella-media's initialize_journal_database, whose doc comment already described this exact hazard. The main store never got the same treatment.
4. The generated .stella/.gitignore used check-then-rename, so concurrent openers all renamed over the target; a racer validating the winner's file saw nlink() == 0 and failed closed with "must be an owner-controlled single-link regular file" — for a file nothing had attacked. Now O_EXCL | O_NOFOLLOW exclusive create; losers read the settled file. Two new tests pin that a planted symlink is still rejected and that the file stays committable-mode (0644), not 0600.

Verified end-to-end, not just in tests. Five concurrent stella init runs against one fresh workspace:

	database is locked
origin/main	3
this branch	0

Every fix was negative-controlled by reverting it and confirming the specific failure came back. The four-thread test went 40/40 and the store suite 6/6 under full parallel load.

The same version-read-outside-the-transaction shape was sitting as an unfixed audit note in stella-context and stella-fleet — the fleet one explicitly warning that a future non-replay-safe step would be applied twice. Both now use IMMEDIATE, and the notes describe what the code does.

#615 — the shipped docs inverted a security claim

#710 moved bash and the key-free web family to registered-by-default and never swept the prose. Every release since has told operators the opposite of the surface they actually had:

• README.md ("Bash is opt-in. The default tool surface has no free-form shell tool")
• AGENTS.md, stella-tools/README.md, docs/why-stella.md, docs/design/semantic-resolution-bridge.md
• docs/design/threat-model.md — "bash is opt-in… the default surface has no shell at all"
• the model's own system prompt — tool_steering! carried "There is no shell unless the workspace enables it", and prompt.rs's own test pinned that false sentence in place

A false claim in a threat model is worse than no claim, and a false one in the system prompt is read every turn.

web.rs is the sharpest case: it justified having no SSRF guard (localhost, cloud metadata endpoints reachable) on the grounds that the family was opt-in — "the gate is the settings opt-in, not a network allowlist." That gate no longer exists. R4 in the threat model says the same. Both now record it as an open ruling rather than presenting a retired guarantee as current. I did not add an SSRF guard: that's strict-vs-permissive and would break "fetch my internal dev server."

New stella-tools/tests/doc_truth.rs gates all of it, so the claim cannot invert silently again. It caught one instance I'd missed while I was writing it.

Item 1 (the run_tests → bash -c injection) was already fixed in #624. I verified it properly rather than taking the checkbox's word: per-token shell_quote, and the test runs a real bash -c with a negative control. Tick it and don't reopen.

Not fixed, recorded instead: screenshot, ci_status and start_work_on_issue compose bash -c lines that never reach command_line_for, so a policy denying command execution cannot see three real shell invocations — contradicting the MUST stated right there in the doc comment. None is injectable (all shell-quote). I left it as a note at that function because closing it makes those tools newly deniable by an existing operator policy, which is an operator-visible change that should land deliberately.

Three small ones

• P1: public API: schedule the one coordinated protocol break, or declare these types frozen #618 item 5 — journal failures were reported as MediaError::Artifact, pointing operators at .stella/artifacts/ for a fault in the journal sidecar. New Journal variant. (Safe: MediaError isn't Serialize and has no exhaustive match anywhere.)
• P1: public API: schedule the one coordinated protocol break, or declare these types frozen #618 item 17 — the speculation select! had unreachable! on a provider-driven path. Now a typed terminal error; invariant 5 outranks asserting a structural claim. ProviderError::Terminal already existed, so the "fresh error-semantics decision" this row was deferred on was already moot.
• P1: budgets: fill in every deferred cap, deadline, and retention window in one sitting #616 item 8 — upsert handed the embedder every missing body in one request, so a full-workspace first index put the whole corpus in a single call. Batched at warm::BATCH, the width the warm indexer has always used. Witness test asserts no request exceeds it; negative-controlled at 133 texts.

Things worth your attention

• P1: on-disk state: close the residual schema and race gaps (durability contract ratified in #699) #617 item 12 is destructive and I did not touch it. The orphan-sidecar sweep would remove_dir_all session dirs containing history.json — the LLM conversation, irrecoverable, and the only thing making a session continuable. The false-positive path is concrete: list() swallows unparsable records with .ok()?, and upsert's own comment documents that a power cut can leave a zero-length <id>.json. The exact failure the fsync work was added to survive is the one that makes a live session look orphaned and get deleted. It needs an explicit go/no-go.
• P1: on-disk state: close the residual schema and race gaps (durability contract ratified in #699) #617 item 5 is destructive too, and the issue doesn't say so. Retrofitting FKs onto a deployed fleet.db can't succeed while orphan rows exist — apply_migration hard-fails on pragma_foreign_key_check — so the only way through is deleting fleet history.
• P1: public API: schedule the one coordinated protocol break, or declare these types frozen #618's break-vs-freeze decision has a cheaper answer than it looks. publish = false is set workspace-wide, so "public API break" always means an in-repo compile-and-fix, never an external consumer — the same premise that already retired three of that issue's four checked rows.
• 9 rows are already done and should just be ticked: P1: public API: schedule the one coordinated protocol break, or declare these types frozen #618 items 23, 29 · P1: on-disk state: close the residual schema and race gaps (durability contract ratified in #699) #617 item 9 · P1: budgets: fill in every deferred cap, deadline, and retention window in one sitting #616 item 29 · P1: trust boundary: decide what gets scrubbed, gated, escaped, and confined before untrusted text reaches a shell or a page #615 item 1, U7 · P1: async boundary: decide where blocking work runs, how wide it fans out, and what we rebuild every step #612 items 13, 18, 19, 30.
• Duplicate rows: P1: budgets: fill in every deferred cap, deadline, and retention window in one sitting #616 9/10 are one defect and 13/21 are another. Two rulings close four rows.
• Half the rows cite drifted line numbers. Anyone landing these should re-locate by symbol.
• P1: CI: merge queue for main is blocked on org ownership — strict status checks adopted as substitute (ci.yml half shipped in #508) #306 (merge queue) is a repo-settings change — branch protection → "Require merge queue" — that changes how everyone lands PRs, plus a merge_group trigger in ci.yml. Not something I should flip unilaterally, and the issue itself says to verify the auto-merge interaction against GitHub's current UX rather than assume.

Gate

make gate exits 0 (checked directly — piping it to tail masks the exit code). Rebased onto eb663f0d; origin/main had moved 0.5.52 → 0.5.56 under me and #760 restructured AGENTS.md, so the doc-truth test and baseline were re-verified after the rebase.

The file-size baseline moves three lines. driver.rs (+11), stella-store/src/lib.rs (+13) and registry.rs (+5) are in-place code plus the rationale the surrounding style expects, and can't be extracted. The 50-line doc-truth test was extracted to its own file rather than raising a baseline for it.

---

Update — #617 items 12 and 5, per the owner's ruling

Both destructive rows are now handled, neither by deleting anything.

Item 12: fix the conflation, then sweep

The sweep was unsafe because SessionRegistry::list cannot tell "no record" from "unreadable record" — it drops whatever fails to parse, and upsert's own comment records that a power cut leaves a zero-length <id>.json. Anything keyed on what list accounts for therefore deletes the sidecar of a session whose record was merely truncated.

So the conflation is fixed first. scan() now reports three states where list reported one:

state	meaning	swept?
healthy	record parsed	no
damaged	<id>.json present but unusable (zero-length / truncated) — sidecar intact	never
orphan_sidecars	no <id>.json beside it at any readability	yes

The orphan test is answered from the directory entry, never from a parse. list() delegates to scan().healthy, so there is one implementation rather than two definitions that can drift. prune_orphan_sidecars() re-checks for a record under the same name immediately before removing, so a session that started since the scan is safe.

Damaged records are reported, not repaired, and the type documents why: id, pid and started_at_ms are recoverable from the filename (ids are self-minted ses-<ms>-<pid>), but workspace is held nowhere else on disk — not in journal.jsonl, not in history.json — and resume needs it. Silently hiding the session (what happens today) and silently deleting it (as filed) are both wrong; surfacing it is what's actually available.

The witness test asserts the full chain — list can't see the record, scan calls it damaged with its sidecar noted, the sweep is a no-op, history.json survives — and is negative-controlled against the filed definition (orphan = "not in list"), which classifies that session as an orphan and deletes it.

Item 5: constrain new ledgers, report on old ones

Fresh fleet.db files get the four REFERENCES runs (id) constraints via a new FRESH_SCHEMA, so a ledger created by this build cannot acquire an orphan. Existing files take the unchanged ladder and are never rebuilt — no migration can brick a fleet.db, and no history is deleted.

The tradeoff is stated at FRESH_SCHEMA rather than left implicit: SCHEMA_VERSION no longer determines shape by itself, since both variants are v2. A future step that rebuilds one of these four tables must reproduce the right variant or read the DDL from sqlite_master. That's a real wart; the alternatives were deleting history or leaving new databases unconstrained forever.

Ledger::orphan_rows() reports what an unconstrained file holds and enforces_run_references() says which variant a file is. Rather than land as uncalled API, it's surfaced as a second stella doctor check. It passes either way — inert rows on an old ledger aren't a fault, and failing the command over them would make doctor unusable on exactly the workspaces with history worth keeping. --repair deliberately does not act on it, asserted by a test that checks the rows are still present afterwards.

Driven against the real binary on a legacy ledger holding 4 orphans across 3 tables:

✓ fleet ledger — .stella/private/fleet.db: 4 row(s) reference a run that is no longer recorded
    tasks.run_id: 2 row(s)
    attempts.run_id: 1 row(s)
    spend.run_id: 1 row(s)
    → nothing to do — these rows are inert (no query resolves a row by its run) and they are
      kept because removing them would delete fleet history
    → this ledger predates the run-reference constraints; a fleet.db created by this build
      cannot acquire new orphans

2 checks: 2 ok, 0 failed

After stella doctor --repair: all 4 rows and the $4.20 of recorded spend still there.

doctor's module doc claimed "today there is exactly one check" — corrected, and its test now asserts check names rather than a count.

make gate exits 0. Two rustdoc failures caught on the way (a public doc linking a private FRESH_SCHEMA, and a FleetLedger type that is actually Ledger) are fixed.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
stella-cli-docs	Ignored	Preview	Jul 27, 2026 7:38am

[sourcery-ai]

Sorry @macanderson, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery