fix(store-turso): add versioned migration ledger (ARN-242) - #389
fix(store-turso): add versioned migration ledger (ARN-242)#389nerdsane wants to merge 20 commits into
Conversation
|
RED EVIDENCE · commit 47a6655 Command: Result: exit 101, 0 passed, 1 failed. Failure reason: The current store returned Ok and reached readiness even though tenant_installed_apps was a view rather than the required durable table. No implementation change is present in this commit. |
LIVE LOCAL E2E · RED revision versus PR headBoth runs used the assigned worktree and locally built Before fix — committed RED revision
|
|
Reviewed the complete open GitHub diff at pushed head
Verdict: FAIL |
Reviewer FAIL remediation · current head
|
|
Reviewed the complete open GitHub patch at exact head The two prior concrete reproductions are fixed on this head: unexpected required columns/UNIQUE/FK/CHECK/generated/default semantics now fail before the owning ledger row, and the current-shape OTS path validates modifiers before returning. I still found two release-blocking durability issues:
Verdict: FAIL |
LIVE LOCAL E2E · exact current head
|
ADR namespace audit · exact head
|
LIVE LOCAL E2E · exact current head
|
|
Fresh independent review of the complete open 23-file GitHub patch and PR evidence at exact head
Verdict: FAIL |
Reviewer FAIL remediation · SQLite trigger-owner case semanticsRemediates the exact current-review finding from #389 (comment). RED — behavioral/durability regressions onlyCommit: 50d8177 $ cargo test -p temper-store-turso differently_cased -- --nocapture
running 3 tests
test migrations::ledger_tests::differently_cased_ledger_trigger_owner_is_rejected_before_migration ... FAILED
test migrations::ots_rebuild_tests::differently_cased_ots_trigger_owner_is_preserved_when_probe_rolls_back ... FAILED
test migrations::schema_verify_tests::runtime_extensions::differently_cased_trigger_owner_cannot_bypass_inventory ... FAILED
test result: FAILED. 0 passed; 3 failedEach regression failed at its GREEN — root-cause correctionCommit: d5069b6
$ cargo test -p temper-store-turso differently_cased -- --nocapture
test result: ok. 3 passed; 0 failed
$ cargo test -p temper-store-turso
test result: ok. 109 passed; 0 failed; 1 ignored
$ git status --short --branch
## codex/arn-242-turso-migration-ledger...origin/codex/arn-242-turso-migration-ledger
$ git rev-parse HEAD
d5069b654dcc287bb360f8b55639ff471114603a
$ git rev-parse origin/codex/arn-242-turso-migration-ledger
d5069b654dcc287bb360f8b55639ff471114603aThe unbypassed push completed with: That includes |
LIVE LOCAL E2E · exact remediated head
|
|
Fresh independent review of the complete 23-file open GitHub diff from base
Verdict: FAIL |
|
ARN-242 persisted-replacement remediation evidence · 2026-07-19T03:42:39Z Reviewer finding: #389 (comment) Current open-PR head: RED — test-only commit
GREEN — root-cause commit
Verification on the exact pushed head:
|
LIVE LOCAL E2E · exact current head
|
|
Independent exact-head review of the complete GitHub base
Verdict: FAIL |
|
Reviewed head SHA: 0535156 Verdict: FAIL
Validation: cargo test -p temper-store-turso passed all 117 executed tests; the live Hrana test and helper-process test remain ignored by default. |
ARN-242 remediation evidence · exact head
|
|
Fresh independent review of the complete open GitHub PR diff from base a28fdb2 through reviewed head a44ded0. GitHub reports the PR open, and the remote per-file blob SHAs match the exact local head sources inspected. No actionable findings. I reviewed the ordered catalog, canonical checksums and released checksum pins, ledger schema and row validation, immediate per-version transactions, retained-row checks, rollback and commit paths, concurrent replay behavior, and final catalog-head verification. I also checked the exact legacy OTS rebuild gates and dependent-object preservation, inbound FK handling, current-shape CASCADE and RESTRICT behavior under the production UPSERT, SQLite identifier owner matching, parsed unconditional audit-trigger contract, one-column sink restrictions, opaque database-generated probe identities, and savepoint rollback inside the real transaction-pinned Hrana path. Validation on this head: cargo test -p temper-store-turso passed 116 unit tests and all package integration tests, with only the isolated-Hrana test and helper-process entrypoint intentionally ignored; cargo fmt --all -- --check and git diff --check passed. GitHub Compile and Lint, Spec Verification, Integrity and DST Patterns, Instrumentation Hygiene, Verification Contract, and platform-random checks were green when reviewed; remaining matrix jobs were still running and produced no review finding. Verdict: PASS |
|
@greptile review |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
ARN-242 final-head remediation and live E2E evidence ·
|
|
Reviewed the complete open GitHub PR diff at exact head Finding
Review summaryThe per-version immediate transactions, ledger insert/retained-row checks, final pinned transaction, OTS rebuild rollback, Hrana savepoint containment, and production UPSERT preservation of inbound Verdict: FAIL |
|
Fresh independent GPT-5.6 review — exact GitHub head\n\nPR state verified: OPEN\nReviewed head: cc75b4a\nReview source: complete open PR diff fetched from GitHub, not merely local files.\n\nFindings: none.\n\nThe review specifically rechecked migration-ledger durability and replay semantics; transactional rollback; Hrana/Turso behavior; CASCADE and RESTRICT preservation; case-insensitive trigger/index owner inventory; and the final remediation that versions the durable extra-index policy as sqlite-identifier-owners-v2, rotates and pins all seven released prefix checksums, adds the RED regression, and documents the compatibility consequence in ADR-0180. No correctness, durability, regression, missing-test, or ADR-alignment defect was found on this exact head.\n\nVerdict: PASS |
|
Exact-head local E2E evidence\n\nPR head:\n git rev-parse HEAD\n cc75b4a\n\nRelink the existing assigned-worktree CLI target:\n CARGO_INCREMENTAL=0 cargo build -p temper-cli --bin temper\n Finished dev profile [unoptimized + debuginfo] target(s) in 2m 21s\n\nSupported audit-trigger replay and live HTTP health:\n RUST_LOG=error TURSO_URL=file:/tmp/arn242-pr389-cc75b4a9-current-live-20260720T053414Z.db target/debug/temper serve --port 43148 --no-observe\n\n Storage: turso (file:/tmp/arn242-pr389-cc75b4a9-current-live-20260720T053414Z.db)\n Restored 21 specs from Turso.\n Temper Data API: http://localhost:43148/tdata\n Listening on http://0.0.0.0:43148\n\n curl -sS -i http://127.0.0.1:43148/observe/health\n\n HTTP/1.1 200 OK\n content-type: application/json\n {"status":"healthy","uptime_seconds":13,"specs_loaded":21,"active_actors":0,"indexed_entities":0,"transitions_total":0,"errors_total":0,"event_store":"turso","cross_invariant_enforce":true,"cross_invariant_eventual_enforce":true}\n\nExact durable ledger and rollback-only trigger-probe residue:\n sqlite3 -header -column /tmp/arn242-pr389-cc75b4a9-current-live-20260720T053414Z.db "SELECT version, checksum FROM temper_schema_migrations ORDER BY version; SELECT 'ledger_rows' AS metric, COUNT() AS value FROM temper_schema_migrations UNION ALL SELECT 'ots_rows', COUNT() FROM ots_trajectories UNION ALL SELECT 'audit_rows', COUNT(*) FROM ots_probe_audit;"\n\n 1 aa159f54e46819312662448552d6ebdd56e5fe0e31d4f7619b28c3c9272521d2\n 2 5ee4fcfe1f9ff6a7b0d1d4c4081a7ec375491abc0f5ba96ec42d1eb405f137cb\n 3 1634a9cdd48acff70f20d2ef78a6b91285a016bd53dbd459b7e9c663650f691c\n 4 e25ad7da3742dc84c0a2ef1b0713b1858b82af040fd503f5cfd3aed4e7151bef\n 5 5606020fefa283e048c7a72ae8bd7db3dc91336a946e8305ac6993e606f50aeb\n 6 78aa56edc91e690e39ffeacc46c85804b3034d2368a480eae1769dfaa516b111\n 7 1e4e7af4c85e857bf0a55890d4c4142f95cc5053a252c70309794599c481603b\n ledger_rows 7\n ots_rows 0\n audit_rows 0\n\nRecognizable historical-probe bypass fails closed without mutation:\n RUST_LOG=error TURSO_URL=file:/tmp/arn242-pr389-cc75b4a9-probe-bypass-20260720T053414Z.db target/debug/temper serve --port 43146 --no-observe\n\n Error: Failed to connect to Turso/libSQL: storage error: Turso migration 7 (declared-key-and-vector-indexes) failed while attempting to verify catalog head schema: storage error: Turso schema compatibility check failed: table 'ots_trajectories' has unsupported executable trigger extension 'reject_non_probe_tenant'; legacy extensions must be an unconditional AFTER INSERT audit trigger with exactly one INSERT INTO <audit_table>(trajectory_id) VALUES (NEW.trajectory_id) statement\n\n ledger_rows 7\n ots_rows 0\n trigger_rows 1\n\nIncompatible required object fails at migration 4 and preserves the view:\n RUST_LOG=error TURSO_URL=file:/tmp/arn242-pr389-cc75b4a9-incompatible-20260720T053414Z.db target/debug/temper serve --port 43147 --no-observe\n\n Error: Failed to connect to Turso/libSQL: storage error: Turso migration 4 (apps-platform-and-secrets) step 1 schema compatibility check failed: capability 'tenant_installed_apps' must be a table, found view\n\n 1 event-journal-and-snapshots aa159f54e46819312662448552d6ebdd56e5fe0e31d4f7619b28c3c9272521d2\n 2 specs-constraints-and-integrations 5ee4fcfe1f9ff6a7b0d1d4c4081a7ec375491abc0f5ba96ec42d1eb405f137cb\n 3 authorization-and-artifacts 1634a9cdd48acff70f20d2ef78a6b91285a016bd53dbd459b7e9c663650f691c\n view tenant_installed_apps\n\nThe live server was stopped with Ctrl-C after the health and durable-state checks. All database files are retained under /tmp; no cleanup or deletion was performed. |
|
ARENA SHIPPABLE · GPT-5.6 · 2026-07-20T16:18:57Z Exact open PR head: cc75b4a Ordered review gate:
Exact-head verification:
Scope audit:
|
2 participants
Objective
Fix ARN-242 by replacing ad hoc Turso startup DDL with a durable, ordered, checksummed migration ledger that prevents readiness when the schema is incompatible or a migration cannot complete.
Root cause
TursoEventStore::newand router startup issued schema changes procedurally. Some migration errors were discarded, there was no durable version/checksum record, and startup could report healthy while required capabilities—or executable schema extensions—could not serve the first runtime write.Design
origin/mainand every open PR.temper_schema_migrationsrecords version, unique name, SHA-256 checksum, and applied time.BEGIN IMMEDIATE, rechecks the ledger, applies the migration, verifies the cumulative prefix, records the version, rereads it, and commits atomically.Behavioral and durability coverage
sqliteX...names cannot bypass inventory; plain safe non-unique indexes remain compatible.Evidence
47a6655cand failing behavior.15c85ec1(four behavioral/durability regressions failed before the fix).sqliteX...trigger refuse readiness; fresh install and replay return 200 with the complete pinned ledger.fe3a942d9c7dfc45f486a262afb2ba6375313879.cargo test --workspace, all DST suites, and doc-tests.Scope
This PR does not touch
crates/temper-actor-runtime; outside-contributor sign-off is not required.The required exact-head fresh-context GPT-5.6 review, Greptile review/remediation, final fresh review, GitHub CI, and shippable evidence remain gates. This PR stays draft until all are complete.
Greptile Summary
This PR adds a durable migration path for Turso schema startup. The main changes are:
temper_schema_migrationsledger and validation flow.Confidence Score: 4/5
The Turso schema verifier needs a fix before merging.
crates/temper-store-turso/src/migrations/schema_verify.rs
Security Review
Extra executable index definitions can bypass the new Turso readiness check when they are attached through a case-variant table owner.
Important Files Changed
Sequence Diagram
%%{init: {'theme': 'neutral'}}%% sequenceDiagram participant Store as TursoEventStore::new participant Runner as migrations::migrate participant Ledger as temper_schema_migrations participant Catalog as Migration catalog participant Verify as Schema verifier Store->>Runner: run startup migration Runner->>Ledger: create and validate ledger Runner->>Ledger: load versions and checksums loop pending versions Runner->>Catalog: apply steps in transaction Runner->>Verify: verify expected schema Runner->>Ledger: record version checksum end Runner->>Verify: verify catalog head Runner-->>Store: report ready%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%% sequenceDiagram participant Store as TursoEventStore::new participant Runner as migrations::migrate participant Ledger as temper_schema_migrations participant Catalog as Migration catalog participant Verify as Schema verifier Store->>Runner: run startup migration Runner->>Ledger: create and validate ledger Runner->>Ledger: load versions and checksums loop pending versions Runner->>Catalog: apply steps in transaction Runner->>Verify: verify expected schema Runner->>Ledger: record version checksum end Runner->>Verify: verify catalog head Runner-->>Store: report readyPrompt To Fix All With AI
Reviews (1): Last reviewed commit: "fix(store-turso): constrain legacy OTS t..." | Re-trigger Greptile
Context used (3)