From e47b00358e19680769ad4aea39e3208933396195 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Sat, 2 May 2026 10:33:51 +0000 Subject: [PATCH] docs: fix schema link and expand storage/ledger reference - Fix broken JSON Schema link in release-artifact.md: release_artifact.schema.json -> release.schema.json (actual file name) - Add pricing_import_audit to the SQLite table list in operations-and-policy.md (7 tables, not 5) - Document Storage connection settings (WAL, IMMEDIATE transactions, busy_timeout, foreign_keys pragma) - Document idempotent run event ingestion: run_id PRIMARY KEY deduplication, per-row insert loop, safe re-ingestion semantics - Document schema migration versions 1-3 with change summary and guidance for adding future migrations - Add Rollup semantics section: how runs/cost/latency/error_rate are computed, run_start vs run_end event counting, latency_ms_avg=None meaning, delta_cost_per_run_pct and delta_latency_ms_avg None guards - Document active_policy single-row upsert behavior and get_active_policy tie-breaking on updated_at Co-authored-by: Gottam Sai Bharath --- docs/operations-and-policy.md | 73 ++++++++++++++++++++++++++++++++++- docs/release-artifact.md | 2 +- 2 files changed, 73 insertions(+), 2 deletions(-) diff --git a/docs/operations-and-policy.md b/docs/operations-and-policy.md index c88e59a..b9107dc 100644 --- a/docs/operations-and-policy.md +++ b/docs/operations-and-policy.md @@ -85,6 +85,30 @@ Runs are averaged across all events in the window to produce `cost_per_run_usd`. spec *before* querying events. This is checked again inside `diff_releases` if run events from both sides are non-empty. +### Rollup semantics + +`ledger.compute_rollup` aggregates a list of `RunEvent` objects into a `Rollup`: + +| Field | How it is computed | +|-------|--------------------| +| `runs` | Total number of events in the window | +| `cost_per_run_usd` | Average of `estimate_cost_usd(event, pricing_table)` across all events | +| `latency_ms_avg` | Average of `metrics.latency_ms` across events **where latency is present**; `None` when no event has latency data | +| `error_rate` | Fraction of events where `metrics.success == False` | + +**All events in the query window count** — including `type: run_start` events. The +`run_id` is the deduplicated key; if an agent emits both `run_start` and `run_end` for +the same logical run, **both** are stored and counted unless they share the same `run_id`. +Best practice is to ingest only `run_end` (the default `type`) when a single-event +model is used, or use distinct `run_id` values when emitting both start and end events. + +`latency_ms_avg` is `None` (not zero) when the window has no events with latency data. +Policy's `max_latency_ms` check is **skipped** when `latency_ms_avg` is `None`. + +`delta_cost_per_run_pct` in `DiffResult` is `None` when `baseline.cost_per_run_usd == 0` +(division by zero guard). Similarly, `delta_latency_ms_avg` is `None` when either side +has no latency data. + --- ## `promote_release` / `rollback_release` @@ -169,6 +193,12 @@ All `min_*` fields default to `None` (defer to `WorkspaceConfig.diff` defaults). ### Setting the active policy +`active_policy` is a single-row table keyed on `policy_id`. `policy set` uses an +`INSERT … ON CONFLICT(policy_id) DO UPDATE` upsert, so calling it repeatedly with +the same `policy_id` always overwrites in place. Changing `policy_id` between calls +creates a second row; `get_active_policy` resolves ambiguity by returning the row +with the most recent `updated_at`. + ```bash flightdeck policy set examples/quickstart/policy.yaml flightdeck policy show @@ -291,12 +321,13 @@ endpoints (`GET /v1/releases`, `GET /v1/promoted`, `GET /v1/actions`) and intern ## SQLite storage schema -The operations layer reads and writes five tables (via `src/flightdeck/storage.py`): +The operations layer reads and writes seven tables (via `src/flightdeck/storage.py`): | Table | Purpose | |-------|---------| | `releases` | Immutable release records keyed by `release_id` | | `pricing_tables` | Pricing data keyed by `(provider, pricing_version)` | +| `pricing_import_audit` | Append-only log of every `pricing import` operation (insert or replace) | | `run_events` | Ingested runtime evidence indexed by `(release_id, timestamp)` | | `active_policy` | Single-row table holding the active `Policy` JSON | | `promoted_releases` | Current promoted pointer per `(agent_id, environment)` | @@ -306,6 +337,46 @@ The operations layer reads and writes five tables (via `src/flightdeck/storage.p that migrations are applied through `LATEST_SCHEMA_MIGRATION_VERSION` and that `audit_seq` has no gaps. +### Storage connection settings + +Every connection is configured with four pragmas before any statement runs: + +| Pragma | Value | Effect | +|--------|-------|--------| +| `foreign_keys` | `ON` | Referential integrity enforcement | +| `journal_mode` | `WAL` | Write-ahead logging; multiple readers can co-exist with a writer | +| `synchronous` | `NORMAL` | Durable enough for power-loss safety without `FULL` fsync overhead | +| `busy_timeout` | `5000` | Wait up to 5 s for a lock before returning `SQLITE_BUSY` | + +Write operations that must be atomic (promote/rollback, pricing import) use +`BEGIN IMMEDIATE` transactions, which acquire the write lock upfront and prevent +`SQLITE_BUSY` races between concurrent writers. + +### Idempotent run event ingestion + +`insert_run_events` inserts rows one at a time and **silently ignores** +`sqlite3.IntegrityError` on `run_id` PRIMARY KEY conflicts. This means: + +- Re-ingesting a JSONL file is safe; duplicate events are skipped. +- The return value is the number of **newly inserted** rows (not the total count + in the input). +- Events are not batched in a single transaction, so a partial failure leaves + already-inserted rows in place. Re-running the ingest picks up where it left + off because duplicates are skipped. + +### Schema migrations + +Migrations are numbered and forward-only; they are never reversed. + +| Version | Change | +|---------|--------| +| 1 | Initial schema (all base tables via `CREATE TABLE IF NOT EXISTS`) | +| 2 | `CREATE INDEX … ON run_events(release_id, timestamp)` — speeds up diff/query | +| 3 | `ALTER TABLE release_actions ADD COLUMN audit_seq INTEGER`; backfill existing rows; add unique index | + +New migrations must increment `LATEST_SCHEMA_MIGRATION_VERSION` in `storage.py` and add a +corresponding check in `test_schemas.py` (or `test_doctor.py`). + --- ## Common errors and remedies diff --git a/docs/release-artifact.md b/docs/release-artifact.md index f5c4a73..1f42b15 100644 --- a/docs/release-artifact.md +++ b/docs/release-artifact.md @@ -61,7 +61,7 @@ spec: All field values are stored verbatim in the `releases` SQLite table as `artifact_json`. The `spec.agent.agent_id` is the primary grouping key used by diff, promote, and rollback. -**JSON Schema:** [`schemas/v1/release_artifact.schema.json`](../schemas/v1/release_artifact.schema.json) +**JSON Schema:** [`schemas/v1/release.schema.json`](../schemas/v1/release.schema.json) ---