mdtazizulislam · mdtazizulislam · Jul 6, 2026
@@ -0,0 +1,64 @@
+# Phase 3B — Baseline Drift Finding (why PR #248 is replaced)
+
+> **EVIDENCE — DOCS ONLY.** Establishes that production has schema drift not captured by
+> `initDatabase()` or the migration files, so the PR #248 baseline is unfaithful. All
+> diagnostics ran on **throwaway PG16** (torn down); production untouched.
+
+## 1. Apples-to-apples counts (same catalog queries both sides)
+
+The B3 operator report and PR #248 originally used **different** count methods (drizzle
+introspect summary vs SQL catalog). To settle whether the gap was real or a measurement
+artifact, `initDatabase()` was re-materialized on a throwaway PG16 and counted with the
+**exact catalog queries the operator used**:
+
+| Metric (catalog query) | `initDatabase()` | Real prod (operator, PG18) | Delta |
+|---|---|---|---|
+| `information_schema.tables` (BASE, public) | 125 | 125 | **0** |
+| `information_schema.columns` (public) | **1595** | **1649** | **+54** |
+| `pg_indexes` (public) | 400 | 402 | +2 |
+| non-PK indexes (`pg_indexes`, not `%_pkey`) | 275 | 277 | +2 |
+| FKs (`table_constraints` FK, public) | 267 | 266 | −1 |
+
+The `+54` columns are **not** a measurement artifact — measured identically, `initDatabase()`
+genuinely produces 54 fewer columns than production has.
+
+## 2. Do the hand-written migrations explain it? No.
+
+All **24** hand-written migration `.sql` files
+(`packages/db-client/src/postgres/migrations/*.sql` + `apps/backend/migrations/*.sql`) were
+applied on top of the `initDatabase()` base and columns re-counted:
+
+```
+initDatabase() alone ................... 1595 columns / 267 FKs
++ all 24 migration .sql files .......... 1596 columns / 267 FKs   (+1 column only)
+real production (Gate 1 / PG18) ........ 1649 columns / 266 FKs
+```
+
+The migrations add **one** column. So **~53 production columns and the FK/index deltas exist
+in NO source of truth** — not the current `initDatabase()` code, not the migration files.
+
+## 3. Interpretation
+
+This is **undocumented production schema drift** — columns/indexes added directly to prod (or
+by since-removed code) that the current source no longer reproduces, minus one FK the source
+declares but prod lacks. It is precisely the class of drift Phase 3 exists to eliminate, and
+it means:
+
+- ❌ A baseline generated from `initDatabase()` (PR #248) **misrepresents** production by ~54
+  columns — adopting it (mark-as-applied) would record a false "this is prod" baseline and
+  desync future `drizzle-kit generate` diffs.
+- ✅ The **only** faithful baseline is one **introspected from the real production schema**
+  (the Gate 1 snapshot) — which the operator already produced on PG18 in B3 (1649 columns).
+
+## 4. Consequence
+
+- **PR #248 baseline → superseded.** Do not adopt it.
+- **Authoritative baseline = the PG18 introspect of the real Gate 1 snapshot** (doc 02).
+- The **exact 54-column / 2-index / 1-FK delta list** should be captured (normalized diff,
+  doc 02 Step 3) and filed — it is a valuable inventory of the undocumented prod drift and
+  feeds the Phase 3C convergence work.
+
+## 5. Guardrails
+
+Diagnostics ran on throwaway PG16 only (dropped after). No production deploy, migration
+execution against prod, schema application, backend wiring, or feature-flag activation.
@@ -0,0 +1,115 @@
+# Phase 3B — Baseline Replacement Runbook (operator, PG18)
+
+> **OPERATOR RUNBOOK — DOCS ONLY.** Regenerates the authoritative `0000_prod_baseline` from
+> the **real Gate 1 production snapshot on PostgreSQL 18**, diffs it against the superseded
+> PR #248 candidate, builds the journal, and proves seeded `db:migrate` skips with zero DDL.
+> Runs on throwaway PG18; **production is never touched.**
+
+Inputs: `postgres:18-alpine` · Gate 1 snapshot
+`C:\Users\vitor\zf-phase3b\20260705T215941Z\zonforge_prod_schema.sql` · repo checkout.
+Reference (real prod, from B3): **125 tables · 1649 columns · 402 indexes · 266 FKs**.
+
+## Step 1 — Regenerate the authoritative baseline (PG18, real snapshot)
+
+```powershell
+docker run -d --name zf-repl -e POSTGRES_PASSWORD=repl -p 55451:5432 postgres:18-alpine
+Start-Sleep 6
+docker cp "C:\Users\vitor\zf-phase3b\20260705T215941Z\zonforge_prod_schema.sql" zf-repl:/snap.sql
+docker exec zf-repl psql -U postgres -v ON_ERROR_STOP=1 -f /snap.sql *> $null   # load real prod schema
+
+# introspect into the AUTHORITATIVE baseline
+@'
+import type { Config } from "drizzle-kit"
+export default { schema: "./packages/db-client/src/postgres/prod-baseline-v2-schema.ts",
+  out: "./packages/db-client/src/postgres/prod-baseline-v2", dialect: "postgresql",
+  dbCredentials: { url: process.env.DATABASE_URL ?? "" }, strict: true } satisfies Config
+'@ | Set-Content packages/db-client/drizzle.config.replace.ts
+$env:DATABASE_URL = "postgresql://postgres:repl@localhost:55451/postgres"
+Remove-Item -Recurse -Force packages/db-client/src/postgres/prod-baseline-v2 -ErrorAction SilentlyContinue
+npx drizzle-kit introspect --config packages/db-client/drizzle.config.replace.ts
+```
+Expect the introspect summary to reflect the **real prod** (≈125 tables / 1649 cols /
+402 idx / 266 FK). Rename the generated `0000_*.sql` → `0000_prod_baseline.sql` and set the
+`meta/_journal.json` tag to `0000_prod_baseline`.
+
+## Step 2 — Fix the introspect artifact
+
+Correct the known `int4_ops`-on-`timestamptz` index (drizzle-kit introspect quirk) so the
+DDL is valid if ever applied to a fresh DB. Grep the generated SQL for `int4_ops` and set the
+correct operator class (or drop the erroneous `USING` clause) for that index.
+
+## Step 3 — Normalized structural diff vs PR #248 (capture the drift inventory)
+
+```powershell
+$V2 = "packages/db-client/src/postgres/prod-baseline-v2/0000_prod_baseline.sql"
+$PR248 = "docs/audit/phase3b-promotion/exec/prod-baseline/0000_prod_baseline.sql"   # PR #248 branch
+function Norm($f){ (Get-Content $f) -match '^\s*(CREATE TABLE|\t"|ALTER TABLE|ADD CONSTRAINT|CONSTRAINT|FOREIGN KEY|PRIMARY KEY)' | Sort-Object }
+Compare-Object (Norm $PR248) (Norm $V2) | Format-Table
+```
+- `=>` rows = present only in **real prod** (the ~54 drift columns / +2 indexes) — **save this
+  list** as the drift inventory (feeds Phase 3C).
+- `<=` rows = present only in **PR #248** (e.g., the 1 FK prod lacks) — record for reconciliation.
+
+## Step 4 — Build + confirm the journal
+
+`meta/_journal.json` (from introspect) must have one `0000_prod_baseline` entry. Record its
+`when` and `sha256sum 0000_prod_baseline.sql` (both feed the seed).
+
+## Step 5 — Prove seeded `db:migrate` skips with ZERO DDL (PG18)
+
+The B3 attempt was blocked because the live `meta/_journal.json` was absent. Point the
+migrator at the new baseline folder:
+
+```powershell
+# copy the v2 baseline into the path migrate.ts reads, in a THROWAWAY checkout/build
+Copy-Item packages/db-client/src/postgres/prod-baseline-v2/0000_prod_baseline.sql packages/db-client/src/postgres/migrations/
+New-Item -ItemType Directory -Force packages/db-client/src/postgres/migrations/meta | Out-Null
+Copy-Item packages/db-client/src/postgres/prod-baseline-v2/meta/_journal.json packages/db-client/src/postgres/migrations/meta/
+npm run build --workspace=packages/db-client   # copies migrations into dist (add copy step if needed)
+
+$U = "postgresql://postgres:repl@localhost:55451/postgres"   # PG18 with the 125 real tables
+# F1
+docker exec zf-repl psql -U postgres -At -c "SELECT md5(string_agg(table_name||'.'||column_name||':'||data_type,',' ORDER BY table_name,ordinal_position)) FROM information_schema.columns WHERE table_schema='public';"
+# seed the journal row (created_at = journal when), then run db:migrate -> must SKIP
+docker exec zf-repl psql -U postgres -c "CREATE SCHEMA IF NOT EXISTS drizzle; CREATE TABLE IF NOT EXISTS drizzle.__drizzle_migrations(id serial primary key, hash text not null, created_at bigint); INSERT INTO drizzle.__drizzle_migrations(hash,created_at) VALUES ('<SHA256>', <WHEN>);"
+$env:ZONFORGE_POSTGRES_URL = $U
+npm run db:migrate --workspace=packages/db-client   # expect: nothing applied (skip) -> ZERO DDL
+# F2
+docker exec zf-repl psql -U postgres -At -c "SELECT md5(string_agg(table_name||'.'||column_name||':'||data_type,',' ORDER BY table_name,ordinal_position)) FROM information_schema.columns WHERE table_schema='public';"
+```
+Pass: **F1 == F2**, 125→125 tables, `db:migrate` applied nothing. ⚠️ Seed **before** migrate.
+Do this in a **throwaway checkout** — do NOT commit the copy into the live `migrations/` path
+until the separately-approved promotion (PR #247 governs).
+
+## Step 6 — Teardown + deliver to the replacement PR
+
+```powershell
+docker rm -f zf-repl
+Remove-Item -Recurse -Force packages/db-client/src/postgres/prod-baseline-v2, packages/db-client/drizzle.config.replace.ts -ErrorAction SilentlyContinue
+```
+Commit to the replacement PR (this branch): the authoritative `0000_prod_baseline.sql` +
+`meta/_journal.json` (staged under `docs/audit/phase3b-promotion/replacement/prod-baseline/`,
+quarantined like PR #248's), the **drift inventory** from Step 3, and the filled Step 6 report:
+
+```
+REPLACEMENT BASELINE (PG18, real snapshot) — <UTC>
+ engine ........... : 18.4
+ tables ........... : 125
+ columns .......... : 1649   (authoritative)
+ indexes / FKs .... : 402 / 266
+ int4_ops fixed ... : yes/no
+ diff vs PR #248 .. : +__ cols / +__ idx / -__ fk  (inventory attached)
+ ZERO_DDL (PG18) .. : F1==F2? yes/no ; db:migrate applied nothing? yes/no
+ RESULT ........... : PASS -> supersede PR #248
+```
+
+## Decision
+
+- **PASS** → the snapshot-derived baseline supersedes PR #248; **close PR #248** in favor of
+  this replacement PR. B3 cleared. Proceed to B2 (restore/PITR), still separately approved.
+- **FAIL** → iterate on the baseline; PR #248 stays superseded, neither adopted.
+
+## Guardrails
+
+No production deploy · migration execution against prod · schema application to prod ·
+backend wiring · feature-flag activation. Throwaway PG18 only; production never targeted.
@@ -0,0 +1,52 @@
+# Phase 3B — 0000_prod_baseline Replacement Plan (supersedes PR #248)
+
+> **PLAN / EVIDENCE — DOCS ONLY.** B3 verification proved PR #248's baseline (generated from
+> `initDatabase()`) is **not faithful to production**. This package documents the confirmed
+> drift and the plan to regenerate the authoritative baseline from the **operator's real
+> Gate 1 snapshot on PG18**. **No production deploy, migration execution, schema application,
+> backend wiring, or feature-flag activation.** PR #248 stays **unmerged / superseded**.
+
+## Why PR #248 is superseded
+
+B3 (PG18, real Gate 1 snapshot) vs PR #248 (initDatabase, PG16), measured with **identical
+catalog queries** (`information_schema` / `pg_indexes`):
+
+| Metric | PR #248 (`initDatabase()`) | Real prod (Gate 1 / PG18) | Delta |
+|---|---|---|---|
+| Tables | 125 | 125 | 0 ✅ |
+| **Columns** | **1595** | **1649** | **+54 (real)** |
+| Indexes (`pg_indexes`) | 400 | 402 | +2 |
+| FKs | 267 | 266 | −1 |
+| Non-PK indexes | 275 | 277 | +2 |
+
+Applying all **24 hand-written migration `.sql` files** on top of `initDatabase()` moved
+columns only **1595 → 1596** — so the migrations do **not** explain the gap. **~53 production
+columns exist in no source of truth** (not `initDatabase()`, not the migration files) — plus
++2 indexes and a −1 FK. This is **undocumented production drift** (the core Phase-3 problem),
+and it makes the `initDatabase()`-derived baseline unsafe to adopt.
+
+**Conclusion:** the authoritative `0000_prod_baseline` must be introspected from the **real
+production schema** (the Gate 1 snapshot), not from `initDatabase()`. PR #248's candidate is
+**suspect and replaced**.
+
+## Documents
+
+| Doc | Purpose |
+|---|---|
+| [`01_DRIFT_FINDING.md`](./01_DRIFT_FINDING.md) | Full evidence: apples-to-apples counts + migration test |
+| [`02_REPLACEMENT_RUNBOOK.md`](./02_REPLACEMENT_RUNBOOK.md) | Operator steps to regenerate + diff + journal + zero-DDL on PG18 |
+
+## Status
+
+- **B3:** CONDITIONAL/PARTIAL PASS only — table count (125) verified on PG18; column/index/FK
+  drift **confirmed real**; ZERO-DDL migrate-skip **not yet proven** on PG18.
+- **PR #248:** unmerged; to be **superseded** by the snapshot-derived baseline (this plan).
+- **This is the replacement/superseding track** requested. The authoritative baseline
+  artifacts are produced by the operator on PG18 from the real snapshot (runbook doc 02) and
+  committed to the replacement PR.
+
+## Guardrails
+
+❌ No production deploy · migration execution · schema application · backend wiring ·
+feature-flag activation. ✅ Diagnostics ran on throwaway PG16 (torn down); production never
+touched; live migrations path unchanged; `db:migrate` still no-op.