W3: scheduled-publish + ingested-sortAt E2E + full-scale proof (all gates PASS) - #329
Merged
Conversation
…harness Pins the W3 critical cases (scheduled-publish-design §2.D, #325+#326): - tests/e2e/e2e-scheduled-publish-sortat.mjs — self-contained (no PG) Node E2E driving /ops with the exact #326 trigger payloads: ingested sortAtUnix(ms)->sortAt(s), scheduled->deferred, the CRITICAL no-op sweep activation at Tf (ingested future sortAt preserved through activation), unpublish/republish. 14/14 PASS. - tests/w3/ — full-scale reproducible harness (prep/dedup/dump-driver/assert, local-PG schema + model-share sortAt triggers, minimal poller, lifecycle, latency, re-emitter) + README with the gate evidence (all gates PASS: 114.8M dump 23m13s no-regression, sortAt invariant 600/600, model3dId 108,567, lifecycle incl. no-op sweep, heal-path, latency, PR-B3). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…column self-heal, widened [PR-M2] - README: gate 1 remains valid without re-dump (prep computes pure GREATEST, never reads Image.sortAt — identical to #328's amended copy_query). Emptied-originals + stage_dir isolation warnings. - w3-stale-column.sql: proves the widened BEFORE trigger heals a stale sortAt column — an nsfwLevel-only update emits the CORRECT recomputed sortAtUnix (verified: stale 2020 -> correct). - w3-m2-latency.sql: widened-trigger per-update overhead +47us P50 / +390us P99 (sub-ms). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
JustMaier
added a commit
that referenced
this pull request
Jul 17, 2026
…ortAt, model3dId (#332) * docs: scheduled-publish wrap-up — per-image fan-out, ingested sortAt, model3dId (#13) Documents the shipped scheduled-publish effort (PRs #325/#326/#328/#329/#330) across the doc set: - pg-sync-v2-final.md: Post fan-out is now per-image materialized (type: fan_out_per_row) not queryOpSet; shared PG functions bitdex_post_fanout_ops / bitdex_image_sortat_ops; sortAt ingested (computed: dropped), dump never trusts the column; ModelVersion/Model stay queryOpSet. - CLAUDE.md: Sync V2 section reflects new fan-out shape + ingested sortAt + model3dId. - scheduled-publish-design.md: §1 history corrected (2024 migration triggers, Post-side neutered by name-collision, new_image_sort_at absent from prod pg_trigger per zuri's live verification); §2.D marked EXECUTED with the no-backfill decision and W1-4 dominant-class (visible-but-misordered) finding. - execution-plan: W4 step 2 no-backfill correction note. - FOLLOWUP.md: 7 residuals (over-cap fan-out skip, flaky flush tests, retired combinedNsfwLevel, frankenstein docs, MV cascade cost, stage_dir isolation, E2E population gap). - HANDOFF.md: scheduled-publish design chain pointer + W4 rollout state. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(reemitter): correct Tf-activation model to sweep-driven shadow flip Prod evidence (sortat-divergence-specimens-2026-07-16.md PR-m5 + execution plan corrected [PR-B2]) established that scheduled slots are ALIVE with isPublished=false (quarantine branch), NOT deferred-map-resident, so Tf activation is an overdue-sweep shadow flip, not activate_due. The re-emitter is therefore a co-driver of activation, not a pure safety net. W3 confirmed the no-op case (sweep alone flipped isPublished + preserved ingested sortAt, 14/14 E2E). Fixes fact #1 + the §7 'does not drive activation' bullet. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
W3 local full-scale gate for #325+#326 (scheduled-publish-design §2.D). All gates PASS.
What's here
tests/e2e/e2e-scheduled-publish-sortat.mjs— CI-friendly self-contained Node E2E (no PG). Creates its own index and drives/opswith the exact W2-1: sync/index config — ingested sortAt + model3dId, per-image Post fan-out #326 trigger payloads. Pins: ingestedsortAtUnix(ms)→sortAt(s) exactly; scheduled→deferred; the CRITICAL no-op sweep activation at Tf (ingested future sortAt preserved through activation); unpublish/republish; model3dId. 14/14 PASS. Run:node tests/e2e/e2e-scheduled-publish-sortat.mjs --url http://localhost:<empty-server>.tests/w3/— full-scale reproducible harness + README with all gate evidence.Gate results
sortAtinvariant 600/600;model3dId Eq 42= 108,567; isPublished 102.6M/6.56M. Ingested seconds land in the sort layers with no unit corruption.sortAt== trigger-written column == BitDex-stored, on the same rows.Notes
Two test-data quirks fixed in the harness (not BitDex defects): staged
images-small.csvduplicates every id (prod ids are unique → dedup); thehashblurhash alphabet includes,so PG quotes it (quote-aware splice). Ops note:bitdex-sync pg's boot-dump writes intostage_dir— keep it isolated from a live dump'sload_stage.Sync-v2 change — requires Justin's approval before merge.
🤖 Generated with Claude Code