fix: repair the June-24 identity fork with audited dedupe migrations, alias-first import identity, and doctor parity - #159
Merged
Merged
Conversation
Promote the captured brief to a full Change contract: shape.md with Product and Planning Contracts, plus four task packets — alias-orphan repair migration, importer alias-first identity resolution, doctor alias-parity diagnostic, and the production repair ceremony. Claude-Session: https://claude.ai/code/session_011wW8VzMJvENoeSY6QxWYkK
loaf state migrate alias-orphans repairs the June-24 duplication damage with the full preview/apply/rollback ceremony, sweeping every project in the global database. Classification runs per project, per entity table (tasks, specs, reports, ideas, sparks, brainstorms): an orphan is an entity row with no matching aliases row. Twins are proven by recomputing the historical derived ID stableMigrationID(kind, hex(sha256(current_path)), alias) for the project's alias holders; exact-title match against a single alias holder inside the June-24 event cluster is a distinctly-labeled content-identity fallback. Everything else is unproven and untouched unless the operator supplies an explicit per-row disposition via repeatable --retire <entity-id> / --realias <entity-id>=<alias> flags, recorded verbatim in the manifest. Preview (the default) classifies against a temporary database copy and mutates nothing. Apply takes a mandatory backup first, applies in one transaction under PRAGMA defer_foreign_keys = ON, snapshots every deleted row into a JSON rollback manifest written beside the backup, and verifies after commit that zero retire-class orphans and dangling aliases remain. Retirement generalizes the spec-delete reference-table sweep across entity kinds: artifact bodies and FTS, events, entity_tags, bundle_members, backend_mappings, exports, relationships, then the row, then sources left unreferenced. Dangling aliases are deleted; spec/task foreign keys pointing at retired rows are nulled and recorded for rollback. The broken-evidence report (report:7644bb23d2664de93b6cb6a5) carries a named disposition: archived as moot with an event recording the unrecoverable evidence and the SPEC-047 rationale. Rollback restores deleted rows, statuses, aliases, and unlinked references from the manifest. Registered as alias-orphans in the stateMigrateSources registry with help routing and human/JSON output, and documented in the generated CLI reference. Tests cover proof labeling, apply/rollback round-trip, idempotency, residue removal, operator dispositions, and preview isolation, all against temp databases via t.Setenv. Claude-Session: https://claude.ai/code/session_011wW8VzMJvENoeSY6QxWYkK
… ids The markdown importer now looks up (project_id, namespace, alias) in the aliases table before minting a derived ID: when the alias already names an entity of the imported kind, that entity's ID is reused so the entity upsert hits ON CONFLICT(id) DO UPDATE instead of inserting a twin, and the alias upsert rewrites the same entity_id it resolved — orphaning and re-pointing become impossible regardless of project-ID changes. stableMigrationID remains the fallback for genuinely new entities only. resolveImportedEntityID backs every derived-ID call site: specs, tasks, ideas, brainstorms, shaping drafts, reports, session-journal sparks with a slug alias, spec placeholders, task dependency targets, and frontmatter relationship targets. The task→spec relationship now points at the ID ensureSpecPlaceholder resolved rather than an independently derived one. Sources carry no aliases, so resolveSourceID reuses the existing row keyed by (project_id, path) before deriving, closing the source-doubling path. Regression tests simulate the historical damage sequence — import, rewrite project_id columns exactly as rekeyLegacyProjectTx does, re-import — and assert zero new entity rows, zero new source rows, zero alias-orphans, stable alias→entity mappings, and no twin rows under the new project ID. An idempotency test asserts the canonical business dump is byte-stable across a no-change re-import, and a re-point test pins the live alias to its original entity. All state is isolated via temp dirs. Claude-Session: https://claude.ai/code/session_011wW8VzMJvENoeSY6QxWYkK
InspectAliasParity sweeps every project in the global database and, for each of the six entity tables in the aliasOrphanEntityTables registry, compares raw row counts to alias-reachable counts and counts dangling alias rows. The check is read-only and runs inside the operational invariants pass of state.Inspect, so both the human and the JSON doctor surfaces carry it. Parity always renders: a green sweep emits an info alias-parity-clear diagnostic with aggregate counts, and any orphan delta or dangling alias emits an error alias-parity-diverged diagnostic carrying the per-project, per-table breakdown in its details and naming loaf state migrate alias-orphans as the repair. The repair plan maps the divergence code to a migrate-alias-orphans action with that command. Following the journal-search-divergence precedent, the error severity fails doctor without flipping the database mode to invalid — identity damage is detectable while the database stays usable. Tests cover the clean fixture, orphan and dangling-alias findings with exact counts, and byte-identical database files before and after inspection. CLI doctor fixtures that seeded tasks without aliases now seed matching alias rows so they exercise their original subjects. Claude-Session: https://claude.ai/code/session_011wW8VzMJvENoeSY6QxWYkK
Retirement was reachable without a proof and incomplete once reached. Proof. Content identity gated on a June-24 timestamp appearing on *either* side of the pair, so a bare title match against any same-titled alias holder scheduled a deletion. It now requires the surviving holder to be the re-import, the orphan to predate it, exactly one candidate on each side, a non-empty title, and identical stored bodies. Legacy-salt recomputation consults every path in project_paths, not just current_path, so a project that moved still earns the strong proof; sparks — minted from (path, line) and never from an alias — earn it through their own source ID. The manifest records which historical path produced each match. Sweep. findings and verdicts hang off a report by a NOT NULL foreign key that no polymorphic (entity_kind, entity_id) sweep reaches, so retiring an orphan report aborted the whole migration at COMMIT with an unattributable error. They now retire with their report. journal_deferrals.spark_id and intent_operations.spark_id are NOT NULL with no constraint at all; they repoint at the proven twin, or are captured and deleted when there is nowhere to go. Realias. --realias onto a claimed alias silently stole it, manufacturing a fresh alias-orphan of exactly the class this migration repairs. It is refused, naming both rows. Dangling aliases are cleared first so a target freed by the same run stays available. Ceremony. The rollback manifest is written inside the transaction, before COMMIT, so no deletion is ever visible without its restore record; a status change now restores updated_at as well as status; preview runs the repair against its disposable copy so it can report the source rows the retire set will strand; and a --retire/--realias flag that matches no orphan is an error instead of a silent no-op. The DB-level and transaction-level classifiers were duplicated, which is why the timestamp gate was wrong in two places. There is now one. Claude-Session: https://claude.ai/code/session_011wW8VzMJvENoeSY6QxWYkK
A spark's alias is the first word of its message, so unrelated sparks share one routinely. Resolving spark identity through that alias made two journal lines collapse onto a single row, and the upsert then overwrote the first spark's text with the second's — silently, on every import. Identity for a spark stays the journal line it came from. The alias is only allowed to name the reused entity when the row behind it is unmistakably that same line: same source file, same text. A rekey and re-import still resolve to the existing row, and two sparks that merely start with the same word stay two sparks. Claude-Session: https://claude.ai/code/session_011wW8VzMJvENoeSY6QxWYkK
…ctor path The parity check counted entities holding at least one alias, but the list surfaces INNER JOIN through aliases and return one row per alias. An entity with two aliases in a namespace therefore read as clean while `loaf task list` returned two entries — the exact scanner-versus-list disagreement the check exists to certify. It now compares raw rows against the count of resolving alias rows and reports multi_alias alongside orphan_delta, so both directions of divergence are visible. The scan was also wired into the invariants every Inspect runs, which is ~41 CLI paths including `loaf task list`, at four COUNT queries per project per table. Expensive whole-database diagnostics are opt-in now, and only `loaf state doctor` opts in. Alias-orphan preview output additionally names the unproven orphans the operator has to disposition and reports the orphan-referenced source rows per table; both previously lived in --json alone. Claude-Session: https://claude.ai/code/session_011wW8VzMJvENoeSY6QxWYkK
…mits Six corrections to the repair migration and the parity diagnostic, all of them cases where the classifier claimed more authority than it had. An explicit --realias now outranks automatic classification. A row the operator names for preservation was being retired anyway whenever a twin proof happened to reach it — silently, since the ID did match a classification, and with a manifest that recorded the realias and the retirement of the same row. Explicit dispositions are consulted before the proof branch, and the proof travels with the disposition for audit. Rerunning the exact apply command the ceremony records is a no-op again. Every disposition from the first run matched nothing on the second, and unmatched dispositions are a hard error. A retire whose row is gone and whose entity a previous rollback manifest records retiring, and a realias whose alias already names the entity, are satisfied rather than unmatched. A typo still aborts, and it still aborts before any mutation. The source-salt twin proof is labeled source-derivation, not derivation: it recomputes the orphan's own source row under a legacy salt but binds orphan to twin by title and source path, which is content identity. It also required uniqueness only among holders, so any number of orphans could collapse onto one holder under the strongest proof class. Uniqueness is now required on the orphan side too; a merge stays unproven. shaping_drafts joins the classified tables. The importer aliases them and the housekeeping scanner counts them, so they orphan by the same mechanism the other six do, and the count-agreement receipt has to cover them. A dangling alias is damage only when it is dead — the entity row missing and nothing left naming that entity. The importer registers the alias of a referenced-but-unimported artifact so a depends_on renders as TASK-000 rather than an opaque ID, and deletes the edge when the reference leaves the markdown; treating those as damage made import, repair, import loop without converging. Detector and repair share one predicate. Failures after the backup hand the result back instead of a zero value, so the backup and the rollback manifest reach the operator on the exact paths where they are needed. Claude-Session: https://claude.ai/code/session_011wW8VzMJvENoeSY6QxWYkK
A spark alias is the message's first word, so two unrelated journal sparks share one routinely. Refusing to reuse the aliased row closed the read path, but the write that followed still re-pointed the alias at the new spark: two rows, one alias, and the earlier spark orphaned on an ordinary first-time import — the exact damage this Change exists to end, reported by the new parity diagnostic as an error the repair cannot converge on. The later spark now takes the next free numbered alias instead of the claimed one, and treats an alias it already holds, or one whose spark is gone, as free. Identity resolution moves off the base alias and onto content — same text, same source file, exactly one candidate — so a spark carrying a disambiguated alias is still recognized after a rekey instead of being minted a second time. Claude-Session: https://claude.ai/code/session_011wW8VzMJvENoeSY6QxWYkK
Two ways a single apply of the alias-orphan repair could go wrong. Retiring an orphan deletes the relationship edges keeping a forward-declared alias alive, so an alias that was a live reference when its table was swept becomes dead moments later in the same transaction. The per-table sweep ran before that table's own retirements — and before every later table's — so it could never see them: the alias survived, the run's own post-apply verification then failed on a repair that had actually succeeded, and the error named the rollback manifest, inviting the operator to undo it. The sweep is now two whole-database passes, one before the retirements so a --realias target freed by this run is available, one after them. The second is the fixed point, because deleting an alias cannot kill another one. Preview reads that pass off its simulation, so the go/no-go number counts the aliases apply will delete. Legacy-salt recomputation proves an orphan was minted for an alias, not that the row now holding that alias is the orphan's duplicate. A reused alias number recomputes to exactly the orphan's ID, so a genuinely different artifact was retired on ID match alone. The proof now also requires the two rows to agree on their title and the orphan to predate its holder; anything short of that stays unproven and waits for an explicit operator disposition. Claude-Session: https://claude.ai/code/session_011wW8VzMJvENoeSY6QxWYkK
A journal file can carry the same spark line twice. Identity resolution matches on (text, source) and processing is sequential inside one import transaction, so the second line found the row the first line had just minted and upsertSpark wrote straight over it — one of the two intake items vanished on a first import. Sparks written earlier in the pass are now excluded from the candidate query. A rekey re-import is unaffected: the rows it has to find were written by an earlier run, not this one. Claude-Session: https://claude.ai/code/session_011wW8VzMJvENoeSY6QxWYkK
Classification gated content-identity and source-derivation on an orphan being the only candidate for its title or source key, and counted those candidates once over the whole table. Retiring an orphan shrinks the count, so a row that was unproven when the run started could be provable by the time the run finished — and the apply loop, having classified each table exactly once, stopped without it. Post-apply verification then re-planned and failed after the destructive transaction had already committed, on a repair that was correct. Against a copy of the production database a bare --apply exited 1, and the same apply run a second time exited 0. Classification now iterates to a fixed point: uniqueness is counted over the orphans not yet retiring, and each pass that promotes a row recomputes the counts over the smaller pool. Every pass strictly shrinks that pool, so it terminates. The loop lives in the per-table classifier that preview, apply, and verification all share, so a preview can no longer report a row as requiring operator disposition while apply quietly retires it. An operator --retire seeds the retiring set; --realias never does, so a row named for preservation cannot free a competitor's uniqueness. The twin proofs that auto-retire now carry the same gates as the derivation proof. Cluster membership is a timestamp window rather than a date prefix: the 2026-06-24 re-import wrote its rows inside one minute, and real work created later that same day was passing a date-prefix test as if it were a re-import twin. Two bodyless rows fingerprint identically as the empty string, which is not evidence, so a bodyless pair also requires the orphan to sit in the 2026-06-13 original-import window; a bodyful pair still requires equal fingerprints, and one body against none stays unproven. The source-salt proof gains the ordering guard — an orphan that postdates its candidate twin is not the pre-rekey original. The derivation proof carries a note that content equality must never be added to it: the production pairs it proves genuinely differ in body, because the artifacts changed between the two imports. Retiring a report sweeps the residue of the findings and verdicts that retire with it: their aliases, relationship edges, events, and tags outlived the subtree because the polymorphic sweep only reaches rows whose endpoint is the report itself. The rollback manifest is fsynced, file and parent directory both, before the destructive transaction commits, so a crash cannot leave committed deletions with no durable record. Creation moved to O_EXCL so the suffix search cannot race. The archive event is recorded for rollback only when the insert actually inserted it, so rolling back never deletes an event that was already there. Apply reports the source rows it stranded instead of a hard zero. Preview accepts --retire and --realias so the ceremony's exact invocation can be rehearsed before it runs; dispositions are recorded as typed, conflicting dispositions for one entity are refused rather than silently last-winning, and the rollback line in human output prints the manifest path the flag requires. Claude-Session: https://claude.ai/code/session_011wW8VzMJvENoeSY6QxWYkK
Journal entry IDs derived from the live project ID, and the upsert conflicts on id alone, so a rekey followed by a re-import inserted a second copy of every line. Import now resolves an existing entry by its natural identity — entry type, scope, and message — and reuses that row's ID, deriving one only for genuinely new entries. The occurred timestamp cannot participate: it is parsed from the line but never stored, since created_at records the import instant. Only rows a previous markdown import wrote are eligible, proven by a journal_origins row recording source_event = markdown_import, so a re-import can never absorb an entry written by `loaf journal log`. Repeated identical lines stay distinct: IDs consumed earlier in the pass are excluded from the lookup, so the second occurrence binds to the second existing row instead of collapsing onto the first. A spark whose message normalizes to an empty slug — punctuation only — was created with no alias at all, an alias-orphan born at import that a rekey then twinned. Such a spark now falls back to a slug derived from a hash of the message, and every imported spark gets an alias. The fallback hashes content alone: mixing in the project or source ID would shift the alias across the rekey this fix exists to survive. The spark resolver's comment overstated its guarantee, so it now records the trade it actually makes. A distinct spark line reuses its row across a rekey re-import. A verbatim-duplicated line forks once — two rows become four, because after the rekey the source salt matches neither and the resolver's exactly-one-candidate gate declines to guess — and then stays at four, all alias-reachable, without reintroducing alias-orphan damage. That gate is what keeps unrelated sparks sharing a first word from collapsing into one row. Claude-Session: https://claude.ai/code/session_011wW8VzMJvENoeSY6QxWYkK
The parity scan counted a table with five independent queries, so a concurrent write could yield a set of counts that never coexisted. The counts for one table now come from a single read transaction. When the counts still failed their internal identity check the scan returned an error, which propagated out of the invariant inspection and flipped doctor into invalid mode — the opposite of the contract that identity divergence is detectable, never invalidating. The inconsistency is now carried on the result, reported through the alias-parity-diverged diagnostic at error severity, and the mode stays ready. Multi-alias no longer gates readiness. It blocked Ready while the repair the diagnostic names, the alias-orphan migration, never removes a duplicate alias, so a database in that state had no route back. It becomes its own warning-severity diagnostic that names no repair command; the count and its per-table detail are unchanged. The no-writes proof, which hashes the database around the diagnostic, now covers the multi-alias fixture as well as the orphan and dangling one. Claude-Session: https://claude.ai/code/session_011wW8VzMJvENoeSY6QxWYkK
The June-24 identity fork also re-minted ~1,020 journal entries, invisible to the alias-orphan lens because journal rows carry no aliases. TASK-005 plans a sibling journal-duplicates migration on the same triad: window-gated natural-key pairing, June-13-copy retirement with reference sweep and FTS parity, refuse-by-default ambiguity handling. The ceremony packet gains the rehearsed-preview discipline, the realias-not-retire rule for the ten June-24-born spark collision victims, and the journal-duplicates step; the contract records the fixed-point and derivation-calibration decisions from the round-4 confirmation review. Claude-Session: https://claude.ai/code/session_011wW8VzMJvENoeSY6QxWYkK
`loaf state migrate journal-duplicates` joins alias-orphans on the same preview/backup/manifest/apply/rollback triad, repairing the unaliased half of the June-24 identity fork. Journal rows carry no aliases, so a pair is an identical (entry_type, scope, message) triple with exactly one row in the June-13 import window and one in the June-24 reimport window, reusing the window constants and inTimestampWindow already shared with alias-orphans. The June-24 row survives. A triple matching more than one row on either side classifies unproven and is refused unless the operator names it with --retire; --realias is rejected outright, since there is nothing to alias. Retirement sweeps every schema reference to journal_entries.id, enumerated in the migration's doc comment: journal_search, journal_origins, journal_deferrals, intent_operations, and journal_conversation_handles. The soft references that cannot be NULLed - journal_deferrals, intent_operations (whose CHECK ties projection_version=1 to a non-NULL journal_entry_id) and the conversation-handle association - repoint to the surviving twin, falling back to capture-and-delete when the target is already taken. journal_search stays consistent through targeted in-transaction deletes rather than a post-apply rebuild, which keeps apply and rollback symmetric. Apply takes a mandatory backup, records operator flags verbatim, and fsyncs the JSON rollback manifest and its parent directory before COMMIT. Rollback restores both the rows and the reference edges. Against a copy of the production database, 1,019 triples appear in both windows: 866 are unambiguous pairs and 153 are multi-candidate groups spanning 614 rows. Applying retires exactly 866 entries, a second apply is a no-op, journal-search parity holds, and rollback restores the original 8,701 rows. Claude-Session: https://claude.ai/code/session_011wW8VzMJvENoeSY6QxWYkK
…data The production-copy rehearsal showed the unproven set is 23 rows, not 13: 3 task orphans plus 20 sparks forming 10 both-orphan message pairs with no surviving twin. Each pair takes one realias and one retire at the operator's judgment. Claude-Session: https://claude.ai/code/session_011wW8VzMJvENoeSY6QxWYkK
…ries The journal-duplicates retirement swept only the reference sites named after journal_entry_id: journal_search, journal_origins, journal_deferrals, intent_operations, and journal_conversation_handles. But journal_entry is a registered entity kind, and the schema exposes polymorphic (entity_kind, entity_id) sites that can cite a journal row — intent conversion demonstrably writes relationships edges with from_entity_kind = 'journal_entry'. Retiring a June-13 twin stranded any such row as a dangling reference, and because none of them reached the rollback manifest, rollback could not restore them either. Retirement now sweeps events, entity_tags, bundle_members, backend_mappings, exports (source_entity_kind / source_entity_id), and relationships (from or to), plus artifact_bodies with its artifact_search mirror and aliases. The alias-orphan dead-alias walk only visits the seven non-journal entity tables, so a journal alias had nothing to collect it. These rows are captured into the rollback manifest and deleted rather than repointed at the surviving twin, matching the residue policy alias-orphans already applies; only the NOT NULL soft references keep their repoint-or-delete behaviour. The table enumeration lives in one place. polymorphicEntityReferenceSweeps returns the six sites as where-clause fragments, and both retireEntityWithResidueTx and the journal path consume it, so the two migrations cannot drift from each other or from the schema. retireEntityWithResidueTx keeps its capture-all-then-delete-all shape and manifest ordering. restoreArtifactSearchTx is factored out of restoreArtifactSearchForRowTx so both rollback paths rebuild the FTS mirror through the same code instead of converting between their manifest row types. TestJournalDuplicatePolymorphicResidueSweep covers all eight tables in both relationship directions: apply leaves zero rows citing the retired ID, every swept row appears in the manifest, rollback restores them byte-identically, the twin's own residue is untouched, the artifact_search body disappears and returns, and FTS parity and apply idempotency stay green. Rehearsed against a copy of the production database: 866 twins retired, journal search parity exact at 7848/7848, zero dangling references across all fourteen checked sites, integrity ok, second apply a no-op, and rollback restoring 1732 rows to the original 8714. Claude-Session: https://claude.ai/code/session_011wW8VzMJvENoeSY6QxWYkK
The production-copy rehearsals answered them: all 191 alias-orphans belong to this project (doctor clear across all 27 projects post-apply), and the journal twins decompose into 866 clean pairs plus 153 ambiguous groups spanning 614 rows that require ceremony dispositions. Claude-Session: https://claude.ai/code/session_011wW8VzMJvENoeSY6QxWYkK
artifact_search is an external-content FTS5 table over artifact_bodies, and deleteArtifactSearchTx removed an index entry with the FTS5 'delete' command using values read from artifact_bodies — the mirror it expects to be indexed, not the one that is. When the index entry was already missing, SQLite answered "database disk image is malformed" and the error propagated out of the journal-duplicates sweep, so a pre-existing desync aborted the whole migration. Probe the index for the rowid before deleting, and treat absence as an ordinary outcome. The probe is a rowid-scoped MATCH on body_kind, since a plain SELECT from an external-content table reads through to the content table and can never report index state. Healthy-path behaviour is unchanged: the same 'delete' command runs with the same values whenever the entry is present. The fix sits in the shared helper, so the alias-orphan sweep, spec delete, and the live upsert path shed the same defect. The FTS mirror stays derived data — rollback re-derives it from the restored body rather than capturing artifact_search bytes in the manifest, and the rollback site now says so. The residue test claimed byte-identical rollback while checking only manifest presence and one MATCH count, through a helper that collapsed NULL to "". It now snapshots full artifact_bodies rows with source_id as *string, and captures artifact_search membership via MATCH, diffing both after rollback; restored rows draw fresh rowids, so membership is compared by logical columns. A new test removes the mirror entry before apply and asserts that apply succeeds and that rollback re-derives a readable index. Reverting the probe fails that test with the malformation error; disabling re-derivation fails the fidelity diff. Claude-Session: https://claude.ai/code/session_011wW8VzMJvENoeSY6QxWYkK
All five verification-contract criteria pass at 6ddf695; the receipt binds the criteria and scope digests to that tree. Claude-Session: https://claude.ai/code/session_011wW8VzMJvENoeSY6QxWYkK
…udit Both PR-gate reviews found only documentation drift: stale six-table language corrected to seven, the Hypothesis qualified with the documented alias-safe spark rekey exception, the twin-proof description expanded to match the implemented gate stack (historical-path derivation, windowed content identity, the source-derivation proof), and the importer packet gains a delivered-variances record. Claude-Session: https://claude.ai/code/session_011wW8VzMJvENoeSY6QxWYkK
Merged
levifig
added a commit
that referenced
this pull request
Aug 10, 2026
## [0.2.21] - 2026-08-10 ### Added - `loaf state migrate alias-orphans` retires entity rows orphaned from the alias registry — the damage left when a project rekey invalidated every derived ID and a later import re-minted the tree. Classification proves twin-ship (legacy-salt derivation, or window-gated content identity) and iterates to a fixed point; unproven rows refuse by default and take explicit `--retire`/`--realias` dispositions that rehearse in preview; apply takes a backup first and writes an fsynced rollback manifest that restores every deleted row ([#159](#159)). - `loaf state migrate journal-duplicates` is the sibling repair for journal entries, which carry no aliases: identical `(type, scope, message)` twins across the two import windows retire their older copy, ambiguous groups refuse, and the reference sweep shares one polymorphic-table enumeration with alias-orphans so the two repairs cannot drift ([#159](#159)). - `loaf state doctor` now checks alias parity: for every project and aliased entity table, raw row counts must equal alias-reachable counts with zero dead aliases. Divergence reports as an error diagnostic naming the repair command — identity damage is detectable the day it happens instead of discoverable by accident at housekeeping ([#159](#159), ADR-028). - `loaf hooks list`, `loaf hooks enable`, and `loaf hooks disable` operate individual hooks from the installed catalog. Enablement lives in the global database, user-scoped and host-local — absence means enabled, disable records tombstone rather than delete — and every verb reprojects the target's hook file through the full reconciler under a per-target lock ([#161](#161)). ### Changed - Codex and Cursor hook files are no longer whole-file managed artifacts guarded by a digest. Install and upgrade now converge Loaf-owned entries per hook point — adding missing ones, updating changed ones, removing retired ones, and absorbing an operator-deleted entry once as a disabled record — while every non-Loaf entry survives value-identical and order-stable. Drift refusals are gone as a class, including the refusal on user-modified `hooks.json` files frozen since the pre-reset releases; integrity violations still fail closed and preserve the file ([#161](#161)). - Markdown import resolves identity through the alias registry before deriving an ID — existing entities, sources, and journal rows are reused instead of re-minted, colliding sparks receive numbered aliases, and a spark whose message normalizes to an empty slug gets a content-hash alias so no row is born unreachable. Derived IDs are now mint-once opaque keys (ADR-028); for healthy databases a re-import is byte-stable ([#159](#159)). ### Fixed - A project rekey followed by a markdown re-import could silently fork the identity space — every artifact re-inserted as an invisible twin, aliases re-pointed to the copies, and the list surfaces disagreeing with the housekeeping scanner about how much work exists. The importer fix removes the cause; the migrations repair existing damage; doctor parity detects any recurrence ([#159](#159)). Claude-Session: https://claude.ai/code/session_011wW8VzMJvENoeSY6QxWYkK
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Change
docs/changes/20260807-state-dedupe/
What & Why
The 2026-06-24 markdown import ran after migration 3 had rekeyed the project ID, and because entity primary keys were derived from
sha256(kind, project_id, alias), every artifact was re-inserted as a twin and the alias upsert re-pointed each alias to the new copy. The June-13 originals became alias-orphans — invisible to every list command, visible to the housekeeping scanner — and the same fork silently duplicated ~1,020 journal entries, which carry no aliases at all. Scanner and list surfaces disagreed about how much work exists (299/38/14 vs 233/26/11 for tasks/specs/reports),loaf task list --status done --jsonreturned zero rows while 66 done tasks existed, and one report row cited evidence with no git history.This PR ships the complete repair and prevention:
loaf state migrate alias-orphans— audited preview/backup/manifest/apply/verify/rollback migration across all seven aliased entity tables in every project. Twin-ship is proven (legacy-salt ID recomputation, or window-gated content identity), never guessed; unproven rows refuse by default and take explicit--retire/--realiasdispositions, rehearsable in preview. Classification iterates to a fixed point so a retirement that unlocks another proof cannot fail post-apply verification. The broken-evidence report archives as moot with a recorded rationale.loaf state migrate journal-duplicates— sibling migration for the unaliased half: window-gated natural-key pairing retires the June-13 copies of 866 clean twin pairs; 153 ambiguous groups refuse and await dispositions. A shared polymorphic-reference sweep enumeration keeps both migrations aligned with the schema and each other.loaf state doctoralias-parity diagnostic — read-only per-project, per-table raw-vs-reachable counts and dead-alias detection, so identity divergence is detected the day it happens instead of discovered at housekeeping. Damage reports as an error diagnostic without condemning the database.Review focus
internal/state/alias_orphan_migration.go): derivation is title + recency by calibrated design — all 132 production derivation-proven pairs have mismatched body fingerprints, so a fingerprint guard would break the repair (comment at the proof site).polymorphicEntityReferenceSweepsenumeration consumed by both migrations.RepairJournalSearch).Verification
loaf change verify docs/changes/20260807-state-dedupe— V1–V5 all pass; receipt committed (criteria digest9947e964…, scope digestdaa9ae95…).loaf change check— no violations, executable.go test ./...,npm run typecheck,gofmt -l internal cmd— green/clean.Migration / breaking changes
Two behavior changes ship immediately on normal CLI paths; the destructive repair does not.
loaf state migrate markdown --apply) now resolves identity before deriving IDs: existing entities, sources, and journal rows are reused instead of re-minted, colliding sparks receive numbered aliases, and empty-slug sparks get content-hash aliases. For healthy databases the import is byte-stable; the change is visible only where the old behavior would have created duplicates.loaf state doctornow runs the alias-parity scan (read-only) and exits non-zero when it finds identity damage — a database with pre-existing orphans reports an error diagnostic naming the repair command instead of passing silently.alias-orphans,journal-duplicates) run only when invoked deliberately. The production repair is a separate operator ceremony (TASK-004 in the Change folder): backup, rehearsed dispositions, both applies, first run of the existing lifecycle-statuses migration, doctor-green verification, and count-agreement receipts.Deferred
artifact_searchhas no rebuild path analogous toRepairJournalSearch(stale-mirror residue) — captured as a spark for triage.specTaskCountsjoin asymmetry and raw status literals — truthful post-dedupe; revisit only if doctor parity ever shows drift.https://claude.ai/code/session_011wW8VzMJvENoeSY6QxWYkK