-
Notifications
You must be signed in to change notification settings - Fork 0
[Epic] E4 — Confidence + provenance instrumentation across LML, Backend, semantic-index #664
Copy link
Copy link
Closed
1 / 21 of 2 issues completedClosed
1 / 21 of 2 issues completed
Copy link
Labels
concern:observabilitySentry, breadcrumbs, audit reports, time-series viewsSentry, breadcrumbs, audit reports, time-series viewscross-cache-identityProject tag for the cross-cache-identity initiative (library hook + identity record + normalization)Project tag for the cross-cache-identity initiative (library hook + identity record + normalization)enhancementNew feature or requestNew feature or requestepic:e4-instrumentationParent epic E4 — confidence + provenance instrumentationParent epic E4 — confidence + provenance instrumentationkind:epicParent epic issueParent epic issuephase:1Mojibake phase 1 — fix tubafrenzyMojibake phase 1 — fix tubafrenzy
Metadata
Metadata
Assignees
Labels
concern:observabilitySentry, breadcrumbs, audit reports, time-series viewsSentry, breadcrumbs, audit reports, time-series viewscross-cache-identityProject tag for the cross-cache-identity initiative (library hook + identity record + normalization)Project tag for the cross-cache-identity initiative (library hook + identity record + normalization)enhancementNew feature or requestNew feature or requestepic:e4-instrumentationParent epic E4 — confidence + provenance instrumentationParent epic E4 — confidence + provenance instrumentationkind:epicParent epic issueParent epic issuephase:1Mojibake phase 1 — fix tubafrenzyMojibake phase 1 — fix tubafrenzy
Type
Fields
Give feedbackNo fields configured for issues without a type.
Projects
StatusShow more project fields
Done
Summary
Add confidence and provenance instrumentation everywhere identity is computed or stored: §3.4 columns on the new
library_identity+library_identity_sourcetables (mandatoryconfidence+methodfrom day one, retroactively backfilled per §3.4.1's matrix); LML returns confidence per the §3.2.2 write contract; semantic-index'sreconciliation_log(53,849 rows, allconfidencecurrently NULL) gets backfilled.Today, identity decisions vanish without an audit trail. No
confidenceis stored onreconciliation_log. LML's matcher emits a confidence internally but doesn't always persist it. Reruns silently overwrite without supersedure tracking. This epic plugs every leak.Scope
Backend-Service
confidence+methodonlibrary_identity(NOT NULL with CHECK).library_identity_historypopulated on every supersedure withsuperseded_reason(§3.2.0 retention policy).>=0.85→ write authoritatively,0.70-0.85→ write but flag,<0.70→ history-only (don't promote to live).library-metadata-lookup
/lookupresponse carries per-sourceconfidence+method(per §3.2.2).entity.identity.reconciliation_logpopulated on every reconciliation (53,849 existing NULL-confidence rows backfilled where evidence permits).semantic-index
reconciliation_log.confidenceno longer permitted NULL on new writes.method='inherited'with the source row's confidence.Dependencies
Existing issues folded in
WXYC/library-metadata-lookup#213— Project cache_stats onto Sentry transaction for E2E tracing → child. Already shipping; provides observability for confidence telemetry.WXYC/Backend-Service#646— WraplookupMetadatain a Sentry span and project LML cache_stats onto it → child. Same wrap-at-chokepoint + project-onto-span pattern.Phase
Phase 1+: instrumentation lands incrementally as each surface (LML response, BS writer, semantic-index reconciler) gains its column or refactor.
Acceptance
confidencerows on any new identity write.cache_statsprojected onto LML span; reachable from thelibrary-metadata-lookupproject's trace explorer.apps/backend/tests/integration/library-identity-composition.spec.jscovering all five Rules + 7 worked examples).Plan reference
plans/library-hook-canonicalization.md§3.4 (full subsection), §3.4.1 (confidence matrix), §3.4.1.1 (composition rules), §3.2.2 (LML write contract).