[§4 step 2 post-pivot] Consume LML POST /api/v1/identity/bulk-resolve-libraries

[jakebromberg]

Goal

Replace the retired source-leg backfill job (jobs/library-identity-backfill/) with a thin consumer of LML's new POST /api/v1/identity/bulk-resolve-libraries endpoint. Per the architecture pivot (#800), Backend stops reading LML's discogs-cache PG directly; it caches LML's verdict via the HTTP contract.

Scope

Build a one-shot ETL job (in the same jobs/ shape as the existing artist-identity-etl) that:

1. Selects libraries needing identity refresh: WHERE library.canonical_entity_id IS NOT NULL OR library.id IN (SELECT library_id FROM library_identity WHERE last_refreshed_at < NOW() - interval '7 days') — incremental, bounded.
2. POSTs them to LML in batches of 500 with (artist_name, album_title) denormalized.
3. UPSERTs the response into library_identity (one row per library_id) and library_identity_source (one row per provenance entry).
4. Emits Sentry-traced metrics: rows_resolved / rows_unresolved / rows_skipped / lml_latency.

What we keep from the retired job

• The library_identity + library_identity_source schemas (Backend's cache of LML's verdict).
• The artists table mirror shape (for in-process reads from the request hot path).
• The DRY_RUN env var pattern (locked JSON output for stage/prod parity).

What we remove

• jobs/library-identity-backfill/ and its source-leg readers.
• The BACKFILL_LEG dispatcher.
• All cross-DB connection setup (DATABASE_URL_DISCOGS).

Acceptance criteria

• New job in jobs/library-identity-consumer/ (name TBD) wired into Manual Build & Deploy.
• Consumes POST /api/v1/identity/bulk-resolve-libraries per the v0.7 contract.
• Idempotent: rerunning is safe; UPSERT prevents dup rows.
• DRY_RUN locked JSON schema; integration test covers happy path + retry path + unresolved row.
• Old jobs/library-identity-backfill/ deleted in the same PR (no tombstone scripts).
• CLAUDE.md + docs/env-vars.md updated.

Sequencing

Blocked by:

• api.yaml v0.7 — POST /api/v1/identity/bulk-resolve-libraries schema wxyc-shared#103 — endpoint schema (api.yaml v0.7 lands first; generated types regenerate)
• Implement POST /api/v1/identity/bulk-resolve-libraries (cross-cache-identity pivot) library-metadata-lookup#272 — endpoint deployed to staging

Cannot ship until both are in place.

Related

• plans: cross-cache-identity architecture pivot decision record #800 — architecture pivot
• [Epic] E2 — Canonical identity record (library_identity) + LML write contract #663 — cross-cache-identity epic
• feat(jobs): library-identity-backfill skeleton + S1 source (§4 step 2 sub-PR 2.0) #790 — sub-PR 2.0 (S1 substrate, kept on disk as reference; this job removes it)
• plans: §4 sub-PR 2.2 spike memo (discogs-cache match score shape) #794 — spike memo (reference for LML's resolution layer)