Skip to content

Make IV-MTR letter tracking work end-to-end#2

Merged
ak2k merged 1 commit into
mainfrom
fix/iv-mtr-tracking-pipeline
Jun 1, 2026
Merged

Make IV-MTR letter tracking work end-to-end#2
ak2k merged 1 commit into
mainfrom
fix/iv-mtr-tracking-pipeline

Conversation

@ak2k

@ak2k ak2k commented Jun 1, 2026

Copy link
Copy Markdown
Owner

Summary

Tracking never surfaced a single scan event. Diagnosing a real mailed letter against the live IV-MTR API turned up a chain of independent defects, each of which alone keeps /tracking empty. This fixes all of them.

Verified against a live piece: the full 31-digit IMb returns 5 real scans from USPS, where the existing code raised, mis-keyed, or never queried.

The defects (each independently fatal)

Pull response models didn't match the live API. IV-MTR returns snake_case keys (scan_date_time, scan_event_code, scan_facility_*), carries no per-scan imb, and reports a miss as {"message": "Barcode not found.", "data": null}. The models required camelCase fields and a per-scan imb, so every populated response failed validation (15 errors) and was swallowed by the poll/track-ws except handlers. Field aliases + populate_by_name parse the real schema while staying backward compatible; message maps to error; expected_delivery_date is surfaced.

Serials could collide → duplicate IMbs. The serial was a per-UTC-day counter that reset daily, and the IMb encoded no date, so serial 1 on two different days produced identical barcodes — and the counter row was purged after 48h. Replaced with a single global monotonic counter (serial_state) that is never reset or purged, seeded above any serial issued under the old scheme so a fresh counter can't collide with a piece already in the mail stream.

/track-ws couldn't build the IMb IV-MTR keys on. IV-MTR keys on the full 31-digit IMb (20-digit tracking + the exact 11-digit routing, including the 2-digit delivery point). The route rebuilt routing from a typed ZIP, which the form caps at 9 digits and a human can't reproduce. A tracked_imbs registry written at /generate stores the full queryable IMb; /track-ws recovers it by serial, falling back to the reconstructed key for pre-registry letters.

The poller couldn't discover a new letter. get_in_flight_imbs only re-polled IMbs that already had a scan, so a freshly mailed letter was never fetched. get_pollable_imbs unions that scan-driven set with registered-but-unscanned IMbs, bounded by a max-age window and dropped on delivery.

Token renewal crashed. USPS omits refresh_token on renewal; persisting the null into the NOT NULL app_state.value raised IntegrityError and aborted every renewal. The refresh token is now only overwritten when the response carries one.

Notes

  • The IV-MTR push-feed model (PushFeedEvent) is left unverified and flagged in a docstring — there's no captured real webhook payload yet, and live pull data suggests handling_event_type uses values like "A" rather than the "L" the handler filters on. Worth re-checking against the first real push.

Test plan

  • 268 tests pass; nix flake check green.
  • New coverage: live snake_case parsing + "Barcode not found" mapping; null-refresh renewal; global-monotonic serial + floor; registry register/lookup; poll bootstrap (registered-but-unscanned) + delivery/max-age exclusion; /generate registration and serial-keyed /track-ws lookup overriding a wrong ZIP.

Tracking never surfaced a single scan. Diagnosing a real mailed letter
against the live IV-MTR API turned up a chain of independent defects, each
of which alone is enough to keep /tracking empty. This fixes all of them.

Verified against a live piece: the 31-digit IMb returns 5 scans from USPS,
where the old code raised, mis-keyed, or never queried.

Models (the deepest bug): the IV-MTR pull response is snake_case
(scan_date_time, scan_event_code, scan_facility_*), carries no per-scan
imb, and wraps misses as {"message": "Barcode not found.", "data": null}.
The models required camelCase + a per-scan imb, so every populated response
failed validation (15 errors) and was swallowed by the poll/track-ws
except handlers. Add field aliases + populate_by_name so the models parse
the real schema while staying backward compatible; map message -> error;
surface expected_delivery_date. The push-feed model is flagged unverified
(no real webhook sample yet).

IMb identity (correctness): serials were a per-UTC-day counter that reset
each day, and the IMb encoded no date — so serial 1 on two different days
produced identical barcodes, and the counter row was purged after 48h.
Replace it with a single global monotonic serial (serial_state) that is
never reset or purged, seeded above any serial issued under the old scheme
so a fresh counter can't collide with a piece already in the mail stream.

Tracking lookup: IV-MTR keys on the full 31-digit IMb (20-digit tracking +
the exact 11-digit routing incl. 2-digit delivery point). /track-ws rebuilt
the routing from a typed ZIP, which the form caps at 9 digits and a human
can't reproduce. Add a tracked_imbs registry written at /generate that
stores the full queryable IMb; /track-ws now recovers it by serial (falling
back to the reconstructed key for pre-registry letters).

Poll bootstrap: get_in_flight_imbs only re-polled IMbs that already had a
scan, so a freshly mailed letter was never discovered. get_pollable_imbs
unions that scan-driven set with registered-but-unscanned IMbs (bounded by
a max-age window and dropped on delivery).

Token refresh: USPS omits refresh_token on renewal; persisting the null
into the NOT NULL app_state.value raised IntegrityError and aborted every
renewal. Only overwrite the refresh token when the response carries one.
@ak2k ak2k merged commit 91ee856 into main Jun 1, 2026
4 checks passed
@ak2k ak2k deleted the fix/iv-mtr-tracking-pipeline branch June 1, 2026 19:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant