Make IV-MTR letter tracking work end-to-end#2
Merged
Conversation
Tracking never surfaced a single scan. Diagnosing a real mailed letter
against the live IV-MTR API turned up a chain of independent defects, each
of which alone is enough to keep /tracking empty. This fixes all of them.
Verified against a live piece: the 31-digit IMb returns 5 scans from USPS,
where the old code raised, mis-keyed, or never queried.
Models (the deepest bug): the IV-MTR pull response is snake_case
(scan_date_time, scan_event_code, scan_facility_*), carries no per-scan
imb, and wraps misses as {"message": "Barcode not found.", "data": null}.
The models required camelCase + a per-scan imb, so every populated response
failed validation (15 errors) and was swallowed by the poll/track-ws
except handlers. Add field aliases + populate_by_name so the models parse
the real schema while staying backward compatible; map message -> error;
surface expected_delivery_date. The push-feed model is flagged unverified
(no real webhook sample yet).
IMb identity (correctness): serials were a per-UTC-day counter that reset
each day, and the IMb encoded no date — so serial 1 on two different days
produced identical barcodes, and the counter row was purged after 48h.
Replace it with a single global monotonic serial (serial_state) that is
never reset or purged, seeded above any serial issued under the old scheme
so a fresh counter can't collide with a piece already in the mail stream.
Tracking lookup: IV-MTR keys on the full 31-digit IMb (20-digit tracking +
the exact 11-digit routing incl. 2-digit delivery point). /track-ws rebuilt
the routing from a typed ZIP, which the form caps at 9 digits and a human
can't reproduce. Add a tracked_imbs registry written at /generate that
stores the full queryable IMb; /track-ws now recovers it by serial (falling
back to the reconstructed key for pre-registry letters).
Poll bootstrap: get_in_flight_imbs only re-polled IMbs that already had a
scan, so a freshly mailed letter was never discovered. get_pollable_imbs
unions that scan-driven set with registered-but-unscanned IMbs (bounded by
a max-age window and dropped on delivery).
Token refresh: USPS omits refresh_token on renewal; persisting the null
into the NOT NULL app_state.value raised IntegrityError and aborted every
renewal. Only overwrite the refresh token when the response carries one.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Tracking never surfaced a single scan event. Diagnosing a real mailed letter against the live IV-MTR API turned up a chain of independent defects, each of which alone keeps
/trackingempty. This fixes all of them.Verified against a live piece: the full 31-digit IMb returns 5 real scans from USPS, where the existing code raised, mis-keyed, or never queried.
The defects (each independently fatal)
Pull response models didn't match the live API. IV-MTR returns snake_case keys (
scan_date_time,scan_event_code,scan_facility_*), carries no per-scanimb, and reports a miss as{"message": "Barcode not found.", "data": null}. The models required camelCase fields and a per-scanimb, so every populated response failed validation (15 errors) and was swallowed by the poll/track-wsexcepthandlers. Field aliases +populate_by_nameparse the real schema while staying backward compatible;messagemaps toerror;expected_delivery_dateis surfaced.Serials could collide → duplicate IMbs. The serial was a per-UTC-day counter that reset daily, and the IMb encoded no date, so serial 1 on two different days produced identical barcodes — and the counter row was purged after 48h. Replaced with a single global monotonic counter (
serial_state) that is never reset or purged, seeded above any serial issued under the old scheme so a fresh counter can't collide with a piece already in the mail stream./track-wscouldn't build the IMb IV-MTR keys on. IV-MTR keys on the full 31-digit IMb (20-digit tracking + the exact 11-digit routing, including the 2-digit delivery point). The route rebuilt routing from a typed ZIP, which the form caps at 9 digits and a human can't reproduce. Atracked_imbsregistry written at/generatestores the full queryable IMb;/track-wsrecovers it by serial, falling back to the reconstructed key for pre-registry letters.The poller couldn't discover a new letter.
get_in_flight_imbsonly re-polled IMbs that already had a scan, so a freshly mailed letter was never fetched.get_pollable_imbsunions that scan-driven set with registered-but-unscanned IMbs, bounded by a max-age window and dropped on delivery.Token renewal crashed. USPS omits
refresh_tokenon renewal; persisting the null into theNOT NULLapp_state.valueraisedIntegrityErrorand aborted every renewal. The refresh token is now only overwritten when the response carries one.Notes
PushFeedEvent) is left unverified and flagged in a docstring — there's no captured real webhook payload yet, and live pull data suggestshandling_event_typeuses values like"A"rather than the"L"the handler filters on. Worth re-checking against the first real push.Test plan
nix flake checkgreen./generateregistration and serial-keyed/track-wslookup overriding a wrong ZIP.