Backup browser-driven address resolver (free USPS standby)#3
Merged
Conversation
Add a standby address-standardization path for when apis.usps.com is unavailable (e.g. after USPS's 2026-07-12 move of the Addresses API to a paid, licensed tier). mailwatch.web_lookup drives USPS's free consumer ZIP-lookup (tools.usps.com) with nodriver + Chromium, headful under xvfb. That endpoint is behind Akamai Bot Manager, which requires a live JS-sensor browser session: plain HTTP clients — including TLS-fingerprint- impersonating ones carrying a browser-minted _abck cookie — are blocked on the data XHR. mailwatch.cli_resolve seeds the address_cache keyed identically to what /generate looks up, so the service and its request path are unchanged: they just get a cache hit and skip the API. The browser stays out of the hardened gunicorn workers entirely. - web_lookup: pure parse_address_list (unit-tested off a captured payload) plus a lazy-imported nodriver driver behind a thin seam (engine swappable) - cli_resolve: seeds address_cache; cache-key alignment unit-tested - nodriver in an optional `browser` dependency group, not main deps - apps.resolve-address (Linux): Chromium + xvfb-run wired; not in the service - integration test is opt-in (-m integration) Verified end-to-end on aarch64 NixOS against live USPS from a datacenter IP: full ZIP+4 + delivery point returned.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Address standardization normally goes through the official
apis.usps.comAddresses API (NewApiClient). USPS is moving that API to a paid, licensed tier on 2026-07-12 (the legacy free Web Tools API was already retired 2026-01-25). This adds a $0 standby so/generatekeeps producing full ZIP+4 + 11-digit-routing barcodes when the API is unavailable.How
USPS's free consumer ZIP-lookup (
tools.usps.com/.../ziplookup/zipByAddress) returns everything we need (zip5/zip4/deliveryPoint/dpvConfirmation) but sits behind Akamai Bot Manager. Empirically, a live browser running Akamai's JS sensor in-request is mandatory — vanilla curl,curl_cffiwith exactfirefox135TLS impersonation, and evencurl_cfficarrying a browser-minted_abckcookie + refreshedbm_*tokens are all blocked on the data XHR (the static page passes; the API XHR does not). So the resolver usesnodriver+ Chromium, headful underxvfb.It's a standby, not a replacement:
mailwatch/web_lookup.py— pureparse_address_list()(USPS web JSON → existing models) behind a lazy-importednodriverdriver. Engine isolated to one function; swappable (e.g. Camoufox) without touching callers.mailwatch/cli_resolve.py— seedsaddress_cachekeyed identically to what/generatecomputes (db.hash_address_dictof the rawAddressRequest), so the service and its request path are unchanged — they just get a cache hit and skip the API. The browser never runs in the gunicorn workers.nodriverlives in an optionalbrowserdependency group (not main deps).apps.resolve-address(Linux-only): wirespkgs.chromium+xvfb-run; deliberately not added to the NixOS service.Usage
Verification
nix flake check -Lgreen: lint, mypy (20 files), 276 tests, coverage 95.9%, nixfmt/statix/deadnix.nodriverbuilds under uv2nix.StandardizedAddressResponse.model_validate_json), and cache-key alignment with the service's construction.nix run .#test -- -m integration).1600 PENNSYLVANIA AVE NW | 20500-0005 | dp=00 | dpv=Y.Caveat
Automates a free public tool — a gray area in USPS's terms of use. It's a low-volume, cached fallback; keep volume low.