catch 503 error on initial scrape parsing

[HongPong]

I ran into this at scraping page 1/7

Error parsing https://www.ice.gov/detention-facilities?page=0&exposed_form_display=1: 503 Server Error: Service Unavailable for url: https://www.ice.gov/detention-facilities?page=0&exposed_form_display=1

Confirmed with network inspector that it was a 503 response. not sure what the best way to catch this error is but i suspect it will happen again so we may wish to do something. this was just after starting

 uv run python main.py --scrape

on head.

[johnseekins]

We actually already have retries built in to the session object (which all requests should be using):

https://github.com/Open-Security-Mapping-Project/ice_detention_scraper/blob/main/utils.py#L14

We definitely don't want infinite retries (as that will appear like a hung run from a user's perspective). And at some point, we do need to know that upstream services are failing. 4 retries felt like enough, but exactly how we handle retry logic is obviously open for debate.

[HongPong]

I figure just reporting something on the status line and quitting if it is a show stopper is probably fine otherwise saying it will proceed. It seemed like it was just proceeding but some kind of If statement for that response code would be a little more robust. Hopefully not too small of a nit but i wanted to make sure it wasn't lost in the shuffle.

…

On Fri, Dec 5, 2025, 10:14 AM John Seekins ***@***.***> wrote: *johnseekins* left a comment (Open-Security-Mapping-Project/ice_detention_scraper#84) <#84 (comment)> We actually already have retries built in to the session object (which all requests *should* be using): https://github.com/Open-Security-Mapping-Project/ice_detention_scraper/blob/main/utils.py#L14 We definitely don't want infinite retries (as that will appear like a hung run from a user's perspective). And at some point, we do need to know that upstream services are failing. 4 retries *felt* like enough, but exactly *how* we handle retry logic is obviously open for debate. — Reply to this email directly, view it on GitHub <#84 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAA322Y2EFOVWBOK6PMWZYL4AGOORAVCNFSM6AAAAACODOL632VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTMMJXGMZDEMRSGI> . You are receiving this because you authored the thread.Message ID: <Open-Security-Mapping-Project/ice_detention_scraper/issues/84/3617322222@ github.com>

[johnseekins]

Part of the inspection PR actually added an additional wrapper that might slightly address this concern.