LCE (Leicester City Council) scraper failing — TCP connection timeout, IP-level block on Lambda

[symroe]

Error (2026-06-30)

is_connect error: wreq::Error { kind: Request, uri: https://cabinet.leicester.gov.uk//mgWebService.asmx/GetCouncillorsByWard, source: Error { kind: Connect, source: Some(ConnectError("tcp connect error", 212.50.184.183:443, Os { code: 110, kind: TimedOut, message: "Connection timed out" })), connect_info: None } }

Consecutive failures: 10 days (June 19–28).

Investigation

From our proxy environment (non-Lambda IP):

• https://cabinet.leicester.gov.uk/mgWebService.asmx/GetCouncillorsByWard → 200 OK with valid XML containing 61 councillors across 21 wards — server is working correctly and serving data

From Lambda (direct, no proxy):

• TCP connect timeout to 212.50.184.183:443 — SYN packets are not being acknowledged

The server IS working (our proxy gets a clean 200 OK with councillor data), but Lambda's IP range (AWS, likely 3.x.x.x or 18.x.x.x) appears to be blocked at the TCP firewall level. cabinet.leicester.gov.uk resolves to 212.50.184.183.

Fix patterns ruled out

1. HTTPS migration — already HTTPS
2. verify_requests = False — TCP connect fails before TLS; cert verification is not the issue
3. http_lib = "playwright" — Playwright runs inside Lambda with the same blocked IP; a TCP firewall block cannot be bypassed by changing the HTTP client
4. URL changes — DNS resolves fine; the URL hasn't moved; the server actively serves valid data to non-Lambda IPs
5. Proxy — use_proxy is not set; a proxy infrastructure solution would be needed

What needs to happen

This is an infrastructure fix, not a scraper code change. The same pattern as SGC (issue #404):

• Either the Lambda IP range needs to be allowlisted with Leicester's hosting provider / firewall admin
• Or the scraper needs to route through a working egress proxy that has a non-blocked IP
• Or the council could be contacted to allowlist the Lambda IP range

Similar open issues

• SGC (South Gloucestershire) scraper failing — TCP connection timeout, IP-level block on Lambda #404 — SGC (South Gloucestershire), same TCP connect timeout / IP-level firewall block pattern