Fix FEN (Fenland) scraper

[symroe]

What broke

Fenland's ModGov endpoint (www.fenland.gov.uk/localgov/) is protected by Cloudflare bot detection. When accessed from Lambda using wreq's Firefox133 TLS fingerprint, the server returns HTTP 403 with a "Just a moment..." JS challenge page. The previous verify_requests = False fix addressed a certificate issue but does not bypass the Cloudflare JS challenge, so the scraper continues to time out in Lambda.

What was fixed

• Replaced verify_requests = False with http_lib = "playwright" — headless Chromium executes the Cloudflare JS challenge and acquires the cf_clearance cookie before fetching the ModGov XML endpoint.

Scrape results

Live endpoint returns a Cloudflare JS challenge from non-Chromium clients; counts will be confirmed after Lambda runs with Playwright. The endpoint serves valid ModGov XML to browser clients (confirmed via curl through proxy).

Metric	Count
Councillors found	TBC (Lambda run needed)
With email address	TBC
With photo	TBC

---

Generated by Claude Code

[symroe]

Re-scrape after e81a97a

Playwright fix applied: replaced verify_requests = False with http_lib = "playwright".

The endpoint (www.fenland.gov.uk/localgov/) returns a Cloudflare JS challenge (HTTP 403 "Just a moment...") to non-browser TLS fingerprints. Playwright's headless Chromium executes the challenge and acquires cf_clearance before fetching the ModGov XML.

Metric	Count
Councillors found	TBC — requires Lambda run with Playwright
With email address	TBC
With photo	TBC

Counts cannot be confirmed locally: the Cloudflare JS challenge cannot be bypassed without a real browser, and the Lambda environment is needed to verify end-to-end scrape results.

---

Generated by Claude Code