fix(stats): vlr.gg page revamp broke /stats — header-keyed data-col parsing, new region vocabulary, session priming

[aLaaah]

vlr.gg revamped its /stats page and it broke the scraper in four independent ways. Each defect below was reproduced against live vlr.gg and, where relevant, the Wayback archive of the old layout. The fix keys the parser off the semantic data-col attributes vlr.gg now emits, so the whole class of "vlr.gg added a column" breakage becomes impossible going forward.

The four defects (all reproduced)

(a) The table went 21 → 23 columns — every positional index after agents reads the wrong stat.
vlr.gg inserted a maps column at td[2] and a fk:fd column at td[11]. _parse_stats_row reads hardcoded positional cells, so fields 2–10 shift by one and 11+ shift by two. Concretely, rounds_played ends up reading a Maps count (e.g. "14") and rating reads an ACS (e.g. "280"). Shifting today's payload back restores sanity (rounds≈280, rating≈1.40, acs≈210).
Archive proof of the old 21-column layout: https://web.archive.org/web/20250915134359id_/https://www.vlr.gg/stats/ — there td[2]=Rnd, td[3]=R2.0, td[4]=ACS, and .stats-player-country is present.

(b) The org selector no longer exists.
.stats-player-country returns zero matches on the current markup — it is now .st-pl-country. The if not org: org = "N/A" default therefore fires for 100% of rows.

(c) The /stats region vocabulary changed.
<select name="region"> now offers all | americas | emea | pacific | china | intl. The old na | eu | ap | … keys are no longer recognized and silently fall back to all rather than erroring — so a caller passing region=na unknowingly gets the global list. (Note: /rankings is untouched and still uses the old na|eu|ap|… keys — this taxonomy change is scoped to the /stats page only.)

(d) The finding I haven't seen reported anywhere: /stats binds its filter to a PHPSESSID session.
The first request on a cold client returns the unfiltered global list regardless of region= — the filter only takes effect once vlr.gg has set a session cookie. Reproduced deterministically with a fresh cookie jar:

FRESH cookie jar, same region= URL requested twice:
  request #1 (no PHPSESSID)  -> 100 rows, global default   <- region= ignored
  request #2 (PHPSESSID set) ->  15 rows, correct americas  <- filter honoured

This is quietly dangerous in a long-running service: with a singleton httpx.AsyncClient (cookies persist for the process lifetime) plus CACHE_TTL_STATS = 1800, the first region requested after any restart receives the global list, and that wrong response is cached for 30 minutes under that region's key. One priming GET to https://www.vlr.gg/stats/ before the first fetch fixes it deterministically, and the page conveniently echoes the applied region as the selected option of select[name=region], which gives a reliable response-level correctness signal.

The fix

• Header-keyed column map (fixes a permanently). Build a {data-col → index} map from <thead> once per parse and read cells by key, never by literal index. The player cell is unlabelled and stays positional td[0]; the previously mis-indexed clutch_attempts is folded into the map under key cl.
  • Fail-closed: REQUIRED_KEYS = {rnd, rating2, acs, kd, adr}. If <thead> has ≥1 data-col but any required key is missing, the parser raises instead of shipping shifted numbers. A future layout change surfaces loudly rather than silently.
  • Legacy fallback: the positional path fires only when <thead> yields zero data-col attributes (old markup), and logs a warning when it does — so this stays backward-compatible with the pre-revamp page.
• Org selector (b). Prefer .st-pl-country, fall back to the old .stats-player-country, keep the "N/A" sentinel. Works against either markup.
• Region vocabulary (c). New /stats-specific set all, americas, emea, pacific, china, intl, validated by validate_stats_region. The legacy keys are accepted as deprecated aliases and normalized (na/br → americas, eu → emea, ap/kr/jp/oce → pacific, cn → china) before the cache key is formed, so an alias and its canonical don't hold duplicate cache entries. The shared /rankings region dict is left completely untouched — a test asserts /rankings?region=americas still 400s.
• Session priming (d). One throwaway GET per client to prime PHPSESSID, guarded by an asyncio once-lock (the singleton client is shared across concurrent requests). Prime failure raises rather than caching a cold response. Correctness is validated at the response level: parse the selected <option> of select[name=region]; on mismatch, re-prime + refetch once, then raise on a second mismatch.
• timespan and MIN_ROUNDS are deliberately left alone.

Why key off data-col

This is the third recurrence of the same positional-index fragility — issue #4 (2022) and issue #14 (2024) were both "vlr.gg added a column", and both were fixed by re-hardcoding the new indices. Keying off the data-col attributes vlr.gg now emits, with fail-closed REQUIRED_KEYS, retires the class of bug: a future column insertion can no longer shift stats into the wrong fields, and if a required column ever disappears the parser raises instead of returning quietly wrong numbers.

Tests

+35 tests (suite 92 → 127, all passing on this branch):

• new 23-column fixture asserting rounds_played ≈ 228/rating ≈ 1.37 (not the shifted 11/228);
• legacy 21-column fixture asserting the positional fallback still parses + logs its warning;
• missing-required-key fixture asserting the parser raises rather than partial-parsing;
• org-selector matrix (new class preferred, old class fallback, missing → N/A);
• priming order + priming-failure (raises, caches nothing) tests;
• region-mismatch retry (re-prime + refetch once, then raise) and mismatch-recovers-on-reprime;
• alias-normalizes-before-cache-key (alias and canonical share one cache entry);
• empty-region returns cleanly with no spurious retry;
• a /rankings?region=americas still-400 guard proving the shared region dict is untouched.

Related observations (reproducible; fixes intentionally out of scope here)

These look like the same revamp bleeding into other endpoints. I've kept this PR focused on /stats, but I'm happy to open separate issues:

1. /match/details → maps[].players returns empty scoreboards on the current markup (verified on matches spanning May 31 – Jul 9 2026). Almost certainly the same page revamp changed the scoreboard selectors.
2. /team?q=transactions rows are field-shifted — the date column carries the player name and the role column carries the announcement URL. Same positional-parsing disease as (a) above.

Let me know if you'd like these as their own issues, or folded into a follow-up.

---

🤖 Generated with Claude Code