Wire PointsPath matched-id join through gflight backend

docs/memories/MEMORY.md

@@ -39,6 +39,12 @@ go into detail and are loaded on demand.
   rides both backends. Decision: `enable_matching=False` + the existing
   matcher keys, not the matched-Google-flight-id path. Empirical evidence
   behind the choice and the upgrade path if it turns out worth it later.
+- [pp_matched_id_recipe.md](pp_matched_id_recipe.md) — **Supersedes the
+  "dead end" framing in `pp_on_gflight.md`.** The matched-id join *does*
+  work; the previous "no" was because we sent synthetic flight_ids and
+  the wrong hint shape. Recipe (real `data[0][17]` from fli + IATA-prefixed
+  flight#, human-readable airline name, space-separated times), empirical
+  proof, and wire-through implementation notes.
 
 ## When to add a new memory file
 

docs/memories/pp_matched_id_recipe.md

@@ -0,0 +1,112 @@
+# PointsPath matched-id join — empirical recipe (2026-05-15)
+
+Settled: PP's `enableGoogleFlightMatching=true` mode **works** and gives a
+robust cash↔award join key (`matchedGoogleFlightId`), better than the
+(flight#, date) + (route, time) fallbacks today. The earlier doc
+(`pp_on_gflight.md`) ruled this out as a dead end; that conclusion was
+based on a probe with a synthetic `flight_id`. With **real** Google
+Flights opaque IDs, PP echoes them back populated.
+
+## What unlocked it
+
+Captured the live PP browser-extension traffic via Patchright + a snapshot
+of the user's Chrome profile (with PP extension registered + Pro session
+cookies). 549 events; 87 successful `/api/airline-search` responses, every
+returned flight with `matchedGoogleFlightId` populated.
+
+The capture also disproved the `store-flights` hypothesis from
+`pp_on_gflight.md`: the extension does NOT pre-register flights via
+`/api/store-flights` before querying. It goes straight to `/api/airline-search`
+with `enableGoogleFlightMatching=true` and the hints carry their own opaque
+IDs. PP looks the IDs up in its catalog at query time.
+
+## The recipe
+
+1. **Extract `data[0][17]`** from each flight in Google Flights' raw
+   API response. fli's `_parse_flights_data` reads `data[0][2]` (legs),
+   `data[0][9]` (duration), `data[0][1][−1]` (price) — but drops index `[17]`
+   on the floor. We added `src/flight_cli/_gflight_ids.py` which reuses fli's
+   encoder + curl_cffi client but parses our own response, capturing flightId.
+
+2. **Hint payload shape** (per `CashFlightHint.to_payload()`):
+
+   ```json
+   {
+     "origin": "MIA",
+     "dest": "LHR",
+     "startDateTime": "2026-06-30 18:05",     // YYYY-MM-DD HH:MM, no T, no TZ
+     "endDateTime":   "2026-07-01 08:05",
+     "flightId": "NbXSYb",                     // 5-6 char opaque from data[0][17]
+     "airline": "Virgin Atlantic",             // HUMAN-READABLE name, not IATA
+     "googleAirlines": ["Virgin Atlantic"],
+     "numConnections": 0,
+     "hasCarryOnBaggage": false,
+     "firstFlightNumber": "VS6",               // IATA prefix + flight# concatenated
+     "cashPrice": 802,
+     "rawCashPriceString": "$802"
+   }
+   ```
+
+   The two formatting gotchas that produce 0-flight responses if missed:
+   - `firstFlightNumber` MUST be IATA-prefixed (`VS6` not `6`)
+   - `airline` MUST be the human-readable name (`"Virgin Atlantic"` not `"VS"`)
+
+3. **PP airline-search request** (`SearchSpec` with `enable_matching=True`):
+   `airline` is the PP-side program being queried (`"American"`, `"Delta"`,
+   etc.). PP returns flights matching ANY hint, regardless of whether the
+   operating carrier matches the queried airline — useful for codeshare
+   bridging (querying American can return a BA-operated flight if the hint
+   refers to it).
+
+4. **Response** has flights with `matchedGoogleFlightId` set to whichever
+   hint `flightId` they matched. Join cash→award by string equality on that.
+
+## Empirical evidence
+
+`research/probe_pp_real_ids.py` produces:
+```
+PP query=American   → 2 flights, 2 matched
+  *MATCHED*  AA38 matched='A1oq6c'      ← our supplied flightId
+  *MATCHED*  BA208 matched='KTfE0c'     ← codeshare picked up via American
+PP query=VirginAtlantic → 1 matched
+  *MATCHED*  VS118 matched='w8PDCc'
+```
+
+Live capture saved at `research/capture/pp_extension_capture.json`
+(gitignored). 87 successful airline-search responses for cross-reference.
+
+## Why we'd want it
+
+The (flight#, date) + (route, time) fallback we ship today works fine for
+most flights but degrades on:
+- Codeshares with marketing flight# != operating flight# (work-22az fix
+  handles the common AA/BA case via route+time; not all edge cases)
+- Flights with the same route+time on the same day but a different metal
+  (rare but real — adjacent slot codeshares)
+
+The matched-id path collapses all of that to string equality on opaque IDs
+PP has already done the heavy lifting on.
+
+## Implementation status
+
+- ✅ `_gflight_ids.py` — fli wrapper that captures the opaque flightId
+- ✅ Format recipe verified empirically (probe above)
+- ⏳ Wire through to provider + matcher (work-?? follow-up):
+  - Add `flight_id: str | None` field to `Slice` (or per-itinerary metadata)
+  - Populate from fli via `gflight_adapter`
+  - Provider takes `cash_hints` and sends with `enable_matching=True`
+  - Provider populates `AwardFlight.matched_google_flight_id` from response
+  - Matcher adds a third join index keyed on flight_id (highest priority)
+  - The existing (flight#, date) + (route+time) keys stay as fallback
+
+## Airline IATA → name mapping
+
+PP's hint field `airline` requires the human-readable name. fli only exposes
+the IATA code via its `Airline` enum. Two paths:
+- Hardcode a mapping for the airlines PP services (`pp_pricing_info` already
+  uses these names, e.g. `Delta`, `American`, `VirginAtlantic`)
+- Extract `fl[22][3]` from Google's response (the airline name string)
+
+The hardcoded mapping is simpler and PP's catalog is small + stable. Future
+spike: extract `fl[22][3]` so the data flows from Google's truth rather than
+a static lookup.

src/flight_cli/_airline_names.py

@@ -0,0 +1,48 @@
+"""IATA airline code → human-readable name (PP's hint payload format).
+
+PP's `/api/airline-search` hint object expects `airline` and `googleAirlines`
+fields as the human-readable name (e.g. "Virgin Atlantic"), NOT the IATA
+code ("VS"). With the IATA code, PP returns 0 flights — verified empirically
+via the browser-extension capture (research/capture/pp_extension_capture.json).
+
+We mirror the names PP uses in its own pricing-info catalog so the hint
+metadata aligns with PP's internal records. Coverage is the airlines PP
+supports — for anything else we fall back to the IATA code, accepting that
+PP may not return a match (which is fine; the matcher falls back to
+flight#+date / route+time).
+"""
+
+# Keys are fli's `Airline.name` (== IATA two-letter), values are PP's
+# canonical airline name from its /api/pricing-info response.
+_IATA_TO_PP_NAME: dict[str, str] = {
+    "AA": "American",
+    "AC": "Air Canada",
+    "AF": "Air France",
+    "AS": "Alaska",
+    "B6": "JetBlue",
+    "BA": "British Airways",
+    "DL": "Delta",
+    "EI": "Aer Lingus",
+    "EK": "Emirates",
+    "EY": "Etihad",
+    "IB": "Iberia",
+    "KL": "KLM",
+    "LH": "Lufthansa",
+    "NK": "Spirit",
+    "QF": "Qantas",
+    "QR": "Qatar",
+    "SQ": "Singapore",
+    "TP": "Tap Air Portugal",
+    "UA": "United",
+    "VA": "Virgin Australia",
+    "VS": "Virgin Atlantic",
+}
+
+
+def pp_airline_name(iata: str) -> str:
+    """Map a 2-letter IATA airline code to PP's human-readable name.
+
+    Falls back to the input on unknown codes — PP will likely return no
+    match for those, which the matcher tolerates (flight#+date and
+    route+time stay available as fallback join keys)."""
+    return _IATA_TO_PP_NAME.get(iata.upper(), iata)

src/flight_cli/_gflight_ids.py

@@ -0,0 +1,146 @@
+"""Wrapper around fli's Google Flights search that ALSO captures the opaque
+`flightId` Google emits at index [17] of each flight row.
+
+fli's `SearchFlights._parse_flights_data` parses legs, price, duration, stops —
+but drops `data[0][17]`. PointsPath's `enableGoogleFlightMatching` mode joins
+its award catalog against exactly that opaque ID (see PP browser extension
+chunk-5KW5VSHS.js: `flightId: a` where `a = n[17]`). Without it, PP returns
+an empty result for hint-based queries; with it, `matchedGoogleFlightId`
+echoes back populated.
+
+We re-use fli's `FlightSearchFilters.encode()` + curl_cffi client so the
+request shape stays in lockstep with upstream; only the response parser
+diverges (extends fli's by one field).
+"""
+
+from __future__ import annotations
+
+import json
+import logging
+from copy import deepcopy
+from dataclasses import dataclass
+from typing import TYPE_CHECKING, Any
+
+from fli.models import (  # pyright: ignore[reportMissingTypeStubs]
+    FlightLeg,
+    FlightResult,
+)
+from fli.models.google_flights.base import TripType  # pyright: ignore[reportMissingTypeStubs]
+from fli.search.client import get_client  # pyright: ignore[reportMissingTypeStubs]
+from fli.search.flights import SearchFlights  # pyright: ignore[reportMissingTypeStubs]
+
+if TYPE_CHECKING:
+    from fli.models.google_flights.flights import (  # pyright: ignore[reportMissingTypeStubs]
+        FlightSearchFilters,
+    )
+
+log = logging.getLogger(__name__)
+
+_BASE_URL = SearchFlights.BASE_URL
+
+# Position of the opaque per-flight ID in Google Flights' API row array.
+# Mirrors the PP browser extension's parser (chunk-5KW5VSHS.js: `a = n[17]`).
+_FLIGHT_ID_IDX = 17
+
+
+@dataclass
+class GFlightWithId:
+    """fli's FlightResult plus Google's opaque flight_id for PP matching."""
+
+    flight: FlightResult
+    flight_id: str
+
+
+def _parse_flight_with_id(data: list[Any]) -> GFlightWithId:
+    """Mirror of fli's `_parse_flights_data` but also reads `data[0][17]`.
+
+    Indices match the PP extension's parser (chunks/chunk-5KW5VSHS.js): n[17]
+    is the per-flight opaque ID; n[2] legs; n[9] duration; t[0][-1] price."""
+    price, currency = SearchFlights._parse_price_info(data)  # pyright: ignore[reportPrivateUsage, reportUnknownMemberType]
+    flight_id = data[0][_FLIGHT_ID_IDX] if len(data[0]) > _FLIGHT_ID_IDX else ""
+    flight = FlightResult(
+        price=price,
+        currency=currency,
+        duration=data[0][9],
+        stops=len(data[0][2]) - 1,
+        legs=[
+            FlightLeg(
+                airline=SearchFlights._parse_airline(fl[22][0]),  # pyright: ignore[reportPrivateUsage]
+                flight_number=fl[22][1],
+                departure_airport=SearchFlights._parse_airport(fl[3]),  # pyright: ignore[reportPrivateUsage]
+                arrival_airport=SearchFlights._parse_airport(fl[6]),  # pyright: ignore[reportPrivateUsage]
+                departure_datetime=SearchFlights._parse_datetime(fl[20], fl[8]),  # pyright: ignore[reportPrivateUsage]
+                arrival_datetime=SearchFlights._parse_datetime(fl[21], fl[10]),  # pyright: ignore[reportPrivateUsage]
+                duration=fl[11],
+            )
+            for fl in data[0][2]
+        ],
+    )
+    return GFlightWithId(flight=flight, flight_id=flight_id)
+
+
+def _one_call(filters: FlightSearchFilters) -> list[GFlightWithId]:
+    """Single HTTP round-trip to Google's endpoint; flat list of one leg's flights."""
+    client = get_client()
+    encoded = filters.encode()
+    resp = client.post(
+        url=_BASE_URL,
+        data=f"f.req={encoded}",
+        impersonate="chrome",
+        allow_redirects=True,
+    )
+    resp.raise_for_status()
+    parsed = json.loads(resp.text.lstrip(")]}'"))[0][2]
+    if not parsed:
+        return []
+    inner = json.loads(parsed)
+    flights_data: list[Any] = [
+        item for i in (2, 3) if isinstance(inner[i], list) for item in inner[i][0]
+    ]
+    out: list[GFlightWithId] = []
+    for fd in flights_data:
+        try:
+            out.append(_parse_flight_with_id(fd))
+        except (AttributeError, KeyError, ValueError, IndexError) as e:
+            log.debug("skipping flight with unparseable data: %s", e)
+            continue
+    return out
+
+
+def search_with_ids(
+    filters: FlightSearchFilters,
+    *,
+    top_n: int = 5,
+) -> list[GFlightWithId | tuple[GFlightWithId, ...]] | None:
+    """Drop-in for fli's `SearchFlights().search()` but each result carries
+    its Google Flights opaque flight_id.
+
+    Round-trip / multi-city follow the same iterative leg-selection pattern
+    as fli: query first leg, pick top_n, drive each through the rest. Each
+    `GFlightWithId` in a returned tuple has its own per-leg flight_id."""
+    first = _one_call(filters)
+    if not first:
+        return None
+
+    if filters.trip_type == TripType.ONE_WAY:
+        return list(first)
+
+    num_segments = len(filters.flight_segments)
+    selected_count = sum(1 for s in filters.flight_segments if s.selected_flight is not None)
+    # Last leg already — no further iteration.
+    if selected_count >= num_segments - 1:
+        return list(first)
+
+    combos: list[GFlightWithId | tuple[GFlightWithId, ...]] = []
+    for picked in first[:top_n]:
+        next_filters = deepcopy(filters)
+        next_filters.flight_segments[selected_count].selected_flight = picked.flight
+        nxt = search_with_ids(next_filters, top_n=top_n)
+        if nxt is None:
+            continue
+        for nx in nxt:
+            if isinstance(nx, tuple):
+                combos.append((picked, *nx))
+            else:
+                combos.append((picked, nx))
+    return combos or None

src/flight_cli/cli.py

@@ -502,14 +502,18 @@ def _run_gflight_path(
     (origin, dest, date) per leg as the matrix path.
     """
     # fli is heavy (selenium/selectolax); lazy-import so the rest of flight_cli
-    # doesn't pay the startup cost when not used.
-    from .fli_bridge import run_gflight_search  # noqa: PLC0415
+    # doesn't pay the startup cost when not used. `search_with_ids` wraps fli's
+    # encoder + client but parses the response ourselves to capture the opaque
+    # per-flight ID (data[0][17]) — that's what PP's enableGoogleFlightMatching
+    # joins against to produce matchedGoogleFlightId in its response.
+    from ._gflight_ids import search_with_ids  # noqa: PLC0415
+    from .fli_bridge import to_fli_filter  # noqa: PLC0415
 
     search = SpecificDateSearch(legs=legs, options=opts)
     try:
-        # fli has no type stubs; treat its return as opaque at this boundary
-        # and use duck-typed attribute access below.
-        results: list[Any] = run_gflight_search(search, top_n=top_n)
+        # Returns GFlightWithId | tuple[GFlightWithId, ...]. The .flight attribute
+        # exposes fli's FlightResult; .flight_id is Google's opaque ID.
+        results: list[Any] = search_with_ids(to_fli_filter(search), top_n=top_n) or []
     except Exception as e:
         err.print(f"[red]Google Flights query failed:[/] {e}")
         raise typer.Exit(1) from e
@@ -526,10 +530,9 @@ def _run_gflight_path(
     if json_out and not run_pp:
         out: list[Any] = []
         for r in results:
-            if isinstance(r, tuple):
-                out.append([fr.model_dump(mode="json") for fr in r])  # pyright: ignore[reportUnknownMemberType, reportUnknownVariableType]
-            else:
-                out.append(r.model_dump(mode="json"))
+            items: list[Any] = list(r) if isinstance(r, tuple) else [r]  # pyright: ignore[reportUnknownArgumentType]
+            dumped = [{**g.flight.model_dump(mode="json"), "flight_id": g.flight_id} for g in items]
+            out.append(dumped if isinstance(r, tuple) else dumped[0])
         sys.stdout.write(json.dumps(out, indent=2, default=str))
         return
 
@@ -553,7 +556,9 @@ def _run_gflight_path(
 
 
 def _render_gflight_table(results: list[Any], *, legs: tuple[Leg, ...], top_n: int) -> None:
-    """Render fli results as a rich table. Duck-typed: fli has no type stubs."""
+    """Render fli results as a rich table. Duck-typed: fli has no type stubs.
+
+    Accepts our `GFlightWithId` wrappers — `.flight` is fli's FlightResult."""
     origin = legs[0].origins[0] if legs[0].origins else "?"
     destination = legs[0].destinations[0] if legs[0].destinations else "?"
     has_return = len(legs) >= _ROUND_TRIP_LEGS
@@ -569,7 +574,8 @@ def _render_gflight_table(results: list[Any], *, legs: tuple[Leg, ...], top_n: i
     t.add_column("legs")
     for i, r in enumerate(results[:top_n], 1):
         items: list[Any] = list(r) if isinstance(r, tuple) else [r]  # pyright: ignore[reportUnknownArgumentType]
-        for j, fr in enumerate(items):
+        for j, g in enumerate(items):
+            fr = g.flight  # unwrap GFlightWithId → fli FlightResult
             label = f"{i}{'a' if j == 0 else 'b'}" if len(items) > 1 else str(i)
             legs_str = " → ".join(
                 f"{getattr(leg.airline, 'name', leg.airline)} {getattr(leg, 'flight_number', '?')}"

src/flight_cli/models.py

@@ -89,6 +89,12 @@ class Slice(_Loose):
     duration: int | None = None
     origin: SliceEndpoint | None = None
     destination: SliceEndpoint | None = None
+    # Provider-supplied opaque ID for this leg (Google Flights data[0][17]).
+    # Populated when the cash side is built from fli's response (gflight backend);
+    # used by PP's enableGoogleFlightMatching to mint the matchedGoogleFlightId
+    # join key. None for Matrix cash itineraries — they don't expose an ID
+    # PP recognizes, so those fall back to flight#+date / route+time joins.
+    flight_id: str | None = None
 
 
 class SliceCarrier(_Loose):