Add full-flow order tracing and per-section selector discovery

.claude/commands/bring-up.md

@@ -9,9 +9,12 @@ Guide me through bring-up for a store, in order, pausing for me between steps
 2. `roomieorder doctor` — confirm profile / display / chrome are green.
 3. `roomieorder verify-selectors --provider <store>` — confirm the price and
    add-to-cart selectors match; fix any MISS off the dom dump first.
-4. `roomieorder dry-run <item> --provider <store>` — confirm it reaches the
+4. `roomieorder trace-order <item> --provider <store>` — walk the whole flow and
+   confirm the cart/checkout/review selectors (incl. `place-order`/`order-total`/
+   payment) resolve; fix any MISS off that step's dom dump before the first order.
+5. `roomieorder dry-run <item> --provider <store>` — confirm it reaches the
    review page; `Read` the screenshot.
-5. Only after a clean dry-run on a cheap item: flip `DRY_RUN=false` and place
+6. Only after a clean dry-run on a cheap item: flip `DRY_RUN=false` and place
    one real order, then `roomieorder queue` to confirm `placed`.
 
 Never flip `DRY_RUN` or place a real order without my explicit go-ahead.

.claude/commands/trace-order.md

@@ -0,0 +1,19 @@
+---
+description: Dump DOM + selector probe + screenshot at every checkpoint of the buy flow.
+argument-hint: "<item_key> [--provider costco|amazon]"
+---
+Run `roomieorder trace-order $ARGUMENTS`.
+
+This always forces DRY_RUN — it walks the real buy flow to the review page and
+NEVER places an order. At each checkpoint (product → cart → cart view → delivery
+→ payment → review) it writes a rendered `*_dom.html`, a selector `*_probe.txt`,
+and a screenshot to the shots dir, and prints a per-step PASS/MISS digest.
+
+Unlike `dump-dom`/`verify-selectors` (which stop at the product page), this
+reaches the checkout/review surface where the `place-order`, `order-total`, and
+payment selectors finally render. For any group still MISS at a checkout step,
+`Read` that step's `*_dom.html` and find the real selector on the live page (per
+AGENTS.md §1), then propose the corrected selector(s) for `purchase.py`.
+
+Do NOT edit `purchase.py` unless I explicitly ask — the buy flow is
+additive-only and can only be validated against live DOM during bring-up.

.claude/commands/triage-failure.md

@@ -7,6 +7,8 @@ newest screenshot (and `*_dom.html` / `*_probe.txt` if present).
 Classify the failure using AGENTS.md §1–§3: selector drift, logged-out /
 sign-in wall, CAPTCHA/OTP challenge, or an outright Akamai block. State which
 stage died (from the shot tag) and recommend exactly one next command — e.g.
-`dump-dom`, `verify-selectors`, `login`, or `resume`.
+`dump-dom`, `verify-selectors`, `login`, or `resume`. For a checkout-stage death
+(`no_place_order`, `left_checkout`, `cart_mismatch`), recommend `trace-order`:
+it's the only tool that dumps the cart/review DOM where those selectors live.
 
 Do not order or log in yourself; just diagnose and recommend.

AGENTS.md

@@ -9,13 +9,14 @@ before touching the buy flow, the catalog, or login/bot-detection logic.
 Start every "why did X break" investigation with the read-only diagnostics
 below — they're safe (no browser, no spend) and tell you where to look. The
 `.claude/commands/` slash commands (`/diagnose`, `/triage-failure`,
-`/verify-selectors`, `/bring-up`) chain these for you.
+`/verify-selectors`, `/trace-order`, `/bring-up`) chain these for you.
 
 | Symptom | First command | Then read |
 | --- | --- | --- |
 | "is anything wrong?" / cold start | `roomieorder doctor` | its own output (config, Chrome, display, profiles, DB, catalog) |
 | "the order didn't place" | `roomieorder failures` | the newest `*.png` it lists, plus the row's `notes` |
 | selector miss / store redesign | `roomieorder verify-selectors [item]` | the `*_dom.html` it points at, then read the live selector off it (§1) |
+| checkout/review selector miss (`no_place_order` / `left_checkout` / `cart_mismatch`) | `roomieorder trace-order <item>` (DRY_RUN walk, never orders) | the `*_checkout_landed_dom.html` it dumps — the only place `place-order`/`order-total`/payment selectors render (§1) |
 | logged out / sign-in wall | `roomieorder dump-dom <item>` | §2 — prefilled ≠ logged in; check the **logon URL**, not header text |
 | CAPTCHA / OTP challenge | (worker auto-pauses) `roomieorder status` | §1, §3 — Akamai may be blocking; this is expected-until-verified |
 | Sheets row never appeared | `roomieorder test-sheet` | the gspread error (`-v`); a no-op logger silently "succeeds" otherwise |
@@ -26,7 +27,10 @@ below — they're safe (no browser, no spend) and tell you where to look. The
 are read-only and allow-listed in `.claude/settings.json`, so they run without a
 permission prompt. `verify-selectors` (and `dump-dom`) hit live store pages
 read-only and need a logged-in profile + network — they're operator-run, not
-CI.
+CI. `trace-order` is the same footprint but walks the *whole* flow to the review
+page (always DRY_RUN — never orders); add
+`Bash(roomieorder trace-order:*)` to the settings allow-list to run it without a
+prompt.
 
 **Queue statuses** (`store.py`, also the Sheets `status` column): `pending` /
 `in_progress` (transient); `placed` (done); `dry_run`; `skipped_cooldown` /
@@ -46,9 +50,11 @@ cart-singleton guard saw more than the intended item — NOT placed) / `signin_*
 / `challenge_*` / `blocked_*` / `left_checkout` / `submitted_unconfirmed` /
 `confirmation` / `review` / `timeout` / `crash` / `dump`. Diagnostic tags are
 captured full-page (below-the-fold banners included); the `review`/`confirmation`
-/`dump` shots stay header-only. `verify-selectors` and `dump-dom` also write
-`*_dom.html` (rendered page) and `*_probe.txt` (per-selector match counts) —
-`Read` those to find the real selector instead of guessing. The shots dir is
+/`dump` shots stay header-only. `trace-order` adds a per-step family tagged
+`trace{HHMMSS}_{NN}_{step}` (full-page, so the whole cart/review page is caught).
+`verify-selectors`, `dump-dom`, and `trace-order` also write `*_dom.html`
+(rendered page) and `*_probe.txt` (per-selector match counts) — `Read` those to
+find the real selector instead of guessing. The shots dir is
 pruned automatically (worker) and via `roomieorder prune-shots`
 (`ROOMIEORDER_SHOTS_RETENTION_DAYS`, default 30).
 
@@ -95,6 +101,19 @@ logged-in profile. (`_PRICE_SELECTORS` already has a structured-data fallback 
 `og:price`/`product:price:amount` meta tags, then JSON-LD `offers.price` — for
 when the visible-price CSS guesses miss on the `/p/-/<slug>/<id>` storefront.)
 
+**Reaching the cart/checkout selectors — `roomieorder trace-order <item>`.**
+`dump-dom` stops at the product page, so the `place-order`/`order-total`/payment
+selector groups always read `count=0` there. `trace-order` (also DRY_RUN, never
+orders — it forces `dry_run` and halts at the review page) attaches a
+`purchase.FlowTracer` that drops the same DOM + probe + screenshot trio at every
+checkpoint of the *real* buy path (`product_loaded` → `cart_added` → `cart_view`
+→ `delivery_continue` → `payment_selected` → `checkout_landed` →
+`review_pre_place`). To fix a checkout selector miss, run it and `Read` the
+`*_checkout_landed_dom.html` — that's the SinglePageCheckoutView where those
+selectors live. The same tracer rides live worker orders when
+`ROOMIEORDER_TRACE_ORDERS=true` (default off — adds per-step I/O; an advanced
+escape hatch for a recurring mid-checkout failure, not a default).
+
 The assistant's own Bash shell on host `link` can reach the graphical session,
 so headed Playwright (`dump-dom`, `dry-run`, `login`) can be driven directly
 from Bash against a logged-in profile dir when faster iteration is wanted — but

src/roomieorder/cli.py

@@ -11,6 +11,7 @@
 * ``login``        — open the profile headed to sign into Costco by hand.
 * ``dry-run KEY`` — drive one item to its review page and screenshot, no order.
 * ``dump-dom KEY`` — read-only DOM dump + selector probe for bring-up.
+* ``trace-order KEY`` — DRY_RUN walk dumping DOM + probe + screenshot per step.
 * ``verify-selectors`` — probe live pages for stale buy-flow selectors.
 * ``doctor``      — one-shot, read-only health check of every subsystem
                     (``--check-login`` adds a per-store signed-in probe).
@@ -304,6 +305,78 @@ def dump_dom(item_key: str, provider: str) -> None:
     click.echo(result.summary)
 
 
+# Selector groups worth a one-line PASS/MISS digest per checkpoint in the
+# trace-order table — the buy-flow groups, skipping the noisier price-meta/signin.
+_DIGEST_GROUPS = ("price", "add-to-cart", "buy-now", "place-order", "order-total")
+
+
+@main.command(name="trace-order")
+@click.argument("item_key")
+@_PROVIDER_OPT
+def trace_order(item_key: str, provider: str) -> None:
+    """Walk ITEM_KEY through the whole buy flow, dumping every step — never orders.
+
+    Forces DRY_RUN (like ``dry-run``) so it always halts at the review page
+    *before* Place Order, then attaches a tracer that writes a rendered DOM, a
+    selector probe, and a screenshot at each checkpoint — product page, cart,
+    cart view, delivery, payment, and the review page. Unlike ``dump-dom`` (which
+    stops at the product page), this reaches the checkout/review surface where the
+    ``place-order``/``order-total``/payment selectors finally render, so they
+    become discoverable. Hits live store pages, so it's operator-run, not CI.
+    """
+    from roomieorder.purchase import FlowTracer, new_run_id
+
+    config = load_config()
+    config = config.model_copy(update={"dry_run": True})
+    items = load_catalog(config.catalog_path)
+    item = items.get(item_key)
+    if item is None:
+        raise click.ClickException(f"unknown item_key: {item_key} (have: {', '.join(items)})")
+    source = _source_for(item, provider)
+
+    store = Store(config.db_path)
+    store.init_db()
+    purchaser = _purchaser_for(config, provider)
+
+    def proceed_check(live_price: float):  # type: ignore[no-untyped-def]
+        ceiling = check_price_ceiling(item.title, source.price_ceiling, live_price)  # type: ignore[attr-defined]
+        if not ceiling.ok:
+            return ceiling
+        return check_spend_cap(store, config, live_price * item.qty)
+
+    tracer = FlowTracer(purchaser, item_key, run_id=new_run_id())  # type: ignore[arg-type]
+    click.echo(f"trace-order {item_key} ({provider}) → {purchaser._resolve_url(source)}")  # type: ignore[attr-defined]
+    result = purchaser.buy(item_key, item, source, proceed_check, tracer=tracer)  # type: ignore[attr-defined]
+    store.close()
+
+    click.echo(f"status:      {result.status}")
+    click.echo(f"unit_price:  {result.unit_price}")
+    click.echo(f"order_total: {result.order_total}")
+    click.echo(f"message:     {result.message}")
+    click.echo("")
+    click.echo(f"steps ({len(tracer.steps)}):")
+    any_artifact = False
+    for step in tracer.steps:
+        hits = _group_hits(step.summary)
+        digest = " ".join(
+            f"{g}={'ok' if hits.get(g) else 'MISS'}"
+            for g in _DIGEST_GROUPS
+            if g in hits
+        )
+        click.echo(f"  {step.idx:02d} {step.name:18} {step.url}")
+        click.echo(f"     {digest}")
+        if step.probe:
+            any_artifact = True
+            click.echo(f"     probe: {step.probe}")
+        if step.html:
+            click.echo(f"     dom:   {step.html}")
+        if step.screenshot:
+            click.echo(f"     shot:  {step.screenshot}")
+    if any_artifact:
+        click.echo("")
+        click.echo("For any group still MISS at a checkout step, Read that step's *_dom.html to find the live selector.")
+
+
 # Statuses that mean an order didn't cleanly place — what `failures` surfaces.
 _TROUBLE_STATUSES = (
     "failed",

src/roomieorder/config.py

@@ -128,6 +128,15 @@ class Config(BaseModel):
     auto_retry: bool = False
     auto_retry_max: int = Field(default=1, ge=0)
 
+    # Opt-in full-flow tracing on *live* worker orders (off by default). When on,
+    # every buy attaches a purchase.FlowTracer that dumps a DOM + selector probe +
+    # screenshot at each checkout step into shots_dir — the same artifacts the
+    # `trace-order` CLI produces, but for real runs, so a mid-checkout failure
+    # leaves the whole trail. Adds page.content()+screenshot I/O per step, so it's
+    # an advanced troubleshooting escape hatch, not a default. The pruner covers
+    # the extra artifacts via shots_retention_days.
+    trace_orders: bool = False
+
     # Dead-man's-switch heartbeat. The worker pings this URL on a timer; a missed
     # ping alerts via whatever push-style monitor it points at — hosted
     # Healthchecks.io or a self-hosted open-source instance, Uptime Kuma push,
@@ -215,6 +224,7 @@ def load_config() -> Config:
         openclaw_channel=_env_str("OPENCLAW_CHANNEL", "telegram"),
         auto_retry=_env_bool("ROOMIEORDER_AUTO_RETRY", False),
         auto_retry_max=_env_int("ROOMIEORDER_AUTO_RETRY_MAX", 1),
+        trace_orders=_env_bool("ROOMIEORDER_TRACE_ORDERS", False),
         heartbeat_url=_env_str("ROOMIEORDER_HEARTBEAT_URL", ""),
         heartbeat_interval_seconds=_env_int("ROOMIEORDER_HEARTBEAT_INTERVAL_SECONDS", 300),
         session_check_hours=_env_float("ROOMIEORDER_SESSION_CHECK_HOURS", 0.0),

src/roomieorder/orchestrator.py

@@ -23,8 +23,10 @@
     AmazonPurchaser,
     BasePurchaser,
     CostcoPurchaser,
+    FlowTracer,
     ProceedCheck,
     PurchaseResult,
+    new_run_id,
 )
 from roomieorder.store import Store
 
@@ -104,7 +106,19 @@ def buy(self, item_key: str, item: CatalogItem) -> PurchaseResult:
 
         for idx, (name, source, purchaser) in enumerate(chain):
             is_last = idx == len(chain) - 1
-            result = purchaser.buy(item_key, item, source, self._proceed_check(item, source))
+            # Opt-in full-flow trace on live orders (config.trace_orders, default
+            # off). Each store-leg gets its own run_id so a Costco→Amazon fallback
+            # keeps its two traces apart. Off → the no-op default keeps the buy
+            # byte-for-byte unchanged.
+            tracer = (
+                FlowTracer(purchaser, item_key, run_id=new_run_id())
+                if self.config.trace_orders
+                else None
+            )
+            kwargs = {"tracer": tracer} if tracer is not None else {}
+            result = purchaser.buy(
+                item_key, item, source, self._proceed_check(item, source), **kwargs
+            )
             result.provider = name
 
             if result.status in _FALLBACK_STATUSES and not is_last: