From 0beb8ad4dac6d86cb9402eee98968f2d1852ddd4 Mon Sep 17 00:00:00 2001 From: Samuel Bharti Date: Thu, 4 Jun 2026 13:10:52 -0400 Subject: [PATCH 1/2] feat(shiny): add reactivity, testing, and agent-driving skills Add three Shiny skills under the shiny plugin: - shiny-reactivity: design and debug the reactive graph (reactive vs observe, reactiveValues, req/bindEvent/bindCache/debounce, reactlog). - shiny-testing: the three testing layers (testthat, testServer, shinytest2/AppDriver) and what to test where. - shiny-for-agents: drive/inspect a running app (R and Python) by waiting on Shiny's busy/idle signal instead of sleeps, and reading server-side errors the browser can't see. Register all three in marketplace.json and document them in the root and shiny category READMEs. --- .claude-plugin/marketplace.json | 5 +- README.md | 3 + shiny/README.md | 33 +++- shiny/shiny-for-agents/SKILL.md | 115 ++++++++++++ .../shiny-for-agents/scripts/launch_app_dev.R | 30 +++ .../scripts/wait_for_shiny_idle.js | 171 ++++++++++++++++++ shiny/shiny-reactivity/SKILL.md | 116 ++++++++++++ shiny/shiny-testing/SKILL.md | 117 ++++++++++++ 8 files changed, 587 insertions(+), 3 deletions(-) create mode 100644 shiny/shiny-for-agents/SKILL.md create mode 100644 shiny/shiny-for-agents/scripts/launch_app_dev.R create mode 100644 shiny/shiny-for-agents/scripts/wait_for_shiny_idle.js create mode 100644 shiny/shiny-reactivity/SKILL.md create mode 100644 shiny/shiny-testing/SKILL.md diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index fb4958d..f9bf183 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -76,7 +76,10 @@ "skills": [ "./brand-yml", "./shiny/shiny-bslib", - "./shiny/shiny-bslib-theming" + "./shiny/shiny-bslib-theming", + "./shiny/shiny-reactivity", + "./shiny/shiny-testing", + "./shiny/shiny-for-agents" ] }, { diff --git a/README.md b/README.md index 4356139..3cf2bf2 100644 --- a/README.md +++ b/README.md @@ -55,6 +55,9 @@ Skills for Shiny app development in both R and Python. - **[brand-yml](./brand-yml/)** - Create and apply brand.yml files for consistent styling across Shiny apps, with support for bslib (R) and ui.Theme (Python), including automatic brand discovery and theming functions for plots and tables - **[shiny-bslib](./shiny/shiny-bslib/)** - Build modern Shiny dashboards using bslib with Bootstrap 5 layouts, cards, value boxes, navigation, theming, and modern inputs. Includes migration guide from legacy Shiny patterns - **[shiny-bslib-theming](./shiny/shiny-bslib-theming/)** - Comprehensive theming for Shiny apps using bslib, covering bs_theme(), Bootswatch themes, custom colors, typography, Bootstrap Sass variables, custom Sass/CSS rules, dark mode, dynamic theming, and R plot theming +- **[shiny-reactivity](./shiny/shiny-reactivity/)** - Design and debug Shiny's reactive graph in R. Choose between reactive(), observe(), observeEvent(), and eventReactive(); manage state with reactiveValues; control execution with req(), bindEvent(), bindCache(), and debounce(); and debug with reactlog, tracebacks, and profvis +- **[shiny-testing](./shiny/shiny-testing/)** - Test Shiny apps in R across three layers: pure functions with testthat, reactive logic with shiny::testServer(), and end-to-end with shinytest2 (AppDriver, snapshots) +- **[shiny-for-agents](./shiny/shiny-for-agents/)** - Drive, inspect, and debug a running Shiny app (R and Python) with a browser tool (Playwright or agent-browser), reading server-side errors the browser can't see and authoring apps to be legible to agents ### Quarto diff --git a/shiny/README.md b/shiny/README.md index 331db31..a69a8d8 100644 --- a/shiny/README.md +++ b/shiny/README.md @@ -76,15 +76,44 @@ Comprehensive theming for Shiny apps using bslib and Bootstrap 5. Use when custo - [Bootswatch themes](https://bootswatch.com/) - [thematic package](https://rstudio.github.io/thematic/) +### `shiny-reactivity` + +Design and debug Shiny's reactive graph in R. Use when choosing between `reactive()`, `observe()`, `observeEvent()`, and `eventReactive()`; managing state with `reactiveValues`; controlling when things run with `req()`, `bindEvent()`, `bindCache()`, or `debounce()`; or debugging reactivity with reactlog, tracebacks, and profvis. + +**Organization**: Single self-contained SKILL.md covering reactive-graph design patterns, common anti-patterns (the observer-writes-reactiveValues trap), and a debugging workflow. + +**Resources**: +- [Mastering Shiny: Reactivity](https://mastering-shiny.org/reactivity-intro.html) +- [reactlog package](https://rstudio.github.io/reactlog/) + +### `shiny-testing` + +Test Shiny apps in R across three layers — pure functions with testthat, reactive logic with `shiny::testServer()`, and end-to-end with shinytest2 (`AppDriver`, snapshots). Use when writing or fixing tests for a Shiny app or module, testing reactives without a browser, setting up shinytest2, or deciding what to test where. + +**Organization**: Single self-contained SKILL.md organized by the three testing layers, cheapest first, with guidance on what to test where. + +**Resources**: +- [shinytest2 package](https://rstudio.github.io/shinytest2/) +- [Mastering Shiny: Testing](https://mastering-shiny.org/scaling-testing.html) + +### `shiny-for-agents` + +Drive, inspect, and debug a running Shiny app (R and Python) with a browser tool (Playwright or agent-browser). Use when launching a Shiny app for inspection, driving its UI without flaky sleeps, reading server-side errors the browser can't see, or authoring an app to be legible to agents. + +**Organization**: SKILL.md explains Shiny's split-runtime model and agent-driving workflow, plus helper scripts: +- `scripts/launch_app_dev.R` - Launch a Shiny app in a dev-friendly mode for inspection +- `scripts/wait_for_shiny_idle.js` - Wait on `shiny:busy`/`shiny:idle` events instead of fixed sleeps + +**Resources**: +- [Shiny JavaScript events](https://shiny.posit.co/r/articles/build/js-events/) + ## Potential Skills This category could include skills for: - Shiny app architecture and best practices -- Reactive programming patterns - UI/UX design for Shiny apps - Performance optimization -- Testing Shiny applications - Deployment strategies - Module development - Extension creation diff --git a/shiny/shiny-for-agents/SKILL.md b/shiny/shiny-for-agents/SKILL.md new file mode 100644 index 0000000..f758310 --- /dev/null +++ b/shiny/shiny-for-agents/SKILL.md @@ -0,0 +1,115 @@ +--- +name: shiny-for-agents +description: Drive, inspect, and debug a running Shiny app (R and Python) with a browser tool (Playwright or agent-browser). Use when launching a Shiny app for inspection, driving its UI without flaky sleeps, reading server-side errors the browser can't see, or authoring an app to be legible to agents. Triggers on driving or screenshotting a Shiny app, the shiny:busy/idle/error JS events, and server errors invisible to the browser. +metadata: + author: Samuel Bharti (@samuelbharti) + version: "1.0" +license: MIT +--- + +# Shiny for Agents + +Shiny is **split-runtime**: the UI (DOM, inputs/outputs) runs in the browser, but the reactive graph, all state, and the logic run in a separate R/Python **server process** over a websocket. Driving the browser (Playwright/agent-browser) shows you only the client half — the half that is *not* the source of truth. + +**What you can see today, and what you can't:** + +| Readable from the browser | NOT readable (lives in the server) | +|---|---| +| DOM, rendered output, input values (`Shiny.shinyapp.$inputValues`) | Current value of any reactive/expression/scoped variable | +| Busy/idle state, the `shiny:*` JS events | Why a reactive fired, the invalidation trace | +| Errors *if* devmode is on (see below) | Server errors/warnings (unless surfaced) | + +So: drive and assert from the browser, but get errors from the **server process output**, and design the app so its logic is testable without a browser at all. The deepest introspection (reading reactive values, tracing invalidation) needs Shiny-side support that doesn't exist yet — see the last section. + +## 0. Setup — install the best-practice tooling + +Install the recommended packages first; the no-install/manual path is a **fallback, not the default**. Use the project's env manager (`renv`, `uv`/`venv`), not global installs; confirm before system/sudo-level steps. + +- **R (install these):** `install.packages(c("shinytest2", "reactlog"))` — `shinytest2` is the official E2E tool (drives real Chrome via `chromote` with idle-waiting built in); `reactlog` is the reactive-graph debugger. See **`shiny-testing`** / **`shiny-reactivity`**. +- **Python (install these):** `uv add --dev pytest playwright pytest-playwright` then `playwright install chromium`; use the `shiny.playwright` controllers for E2E. +- **Interactive driving:** prefer the **Playwright MCP or agent-browser** tool when present (no install). *Alternate* — raw Playwright + the idle helper below. +- **Fallback only** (can't install / locked-down env): drive an already-running app with raw Playwright (§3) and scrape server stdout for errors (§4). On restricted Linux, the official `mcr.microsoft.com/playwright` image ships browsers + OS libs. + +## 1. Authoring for legibility + +Habits that make an app driveable and debuggable without instrumentation: + +- **Stable, explicit IDs.** Hand-write every `inputId`/`outputId` (`"country_filter"`). Never auto-generate them — your selectors and assertions depend on them. +- **Consistent modules.** One `moduleServer(id, ...)` per module, every child ID wrapped in `ns()`, so the DOM id is predictable: `#-`. +- **Logic in named reactives / pure functions, not inside `render*`** — so it's testable without a browser. See **`shiny-reactivity`** for reactive design and **`shiny-testing`** for the headless test layers. + +## 2. Run for inspection + +Launch once, fixed port, dev features on, randomness pinned. Use [scripts/launch_app_dev.R](scripts/launch_app_dev.R) or: + +```r +options(shiny.port = 7654, shiny.host = "127.0.0.1", shiny.fullstacktrace = TRUE) +shiny::devmode(TRUE) # client-side error console + dev defaults +set.seed(1); Sys.setenv(TZ = "UTC") +shiny::runApp("path/to/app", launch.browser = FALSE) +``` + +- Run it in the **background and capture stdout/stderr** — that stream is your source of truth for errors (§3). +- `launch.browser = FALSE` — *your* browser attaches, at `http://127.0.0.1:7654`. +- Determinism: fixed seed/timezone/fixtures so assertions don't flap. + +## 3. Drive the app — never `sleep`, wait on `shiny-busy` + +> Act → wait for the server to go idle → assert. **Never** `waitForTimeout`/`sleep`. + +This section is for **live interactive driving** (Playwright MCP / agent-browser against a running app). For anything you'd *commit as a test*, use `shinytest2` (R) or the `shiny.playwright` controllers (Python) — they encapsulate all of this; see **`shiny-testing`**. The helper below is the fallback for the no-wrapper case. + +Shiny adds the **`shiny-busy` class to ``** while the server works and removes it when idle — the same signal `shinytest2` waits on internally. Wait on that. Full helper (connect, idle-wait, error recorder): [scripts/wait_for_shiny_idle.js](scripts/wait_for_shiny_idle.js). + +```js +// idle = no `shiny-busy` on , stable for 250ms (absorbs the busy-not-yet-set race) +await page.waitForFunction((stableMs) => { + if (document.documentElement.classList.contains('shiny-busy')) { + window.__lastBusy = Date.now(); return false; + } + if (window.__lastBusy === undefined) return true; + return Date.now() - window.__lastBusy >= stableMs; +}, 250, { polling: 50, timeout: 30000 }); +``` + +The act → settle → assert loop: + +```js +await page.fill('#country_filter', 'Canada'); // ACT +await waitForShinyIdle(page); // SETTLE +const txt = await page.textContent('#summary'); // ASSERT +``` + +**Don't** gate idle on the `.recalculating` class globally — suspended/hidden outputs can keep it indefinitely (they don't render until shown), and some hide/`removeUI` sequences leave it stuck (rstudio/shiny#4114), so a global `.recalculating === 0` check can hang. Use it only scoped to one visible output: `page.waitForSelector('#plot:not(.recalculating)')`. + +**Settling races (important).** `waitForShinyIdle` can return *before* a slow round-trip even starts — the stable window elapses against the pre-action quiet period — so a read taken right after may be stale. Two robust fixes (both in the helper): +- **Asserting a change?** Wait for the change itself, not a fixed settle: `assertOutputChanged(page, '#plot img', action, {attr:'src'})` snapshots, acts, and resolves once the output differs. This is the reliable "did it update?" oracle and immune to render nondeterminism. +- **Need to settle, then read?** Use `actAndSettle(page, action)` — it waits for the server to *become busy* (or an output to enter `.recalculating`) before waiting for idle, closing the race. + +## 4. Read errors the browser can't show + +The real traceback is in the **server process stdout/stderr** (`shiny.fullstacktrace = TRUE`) — read it there first. With `devmode(TRUE)` the client also shows the error; catch *which* output failed via the `shiny:error` event (`e.name`, `e.error`). By default client errors are sanitized to "An error has occurred"; the real text is server-side. + +To go beyond observing — reproducing a server error or asserting on a reactive's value — reach for the headless test layers in **`shiny-testing`** (`testServer` for reactive logic, `shinytest2` for end-to-end), and **`shiny-reactivity`** for diagnosing *why* a reactive did or didn't fire. + +## 5. Shiny for Python + +The client JS is identical, so §3 (wait on `shiny-busy`) and the `shiny:*` events apply unchanged. Differences: launch with `shiny run --port 7654 --reload app.py`; server errors go to the `shiny run` process output; there is **no `testServer`** — write logic as pure Python functions and unit-test them; E2E uses `pytest` + `shiny.playwright` controllers (`from shiny.playwright import controller`), which wait on Shiny state for you. + +## 6. The wall, and how to work with it + +There is currently **no API to read a reactive's value or trace invalidation from outside the server** — no dev-mode reactive accessor, no machine-readable invalidation stream, no structured error object over the protocol (not even a `shiny:invalidated` JS event; `shiny:recalculating` lags invalidation). Everything above works *around* this split-runtime wall. So the most reliable way to inspect server state today is to not need to: put logic in named reactives / pure functions and exercise it headlessly (§1, §4) rather than trying to read it through the browser. + +## See also + +- **`shiny-reactivity`** — design and debug the reactive graph (reactlog, tracebacks, "why didn't this update"). +- **`shiny-testing`** — the headless test layers (`testServer`, `shinytest2`). +- **`shiny-bslib`** — building the UI. + +## References + +Fetch only for an edge case or to confirm a current API signature: + +- [Shiny JS events](https://shiny.posit.co/r/articles/build/js-events/) — full `shiny:*` event list + properties. +- [Shiny for Python: end-to-end testing](https://shiny.posit.co/py/docs/end-to-end-testing.html) — `shiny.playwright` controllers + pytest fixtures. +- [Playwright](https://playwright.dev/docs/intro) · [Playwright MCP](https://github.com/microsoft/playwright-mcp) — driving APIs and the MCP server setup. diff --git a/shiny/shiny-for-agents/scripts/launch_app_dev.R b/shiny/shiny-for-agents/scripts/launch_app_dev.R new file mode 100644 index 0000000..a6f2bd1 --- /dev/null +++ b/shiny/shiny-for-agents/scripts/launch_app_dev.R @@ -0,0 +1,30 @@ +#!/usr/bin/env Rscript +# launch_app_dev.R — launch a Shiny app for deterministic agent inspection. +# +# Usage: Rscript launch_app_dev.R [path/to/app] [port] +# Defaults: app ".", port 7654. Run in the background and CAPTURE its +# stdout/stderr — that stream is the source of truth for server errors. +# Your browser attaches to http://127.0.0.1:; no system browser opens. + +if (!requireNamespace("shiny", quietly = TRUE)) { + stop("Package 'shiny' is required. Install with install.packages('shiny').", call. = FALSE) +} + +args <- commandArgs(trailingOnly = TRUE) +app_dir <- if (length(args) >= 1) args[[1]] else "." +port <- if (length(args) >= 2) as.integer(args[[2]]) else 7654L + +options( + shiny.port = port, + shiny.host = "127.0.0.1", + shiny.fullstacktrace = TRUE, # full server traceback on error + shiny.sanitize.errors = FALSE # show real error text in dev +) +shiny::devmode(TRUE) # client-side error console + dev defaults + +# Determinism: pin anything the app might pull from the environment. +set.seed(1) +Sys.setenv(TZ = "UTC") + +message(sprintf("Launching '%s' on http://127.0.0.1:%d", app_dir, port)) +shiny::runApp(app_dir, launch.browser = FALSE) diff --git a/shiny/shiny-for-agents/scripts/wait_for_shiny_idle.js b/shiny/shiny-for-agents/scripts/wait_for_shiny_idle.js new file mode 100644 index 0000000..24bcf1b --- /dev/null +++ b/shiny/shiny-for-agents/scripts/wait_for_shiny_idle.js @@ -0,0 +1,171 @@ +// wait_for_shiny_idle.js — wait on Shiny's own settle signals; NEVER sleep. +// +// Shiny adds class `shiny-busy` to while the server is working and removes +// it when idle; each recalculating output carries class `.recalculating` until it +// re-renders. We wait until BOTH are clear and have *stayed* clear for `stableMs`, +// which absorbs (a) the race where we check before `shiny-busy` is added, and +// (b) multi-step reactive cascades. +// +// Works with any Playwright `page` (Playwright MCP, agent-browser, raw Playwright, +// or Shiny for Python — same client JS). For the Python `sync_api`, call +// page.wait_for_function(...) with the same body. + +/** + * Wait for the app to connect to the Shiny server. + * @param {import('playwright').Page} page + */ +async function waitForShinyConnected(page, { timeout = 30000 } = {}) { + await page.waitForFunction( + () => + typeof window.Shiny !== "undefined" && + window.Shiny.shinyapp && + typeof window.Shiny.shinyapp.isConnected === "function" && + window.Shiny.shinyapp.isConnected(), + undefined, + { timeout } + ); +} + +/** + * Wait until Shiny has been idle for `stableMs` continuous milliseconds. + * + * The GLOBAL idle signal is the `shiny-busy` class on (added while the + * server is working, removed when idle) — this is what shinytest2's + * wait_for_idle() keys on. Do NOT gate global idle on the `.recalculating` + * class: outputs on hidden tabs / suspended outputs keep `.recalculating` + * indefinitely (they never render until shown — see rstudio/shiny#4114), so a + * global `.recalculating === 0` check can hang forever. + * + * To wait for a SPECIFIC output to finish, pass `selector` — we additionally + * require that element to drop its `.recalculating` class. Scope it to a + * visible output, never use it as a global gate. + * + * CAVEAT: `window.__shinyLastBusy` persists on the page across calls and is + * never reset, so a bare second call right after an action can read a stale + * timestamp and report idle before the new round-trip even starts. For the + * "act, then read" pattern use `actAndSettle` (it waits for busy to appear + * first); for "assert it changed" use `assertOutputChanged`. + * + * @param {import('playwright').Page} page + * @param {{ stableMs?: number, timeout?: number, selector?: string }} [opts] + */ +async function waitForShinyIdle(page, { stableMs = 250, timeout = 30000, selector = null } = {}) { + await page.waitForFunction( + ({ stableMs, selector }) => { + const serverBusy = document.documentElement.classList.contains("shiny-busy"); + // Only consider .recalculating for an explicitly-scoped, visible output. + const targetBusy = + selector && document.querySelector(selector) + ? document.querySelector(selector).classList.contains("recalculating") + : false; + if (serverBusy || targetBusy) { + window.__shinyLastBusy = Date.now(); + return false; + } + if (window.__shinyLastBusy === undefined) return true; // never went busy => already idle + return Date.now() - window.__shinyLastBusy >= stableMs; // idle long enough to be settled + }, + { stableMs, selector }, + { polling: 50, timeout } + ); +} + +/** + * Install error/console recorders right after connect, so server-propagated + * output errors (otherwise invisible to the browser) can be read after an action. + * @param {import('playwright').Page} page + */ +async function installShinyErrorRecorder(page) { + await page.evaluate(() => { + if (window.__shinyErrors) return; // idempotent + window.__shinyErrors = []; + // jQuery is always present in a Shiny page. + $(document).on("shiny:error", (e) => + window.__shinyErrors.push({ + output: e.name, + message: e.error && e.error.message, + }) + ); + }); +} + +/** Read any errors captured since installShinyErrorRecorder(). */ +async function getShinyErrors(page) { + return page.evaluate(() => window.__shinyErrors || []); +} + +/** + * Act, then settle — robust against the premature-idle race. + * + * `waitForShinyIdle` alone can return BEFORE a slow round-trip even starts (the + * stable window elapses against the pre-action quiet period), so a read taken + * right after can be stale. This first waits for the server to actually BECOME + * busy (or any output to enter `.recalculating`), then waits for idle. If no + * work starts within `busyTimeout`, that's fine — the action triggered nothing. + * @param {import('playwright').Page} page + * @param {() => Promise} action + */ +async function actAndSettle(page, action, { busyTimeout = 4000, stableMs = 400, timeout = 30000 } = {}) { + await action(); + await page + .waitForFunction( + () => + document.documentElement.classList.contains("shiny-busy") || + document.querySelectorAll(".recalculating").length > 0, + undefined, + { timeout: busyTimeout } + ) + .catch(() => {}); // no work started — not an error + await waitForShinyIdle(page, { stableMs, timeout }); +} + +/** + * When you assert that an action CHANGES an output, wait for the change itself — + * don't settle-then-compare (that races the render). Snapshots the output's + * text/`src`, runs `action`, and resolves true once it differs (false on timeout). + * The most reliable "did this output update?" oracle. `attr` is "src" for plots + * (an ), omit for text outputs. + * @param {import('playwright').Page} page + * @param {string} selector e.g. "#myplot img" or "#summary" + * @param {() => Promise} action + */ +async function assertOutputChanged(page, selector, action, { attr = null, timeout = 15000 } = {}) { + await page.evaluate( + ([sel, attr]) => { + const el = document.querySelector(sel); + window.__before = el ? (attr ? el.getAttribute(attr) : el.textContent) : null; + }, + [selector, attr] + ); + await action(); + return page + .waitForFunction( + ([sel, attr]) => { + const el = document.querySelector(sel); + const now = el ? (attr ? el.getAttribute(attr) : el.textContent) : null; + return now != null && now !== window.__before; + }, + [selector, attr], + { polling: 100, timeout } + ) + .then(() => true) + .catch(() => false); +} + +// --- Typical usage ----------------------------------------------------------- +// await page.goto("http://127.0.0.1:7654"); +// await waitForShinyConnected(page); +// await installShinyErrorRecorder(page); +// await page.fill("#country_filter", "Canada"); // ACT +// await waitForShinyIdle(page); // SETTLE +// const txt = await page.textContent("#summary"); // ASSERT +// console.log(await getShinyErrors(page)); // any server-side output errors + +module.exports = { + waitForShinyConnected, + waitForShinyIdle, + actAndSettle, + assertOutputChanged, + installShinyErrorRecorder, + getShinyErrors, +}; diff --git a/shiny/shiny-reactivity/SKILL.md b/shiny/shiny-reactivity/SKILL.md new file mode 100644 index 0000000..25d0243 --- /dev/null +++ b/shiny/shiny-reactivity/SKILL.md @@ -0,0 +1,116 @@ +--- +name: shiny-reactivity +description: Design and debug Shiny's reactive graph in R. Use when choosing between reactive(), observe(), observeEvent(), and eventReactive(); managing state with reactiveValues; controlling when things run with req(), bindEvent(), bindCache(), or debounce(); or debugging reactivity with reactlog, tracebacks, and profvis. Triggers on reactive bugs, outputs not updating, reactives firing too often, invalidation, infinite loops, and the observer-writes-reactiveValues anti-pattern. +metadata: + author: Samuel Bharti (@samuelbharti) + version: "1.0" +license: MIT +--- + +# Reactive Design & Debugging in Shiny + +Most Shiny bugs are reactive-graph bugs: the wrong things run, run too often, or don't run at all. Write reactivity so the graph is obvious, then debug by inspecting the graph — not by reading code top to bottom. + +## The mental model + +Shiny builds a **dependency graph**: inputs → reactive expressions → outputs/observers. When an input changes, everything downstream is **invalidated** and lazily recomputed. You don't call reactives in order; you declare dependencies and Shiny schedules them. + +## The building blocks — pick the right one + +| Construct | Purpose | Eager/lazy | Returns a value? | +|---|---|---|---| +| `reactive(expr)` | **Compute a value** from inputs/other reactives | lazy + cached | yes — call it like `x()` | +| `eventReactive(event, expr)` | Computed value that updates **only** when `event` fires | lazy | yes | +| `observeEvent(event, handler)` | **Side effect** in response to an event | eager | no | +| `observe(expr)` | Side effect tracking all reactive reads in `expr` | eager | no | +| `reactiveVal()` / `reactiveValues()` | **Mutable state** you set explicitly | — | get/set | + +**Rules of thumb:** +- Use `reactive()` to *compute*; use `observe`/`observeEvent` only for *side effects* (writing files, updating inputs, showing notifications). A construct that returns a value you use elsewhere should almost always be a `reactive()`. +- A module's return value should be a `reactive()` (or a list of reactives), never a plain value. +- Keep reactives small and single-purpose — one job each makes the graph legible and avoids over-broad invalidation. + +## The #1 anti-pattern: observers that write state + +Writing to `reactiveValues` from an `observe` to feed a render **hides the dependency** from the graph — the thing this whole framework exists to make visible: + +```r +# BAD — the nrows -> df edge is invisible; harder to trace, test, and reason about +observe({ r$df <- head(cars, input$nrows) }) +output$plot <- renderPlot(plot(r$df)) + +# GOOD — the dependency is explicit in the graph +df <- reactive(head(cars, input$nrows)) +output$plot <- renderPlot(plot(df())) +``` + +Use `reactiveValues` only when you genuinely need mutable state that several events update (accumulators, "either button sets this", pause/resume). When an observer must update a value it also reads, wrap the read in `isolate()` to avoid an infinite invalidation loop. + +`isolate(x())` reads a reactive's current value **without** taking a dependency on it — the tool for "use this value but don't re-run when it changes." + +## Control when things run + +- **`req(x)`** — stop silently (no output, no error) until `x` is truthy. The right gate for "don't compute until the user has chosen something." A stray `req()` is the most common cause of "output never appears." +- **`validate(need(cond, "message"))`** — like `req()` but shows the user a message. For richer, field-level validation install **`shinyvalidate`** (`install.packages("shinyvalidate")`) and use its `InputValidator`. +- **`x |> bindEvent(input$go)`** — make a reactive/render/observer respond **only** to `input$go` (the modern, composable replacement for `eventReactive`/`observeEvent`'s event arg). Pair with `ignoreInit`/`ignoreNULL` as needed. +- **`x |> bindCache(input$a, input$b)`** — cache results by key; huge speedups for slow computations, shareable across sessions. **Caveat:** the cache key determines dependencies — `reactive({input$x + input$y}) |> bindCache(input$x)` will *not* recompute when only `input$y` changes. Pair `bindCache()` then `bindEvent()` for slow, on-demand work. +- **`debounce()` / `throttle()`** — rate-limit a fast-changing reactive (typing, sliders) so downstream work doesn't fire on every keystroke. + +## Debug the reactive graph + +**See the graph — `reactlog`** (the canonical tool — install it: `install.packages("reactlog")`): + +```r +reactlog::reactlog_enable() # or options(shiny.reactlog = TRUE), before launching +shiny::runApp("app") +# In the running app press Ctrl+F3 (Cmd+F3 on Mac), or after stopping: +shiny::reactlogShow() +``` + +It shows every reactive, its dependencies, and what invalidated what — the direct answer to "why did this (not) update." Dev-only: it exposes reactive source and grows unbounded; never enable in production. + +**See the error — tracebacks:** + +```r +options(shiny.fullstacktrace = TRUE) # full traceback to the console on error +options(shiny.sanitize.errors = FALSE) # show real error text (dev only) +``` + +The real traceback is in the **R console / server stdout**, not the browser (the browser shows a sanitized "An error has occurred"). Drop `browser()` into a reactive to step through it; use `message()`/`cat(file = stderr())` for lightweight tracing without perturbing the graph. + +**Find the slow part — `profvis`:** + +```r +profvis::profvis(shiny::runApp("app")) # interact, then read the flame graph +``` + +## Symptom → fix + +**"Output didn't update."** +1. Is there a `req()`/`validate()` short-circuiting it? Check the gate. +2. Open `reactlog` — does the output actually depend on the input you changed? A missing edge means the value was read in `isolate()`, via the observer-write anti-pattern, or under the wrong ID. +3. Frozen by `freezeReactiveValue()` / an `eventReactive` waiting on a different trigger? + +**"Reactive fires too often / app is slow."** +1. In `reactlog`, find the over-broad dependency — a reactive reading a whole `input`/`reactiveValues` object instead of one field. +2. Split the reactive; depend only on what's needed; gate with `bindEvent()`; rate-limit with `debounce()`; cache with `bindCache()`. +3. Confirm with `profvis`. + +**"An error has occurred."** +1. Read the **console traceback** (`shiny.fullstacktrace = TRUE`) — the browser message is sanitized. +2. Reproduce the failing reactive in `testServer` (see `shiny-testing`) to get a plain R traceback. + +## See also + +- **`shiny-testing`** — test reactives headlessly (`testServer`) and end-to-end (`shinytest2`). +- **`shiny-for-agents`** — drive/inspect a running app from a browser without flaky sleeps. +- **`shiny-bslib`** — building the UI. + +## References + +Fetch only for an edge case or to confirm a signature: + +- [Mastering Shiny — Reactivity](https://mastering-shiny.org/reactivity-intro.html) and [The reactive graph](https://mastering-shiny.org/reactive-graph.html). +- [`bindEvent` / `bindCache`](https://rstudio.github.io/shiny/reference/bindEvent.html) and [caching guide](https://shiny.posit.co/r/articles/improve/caching/). +- [shinyvalidate](https://rstudio.github.io/shinyvalidate/) — input validation. +- [reactlog](https://rstudio.github.io/reactlog/) — reactive graph visualizer. diff --git a/shiny/shiny-testing/SKILL.md b/shiny/shiny-testing/SKILL.md new file mode 100644 index 0000000..66fd6c7 --- /dev/null +++ b/shiny/shiny-testing/SKILL.md @@ -0,0 +1,117 @@ +--- +name: shiny-testing +description: Test Shiny apps in R across three layers — pure functions with testthat, reactive logic with shiny::testServer(), and end-to-end with shinytest2 (AppDriver, snapshots). Use when writing or fixing tests for a Shiny app or module, testing reactives without a browser, setting up shinytest2, or deciding what to test where. Triggers on testServer, shinytest2, AppDriver, snapshot tests, testing modules, and flaky Shiny tests. +metadata: + author: Samuel Bharti (@samuelbharti) + version: "1.0" +license: MIT +--- + +# Testing Shiny Apps + +Test in three layers, cheapest first. Most logic should be covered by the fast lower layers; reserve the slow browser layer for what genuinely needs rendering. + +| Layer | Tool | Tests | Speed | +|---|---|---|---| +| 1. Pure functions | `testthat` | computation with no reactivity | fastest | +| 2. Reactive logic | `shiny::testServer()` | reactives, observers, module server logic | fast (no browser) | +| 3. End-to-end | `shinytest2` | rendering, JS, full round-trip, visuals | slow (headless Chrome) | + +**Design for testability first:** push computation into plain functions (Layer 1) and named reactives (Layer 2). Logic buried inside `renderPlot({...})` can only be reached through the slow Layer 3. + +This skill is **R-specific**. For Shiny for Python there's no `testServer` — unit-test pure functions, and use `pytest` + `shiny.playwright` controllers for E2E (see **`shiny-for-agents` §5**). + +## Layer 1 — pure functions (testthat) + +Extract non-reactive computation into ordinary functions and test them directly. No Shiny needed. + +```r +test_that("filtering selects the chosen country", { + out <- filter_country(sample_data, "Canada") + expect_equal(unique(out$country), "Canada") +}) +``` + +For testthat structure, fixtures, mocking, and snapshots, see the **`testing-r-packages`** skill — it all applies here. + +## Layer 2 — reactive logic (testServer) + +`testServer()` runs the server function in a simulated session, with no browser. Set inputs, then read reactives and outputs directly. + +```r +test_that("summary reacts to the country filter", { + testServer(myAppServer, { + session$setInputs(country_filter = "Canada") # set inputs + expect_equal(filtered_data()$country[1], "Canada") # read a named reactive + expect_match(output$summary, "Canada") # read an output + session$setInputs(country_filter = "Brazil") # multiple steps in one test + expect_match(output$summary, "Brazil") + }) +}) +``` + +- `session$setInputs()` exists only inside `testServer`; it sets inputs and flushes reactivity synchronously — no idle-waiting needed. +- Test a **module**: pass args to its server. `testServer(myModuleServer, args = list(id = "x", data = reactive(df)), { ... })`. +- **Limitation:** `render*` outputs that need a client (e.g. `renderPlot` sizing) don't fully execute headlessly — test the *named reactive* feeding the render, not the rendered image. That's another reason to keep logic out of `render*`. + +### Expose internals for snapshots — `exportTestValues()` + +Register an internal reactive so the **end-to-end** layer can assert on it (active only under `shiny.testmode`, which `shinytest2` sets automatically): + +```r +exportTestValues(result = filtered_data()) # in the server function +``` + +Then in `shinytest2`: `app$get_value(export = "result")`, and exports are included in `app$expect_values()`. In `testServer` you usually don't need this — read reactives directly; use `session$getReturned()` to assert a **module's return value**. + +## Layer 3 — end-to-end (shinytest2) + +**Install it — this is the best-practice E2E path** (prefer it over hand-driving a browser, which is `shiny-for-agents`' fallback): `install.packages("shinytest2")` (pulls `chromote`), then scaffold with `shinytest2::use_shinytest2()`. + +`AppDriver` drives a real app in headless Chrome (via chromote), with idle-waiting built in. + +```r +library(shinytest2) +test_that("app renders the filtered summary", { + app <- AppDriver$new("path/to/app", name = "country-filter", seed = 1) + app$set_inputs(country_filter = "Canada") # auto-waits for the resulting change + app$wait_for_idle() # extra settle for chained reactives + app$expect_values() # JSON snapshot of inputs/outputs/exports + app$expect_screenshot() # visual snapshot (review on first run) +}) +``` + +Key `AppDriver` methods: `set_inputs()`, `wait_for_idle()`, `wait_for_value()`, `get_value(input=/output=/export=)`, `get_values()`, `get_logs()`, `expect_values()`, `expect_screenshot()`, `run_js()`. + +- **Prefer `expect_values()` over `expect_screenshot()`** for data — value snapshots are stable; pixels flap across machines/fonts. +- Review snapshot changes with `testthat::snapshot_review()`. +- The driver waits on Shiny's own busy/idle signal, so you never write sleeps. (For driving a *live* app outside shinytest2, see **`shiny-for-agents`**.) + +## Test behavior, not wiring + +Tests break when they assert *implementation* (internal IDs, intermediate values, exact HTML) instead of *behavior*. Assert what the user observes — the output value, the visible state — so refactors don't break green tests. Snapshot the result, not the plumbing. + +## Determinism (or tests flake) + +- Pass `seed =` to `AppDriver`; `set.seed()` for Layer 1/2. +- Use fixed fixtures, not live data sources; freeze time (`Sys.setenv(TZ=...)`, mock `Sys.time()`). +- One assertion target per behavior; avoid asserting on timing. + +## Running tests + +- App-as-a-package: `devtools::test()` (tests in `tests/testthat/`). +- App directory: `shiny::runTests()` runs the app's top-level test runners in `tests/` (e.g. `tests/testthat.R`), which in turn run the testthat and shinytest2 tests. +- shinytest2 tests live in `tests/testthat/` as `test-*.R` using `AppDriver`. + +## See also + +- **`testing-r-packages`** — testthat 3 patterns (structure, fixtures, mocking, snapshots) underlying Layer 1. +- **`shiny-reactivity`** — design reactives so they're testable; debug failures. +- **`shiny-for-agents`** — drive/inspect a running app from a browser. +- **`review-testing`** — review existing tests for quality and coverage. + +## References + +- [Shiny testing overview](https://shiny.posit.co/r/articles/improve/testing-overview/) and [server-function testing](https://shiny.posit.co/r/articles/improve/server-function-testing/). +- [`testServer`](https://shiny.posit.co/r/reference/shiny/latest/testserver.html) · [shinytest2 `AppDriver`](https://rstudio.github.io/shinytest2/reference/AppDriver.html). +- [Mastering Shiny — Testing](https://mastering-shiny.org/scaling-testing.html). From 919e21378d68c28fecbe0bf714ddf5de3144d478 Mon Sep 17 00:00:00 2001 From: Samuel Bharti Date: Mon, 29 Jun 2026 03:01:52 -0600 Subject: [PATCH 2/2] =?UTF-8?q?docs(shiny):=20address=20PR=20review=20?= =?UTF-8?q?=E2=80=94=20wording,=20uv/chromote,=20drop=20=C2=A7=20refs?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Apply the split-independent review feedback from #67: - shiny-reactivity: move mutable-state row to top of the table; prefer a list of reactiveVal() over a reactiveValues() object (legibility); reword the module-return rule; use reactiveVal() in the anti-pattern example. - shiny-testing: "cheapest" -> "fastest first"; "snapshot app author values, not the plumbing". - shiny-for-agents: `uv run playwright install`; note R has no Playwright (use chromote); stable-ID example `id = "..."`. - Correct the §6 claim — reactive values *can* be exposed opt-in via shiny.testmode/exportTestValues (and py-shiny#2270); only arbitrary no-opt-in access and an invalidation trace are still missing. - Remove all `§` section cross-references (reader-facing, not internal shorthand) in favor of section names. --- shiny/shiny-for-agents/SKILL.md | 15 ++++++++------- shiny/shiny-reactivity/SKILL.md | 18 ++++++++++-------- shiny/shiny-testing/SKILL.md | 6 +++--- 3 files changed, 21 insertions(+), 18 deletions(-) diff --git a/shiny/shiny-for-agents/SKILL.md b/shiny/shiny-for-agents/SKILL.md index f758310..dcc67b6 100644 --- a/shiny/shiny-for-agents/SKILL.md +++ b/shiny/shiny-for-agents/SKILL.md @@ -26,15 +26,16 @@ So: drive and assert from the browser, but get errors from the **server process Install the recommended packages first; the no-install/manual path is a **fallback, not the default**. Use the project's env manager (`renv`, `uv`/`venv`), not global installs; confirm before system/sudo-level steps. - **R (install these):** `install.packages(c("shinytest2", "reactlog"))` — `shinytest2` is the official E2E tool (drives real Chrome via `chromote` with idle-waiting built in); `reactlog` is the reactive-graph debugger. See **`shiny-testing`** / **`shiny-reactivity`**. -- **Python (install these):** `uv add --dev pytest playwright pytest-playwright` then `playwright install chromium`; use the `shiny.playwright` controllers for E2E. -- **Interactive driving:** prefer the **Playwright MCP or agent-browser** tool when present (no install). *Alternate* — raw Playwright + the idle helper below. -- **Fallback only** (can't install / locked-down env): drive an already-running app with raw Playwright (§3) and scrape server stdout for errors (§4). On restricted Linux, the official `mcr.microsoft.com/playwright` image ships browsers + OS libs. +- **Python (install these):** `uv add --dev pytest playwright pytest-playwright` then `uv run playwright install chromium`; use the `shiny.playwright` controllers for E2E. +- **Browser driver:** Playwright is JS/Python; **R has no Playwright** — drive R apps from `chromote` (which `shinytest2` already uses) or from a Playwright/agent-browser process pointed at the running app over HTTP. The `shiny-busy` idle signal (below) is identical either way. +- **Interactive driving:** prefer the **Playwright MCP or agent-browser** tool when present (no install). *Alternate* — raw Playwright (Python) / `chromote` (R) + the idle helper below. +- **Fallback only** (can't install / locked-down env): drive an already-running app and scrape server stdout for errors (see *Read errors the browser can't show*). On restricted Linux, the official `mcr.microsoft.com/playwright` image ships browsers + OS libs. ## 1. Authoring for legibility Habits that make an app driveable and debuggable without instrumentation: -- **Stable, explicit IDs.** Hand-write every `inputId`/`outputId` (`"country_filter"`). Never auto-generate them — your selectors and assertions depend on them. +- **Stable, explicit IDs.** Hand-write every `inputId`/`outputId` (e.g. `id = "country_filter"`). Never auto-generate them — your selectors and assertions depend on them. - **Consistent modules.** One `moduleServer(id, ...)` per module, every child ID wrapped in `ns()`, so the DOM id is predictable: `#-`. - **Logic in named reactives / pure functions, not inside `render*`** — so it's testable without a browser. See **`shiny-reactivity`** for reactive design and **`shiny-testing`** for the headless test layers. @@ -49,7 +50,7 @@ set.seed(1); Sys.setenv(TZ = "UTC") shiny::runApp("path/to/app", launch.browser = FALSE) ``` -- Run it in the **background and capture stdout/stderr** — that stream is your source of truth for errors (§3). +- Run it in the **background and capture stdout/stderr** — that stream is your source of truth for server errors. - `launch.browser = FALSE` — *your* browser attaches, at `http://127.0.0.1:7654`. - Determinism: fixed seed/timezone/fixtures so assertions don't flap. @@ -94,11 +95,11 @@ To go beyond observing — reproducing a server error or asserting on a reactive ## 5. Shiny for Python -The client JS is identical, so §3 (wait on `shiny-busy`) and the `shiny:*` events apply unchanged. Differences: launch with `shiny run --port 7654 --reload app.py`; server errors go to the `shiny run` process output; there is **no `testServer`** — write logic as pure Python functions and unit-test them; E2E uses `pytest` + `shiny.playwright` controllers (`from shiny.playwright import controller`), which wait on Shiny state for you. +The client JS is identical, so the idle-waiting approach (wait on `shiny-busy`) and the `shiny:*` events apply unchanged. Differences: launch with `shiny run --port 7654 --reload app.py`; server errors go to the `shiny run` process output; there is **no `testServer`** — write logic as pure Python functions and unit-test them; E2E uses `pytest` + `shiny.playwright` controllers (`from shiny.playwright import controller`), which wait on Shiny state for you. ## 6. The wall, and how to work with it -There is currently **no API to read a reactive's value or trace invalidation from outside the server** — no dev-mode reactive accessor, no machine-readable invalidation stream, no structured error object over the protocol (not even a `shiny:invalidated` JS event; `shiny:recalculating` lags invalidation). Everything above works *around* this split-runtime wall. So the most reliable way to inspect server state today is to not need to: put logic in named reactives / pure functions and exercise it headlessly (§1, §4) rather than trying to read it through the browser. +You **can** expose chosen reactive values for inspection by opting in: under `shiny.testmode`, `exportTestValues()` (R) registers internals that `shinytest2` reads via `app$get_value(export=)` (Shiny for Python is gaining an equivalent — posit-dev/py-shiny#2270). What there's still **no** API for is reading *arbitrary* reactive state without that opt-in, or getting a machine-readable invalidation trace (not even a `shiny:invalidated` JS event; `shiny:recalculating` lags invalidation). So the most reliable way to inspect server state today is to design for it: put logic in named reactives / pure functions and exercise it headlessly (see *Authoring for legibility* above and **`shiny-testing`**) rather than trying to read it through the browser. ## See also diff --git a/shiny/shiny-reactivity/SKILL.md b/shiny/shiny-reactivity/SKILL.md index 25d0243..6781c6d 100644 --- a/shiny/shiny-reactivity/SKILL.md +++ b/shiny/shiny-reactivity/SKILL.md @@ -19,15 +19,16 @@ Shiny builds a **dependency graph**: inputs → reactive expressions → outputs | Construct | Purpose | Eager/lazy | Returns a value? | |---|---|---|---| +| `reactiveVal()` / `reactiveValues()` | **Mutable state** you set explicitly | — | get/set | | `reactive(expr)` | **Compute a value** from inputs/other reactives | lazy + cached | yes — call it like `x()` | | `eventReactive(event, expr)` | Computed value that updates **only** when `event` fires | lazy | yes | | `observeEvent(event, handler)` | **Side effect** in response to an event | eager | no | | `observe(expr)` | Side effect tracking all reactive reads in `expr` | eager | no | -| `reactiveVal()` / `reactiveValues()` | **Mutable state** you set explicitly | — | get/set | **Rules of thumb:** - Use `reactive()` to *compute*; use `observe`/`observeEvent` only for *side effects* (writing files, updating inputs, showing notifications). A construct that returns a value you use elsewhere should almost always be a `reactive()`. -- A module's return value should be a `reactive()` (or a list of reactives), never a plain value. +- To inspect a module's reactive state, have the module **return** a `reactive()` (or a list of reactives), never a plain value. +- Prefer a **list of `reactiveVal()`** over a single `reactiveValues()` object for explicit state. Get/set on a `reactiveVal()` is visibly a function call (`state$count()` to read, `state$count(n)` to set), whereas `rv$count <- n` reads like ordinary R assignment and hides that it's reactive — easy to misread. - Keep reactives small and single-purpose — one job each makes the graph legible and avoids over-broad invalidation. ## The #1 anti-pattern: observers that write state @@ -35,16 +36,17 @@ Shiny builds a **dependency graph**: inputs → reactive expressions → outputs Writing to `reactiveValues` from an `observe` to feed a render **hides the dependency** from the graph — the thing this whole framework exists to make visible: ```r -# BAD — the nrows -> df edge is invisible; harder to trace, test, and reason about -observe({ r$df <- head(cars, input$nrows) }) -output$plot <- renderPlot(plot(r$df)) +# BAD — the nrows -> data edge is invisible; harder to trace, test, and reason about +data <- reactiveVal() +observe({ data(head(cars, input$nrows)) }) +output$plot <- renderPlot(plot(data())) # GOOD — the dependency is explicit in the graph -df <- reactive(head(cars, input$nrows)) -output$plot <- renderPlot(plot(df())) +data <- reactive(head(cars, input$nrows)) +output$plot <- renderPlot(plot(data())) ``` -Use `reactiveValues` only when you genuinely need mutable state that several events update (accumulators, "either button sets this", pause/resume). When an observer must update a value it also reads, wrap the read in `isolate()` to avoid an infinite invalidation loop. +Use `reactiveVal()`/`reactiveValues()` only when you genuinely need mutable state that several events update (accumulators, "either button sets this", pause/resume). When an observer must update a value it also reads, wrap the read in `isolate()` to avoid an infinite invalidation loop. `isolate(x())` reads a reactive's current value **without** taking a dependency on it — the tool for "use this value but don't re-run when it changes." diff --git a/shiny/shiny-testing/SKILL.md b/shiny/shiny-testing/SKILL.md index 66fd6c7..6b07435 100644 --- a/shiny/shiny-testing/SKILL.md +++ b/shiny/shiny-testing/SKILL.md @@ -9,7 +9,7 @@ license: MIT # Testing Shiny Apps -Test in three layers, cheapest first. Most logic should be covered by the fast lower layers; reserve the slow browser layer for what genuinely needs rendering. +Test in three layers, fastest first. Most logic should be covered by the fast lower layers; reserve the slow browser layer for what genuinely needs rendering. | Layer | Tool | Tests | Speed | |---|---|---|---| @@ -19,7 +19,7 @@ Test in three layers, cheapest first. Most logic should be covered by the fast l **Design for testability first:** push computation into plain functions (Layer 1) and named reactives (Layer 2). Logic buried inside `renderPlot({...})` can only be reached through the slow Layer 3. -This skill is **R-specific**. For Shiny for Python there's no `testServer` — unit-test pure functions, and use `pytest` + `shiny.playwright` controllers for E2E (see **`shiny-for-agents` §5**). +This skill is **R-specific**. For Shiny for Python there's no `testServer` — unit-test pure functions, and use `pytest` + `shiny.playwright` controllers for E2E (see the *Shiny for Python* section of **`shiny-for-agents`**). ## Layer 1 — pure functions (testthat) @@ -89,7 +89,7 @@ Key `AppDriver` methods: `set_inputs()`, `wait_for_idle()`, `wait_for_value()`, ## Test behavior, not wiring -Tests break when they assert *implementation* (internal IDs, intermediate values, exact HTML) instead of *behavior*. Assert what the user observes — the output value, the visible state — so refactors don't break green tests. Snapshot the result, not the plumbing. +Tests break when they assert *implementation* (internal IDs, intermediate values, exact HTML) instead of *behavior*. Assert what the user observes — the output value, the visible state — so refactors don't break green tests. Snapshot app author values, not the plumbing. ## Determinism (or tests flake)