[miniflare] Local dev observability: traces, spans & logs via a SQL query API#14633
Open
nickpatt wants to merge 6 commits into
Open
[miniflare] Local dev observability: traces, spans & logs via a SQL query API#14633nickpatt wants to merge 6 commits into
nickpatt wants to merge 6 commits into
Conversation
…lder collector) Experimental. Add the X_LOCAL_OBSERVABILITY env var (read by both wrangler and the Vite plugin — there is no CLI flag or plugin option), which is translated into a single `unsafeObservability` Miniflare option. When set, Miniflare core attaches an internal trace collector as a streaming-tail consumer of each user worker and adds the compat flags workerd needs to emit that tail — done once in core so the dev tools don't duplicate the wiring. It defaults to off while experimental. The collector is a placeholder for now (no-op tailStream); capturing spans/logs and persisting them to an internal store land in follow-up PRs.
…ad API Adds the internal SQLite-backed `TraceStore` Durable Object (hosted by the collector worker) that owns captured spans + logs, plus its sole read surface: a single read-only `POST /query` SQL endpoint. `query()` binds params, allows only one SELECT/WITH statement (rejecting writes, DDL, PRAGMA, ATTACH, and multi-statement), and caps returned rows. The spans/logs schema is the read contract. tailStream is still a placeholder here (capture lands next).
🦋 Changeset detectedLatest commit: 1e390e7 The changes in this PR will be included in the next version bump. This PR includes changesets to release 6 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
5 tasks
workers-devprod
requested review from
a team and
emily-shen
and removed request for
a team
Contributor
|
Codeowners approval required for this PR:
Show detailed file reviewers
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Wires the local observability collector to capture: it folds each user worker's streaming tail straight into spans + logs — using the Workers Observability attribute conventions (faas.trigger, http.request.method, http.response.status_code, cloudflare.outcome, cpu_time_ms, …) so a local waterfall matches production — and persists them to the internal TraceStore Durable Object. Every invocation emits a synthetic log (so silent workers still show up), console.* output is captured, and uncaught exceptions surface as both a span error and an error-level log. No OpenTelemetry SDK/transport is needed locally, so there are no added runtime dependencies.
…orker Local observability already captures every user worker (the collector is attached to each one), and a request that crosses a service binding is stitched into a single distributed trace. This adds per-worker attribution: each captured span now records the worker (service) that produced it, so multi-worker setups — and cross-worker distributed traces — can be grouped and filtered by worker. Miniflare passes each worker's name to the collector via binding props (workerd doesn't surface it on the tail locally), and the trace store gains a service column plus a service_count per trace in the list view.
…cal Explorer HTTP API Adds a single read endpoint to the Local Explorer API: POST /observability/query, which proxies to the collector's read-only SQL /query. The spans/logs schema and json(attributes) note are published in the endpoint's OpenAPI description so agents can discover the columns. No typed per-view routes — the query surface is the API, mirroring the direction of the production Observability API.
… views
Adds the Observability section to the Local Explorer UI: a traces list and a trace detail view (spans + logs). Data is read via canned SQL sent to POST /observability/query; the UI owns the queries and maps the {columns, rows} result into local Span/Log/TraceSummary shapes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds experimental local-dev observability to Miniflare: each local Worker invocation is captured as a trace (spans + logs) and surfaced in the Local Explorer, read entirely through a single read-only SQL endpoint. This is the clean, SQL-query-only rewrite of the earlier exploration in #14391, which it supersedes.
Architecture
faas.trigger,http.request.method,http.response.status_code,cloudflare.outcome,cpu_time_ms, …) so a local waterfall matches production.TraceStore).POST /cdn-cgi/explorer/api/observability/query— one read-onlySELECT/WITHstatement,paramsbound, rows capped. Thespans/logsschema is the contract and is published in the endpoint's OpenAPI description, so the UI and coding agents read the same way. No purpose-built per-view routes — the query surface is the API, mirroring the direction of the production Observability API.Reviewing
The 6 commits are the review units, in order:
[miniflare]opt-in switch + core wiring (placeholder collector)[miniflare]internal SQLite trace store + SQL read API[miniflare]capture traces into the SQLite store[miniflare]attribute captured traces to their worker[miniflare]expose the SQL query API over the Local Explorer HTTP API[local-explorer-ui]traces list & trace-detail views — also flips the default on (see below)The final commit flips
X_LOCAL_OBSERVABILITYdefaultfalse → true, so once landed, local-dev capture is on by default under bothwrangler devand the Vite plugin (a collector DO + streaming-tail capture + SQLite writes per session). SetX_LOCAL_OBSERVABILITY=falseto opt out. Happy to split this flip into its own commit, or keep it off for the initial landing, if reviewers prefer.Scope / not included
Cross-process trace propagation (spans across separate
wrangler devprocesses) is not included: workerd doesn't carry trace context across the dev-registry proxy hop, so a distributed trace only stitches within a single process today. Single-process multi-worker (service bindings in one dev session) is captured and stitched.X_LOCAL_OBSERVABILITY; user-facing docs will follow when it stabilises.