canopy: M4 — Query console + result views, E2E framework redesign, WHERE coverage#1224
Open
hanahmily wants to merge 55 commits into
Open
canopy: M4 — Query console + result views, E2E framework redesign, WHERE coverage#1224hanahmily wants to merge 55 commits into
hanahmily wants to merge 55 commits into
Conversation
Bring up the M4 query console: server-side no-cache headers, an
m4-seed CLI that installs the canonical service_cpm_minute measure
under the sw_metric group, a QueryConsole page that wires the
BanyanDB HTTP API end-to-end (groups → schema → query), the four
result views (Measure / Stream / Trace / Top-N) with shared
proto-decoding, and a no-skip pixel-regression gate (handoff capture
+ chrome baseline) to lock the visual chrome.
Wiring:
- canopy/server/src/plugins/static.ts: no-cache on HTML and assets
so /query always renders the latest dist after a rebuild.
- cmd/m4-seed/main.go: install service_cpm_minute under sw_metric,
rename m4-measure → sw_metric so the query console's first IN
pick matches the handoff.
- canopy/shared/src/api-dto.ts + canopy/web/src/data/api.ts: Query
proto mapping (criteria, group-by, top-n, trace) and Measure
/Stream /Trace /Top-N response shapes consumed by hooks and
result views.
- canopy/web/src/data/fixtures/query/*: four monorepo wire-shape
fixtures + a fixture test that locks the BanyanDB JSON shape
(no normalization).
- canopy/web/src/query/{bydbql,proto-decoder,role-infer,CodeEditor,
TraceDecoderModal}.ts: BydbQL builder state + compiler, raw
proto-byte decoder, role inference, code-mode editor, trace
decoder modal.
- canopy/web/src/query/QueryConsole.tsx: rail + resizer + result
panel + Builder/Code segmented toggle + page-header chrome
(title left, Run right, kbd hint).
- canopy/e2e/playwright.config.ts + m4-{query,query-chrome,
handoff-capture}.spec.ts: Playwright config + M4 query test +
two no-skip pixel gates (chrome empty-result / sidebar / page-
header, plus handoff screenshots for query-builder and the four
result views).
- implement-m4-note.md: design notes captured during the M4 build
(decisions, tradeoffs, anything not in the spec).
Co-located test suites (bydbql.test.ts, proto-decoder.test.ts,
role-infer.test.ts, result-views.test.tsx, fixtures.test.ts) cover
the builder compiler, decoder, role inference, result views, and
fixture wire shapes — no global state, no mocks, real BanyanDB
shapes end-to-end.
Iterate the QueryBuilder clauses and the qb-foot footer to match the handoff silhouettes, then wire the footer controls. Clauses (FROM / SELECT / WHERE / TIME / GROUP BY / ORDER BY / TOP-N): - 2-column clause layout: keyword on the left, body on the right, with clause content pinned to the bottom of the rail. - FROM: search input + icon catalog tabs (Measure / Stream / Trace / Top-N), MEASURE/IN/IN select row, IN select loads live groups. - SELECT: tag chips, all-tags sentinel, fields aggregation rows with fn / of / field, add-field. - WHERE: nested group tree (AND/OR connectors, recursive conditions), Conditions group tag, Add condition / Add group buttons, remove-condition button, value input, optional sub-label that sits next to WHERE on the same row, fields-pattern sub-label. - TIME: qb-time tablist (Absolute range / Relative) with calendar/clock icons, datetime-local inputs with step=1, FROM < TO validation, relative-time select, clear-range close icon. - GROUP BY: no-grouping chip + per-tag chips (measures only). - ORDER BY: order field + DESC/ASC segmented control + LIMIT number input. qb-foot footer (matches the handoff): - BydbQL pill + generated query preview (ellipsis on overflow). - "Edit as code" qb-eject button → switches to Code mode with the current builder state. - Trace toggle with qb-trace-dot (turns green / glows when on); replaces the inline WITH QUERY_TRACE checkbox. - Reset qb-btn-ghost (icon-only) wired to setState(defaultState(groups)) + clear code/codeDirty/status/ errorMsg/response — full state reset, not just WHERE. - Run qb-btn-primary with Ctrl↵/⌘↵ kbd hint. QueryConsole chrome: - Remove the duplicate "Eject to code" button in qb-rail-foot; remove the page-header Run button. The qb-foot (builder) and a matching rail-foot Run (code mode, with the same qb-btn / kbd hidden via .qb-rail-foot CSS) are now the only Run controls. - .qb-rail-foot layout: flex wrap, nowrap labels, primary button right-aligned via margin-left: auto, kbd hidden to keep the narrow-rail width inside the 348px builder rail. bydbql: small companion tweak to qbNewGroup.
The code-mode card used markup classes that had no CSS, placed the
"← Builder" back button in the footer instead of the toolbar, hid the
Run kbd hint behind `.qb-rail-foot .qb-kbd { display: none }`, and
double-rendered the code area because a syntax-highlight <pre> sat
as a flex sibling of the textarea. Rename markup to the handoff class
names, add a `toolbarRight` slot on CodeEditor for the back button,
switch the footer to `qb-foot` so the handoff Re-sync + Run silhouette
applies, and drop the <pre>.
Add a subsequence fuzzy match (contiguous-run + word-boundary scoring, group-name match weighted 0.4x) plus a QBResourceSearch component with the handoff's input + clear button + dropdown UI, keyboard navigation, and <mark>-based highlight rendering. QueryConsole prefetches resources for every group on initial load so the search spans every catalog/group, not just the currently-selected one. Search keystrokes themselves are pure client-side (zero API calls); the mount-time prefetch is the only upstream load.
- Collapse query builder clauses into accordion summaries after the first run. - Add paging, exec-time tracking, default -24h range, and result panels to QueryConsole. - Fix BydbQL TIME / SELECT projection generation and flatten camelCase protojson values. - Introduce QueryTraceSpan DTO and grouped/raw/export trace views. - Adjust m4-seed measure intervals and base timestamp for the default query window.
- Increase gap between Result and Trace tabs. - Add --info blue variable and use it for execution metadata and the trace-enabled dot. - Include shared ResultPanel, ResultEmpty, and TraceView result components.
- Add page padding and larger Query title to match the handoff. - Render the builder/resizer divider with a visible centered drag handle. - Hide the empty qb-rail-foot strip when there is no code-edited note.
- Introduce stream-specific role inference (service colors, layer labels, smart type detection for ids, numbers, text, endpoint/operation). - Rewrite StreamResultView to auto-recognize tag types and let users override render settings per tag (visibility, role). - Add Console/Table/JSON view switcher, colored one-line pills, and expandable row detail grid. - Wire tag specs from QueryConsole into StreamResultView. - Update role-infer and result-views tests.
- Adopt handoff role names: cat, body, id, numeric, time, list, binary, text. - Treat element_id and timestamp as reserved spine fields: always shown in console/table, excluded from the Tag rendering popover. - Fix Tag rendering popover options/labels (auto · badge, id, number, etc.) and source badges (TYPE/AUTO/CONV/SET). - Use handoff pill styles: color-mix translucent backgrounds, truncated IDs, strong numeric, dim time, and body text. - Update role-infer and result-views tests.
…opover The handoff renders layer-1 (type-fixed) source badges in blue; the local --ok variable is green, so use the handoff's #4a9fe8 text and rgba(74,159,232,.13) background to match the screenshot.
Replace the terse bottom-left error text with a structured error state: - Alert icon + contextual title (Server error, Query error, etc.) - Helpful hint explaining what to check or try - Styled error message block with an ERROR label - Retry button - Collapsible technical details section Add ResultError component + unit tests covering classification, retry, and details toggle.
- Extend StreamResultView Props with hasMore/onLoadMore/isLoadingMore. - Wire paging props from QueryConsole through ResultViewRouter. - Add Load more button to ConsoleView and TableView, reusing the existing .rv-loadmore styles from MeasureResultView. - Update result-views tests to pass paging props and cover the Load more button in both console and table modes.
The BFF overwrites response.totalRowCount with the current page size, so showing '120 of 20' was confusing. Now the stream toolbar shows: - 'X rows loaded' when the reported total is not larger than the loaded count - 'showing X of Y' only when the backend reports a trustworthy total (Y > X)
- Keep ResultViewRouter mounted during Load more by rendering it whenever a response exists, instead of unmounting when status becomes 'running'. This removes the flash and scroll-to-top when loading the next page. - Fix Run buttons in QueryBuilder and code-mode QueryConsole: they were passing the click event object as the loadMore argument (truthy), so Run was appending rows instead of resetting to the first page.
Expose OFFSET in the visual query builder for all catalog types (measure, stream, trace, top-n). The backend already emits OFFSET when QBBuilderState.offset is set; this wires the control into the ORDER BY section next to LIMIT and updates the summary line.
Use a compact ', off N' suffix in the ORDER BY accordion summary so offset remains visible after the builder collapses post-run. The previous ' · offset N' form was truncated by the narrow summary row.
…onses Trace schemas store tags as a flat array (not tagFamilies), so the QueryBuilder SELECT/WHERE chips were empty for trace resources. Update QueryConsole to read both tagFamilies (measure/stream) and tags (trace/property) shapes. Also fix trace query response flattening: trace.v1.QueryResponse uses (each with trace_id + spans), not . Flatten each span into a row carrying the parent trace_id and its own tags. Update the TraceResultView test fixture to match the real wire shape.
Replace the old trace table with the handoff's flat repeated-Span inspector layout: - start_time / span_id / summary columns - expandable rows showing all tags in a grid + opaque span bytes - reuse the stream role-inference engine (type/value/convention/override) - Tag rendering popover to tune summary visibility and override roles - proto binding control + TraceDecoderModal for span bytes - paging support (Load more) wired through QueryConsole Also pass tagSpecs to TraceResultView so schema types drive inference.
- Expand trace SELECT * to explicit tag list so BanyanDB returns tags.
- Omit ORDER BY for trace queries because the default m4-traces schema has
no order-able index for timestamp.
- Parse TIMESTAMP tag values ({"timestamp": "..."}) into epoch ms.
- Exclude the opaque 'span' byte field from configurable trace tags.
- Add semantic role conventions for trace_id/span_id/parent_span_id (id),
endpoint/operation_name/endpoint_name (body).
- Add e2e trace verification script and Playwright test.
BanyanDB's trace query analyzer rejects queries without either a trace_id filter or an ORDER BY. The default m4-traces schema only indexes trace_id, so ORDER BY time returns empty and ORDER BY other tags fails. Enforce a trace_id equality filter in the builder instead. - Add qbHasTraceIdCondition / qbHasTraceIdFilter helpers in bydbql.ts. - Auto-seed an empty trace_id = '' condition when a trace resource is selected. - Validate and block Run in builder mode when the trace_id value is empty. - Update the trace e2e verification script to fill the auto-added condition.
… users The BFF proxy was wrapping every upstream 5xx as a generic 502 upstream_error envelope, hiding the actual BanyanDB error message. Forward 5xx responses verbatim (status + body) so the UI can surface details like "either traceIDs or order must be specified". - Remove the generic 5xx wrapping in proxy.ts. - Update bff.test.ts to expect the upstream status/body to pass through.
The previous message claimed trace_id was the only indexed tag, which is inaccurate. BanyanDB's trace query analyzer requires either a trace_id filter or an ORDER BY clause; update the comment and user-facing message accordingly.
- Remove the builder-side requirement that trace queries must filter on trace_id. BanyanDB accepts any indexed tag filter as long as ORDER BY is present (or a trace_id filter is used). - Emit ORDER BY for trace queries when no trace_id condition is present. The builder's generic 'time' order field maps to the trace timestamp tag. - Default orderField to 'time' when the user switches a trace WHERE clause away from trace_id, keeping the generated query valid. - Add bydbql unit tests covering service-filter + ORDER BY and trace_id-filter without ORDER BY.
The demo's m4-traces/service_spans schema previously only indexed trace_id. Add tree index rules (and bindings) for service, span_id, parent_span_id, duration_ms, status, endpoint, and timestamp so the Canopy builder can exercise WHERE and ORDER BY on non-trace_id tags against the live demo.
trace_id and span_id are now treated as identity tags in the stream result view; update the convention tests to expect 'id' for these names.
String tags such as trace_id, span_id, or status_code can contain only digits. BydbQL was rendering them as unquoted integer literals, which failed for string-typed columns. Pass the schema-known string tags into the query builder so their values are always quoted in WHERE / IN clauses.
The auto-seed effect was running on every WHERE change, so changing the prefilled trace_id condition to service/span_id/status/etc. immediately snapped back to trace_id. Seed trace_id only when a trace resource is picked; after that let the user edit freely.
Creating separate tree index rules for service/status/endpoint/etc. removed those tags from the timestamp SIDX tag filters, so non-trace_id WHERE clauses combined with ORDER BY timestamp returned empty. Keep only the trace_id identity rule and the timestamp ordering rule; other tags remain as SIDX tag filters and can be filtered correctly.
The trace model indexes identity and filter tags automatically, so we only need an explicit index rule for the ordering tag (timestamp) to support ORDER BY. Remove the trace_id rule and keep only the timestamp rule and binding.
Restructure the dirty-code confirmation dialog to use the standard modal layout (modal-overlay / modal-head / modal-body / modal-foot) and copy the handoff wording: 'Back to the builder?', 'You have manual edits in the code editor.', 'Keep my edits' / 'Discard edits'.
Replace the old binding+preview hybrid with the handoff's clean upload/bind flow:
- Title 'Span bytes decoder' with explanatory subtitle
- Drop zone with { } icon, .proto browse hint, and local-parse note
- Body paragraph describing schema-driven decoding
- Footer Cancel / Bind decoder buttons; bind only happens on click
- Unbind option shown when a schema is already bound
- Update TraceResultView caller and result-views tests
flattenTraceSpan was copying name/timestamp/duration from the wire shape even when they are not defined tags on the trace resource, causing the span detail card to show undefined-looking rows. Read only trace_id and span_id (camelCase or snake_case) plus the real tag families, and update the shared DTO comment and test to match.
…sting Generate protobuf-encoded SegmentObject payloads for each seeded trace span so the Canopy span-bytes decoder can be exercised against a real SkyWalking schema. Also fetch and save the upstream Tracing.proto and add a Playwright script that binds it to a span in the UI.
- Rewrite SpanBytesPanel to render the opaque bytes directly instead of only a size pill: hex dump (16 bytes/row) with offset/hex/ASCII columns, a DATA_BINARY byte count, and a base64/hex mode toggle. - Truncate the hex preview to the first 64 bytes and add the handoff-style footer note reporting how many more bytes are opaque to BanyanDB. - Add a small caret to the size chip so the unbound panel matches the handoff screenshot.
… click - SpanBytesPanel now shows only the size chip under SPAN · BYTES by default; clicking the chip toggles the full inspector (hex/base64 dump, DATA_BINARY label, remaining-bytes note). - Rotate the caret icon to indicate open/closed state and add hover styling so the chip feels clickable. - Update verify-decode.mjs to capture both the collapsed and expanded inspector states.
- Refactor SpanBytesPanel to match the handoff SRBinary component: a clickable size chip that opens a fixed-position popover anchored below the chip. - Popover header shows DATA_BINARY byte count and copy buttons for base64/hex; clicking copies the full content to the clipboard and briefly shows 'copied' feedback. - Popover body shows a 12-byte-per-row hex dump; footer reports the remaining opaque bytes. - Close the popover on backdrop click or window scroll/resize. - Update verify-decode.mjs to close the popover before opening the decode modal so the toolbar Decode bytes button remains reachable.
…ON view - Replace regex proto parser with richer TD AST (labels, types, maps, repeated). - TraceDecoderModal now shows parsed message count, root message and field list. - TraceResultView renders the bound proto filename and decoded span JSON with syntax highlighting. - Update verify-decode e2e to handle 'Replace decoder' button label.
- Add an optional className prop to ResultPanel so trace views can mark the card. - Override .result-card.is-trace and its .rv-root to flex:0 1 auto instead of filling the column. - Keep .tin max-height at 460px so many rows scroll inside the compact card.
- Remove the trace result footer (slog-foot) from TraceResultView. - Let the trace result card fill the right column by reverting the compact override and making .tin flex to fill the card. - Restore a 460px cap on .tin in stacked/narrow viewports.
hasRun was tied to status === 'done', so clicking Run flipped it to false during the running state, expanding all accordion sections and then collapsing them again. Use status !== 'idle' instead so the builder stays in post-run accordion mode from the first click through completion.
time.tsx centralises parseTs / parseIntervalMs / formatTs / tipRows
plus useTimestampTooltip, TimestampTooltipPortal, TimestampCell,
and TimestampText components. The result views (Measure, Trace,
Stream) refactored to import from this module instead of each
defining their own helpers — keeping the timezone default ('local')
in one place and matching the handoff's locale-stable rendering.
Adds a hover tooltip on every timestamp cell: ISO + epoch ms, with
a 120ms hide delay so the user can move the cursor onto the tip.
canopy.css gets the matching styles (.ts-cell / .ts-tip / .ts-tip-row
plus a 109 → 60-line reduction in MeasureResultView from the dedup).
time.test.ts: 24 unit tests covering the format functions and
tooltip state.
Renders any long identifier (trace_id, span_id, instance_id, etc.) as
a truncated, ellipsised cell that opens a body-portal popover on
hover and copies the full value to the clipboard on click. Falls back
to document.execCommand('copy') when navigator.clipboard.writeText
is unavailable.
Surface area:
- .ti-id-cell (cursor: copy, ellipsis, inline-block max-width 100%)
- .ti-id-tip (panel-2 / border-strong / rounded, mirrors .ts-tip chrome)
- .ti-id-copy (Copy button → flashes ✓ Copied for 1.2s)
stopPropagation on click so the host row's toggle handler doesn't fire;
keyboard support (Tab focus, Enter/Space copy); 120ms hide delay so
the user can move the cursor onto the popover without it closing.
9 unit tests cover rendering, hover, popover persistence across
mouseleave→tip-mouseenter, click copy, stopPropagation, keyboard
copy, and the empty-value no-op.
The visual builder used to read the measure list for the Top-N tab,
but the BydbQL grammar treats the Top-N FROM as a topn-aggregation
(separate registry, not a measure). This commit switches the FROM
row to fetch and surface topn-aggregation definitions.
shared/src/api-dto.ts:
- TopNAggregationSchema + TopNAggregationListResponse types
- Wire shape: {topNAggregation: [{metadata, sourceMeasure, fieldName,
fieldValueSort, groupByTagNames, ...}]} (camelCase, NOT the
snake_case 'top_n_aggregation' from the proto)
data/api.ts + DataSource.ts:
- listTopNAggregations(group) calls
GET /api/v1/topn-agg/schema/lists/{group} (already proxied by
the BFF at /api/*). Sort by name to match the measure path.
bydbql.ts:
- QB_CATALOG_DEF.uiKw distinct from .kw. .kw stays 'MEASURE' so
the generated BydbQL still says 'FROM MEASURE <name>'; .uiKw
is 'TOPN AGG' for the FROM row's inline label.
- GroupTopnAggMap + qbSearchIndex reworked to source topn entries
from the topn-agg registry (NOT the measure list) so the From-row
fuzzy search surfaces topn-agg names.
QueryBuilder.tsx:
- new topnAggNames prop, used in the FROM row's <select> when
catalog='topn'. setCatalog preserves the current group when
switching to topn (topn-aggs live alongside measures in the
same group; resetting to alphabetical-first would land on a
group with zero topn-aggs).
QueryConsole.tsx:
- new topnAggList + groupTopnAggs state, fetched in parallel with
the measure list, plus a prefetch effect that walks every group
so the From-row fuzzy search has the full index as soon as
possible. tagSpecs/tags derived from groupByTagNames when
catalog='topn' so the WHERE chip dropdown has valid tags.
- topnTimeRangeFor() converts the builder's time shape into a
required RFC3339 time_range for /v1/measure/topn (BanyanDB's
liaison rejects requests with a missing or relative-duration
time_range). Defaults to a 30-minute window when the user
picked a relative range like '-30m'.
- Auto-pick the first topn-agg (alphabetical) when Top-N is
active and the user hasn't picked one yet.
bydbql.test.ts:
- 7 new tests for qbSearchIndex with GroupTopnAggMap, the
'TOPN AGG' uiKw, and QB_CAT preserving MEASURE for the
topn BydbQL grammar keyword.
Verified end-to-end on the live demo (sw_metricsMinute / top_service
+ endpoint_cpm-service both surface the right topn-agg list, the
generated BydbQL parses against the BanyanDB liaison, and the
result view renders bars).
TopNResultView.tsx — full rewrite:
- Ranking / Trace tabs (was Result / Trace) via ResultPanel.tabs
- Status line: 'executed in X ms · N lists · M items' (the
shared ResultPanel doesn't know the per-catalog label, so
Top-N surfaces its own status when multi-list)
- 'Top N of <resource> by value' toolbar with a list-count
pill ('aggregated · 1 list' or 'N lists by time') and a
topN rank badge with the bar-chart icon
- Time-bucket picker (← pills → current ts) when lists.length>1,
hidden in aggregated mode
- Per-list # | ENTITY_ID | topN bar | VALUE leaderboard with
podium styling for top 3 (rank-1 gold, rank-2 silver, rank-3
bronze) and Number.toLocaleString('en-US') values
- ENTITY_ID column header is the wire-format key (NOT derived
from groupByTagNames)
- decodeEntityValue handles three on-the-wire encodings:
1. plain string ('orders', 'checkout' from older top_service)
2. '<base64-name>.<base64-desc>' (newer service_id topn-aggs
— isolate the head and atob; split on '.' because the dot
would otherwise break atob)
3. whole-string base64 → '<hash> <pod-XXX>' (older instance_id
topn-aggs; atob the whole wire, then take the trailing
token after the last space)
The full wire value stays one hover away in the cell's title.
- resolveListTs() falls back to the most-recent item timestamp
when the list-level one is null (BanyanDB sets it to null for
aggregated responses).
TopNTracePanel.tsx (new) — delegates to the shared TraceView so
the disabled-state UI ('WITH QUERY_TRACE is not enabled') lives in
a single place.
ResultPanel.tsx:
- optional tabs?: {result, trace} prop so the shared panel can
label its primary tab 'Ranking' (Top-N) without changing every
other result view.
result-views.test.tsx:
- 5 single-list assertions: leaderboard bars, Ranking tab,
'Top N of <resource> by value' toolbar, topN rank badge,
entity_id column header.
- 6 multi-list assertions: time-bucket picker renders, default
= most recent, pill click switches the active list, nav button
disable state, leaderboard rows update, item-level timestamp
fallback when list-level is null.
- Uses synthetic response fixtures (the cluster's precomputed
topn registry returns single rolled-up lists with null ts).
Also drops the Export button from the Top-N result bar (per user
feedback) — the click-to-export JSON path was unused, and the
M4 handoff's Top-N view never had an export pill.
Four headless Chromium probes against the live demo at
http://localhost:4000/query (session cookie hardcoded; localStorage
cleared at start so each run starts from a fresh default state).
topn-agg-from-row.mjs:
- Click the Top-N tab → FROM row inline label is 'TOPN AGG' (not
'MEASURE'); resource dropdown contains topn-agg names and no
measure names.
- Switch to sw_metricsMinute / endpoint_cpm-service / TIME > '-7d',
run the query, assert the leaderboard renders with bars.
topn-view-redesign.mjs:
- Asserts Ranking tab, 'Top N of <resource> by value' toolbar,
list-count pill, topN badge, ENTITY_ID column header, decoded
cell value (not raw base64, not [object Object]), podium
rank-1/2/3 styling, comma-formatted values, single-bar-per-row,
status header.
topn-bydbql-roundtrip.mjs:
- Captures the BydbQL string the visual builder emits, ejects to
code mode, runs the BydbQL string (so it goes through
/v1/bydbql/query rather than /v1/measure/topn), confirms the
parse error banner is absent.
topn-verify-shots.mjs:
- Captures the four key screenshots (fresh load, Top-N tab,
group + resource picked, result) for visual regression.
All four run on the live demo and pass; the localStorage clear +
fresh session cookie handle the most common 'stale persisted
state' gotcha (a previous test that left a topn/group combo
whose group has zero topn-aggs would otherwise mask the new fix).
Two pre-existing tests asserted against the original proto-decoder
API and the original TraceDecoderModal UI. Both the implementation
and the rest of the test suite had moved on — these were the only
12 failures left in the suite.
result-views.test.tsx > TraceDecoderModal > persists the binding
per traceId in localStorage:
- tdSetBinding takes (traceId, fileName, protoSrc, parsed); the
test was passing only 3 args, so the parsed result landed in
the protoSrc slot and parsed was undefined — surfacing as
'Cannot read properties of undefined (reading primary)' at
proto-decoder.tsx:397. Pass 'span.proto' as the fileName and
move the parsed result to its proper slot.
- The assertion looked for 'Span · bound' + an 'Unbind' button
that don't exist in the current modal; the bound state shows
the file name + root message in the summary and a 'Remove
decoder' button. Update the assertions to match.
proto-decoder.test.ts:
- tdParseProto now returns TDParseResult = { messages, order,
primary, count } (was TDMessage[] in the early revision). The
test was treating the result as an array and calling .map on
it. Switch to accessing parsed.order (discovery sequence) and
parsed.messages[name] for the message map.
- tdPickMessage takes Record<string, TDMessage>, not the array.
Pass parsed.messages.
- tdDecode now takes (binding, span) → string (was (bytes,
binding, traceId) → {kind, ascii, ints, messages, bytes}).
The preview rendering moved into the modal; tdDecode is now a
pure JSON-stringifier. Update the test to assert on the string
return value.
Result: 316/316 web tests pass, 33/33 server tests pass.
…view issues E2E testing framework (see canopy/e2e/TESTING.md): - POM + Playwright fixtures (dependency injection), semantic locators only, web-first assertions (no hard waits) - login-once storageState auth; CANOPY_DEV_NOAUTH kept as the CI fast path - HTTP SeedFactory (dynamic names + auto-cleanup); demo-data seeding wired into global-setup via cmd/m4-seed (BanyanDB writes are gRPC-only) - specs reorganized by domain (boot/auth/shell/schema/index-rule/lifecycle/ query); 10 ad-hoc .mjs probes removed; collapsed to a single playwright.config.ts (handoff/reuse configs deleted) - VRT scoped to stable chrome regions with committed baselines; e2e:vrt + e2e:update-baselines scripts for CI regeneration - additive ARIA landmarks (role/aria-label/data-testid) across shell, forms, pager, and the query builder/results so tests use user-facing handles WHERE clause coverage: - generator supports HAVING/NOT HAVING, MATCH(...) function-call form with analyzer/operator, and bare NULL literals; nested-group double-parens fixed - QueryBuilder exposes analyzer + match-operator inputs for MATCH - where.test.ts (42 UTs) covers every operator and tree shape from the BydbQL conformance corpus; QueryBuilder.test.tsx (4 UTs) covers the MATCH UI M4 review fixes: - api.ts: trace flatten reads .elements (was .traces -> always empty), TopN unwraps protojson values, stream element_id camelCase fallback - bydbql.ts: quote-char escaping, connector-index alignment after pruning - MeasureResultView: fractional number formatting + adaptive chart y-axis - cmd/m4-seed: errors.Is(err, err) tautology + >170-char lines - typecheck + lint clean; 362 web unit tests pass
# Conflicts: # CHANGES.md
Running the canopy.yml checks surfaced two blockers these branches never hit (they'd never gone through the workflow): - Lint: proto-decoder.tsx had an unused `TD_SCALARS` const (no-unused-vars) — removed. - Web typecheck: vite.config.ts failed `tsc` with a vite Plugin type mismatch. web declares vite ^8 while vitest@3 / @vitejs/plugin-react@4 pull vite 7, so two vite majors coexist and their Plugin types collide (build runs on vite 8, tests on vitest's vite 7 — both fine at runtime; only tsc choked). Exclude the build-only vite.config.ts from the app typecheck (vite validates it at build/test time). Follow-up: align the dep set onto one vite major. Full canopy CI green: typecheck (shared/web/server), lint (web/server), server test, web+server build, CSS :has()/color-mix() check, ui/ guardrail; web unit tests 362/362.
CopyableId.{tsx,test.tsx} and time.{tsx,test.ts} had a malformed header line
("...(ASF) to you under" / "(ASF to you under" instead of "...(ASF) licenses
this file to you under"), failing the repo-wide `make license-check` (the Check
job). Corrected to the exact ASF wording license-eye expects.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1224 +/- ##
==========================================
+ Coverage 45.97% 54.95% +8.97%
==========================================
Files 328 615 +287
Lines 55505 93428 +37923
==========================================
+ Hits 25520 51343 +25823
- Misses 27909 37537 +9628
- Partials 2076 4548 +2472
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Contributor
There was a problem hiding this comment.
Pull request overview
Implements the Canopy M4 /query console (builder + code editor) and multiple live-result views, expands shared DTOs to reflect current BanyanDB query/top-N wire shapes, and replaces the legacy E2E suite with a fixture-driven Playwright Page Object Model and deterministic seeding.
Changes:
- Add query-console UI building blocks (editor chrome, empty/error states, trace rendering, trace-bytes decoder modal, copy-to-clipboard ID cell) plus unit tests and recorded-wire fixtures.
- Update shared API DTOs and web data interfaces/hooks to support BydbQL and Top-N execution + response flattening.
- Redesign E2E framework (single Playwright config, auth storageState, HTTP seeding factory, global gRPC demo-data seed) and migrate specs; adjust BFF proxy/static behavior and improve app accessibility/testability.
Reviewed changes
Copilot reviewed 95 out of 113 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| Makefile | Adjust license-check/license-fix to run from repo root with root config. |
| CHANGES.md | Add release-note bullets for Canopy query-console UX improvements. |
| canopy/web/tsconfig.json | Narrow TS include set for the web workspace. |
| canopy/web/src/query/TraceDecoderModal.tsx | Add modal UI to bind a local .proto schema for decoding trace span bytes. |
| canopy/web/src/query/time.test.ts | Add unit tests for timestamp/interval parsing + formatting helpers. |
| canopy/web/src/query/role-infer.test.ts | Add unit tests for stream/trace “semantic role” inference and rendering. |
| canopy/web/src/query/results/TraceView.tsx | Add shared query-trace span-tree renderer + response trace extraction. |
| canopy/web/src/query/results/TopNTracePanel.tsx | Wire Top-N trace panel to shared TraceView. |
| canopy/web/src/query/results/ResultPanel.tsx | Add shared Result/Trace tab chrome and metadata line. |
| canopy/web/src/query/results/ResultError.tsx | Add friendly error-state component with classification + retry + details. |
| canopy/web/src/query/results/ResultError.test.tsx | Add tests for ResultError rendering and interactions. |
| canopy/web/src/query/results/ResultEmpty.tsx | Replace accidental Playwright spec content with a real empty-state component. |
| canopy/web/src/query/QueryBuilder.test.tsx | Add focused unit tests for MATCH analyzer/operator UI in WHERE builder. |
| canopy/web/src/query/proto-decoder.test.ts | Add unit tests for proto parsing/binding lifecycle + decode helpers. |
| canopy/web/src/query/CopyableId.tsx | Add truncated-ID cell with hover popover + click/keyboard copy behavior. |
| canopy/web/src/query/CopyableId.test.tsx | Add tests for popover lifecycle and clipboard-copy behavior. |
| canopy/web/src/query/CodeEditor.tsx | Add handoff-aligned textarea editor with gutter + toolbar slot. |
| canopy/web/src/pages/IndexPage.tsx | Add test IDs and ARIA tab semantics for index rule/binding views. |
| canopy/web/src/pages/HomePage.tsx | Add ARIA region landmark for quick navigation. |
| canopy/web/src/pages/GroupPage.tsx | Add test IDs/ARIA labels; navigate to /query with seeded route state. |
| canopy/web/src/data/hooks.ts | Add TanStack Query mutation hook for running queries. |
| canopy/web/src/data/fixtures/query/trace-query-response.json | Add recorded trace query wire-shape fixture. |
| canopy/web/src/data/fixtures/query/topn-data-response.json | Add recorded Top-N response wire-shape fixture. |
| canopy/web/src/data/fixtures/query/stream-query-response.json | Add recorded stream query wire-shape fixture. |
| canopy/web/src/data/fixtures/query/measure-query-response.json | Add recorded measure query wire-shape fixture. |
| canopy/web/src/data/fixtures/query/manifest.json | Add fixtures manifest describing provenance/versioning. |
| canopy/web/src/data/fixtures/query/fixtures.test.ts | Add tests to sanity-check the recorded fixtures’ shapes and stamps. |
| canopy/web/src/data/DataSource.ts | Extend DataSource interface to list Top-N aggregations. |
| canopy/web/src/components/TraceForm.tsx | Add dialog ARIA role/labels for trace CRUD modals. |
| canopy/web/src/components/StreamForm.tsx | Add dialog ARIA role/labels for stream CRUD modals. |
| canopy/web/src/components/Sidebar.tsx | Add sidebar landmark labeling, status semantics, aria-expanded for collapse. |
| canopy/web/src/components/Shell.tsx | Convert scroll container to <main> landmark. |
| canopy/web/src/components/Pager.tsx | Add pager test ID for stable assertions. |
| canopy/web/src/components/MeasureForm.tsx | Add dialog ARIA role/labels for measure CRUD modals. |
| canopy/web/src/components/IndexRuleForm.tsx | Add dialog ARIA role/labels for index rule CRUD modals. |
| canopy/web/src/components/IndexRuleBindingForm.tsx | Add dialog ARIA role/labels for binding CRUD modals. |
| canopy/web/src/components/GroupForm.tsx | Improve modal ARIA semantics; add labelled interval/number groups for tests/accessibility. |
| canopy/web/src/App.tsx | Route /query to the new QueryConsole. |
| canopy/shared/src/api-dto.ts | Redefine query/top-N DTOs to match current protojson and view-model needs. |
| canopy/server/test/bff.test.ts | Update proxy behavior expectations to forward upstream 5xx verbatim. |
| canopy/server/src/plugins/static.ts | Add explicit caching/revalidation headers and SPA fallback headers. |
| canopy/server/src/plugins/proxy.ts | Forward upstream responses verbatim (including 5xx) instead of wrapping. |
| canopy/package.json | Add scripts for web build and VRT workflows; adjust dependencies. |
| canopy/Makefile | Delegate license-eye to root .licenserc.yaml via explicit -c. |
| canopy/e2e/tests/shell.spec.ts | New shell/home E2E coverage using POM + semantic locators. |
| canopy/e2e/tests/query.spec.ts | New query-console E2E coverage + scoped VRT regions. |
| canopy/e2e/tests/lifecycle.spec.ts | New lifecycle-stage editor E2E coverage using seeded preconditions. |
| canopy/e2e/tests/boot.spec.ts | New boot/health/theme smoke suite. |
| canopy/e2e/tests/auth.spec.ts | New login-form suite (runs unauthenticated) using semantic locators. |
| canopy/e2e/setup/global-setup.ts | Add gRPC demo-data seeding via cmd/m4-seed during global setup. |
| canopy/e2e/playwright.config.ts | Consolidate to a single canonical Playwright config with setup project and VRT defaults. |
| canopy/e2e/m3-metadata.spec.ts | Remove legacy E2E spec (replaced by redesigned suite). |
| canopy/e2e/m3-handoff.spec.ts | Remove opt-in handoff suite (legacy). |
| canopy/e2e/m2-shell.spec.ts | Remove legacy shell spec (replaced by redesigned suite). |
| canopy/e2e/m2-login.spec.ts | Remove legacy login spec (replaced by redesigned suite). |
| canopy/e2e/m1-bff.spec.ts | Remove legacy BFF spec (replaced by redesigned suite). |
| canopy/e2e/handoff-server.cjs | Remove legacy static server for handoff bundle. |
| canopy/e2e/framework/seed/factory.ts | Add HTTP SeedFactory for deterministic schema-level preconditions + cleanup. |
| canopy/e2e/framework/paths.ts | Replace config override with shared constants (storageState path). |
| canopy/e2e/framework/pages/ShellPage.ts | Add shell/home POM. |
| canopy/e2e/framework/pages/SchemaPage.ts | Add metadata CRUD POM. |
| canopy/e2e/framework/pages/QueryConsolePage.ts | Add query console POM with semantic builders/results landmarks. |
| canopy/e2e/framework/pages/LoginPage.ts | Add login-form POM. |
| canopy/e2e/framework/pages/LifecyclePage.ts | Add lifecycle-stage editor POM. |
| canopy/e2e/framework/pages/IndexRulePage.ts | Add index rule/binding POM. |
| canopy/e2e/framework/pages/BasePage.ts | Replace accidental config content with actual BasePage abstraction. |
| canopy/e2e/framework/fixtures.ts | Add DI-style Playwright fixtures for POMs + SeedFactory lifecycle. |
| canopy/e2e/auth/auth.setup.ts | Add “login once” setup project that writes storageState. |
| canopy/.gitignore | Ignore transient E2E artifacts while keeping VRT baselines tracked. |
| .licenserc.yaml | Update ignore patterns for canopy scanning and new runtime artifact exclusions. |
| .gitignore | Ignore handoff import bundle, Playwright MCP artifacts, and debug screenshots. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
315
to
317
| license-fix: $(LICENSE_EYE) ## Fix license header issues | ||
| $(LICENSE_EYE) header fix | ||
| $(LICENSE_EYE) header fix |
| if (m.includes('empty query')) return 'empty'; | ||
| if (m.includes('to must be later than from')) return 'timeRange'; | ||
| if (m.includes('401') || m.includes('403') || m.includes('unauthorized') || m.includes('forbidden') || m.includes('unauthenticated')) return 'auth'; | ||
| if (m.includes('fetch') || m.includes('network') || m.includes('failed to fetch') || m.includes('connection') || m.includes('ENOTFOUND')) return 'network'; |
Comment on lines
+38
to
+44
| // SPA assets must always revalidate. Without these headers the browser | ||
| // caches the index.html + the hash-named bundle, so a `npm run -w web build` | ||
| // that swaps the bundle filename does NOT clear the browser's view of the | ||
| // page — the cached HTML still points at the old hash. Sending | ||
| // no-cache on the HTML and a short max-age on the assets keeps the dev | ||
| // loop tight while still letting the browser avoid refetching on every | ||
| // request within a session. |
Comment on lines
+55
to
+64
| export class SeedFactory { | ||
| private readonly createdGroups: string[] = []; | ||
| private readonly createdResources: TrackedResource[] = []; | ||
|
|
||
| constructor(private readonly request: APIRequestContext) {} | ||
|
|
||
| // Append a monotonic suffix so names are unique across workers and re-runs. | ||
| uniqueName(prefix: string): string { | ||
| return `${prefix}-${Date.now()}`; | ||
| } |
Comment on lines
+127
to
+136
| <div | ||
| className={'tdm-drop' + (dragOver ? ' is-dragover' : '') + (active ? ' has-file' : '')} | ||
| onDrop={handleDrop} | ||
| onDragOver={handleDragOver} | ||
| onDragLeave={handleDragLeave} | ||
| onClick={() => inputRef.current?.click()} | ||
| role="button" | ||
| tabIndex={0} | ||
| aria-label="Upload a .proto file" | ||
| > |
Comment on lines
+2
to
+5
| "description": "Manifest for the query/ fixtures set. The four JSON files in this directory are HAND-AUTHORED to match the BanyanDB 0.9.x wire shape (see implement-m4-note.md decision #24 — live capture requires the M0 seed harness, which is out of this session's scope). Wire shape is pinned per banyandbVersion; when BanyanDB upstream evolves, re-run a live capture (planned M5 helper, see `npm run -w web capture:query-fixtures` after the harness lands) and diff against these committed files; a CI guard would warn/fail on shape drift.", | ||
| "capturedAt": "2026-06-29T12:00:00Z", | ||
| "banyandbVersion": "0.9.x", | ||
| "authorship": "hand-authored (NOT live-captured)", |
- ResultError.classify(): fix dead branch — match lowercased 'enotfound' (was 'ENOTFOUND', never matched a lowercased message), so DNS errors classify as network. - SeedFactory.uniqueName(): add a per-instance counter so same-millisecond calls don't collide (e2e flakiness). - TraceDecoderModal: add Enter/Space keyboard handler to the role="button" drop zone (was focusable but not keyboard-activatable). - Makefile license-fix: drop the duplicated `license-eye header fix` line. - static.ts: align the cache-control comment with the actual headers (index.html no-store, hash assets no-cache/must-revalidate). - query fixtures manifest: banyandbVersion "0.9.x" -> "monorepo-current" to match the fixtures + fixtures.test.ts stamp. typecheck + lint clean, web + server tests pass.
tristaZero
approved these changes
Jul 23, 2026
mrproliu
approved these changes
Jul 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Canopy M4 — Query console + result views, E2E framework redesign, WHERE coverage
Builds the
/queryconsole and its four result views against a live, seeded BanyanDB, plus a redesigned E2E testing framework and full WHERE-clause coverage.canopy/-only change (Go untouched;ui/guardrail enforced).Query console & result views
query-console.jsx/query-builder.jsx→bydbql.tscodegen,role-infer.ts,proto-decoder.tsx..elements(was.traces→ always empty), Top-N protojson value unwrapping, streamelement_idcamelCase, fractional number formatting + adaptive chart axis, andTagType/FieldTypeenums aligned to the proto (the stale*_INT64/*_FLOAT64names were silently dropped toUNSPECIFIED, also affecting measure/stream/trace).WHERE coverage
= != < <= > >= IN "NOT IN" HAVING "NOT HAVING" MATCH(...), NULL literals, and nested AND/OR groups with correct parenthesization.where.test.ts(42 cases) covers every operator and tree shape from thetest/cases/**/*.qlconformance corpus; adds the MATCH analyzer/operator UI.E2E testing framework redesign (see
canopy/e2e/TESTING.md)storageState(withCANOPY_DEV_NOAUTHCI fast path); HTTPSeedFactory+ demo-data seeding wired intoglobal-setup..mjsprobes removed; collapsed to a singleplaywright.config.ts.e2e:update-baselinesregeneration path.CI
Full
canopy.ymlsuite green locally: typecheck (shared/web/server), lint (web/server), server unit/contract test, web+server build, the CSS:has()/color-mix()preservation check, and theui/guardrail; plus 362/362 web unit tests. Branch synced withmain.