viz: scatter, histogram, stacked bar + analyse: filter_rows#25
Merged
Conversation
Three chart types and a filtering primitive, closing gaps that forced agents to hand-roll the same code every run. Plus a docs-accuracy fix that PR #24 surfaced. viz.py (HTML only; the xlsx path refuses them with the existing clear message): - scatter_chart(x, y, ..., trend_line=False) - both axes float via _nice_ticks. The OLS overlay spans only the observed x-range and carries slope / Pearson r / n in its tooltip; a cloud with no x-variance gets no line at all rather than an invented relationship. - histogram(values, bins=10, ...) - bin count or explicit edges. Half-open [lo,hi) except the last bin, which includes its upper bound so the maximum is never silently dropped. Y-axis forced to zero (a floated frequency axis misstates every bar); bars touch. Unparseable values and values outside the caller's own edges are reported separately - they mean different things. - stacked_bar(data, ...) - takes pivot() output directly. Negative segments stack BELOW the zero line, so a credit note reads as the reduction it is instead of inflating the bar it belongs to. All three reuse the engine's parse_number when reachable ('15%' -> 0.15, '(500)' -> -500) rather than inventing a second numeric dialect that would put a different number on the chart than in the working paper. analyse.py: - filter_rows(header, rows, filters) - 12 operators, ANDed, returning (rows, report) with in/out and per-filter removed counts. Two comparison rules exist because the alternative is a plausible WRONG answer, not an error: * Dates compare as dates. parse_number('15/02/2026') returns 15022026 (it strips the separators), so a number-first coercion sorts 15 Feb after 1 Mar. * A type mismatch is incomparable, not a string compare - the fallback would make 'n/a' > 1000 true. Those rows are excluded AND counted. Unknown columns/operators raise; a typo must not look like "no results". Docs: the README's "no network calls / no cloud OCR / no credentials" and DATA-HANDLING's "no external APIs" predated the opt-in vision image extract shipped in #19, which calls a user-configured endpoint. All three now carve out that one exception. SKILL.md and COMPATIBILITY.md already disclosed it correctly. Wired through _viz_block and dashboard-spec.schema.json. Five regression tests, all mutation-checked - each fails when its guard is narrowed. Gates: data-lint OK, 101 pytest, 68 standalone, 19 runtime, 12 schemas, 6/6 golden plans, quickstart smoke. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…rief
The brief arrived after the first pass. Reconciling against it found three
places where my API would have shipped wrong, plus several smaller deviations.
Breaking (relative to the previous commit on this branch, unmerged):
- stacked_bar: the brief's PRIMARY input shape, {category: [v1, v2, ...]},
raised TypeError - I had only implemented the segment-first
{segment: [(cat, value)]} form. _norm_stacked now normalises five shapes:
pivot() output, {category: [values]}, [(category, [values])],
{categories, series}, and the segment-first form. Positional shapes have no
segment names, so segments are numbered rather than left blank.
- scatter_chart: added the missing x_label / y_label axis annotations, and
renamed unit/x_unit -> unit_y/unit_x per the brief. Now also accepts
[(label, value)] pairs via _norm_pairs, and trims to the shorter side on a
length mismatch (counting the unpaired tail).
- filter_rows: filter specs take "col", "values" for membership, and "lo"/"hi"
for ranges; the report key is n_dropped. "column" and a two-item "value"
remain accepted so nothing already written breaks.
Smaller alignments: histogram requires >= 2 parseable values (one point has no
distribution to show), bars carry rx="1", the tooltip reads "{n} values in
[lo, hi)" with an honest closing bracket on the last bin, and ~8 edge labels.
stacked_bar uses the brief's 6px gap and padding, with a "cat . segment: value"
tooltip. scatter's trend line spans the plot area as specified - the axes are
floated to the data, so the edges sit a tick outside the observed range.
The two comparison rules found earlier still hold and are unchanged: dates
coerce before numbers (parse_number strips date separators), and a type
mismatch is incomparable rather than a string compare.
Added: cookbook snippets for all three charts in references/blocks.md (required
by the brief and missed first time), two acceptance-shape regression tests
pinning the documented call signatures, and SKILL.md/schema/runtime updates for
the renamed parameters.
Verified every acceptance checkbox in the brief literally - 41 checks, all pass.
The five-mutant suite still catches all five.
Gates: data-lint OK, 103 pytest, 70 standalone, 19 runtime, 12 schemas,
6/6 golden plans, quickstart smoke.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three chart types and a filtering primitive — the gaps that forced agents to hand-roll the same code every run — plus the docs-accuracy fix that reviewing #24 surfaced.
Charts (
viz.py, HTML only — the xlsx path refuses them with the existing message)scatter_chart(x, y, …, trend_line=False)_nice_ticks(matchingline_chart). The OLS overlay spans only the observed x-range and carries slope / Pearson r / n in its tooltip.histogram(values, bins=10, …)[0,30,60,90,365]. Y-axis forced to 0; bars touch.stacked_bar(data, …)pivot()result directly, or{segment: [(cat, value)]}, or{categories, series}. Legend with swatches.Three judgment calls worth reviewing:
histogramedges are half-open[lo, hi)except the last, which includes its upper bound — otherwise the maximum observation silently vanishes.scatter_chartdraws no trend line when x has zero variance. There is no defensible slope for a vertical cloud, and drawing one would invent a relationship the data doesn't contain.stacked_barstacks negatives below the zero line. Folding a credit note into the positive stack would inflate the very bar it reduces.All three reuse the engine's
parse_numberwhen reachable ('15%'→ 0.15,'1.2m'→ 1200000,'(500)'→ -500) rather than inventing a second numeric dialect — otherwise the chart and the working paper could show different numbers for the same cell. Non-numeric inputs are skipped and reported, never plotted at the origin where they'd read as a real zero.filter_rows(header, rows, filters)(analyse.py)12 operators (
==!=>>=<<=innot_inbetweennot_betweencontainsis_emptynot_empty), ANDed, returning(rows, report)with in/out totals and per-filter removed counts.Two comparison rules exist because the alternative is a plausible wrong answer, not an error — both found by testing, not by reading:
parse_number('15/02/2026')returnsDecimal('15022026')— it strips the separators. A number-first coercion therefore sorts 15 Feb after 1 Mar. Values that look like dates are now tested as dates first.'n/a' > 1000true. Such rows are excluded and counted inreport["incomparable"].betweenis inclusive of both ends (finance quotes ranges that way). Unknown columns and operators raise — a typo must not look like "no results".Docs accuracy
The README's "no network calls / no cloud OCR / no credentials" and
DATA-HANDLING.md's "no external APIs, no third-party uploads" predated the opt-in vision image extract shipped in #19, which calls a user-configured vision endpoint and needs a key. All three claims now carve out that one exception explicitly.SKILL.mdandCOMPATIBILITY.mdalready disclosed it correctly — it was the top-level pitch that overstated, which is the wrong way round.Verification
Wired through
_viz_blockandschemas/dashboard-spec.schema.json(new block types, their properties, and per-type required-property rules;additionalProperties:falsestill holds).Five regression tests, all mutation-checked — each was confirmed to fail when its guard is narrowed. One of them initially escaped: the stacked-negatives test only asserted relative ordering, which holds even when both segments stack upward. It now measures against the zero line.
Gates:
data-lintOK · 101 pytest · 68 standalone · 19 runtime · 12 schemas · 6/6 golden plans · quickstart smoke.🤖 Generated with Claude Code