feat(vibe-tests): add trends dashboard from nightly issue history - #4694
Open
ernestt wants to merge 1 commit into
Open
feat(vibe-tests): add trends dashboard from nightly issue history#4694ernestt wants to merge 1 commit into
ernestt wants to merge 1 commit into
Conversation
Collect nightly vibe-test results from GitHub issues (label:vibe-test) into a normalized time series and render a self-contained HTML dashboard: overall score trend, Astryx-vs-Baseline gap with a moving average, per-dimension small multiples, and a sortable table linking back to each issue. - trends-collect.ts parses scoreboards across historical table layouts, excludes non-score issues, and rescales old 0-10 runs to 0-100 - trends-report.ts injects the data into an inline-SVG dashboard (no deps) - exposed via 'lns trends' plus trends:collect / trends:report scripts
ernestt
requested review from
cixzhang,
ejhammond and
imdreamrunner
as code owners
August 3, 2026 23:57
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
meta-cla
Bot
added
the
CLA Signed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds a Vibe Test Trends dashboard — a longitudinal view of the nightly vibe-test results that, until now, only existed as a stream of individual GitHub issues.
Each nightly run posts an issue labeled
vibe-testwith a scoreboard table, but there was no way to see the scores over time. This wires up the existing data into a tracker:trends-collect.ts— reads everylabel:vibe-testissue viagh, parses the scoreboard across the handful of historical table layouts (transposed, dimension-major, split "Overall Scores" tables, and title-only fallbacks), excludes non-score issues (experiments, audits, failed runs), rescales old 0–10 rubric runs to 0–100, and writes a normalizedtrends.json.trends-report.ts— injects that time series into a single self-contained HTML dashboard. No build step, no external chart deps (inline SVG).lns trends(collect + build in one shot) plustrends:collect/trends:reportscripts.Dashboard
Data
Backfilled from the full issue history: 87 scored runs, 2026-02-22 → 2026-06-26 (23 non-score issues excluded automatically). Parsed overall scores were cross-checked against the numbers embedded in issue titles — zero mismatches.
How to run
trends.jsonandtrends-dashboard.htmlare generated artifacts (gitignored).Testing
eslint(strict/CI mode) andprettierclean on all new files;tsc --strictpassesnode --check