Add eval.news frontend (packages/web)

[vu1n]

Add eval.news frontend (packages/web)

The Frontier of Evals — a Vite + React + TypeScript reading portal for the awesome-evals library, ported from the Claude Design source. Reading path · searchable catalog · landscape map · ten-part curriculum, with localStorage read/bookmark progress and an "Open in LLM" teacher mode on every entry and section. Hash-routed; deploys on Vercel (Root Directory packages/web), no env vars or secrets.

🔗 Live demo: https://vu1n.github.io/awesome-evals/ — GitHub Pages, built from my fork's demo/gh-pages branch (not part of this diff).

The dataset comes from this repo's README.md — two ways

• Build time — a prebuild step parses the in-repo README.md (GitHub-raw fallback) into src/data/evals-data.json, so every deploy reflects merged main. The JSON is committed too, so dev/offline loads work immediately.
• Runtime — on load the app re-fetches README.md from GitHub main and live-swaps the dataset only if it changed since the deploy (best-effort; silent fallback to the bundled data).

A single shared parser (src/lib/parseReadme.js) runs in both Node and the browser. The only layer that isn't derivable from the README — the 10 section titles/subtitles (editorially rewritten for the portal) — lives in src/data/editorial.json and is merged in.

"Open in LLM" — teacher mode

Every entry (in the detail drawer) and every Learn section offers a one-click teaching session: deep-links that pre-fill a teach-style prompt into ChatGPT / Claude, plus a universal Copy prompt for any LLM / Codex / Claude Code. The prompt is Socratic and calibrated to the learner (retrieval practice over summary, ending in an applied "mission"), distilled from Matt Pocock's teach skill. Gemini has no URL-prefill param, so its chip copies the prompt and opens Gemini to paste.

Reviewer guide

~2k lines of hand-written code; ~6.7k of the 8.7k diff is generated. Read in dependency order:

Data pipeline

1. src/lib/parseReadme.js — README → dataset parser (+ flattenEntries, datasetIssues)
2. scripts/build-data.mjs — prebuild generator · scripts/check-readme-sync.mjs — drift check
3. src/lib/dataset.ts — reactive store (useSyncExternalStore) + ingest/lookups
4. src/lib/useLiveUpdate.ts — runtime GitHub fetch → live swap

UI
5. src/App.tsx — layout shell, hash routing, view + selection state
6. src/components/ — Masthead · Home · ReadingPath · Catalog · MapView · Learn · DetailDrawer

Learn-with-LLM
7. src/lib/teachPrompt.ts — teach-prompt builders + provider deep-links
8. src/components/LearnWithLLM.tsx — the control (wired into DetailDrawer + Learn)

Skip — generated / mechanical

• src/data/evals-data.json (~4.8k lines — parser output)
• package-lock.json

One commit

Squashed into a single additive commit; read the files in the dependency order above.

Verify

cd packages/web
npm install
npm run build         # prebuild → tsc (strict) → vite
npm run check:readme  # dataset in sync with README (354 = 354)

Risk

Additive only — a brand-new package; no existing files are touched (rebased onto current main, diff is packages/web/ only). The runtime GitHub fetch and the LLM links are best-effort and degrade gracefully (bundled dataset offline; Copy-prompt backstops every provider).

Known follow-up

The landscape Map is a fixed 1180px canvas (desktop-first, faithful to the design) — it needs a responsive pass before mobile.

[vu1n]

🚀 Deploying the portal

Both options build the same static SPA from packages/web — no server, env vars, or secrets. Hash routing (#/catalog, #/map, …) means deep links work without any SPA-fallback config.

Vercel — recommended (production target)

The package is ready to deploy as-is (vite.config.ts base is /):

1. New Project → import benchflow-ai/awesome-evals
2. Root Directory: packages/web
3. Framework preset Vite — build npm run build, output dist/ (already declared in packages/web/vercel.json)
4. Deploy. Pushes to main auto-redeploy, so the live site tracks the curated README.md.

→ ships to *.vercel.app; attach a custom domain (e.g. eval.news) in project settings.

GitHub Pages

Project Pages serve from a subpath (/awesome-evals/), so the build needs a base path. One small tweak (kept off this PR's diff, but proven on my demo branch):

// packages/web/vite.config.ts
export default defineConfig(({ mode }) => ({
  base: mode === 'pages' ? '/awesome-evals/' : '/', // '/' stays Vercel/local-safe
  plugins: [react()],
}))

Then build with npm run build -- --mode pages and deploy packages/web/dist.

Admin steps: Settings → Pages → Source = GitHub Actions, then add a build+deploy workflow (happy to provide the exact .github/workflows/pages.yml — it's running on my fork).

🔎 Live reference deploy (from my fork, current code): https://vu1n.github.io/awesome-evals/

Note: Pages isn't currently enabled on this repo (benchflow-ai.github.io/awesome-evals/ returns Site not found), so the org-hosted demo needs a maintainer with admin to flip it on. Vercel needs no special repo access.

[xdotli]

First: apologies this sat a month without a reply. It's a serious piece of work and deserved a faster response.

I've now read it properly. The code is good — 2 runtime dependencies, no postinstall hooks, every lock entry resolving to the npm registry with integrity hashes, install scripts only on esbuild and fsevents. The parser is shared cleanly between Node and browser, build-data.mjs regenerates at build time with a GitHub-raw fallback, and useLiveUpdate.ts hot-swaps at runtime. You clearly anticipated the staleness problem rather than ignoring it.

This isn't a quality review, though — it's a scope decision, so let me be direct about the tradeoffs.

The one hard finding. I ran your own check-readme-sync.mjs against current main:

README entries        : 365
Committed dataset     : 359
▸ In README but missing from the dataset (6)

That's scan PR #33, merged two days after you opened this. One month, one merged scan, already drifted — and check:readme isn't wired into anything, because this repo has no CI at all (the two workflows are the daily scan and the comment responder; neither installs Node). Every one of the README PRs in the queue adds another.

The deeper coupling worries me more: datasetIssues() hard-fails the build below 10 sections or 300 unique URLs. Merging this makes README.md load-bearing for a production deploy — in a repo whose entire contribution model is a bot rewrites README.md every morning. A scan run that changes its line format becomes a red deploy.

Where I've landed: I'd rather this live in its own repo.

parseReadme.js and useLiveUpdate.ts already fetch from raw.githubusercontent.com and need nothing local. Split to benchflow-ai/eval.news (or keep it under your account and we link it prominently from the README) and you get the site, README stays the single source of truth, the scan bot can't break a deploy, and this repo stays pure Markdown with no lockfile or toolchain to maintain.

If you'd prefer it merged here instead, make the case — it's not unreasonable, it just needs: check:readme --strict in a pull_request workflow (not pull_request_target, given the lockfile), or npm run data added to the scan recipe; a regenerated dataset; and someone to own a TSX app long-term, which today would be nobody.

Two other things worth knowing: docs/index.html is an existing 2.2MB explorer from the initial commit that nothing links to and Pages doesn't serve — it renders the notes/ and citation graph, which yours doesn't, so this supersedes the idea but not the content. And you flagged the fixed-1180px Map yourself; that'd need fixing either way.

Leaving this open for your response. Whichever way it goes, thank you for building it — and for the live demo on your fork, which is genuinely useful right now.