Collect community benchmark submissions via Issue Forms + validating Action + Pages (no external site)

[PBNZ]

Feature request (follow-up to #37)

Once the benchmark harness in #37 exists, let users submit their own results through a fully GitHub-native pipeline — no separate website or server required. GitHub Pages alone can't accept submissions (it's static hosting: no backend, no database), but the following collects and publishes community benchmarks using only GitHub features.

Proposed pipeline

1. Issue Forms — .github/ISSUE_TEMPLATE/benchmark.yml with structured fields (CPU, GPU, RAM, OS, whisper model, transcribe time, speed-vs-realtime, quality metric, watch-local version). Users submit by opening an issue; data arrives clean, searchable, and labelable with zero infrastructure. The harness (Add a repeatable cross-platform benchmark harness (fixture video → per-model timing + quality) #37) could print a pre-filled issue body to make this one click.
2. GitHub Action — on each new benchmark-labeled issue, parse the form fields and append the entry to a results file (CSV / Markdown table) in the repo.
3. GitHub Pages — render the aggregated results table (per-OS / per-CPU / GPU vs CPU), auto-updated from the results file.

Validation & anti-abuse (required)

Any open submission path attracts spam and fake/inaccurate numbers, so the aggregating Action must gate entries before they reach the published table, e.g.:

• Schema validation — required fields present, correct types/units, known enum values (model name ∈ {tiny,base,small,medium,large-v3}; recognised OS).
• Plausibility bounds — reject impossible values (e.g. negative time, absurd real-time-speed for a CPU, memory beyond stated RAM), and flag statistical outliers for manual review rather than auto-publishing.
• Spam controls — ignore submissions from brand-new/zero-history accounts pending maintainer approval; require a verified/approved label (applied by a maintainer or a trusted-contributor check) before the Action includes a row.
• Provenance — encourage attaching the harness-generated CSV/JSON so numbers are reproducible, not hand-typed; the Action can prefer/soft-verify machine-generated output.
• Invalid entries get an automated comment explaining the rejection and are left out of the table (issue stays open for correction).

Why GitHub-native

Free, no hosting to maintain, and moderation happens in the same place as the data. A real backend (Formspree, Cloudflare Worker + D1/KV, etc.) is only worth it if this pipeline is later outgrown.

Related

• Add a repeatable cross-platform benchmark harness (fixture video → per-model timing + quality) #37 — benchmark harness that produces the submittable numbers
• Publish CPU performance benchmarks & per-model Whisper guidance (README / docs / onboarding) #36 — the reference benchmarks/report this would crowd-source across hardware