Skip to content

bench(#856): add 6 demonstrative cross-runtime workloads (json, regex, sort, typed-arrays, binary-trees, brainfuck)#879

Merged
nickna merged 1 commit into
mainfrom
wrk/856-starter-benchmarks
Jun 22, 2026
Merged

bench(#856): add 6 demonstrative cross-runtime workloads (json, regex, sort, typed-arrays, binary-trees, brainfuck)#879
nickna merged 1 commit into
mainfrom
wrk/856-starter-benchmarks

Conversation

@nickna

@nickna nickna commented Jun 22, 2026

Copy link
Copy Markdown
Owner

Part of #856 (perf epic). Broadens the cross-runtime benchmark suite (benchmarks/) beyond the original 7 arithmetic kernels — which are all tuned to ~parity — with 6 workloads chosen for where SharpTS-compiled, Node (V8), and Bun (JSC) diverge most: builtin hot paths, data-parallel numeric code, GC/allocation pressure, and a dispatch-heavy macro program.

Workloads added

Workload Demonstrates Placement
json builtin gap; biggest Bun/Node spread shared algorithms.ts + BDN
regex Irregexp vs JSC vs .NET Regex standalone script
sort comparator-call + sort algorithm standalone script
typed-arrays data-parallel Float64Array shared algorithms.ts + BDN
binary-trees (CLBG) GC/allocation + recursion shared algorithms.ts + BDN
brainfuck macro / interpreter dispatch standalone script
  • json / typed-arrays / binary-trees are number → number, added to the shared benchmarks/scripts/lib/algorithms.ts (also embedded by SharpTS.Benchmarks), with matching IdiomaticCSharp (native) + EquivalentCSharp (boxed object?) baselines and a new BenchmarkDotNet StarterWorkloadBenchmarks.cs (SharpTS-compiled vs native C# vs boxed C#).
  • regex / sort / brainfuck are self-contained scripts using a seeded Park-Miller LCG so all runtimes process byte-identical data.

No harness plumbing changed — run-benchmarks.ps1 globs scripts/*.ts, format-results.ps1 auto-renders, and BenchmarkSwitcher auto-discovers the new classes.

Results (compiled vs Node, largest param)

Workload Compiled Node Bun Compiled / Node
sort @10k 3235 ms 3.24 ms 3.50 ms 998×
typed-arrays @1m 100.3 ms 4.18 ms 2.23 ms 24×
binary-trees @16 37.5 ms 1.37 ms 1.69 ms 27×
brainfuck @5k 410.7 ms 21.0 ms 6.48 ms 20×
json @10k 40.9 ms 2.55 ms 2.29 ms 16×
regex @10k 6.07 ms 2.72 ms 1.73 ms 2.2×

The new workloads immediately surfaced two compiled-output gaps, now filed:

Verification

  • dotnet build -c Release — clean (0 warnings).
  • All 6 scripts run under both the interpreter and --compile (3 BENCH: lines each).
  • Equal-work parity: every workload produces byte-identical results across interpreter, compiled, Node, and Bun — so any gap is pure codegen, not divergent work.
  • Full run-benchmarks.ps1 completes with all 4 runtimes producing output for all 40 cases.
  • BenchmarkDotNet dry run executes all 27 new benchmarks (3 classes × 3 params × 3 methods); MemoryDiagnoser confirms binary-trees allocates ~6.7× the native version.

Purely additive benchmark/fixture code — no interpreter, compiler, or runtime changes, so Test262 / TS-conformance are unaffected.

Broadens the cross-runtime suite beyond the original arithmetic kernels with
workloads where SharpTS-compiled, Node (V8), and Bun (JSC) diverge most:

  json, regex, sort, typed-arrays, binary-trees, brainfuck

- json/typed-arrays/binary-trees are number->number, added to the shared
  benchmarks/scripts/lib/algorithms.ts (also embedded by SharpTS.Benchmarks),
  with matching Idiomatic/Equivalent C# baselines and a new BenchmarkDotNet
  StarterWorkloadBenchmarks.cs (SharpTS-compiled vs native vs boxed C#).
- regex/sort/brainfuck are self-contained scripts (seeded Park-Miller LCG for
  deterministic, cross-runtime-identical data).

All six produce byte-identical results across interpreter, compiled, Node, and
Bun (equal work), so any gap is pure codegen. They immediately surfaced two
compiled-output gaps now tracked as #877 (Array.sort is O(n^2) insertion sort)
and #878 (Float64Array element access is boxed + virtually dispatched).
@nickna nickna merged commit 33f266f into main Jun 22, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant