Docs: clarify benchmark suites — rename SharpTS.Benchmarks → SharpTS.Microbenchmarks, add READMEs

CLAUDE.md

@@ -159,6 +159,15 @@ Two standalone test projects pin SharpTS against external corpora. Neither is in
 - **`SharpTS.Test262/`** — TC39 ECMA-262 (interpreter + compiled). Update baseline: `SHARPTS_TEST262_UPDATE_BASELINE=1`.
 - **`SharpTS.TypeScriptConformance/`** — `microsoft/TypeScript` conformance (type-checker only). Update baseline: `SHARPTS_TSCONFORMANCE_UPDATE_BASELINE=1`. Bucket model: Pass/Fail/ParseError/TypeCheckError/Skipped/HarnessError. Match strategy: `(line, tsCode)` tuples — `tsCode` is the canonical `TSnnnn` code carried on every type-checker `Diagnostic` (see `Diagnostics/Diagnostic.cs`). See `SharpTS.TypeScriptConformance/README.md`.
 
+## Benchmarking
+
+Two **complementary** benchmark suites measure different things — they are not redundant and cannot be merged (one drives external runtime executables; the other must run in-process against managed code). Pick by the question you're answering.
+
+- **`benchmarks/`** — *external / competitive.* "Are we as fast as Node/Bun?" PowerShell harness (`run-benchmarks.ps1`) runs `scripts/*.ts` whole-program across the SharpTS interpreter, SharpTS compiled, Node.js, and Bun via a shared `scripts/lib/bench.ts`. Wired into CI (`.github/workflows/benchmarks.yml`, `workflow_dispatch` only). Goal: **meet or exceed Node.** See `benchmarks/README.md`.
+- **`SharpTS.Microbenchmarks/`** — *internal / headroom.* "How close are we to the C# ceiling, and where's the overhead?" BenchmarkDotNet project in `SharpTS.sln`; compiles TS in-process and compares against idiomatic C# (native-type ceiling) and "equivalent" C# (`object?`/boxing tax), with `MemoryDiagnoser` allocation profiling. This is the harness behind compiler perf work. See `SharpTS.Microbenchmarks/README.md`.
+
+`benchmarks/scripts/lib/algorithms.ts` is **shared byte-identical** between the two (embedded into the microbenchmark assembly as `SharpTS.Microbenchmarks.algorithms.ts`) so both measure the same source. Embedded-resource names are referenced by string and `RootNamespace`/`AssemblyName` are pinned in the `.csproj` — a wrong name compiles but throws at `[GlobalSetup]`.
+
 ## See Also
 
 - `STATUS.md` - Feature implementation status and known bugs

SharpTS.Benchmarks/Baselines/EquivalentCSharp.cs → SharpTS.Microbenchmarks/Baselines/EquivalentCSharp.cs

@@ -1,4 +1,4 @@
-namespace SharpTS.Benchmarks.Baselines;
+namespace SharpTS.Microbenchmarks.Baselines;
 
 /// <summary>
 /// C# implementations using object/dynamic types to simulate SharpTS runtime overhead.

SharpTS.Benchmarks/Baselines/IdiomaticCSharp.cs → SharpTS.Microbenchmarks/Baselines/IdiomaticCSharp.cs

@@ -1,4 +1,4 @@
-namespace SharpTS.Benchmarks.Baselines;
+namespace SharpTS.Microbenchmarks.Baselines;
 
 /// <summary>
 /// Idiomatic C# implementations using native types (int, long, bool[], etc.).

SharpTS.Benchmarks/Benchmarks/ArrayHelpersBenchmarks.cs → SharpTS.Microbenchmarks/Benchmarks/ArrayHelpersBenchmarks.cs

@@ -1,9 +1,9 @@
 using System.Reflection;
 using BenchmarkDotNet.Attributes;
 using BenchmarkDotNet.Order;
-using SharpTS.Benchmarks.Infrastructure;
+using SharpTS.Microbenchmarks.Infrastructure;
 
-namespace SharpTS.Benchmarks.Benchmarks;
+namespace SharpTS.Microbenchmarks.Benchmarks;
 
 /// <summary>
 /// Iterator-helper benchmarks for the issue #90 hot path. Measures the
@@ -43,7 +43,7 @@ public void Setup()
     {
         var assembly = typeof(ArrayHelpersBenchmarks).Assembly;
         using var stream = assembly.GetManifestResourceStream(
-            "SharpTS.Benchmarks.TypeScriptSources.ArrayHelpers.ts")
+            "SharpTS.Microbenchmarks.TypeScriptSources.ArrayHelpers.ts")
             ?? throw new InvalidOperationException("Could not find embedded resource ArrayHelpers.ts");
         using var reader = new StreamReader(stream);
         var tsSource = reader.ReadToEnd();

SharpTS.Benchmarks/Benchmarks/ComputationalBenchmarks.cs → SharpTS.Microbenchmarks/Benchmarks/ComputationalBenchmarks.cs

@@ -1,10 +1,10 @@
 using System.Reflection;
 using BenchmarkDotNet.Attributes;
 using BenchmarkDotNet.Order;
-using SharpTS.Benchmarks.Baselines;
-using SharpTS.Benchmarks.Infrastructure;
+using SharpTS.Microbenchmarks.Baselines;
+using SharpTS.Microbenchmarks.Infrastructure;
 
-namespace SharpTS.Benchmarks.Benchmarks;
+namespace SharpTS.Microbenchmarks.Benchmarks;
 
 // Computational algorithm benchmarks: SharpTS-compiled TypeScript vs idiomatic
 // C# (native types — the performance ceiling) vs "equivalent" C# (object?/boxing
@@ -23,7 +23,7 @@ namespace SharpTS.Benchmarks.Benchmarks;
 /// <summary>Shared embedded-resource loading + compiled-delegate resolution.</summary>
 public abstract class ComputationalBenchmarkBase
 {
-    private const string ResourceName = "SharpTS.Benchmarks.algorithms.ts";
+    private const string ResourceName = "SharpTS.Microbenchmarks.algorithms.ts";
 
     protected static Func<double, double> LoadCompiled(string functionName)
     {

SharpTS.Benchmarks/Benchmarks/ObjectLiteralsBenchmarks.cs → SharpTS.Microbenchmarks/Benchmarks/ObjectLiteralsBenchmarks.cs

@@ -1,9 +1,9 @@
 using System.Reflection;
 using BenchmarkDotNet.Attributes;
 using BenchmarkDotNet.Order;
-using SharpTS.Benchmarks.Infrastructure;
+using SharpTS.Microbenchmarks.Infrastructure;
 
-namespace SharpTS.Benchmarks.Benchmarks;
+namespace SharpTS.Microbenchmarks.Benchmarks;
 
 /// <summary>
 /// Object-literal allocation benchmarks. Each <c>{ ... }</c> in TS source
@@ -29,7 +29,7 @@ public void Setup()
     {
         var assembly = typeof(ObjectLiteralsBenchmarks).Assembly;
         using var stream = assembly.GetManifestResourceStream(
-            "SharpTS.Benchmarks.TypeScriptSources.ObjectLiterals.ts")
+            "SharpTS.Microbenchmarks.TypeScriptSources.ObjectLiterals.ts")
             ?? throw new InvalidOperationException("Could not find embedded resource ObjectLiterals.ts");
         using var reader = new StreamReader(stream);
         var tsSource = reader.ReadToEnd();

SharpTS.Benchmarks/Benchmarks/PropertyAccessBenchmarks.cs → SharpTS.Microbenchmarks/Benchmarks/PropertyAccessBenchmarks.cs

@@ -1,9 +1,9 @@
 using System.Reflection;
 using BenchmarkDotNet.Attributes;
 using BenchmarkDotNet.Order;
-using SharpTS.Benchmarks.Infrastructure;
+using SharpTS.Microbenchmarks.Infrastructure;
 
-namespace SharpTS.Benchmarks.Benchmarks;
+namespace SharpTS.Microbenchmarks.Benchmarks;
 
 /// <summary>
 /// Property-access benchmarks. Measures cost of <c>obj.foo</c> lookup in
@@ -31,7 +31,7 @@ public void Setup()
     {
         var assembly = typeof(PropertyAccessBenchmarks).Assembly;
         using var stream = assembly.GetManifestResourceStream(
-            "SharpTS.Benchmarks.TypeScriptSources.PropertyAccess.ts")
+            "SharpTS.Microbenchmarks.TypeScriptSources.PropertyAccess.ts")
             ?? throw new InvalidOperationException("Could not find embedded resource PropertyAccess.ts");
         using var reader = new StreamReader(stream);
         var tsSource = reader.ReadToEnd();

SharpTS.Benchmarks/Benchmarks/RegexBenchmarks.cs → SharpTS.Microbenchmarks/Benchmarks/RegexBenchmarks.cs

@@ -1,9 +1,9 @@
 using System.Reflection;
 using BenchmarkDotNet.Attributes;
 using BenchmarkDotNet.Order;
-using SharpTS.Benchmarks.Infrastructure;
+using SharpTS.Microbenchmarks.Infrastructure;
 
-namespace SharpTS.Benchmarks.Benchmarks;
+namespace SharpTS.Microbenchmarks.Benchmarks;
 
 /// <summary>
 /// Measures regex literal compilation overhead. Each TS regex literal
@@ -35,7 +35,7 @@ public void Setup()
     {
         var assembly = typeof(RegexBenchmarks).Assembly;
         using var stream = assembly.GetManifestResourceStream(
-            "SharpTS.Benchmarks.TypeScriptSources.Regex.ts")
+            "SharpTS.Microbenchmarks.TypeScriptSources.Regex.ts")
             ?? throw new InvalidOperationException("Could not find embedded resource Regex.ts");
         using var reader = new StreamReader(stream);
         var tsSource = reader.ReadToEnd();

SharpTS.Benchmarks/Benchmarks/StarterWorkloadBenchmarks.cs → SharpTS.Microbenchmarks/Benchmarks/StarterWorkloadBenchmarks.cs

@@ -1,8 +1,8 @@
 using BenchmarkDotNet.Attributes;
 using BenchmarkDotNet.Order;
-using SharpTS.Benchmarks.Baselines;
+using SharpTS.Microbenchmarks.Baselines;
 
-namespace SharpTS.Benchmarks.Benchmarks;
+namespace SharpTS.Microbenchmarks.Benchmarks;
 
 // Starter-set workloads that broaden coverage beyond the original arithmetic
 // kernels: a builtin-heavy JSON round-trip, a data-parallel typed-array kernel,

SharpTS.Benchmarks/Infrastructure/BenchmarkHarness.cs → SharpTS.Microbenchmarks/Infrastructure/BenchmarkHarness.cs

@@ -3,7 +3,7 @@
 using SharpTS.Parsing;
 using SharpTS.TypeSystem;
 
-namespace SharpTS.Benchmarks.Infrastructure;
+namespace SharpTS.Microbenchmarks.Infrastructure;
 
 /// <summary>
 /// Harness for compiling TypeScript to .NET assemblies and invoking compiled methods

SharpTS.Benchmarks/Infrastructure/CompilationCache.cs → SharpTS.Microbenchmarks/Infrastructure/CompilationCache.cs

@@ -1,6 +1,6 @@
 using System.Collections.Concurrent;
 
-namespace SharpTS.Benchmarks.Infrastructure;
+namespace SharpTS.Microbenchmarks.Infrastructure;
 
 /// <summary>
 /// Manages pre-compilation of TypeScript sources during benchmark initialization.

SharpTS.Benchmarks/Program.cs → SharpTS.Microbenchmarks/Program.cs

@@ -5,7 +5,7 @@
 using BenchmarkDotNet.Columns;
 using BenchmarkDotNet.Order;
 
-namespace SharpTS.Benchmarks;
+namespace SharpTS.Microbenchmarks;
 
 /// <summary>
 /// Entry point for SharpTS benchmark suite.

SharpTS.Microbenchmarks/README.md

@@ -0,0 +1,80 @@
+# Microbenchmarks (`SharpTS.Microbenchmarks/`)
+
+**Internal / headroom benchmarks.** This [BenchmarkDotNet](https://benchmarkdotnet.org/)
+suite answers a different question from the cross-runtime suite: *how close is
+SharpTS-compiled TypeScript to the performance ceiling, and where does the
+overhead go?* It is the harness behind the compiler perf work (object-shape
+structs, typed-array fast paths, the merge-sort `Array.sort`, etc.).
+
+Each workload is measured three ways:
+
+- **SharpTS** — TypeScript compiled to IL by `ILCompiler`, invoked in-process.
+- **Idiomatic** — hand-written C# using native types. The **performance ceiling**.
+- **Equivalent** — C# written with `object?`/boxing to approximate the
+  **dynamic-typing tax** SharpTS pays for JS semantics.
+
+It is part of `SharpTS.sln` and runs **in-process**: the TypeScript is compiled
+once to a DLL at `[GlobalSetup]`, then reached through a cached strongly-typed
+`Func<double,double>` delegate so reflection and argument boxing stay **out of
+the timed region**.
+
+> For the *external* comparison against Node.js and Bun, see
+> [`../benchmarks`](../benchmarks). That suite has a table explaining why the two
+> are kept separate ("Why two benchmark suites?").
+
+## Layout
+
+| Path | Purpose |
+|------|---------|
+| `Program.cs` | BenchmarkDotNet entry point (GitHub-Markdown + HTML exporters, `MemoryDiagnoser`, rank/ops-per-sec columns). |
+| `Benchmarks/*.cs` | One file per workload family (computational, starter workloads, array helpers, property access, object literals, regex). |
+| `Baselines/IdiomaticCSharp.cs` | Native-type C# baselines — the ceiling. |
+| `Baselines/EquivalentCSharp.cs` | `object?`/boxing C# baselines — the dynamic-typing tax. |
+| `Infrastructure/BenchmarkHarness.cs` | Compile TS → DLL, load it, resolve compiled methods/delegates. |
+| `Infrastructure/CompilationCache.cs` | Compile each TS source once, reuse across benchmark classes. |
+| `TypeScriptSources/*.ts` | TS bodies for the non-computational workloads (embedded as resources). |
+
+The computational/starter workloads load their TS from
+`../benchmarks/scripts/lib/algorithms.ts`, embedded as the resource
+`SharpTS.Microbenchmarks.algorithms.ts`. That file is **shared byte-identical**
+with the cross-runtime shell harness, so both suites measure the same source.
+
+## Running
+
+```bash
+# From the repo root. BenchmarkDotNet requires a Release build.
+dotnet run -c Release --project SharpTS.Microbenchmarks
+
+# Interactive picker, or filter to a subset:
+dotnet run -c Release --project SharpTS.Microbenchmarks -- --filter '*Fibonacci*'
+dotnet run -c Release --project SharpTS.Microbenchmarks -- --list flat
+```
+
+Results (Markdown + HTML, plus allocation columns) are written under
+`BenchmarkDotNet.Artifacts/`.
+
+## Conventions
+
+- **One algorithm per class**, each with a single `[Params]` axis — a single
+  class with multiple independent `[Params]` would run BenchmarkDotNet's full
+  Cartesian product and waste ~Nx of the work.
+- Compiled functions are reached via `ComputationalBenchmarkBase.LoadCompiled`,
+  which returns a cached `Func<double,double>` — keep reflection/boxing outside
+  `[Benchmark]` methods so the measurement reflects the generated IL, not the
+  invocation plumbing.
+- Embedded-resource names are referenced **by string** (e.g.
+  `"SharpTS.Microbenchmarks.TypeScriptSources.Regex.ts"`). `RootNamespace`/
+  `AssemblyName` are pinned in the `.csproj` so those names stay stable; if you
+  rename the project, keep the strings and the pinned names in sync.
+
+## Embedded-resource gotcha
+
+A wrong resource name **compiles fine** but throws at `[GlobalSetup]`
+(`GetManifestResourceStream` returns null). After adding a `.ts` source or
+renaming anything, verify the manifest names resolve:
+
+```powershell
+$asm = [System.Reflection.Assembly]::LoadFrom(
+  (Get-ChildItem -Recurse SharpTS.Microbenchmarks/bin -Filter SharpTS.Microbenchmarks.dll)[0].FullName)
+$asm.GetManifestResourceNames()
+```

SharpTS.Benchmarks/SharpTS.Benchmarks.csproj → SharpTS.Microbenchmarks/SharpTS.Microbenchmarks.csproj

@@ -7,6 +7,12 @@
     <Nullable>enable</Nullable>
     <IsPackable>false</IsPackable>
 
+    <!-- Pinned explicitly so embedded-resource names (referenced by string in
+         the benchmark classes, e.g. "SharpTS.Microbenchmarks.TypeScriptSources.Regex.ts")
+         stay stable regardless of the project/folder name. -->
+    <AssemblyName>SharpTS.Microbenchmarks</AssemblyName>
+    <RootNamespace>SharpTS.Microbenchmarks</RootNamespace>
+
     <!-- Required for Release builds with optimizations -->
     <Optimize>true</Optimize>
     <DebugType>portable</DebugType>
@@ -28,7 +34,7 @@
     <EmbeddedResource Include="TypeScriptSources\**\*.ts" />
     <!-- Shared with the cross-runtime shell harness (benchmarks/scripts/lib)
          so both benchmark systems measure byte-identical algorithm sources. -->
-    <EmbeddedResource Include="..\benchmarks\scripts\lib\algorithms.ts" LogicalName="SharpTS.Benchmarks.algorithms.ts" />
+    <EmbeddedResource Include="..\benchmarks\scripts\lib\algorithms.ts" LogicalName="SharpTS.Microbenchmarks.algorithms.ts" />
   </ItemGroup>
 
   <!-- Create output directory for pre-compiled DLLs -->

SharpTS.csproj

@@ -1,4 +1,4 @@
-<Project Sdk="Microsoft.NET.Sdk">
+<Project Sdk="Microsoft.NET.Sdk">
 
   <PropertyGroup>
     <OutputType>Exe</OutputType>
@@ -57,8 +57,8 @@
     <None Remove="SharpTS.Test262.Worker/**" />
     <Compile Remove="SharpTS.TypeScriptConformance/**" />
     <None Remove="SharpTS.TypeScriptConformance/**" />
-    <Compile Remove="SharpTS.Benchmarks/**" />
-    <None Remove="SharpTS.Benchmarks/**" />
+    <Compile Remove="SharpTS.Microbenchmarks/**" />
+    <None Remove="SharpTS.Microbenchmarks/**" />
     <Compile Remove="SharpTS.Example.Interop/**" />
     <None Remove="SharpTS.Example.Interop/**" />
     <Compile Remove="Examples/**" />

SharpTS.sln

@@ -1,4 +1,4 @@
-
+
 Microsoft Visual Studio Solution File, Format Version 12.00
 # Visual Studio Version 18
 VisualStudioVersion = 18.3.11312.210
@@ -9,7 +9,7 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "SharpTS.Tests", "SharpTS.Te
 EndProject
 Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "SharpTS.Example.Interop", "Examples\Interop\SharpTS.Example.Interop.csproj", "{189D6325-E224-0097-E8A5-0FC6C44652A0}"
 EndProject
-Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "SharpTS.Benchmarks", "SharpTS.Benchmarks\SharpTS.Benchmarks.csproj", "{27E0C313-AB29-43BC-AA2C-E52F13E634AE}"
+Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "SharpTS.Microbenchmarks", "SharpTS.Microbenchmarks\SharpTS.Microbenchmarks.csproj", "{27E0C313-AB29-43BC-AA2C-E52F13E634AE}"
 EndProject
 Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "SharpTS.Sdk", "SharpTS.Sdk\SharpTS.Sdk.csproj", "{35E3B550-4DE0-4D5F-ADC3-C80DF7092F17}"
 EndProject

benchmarks/README.md

@@ -0,0 +1,88 @@
+# Cross-Runtime Benchmarks (`benchmarks/`)
+
+**External / competitive benchmarks.** This suite answers one question: *how does
+SharpTS compare to other JavaScript/TypeScript runtimes?* The bar is to **meet or
+exceed Node.js**.
+
+It runs the same TypeScript workloads across four runtimes and prints a
+side-by-side table:
+
+- SharpTS **interpreter** (`dotnet run -- script.ts`)
+- SharpTS **compiled** (`dotnet run -- --compile script.ts` → run the DLL)
+- **Node.js** (with `--experimental-strip-types` on Node < 23)
+- **Bun** (if installed)
+
+Measurement is **whole-program, process-level** (includes startup + the full
+pipeline), so it reflects what a user actually experiences invoking each runtime.
+
+> For the *internal* benchmarks — SharpTS-compiled vs the idiomatic-C#
+> performance ceiling, with allocation/GC profiling — see
+> [`../SharpTS.Microbenchmarks`](../SharpTS.Microbenchmarks). The two suites are
+> complementary and measure different things; see "Why two suites?" below.
+
+## Layout
+
+| Path | Purpose |
+|------|---------|
+| `run-benchmarks.ps1` | Builds SharpTS (Release), runs every `scripts/*.ts` on all runtimes, writes `results.txt`. |
+| `format-results.ps1` | Renders `results.txt` as a Markdown table (used for the CI job summary). |
+| `scripts/*.ts` | One workload per file (fibonacci, sort, json, regex, closures, …). |
+| `scripts/lib/bench.ts` | Shared cross-runtime timing harness (auto-batching, warmup, mean/min/stdev). |
+| `scripts/lib/algorithms.ts` | Algorithm bodies **shared byte-identical** with the microbenchmark suite (embedded there as a resource). |
+
+## Running
+
+```powershell
+# Run everything; results land in $TEMP/bench-results/results.txt
+./benchmarks/run-benchmarks.ps1
+
+# Render the table from a results file
+./benchmarks/format-results.ps1 -ResultsFile $env:TEMP/bench-results/results.txt
+```
+
+Override the output directory with `$env:OUTPUT_DIR`. Node and Bun are detected
+automatically; Bun is skipped if not on `PATH`.
+
+## How timing works
+
+Each workload calls `bench(name, param, fn)` from `scripts/lib/bench.ts`, which:
+
+1. Probes once; if a single call is already ≥ 1 ms it samples one call at a time
+   (honest for slow cases like the tree-walking interpreter on big inputs).
+2. Otherwise warms the JIT, calibrates an inner batch until a sample spans ≥ 1 ms
+   (lifting fast cases above the timer noise floor), then samples to a budget.
+3. Emits one line per case, consumed by `format-results.ps1`:
+
+   ```
+   BENCH:<name>:<param>:<meanMs>:<minMs>:<stdevMs>
+   ```
+
+`performance.now()` (sub-microsecond, monotonic) is used everywhere so the
+methodology is identical across runtimes. A `guard` accumulator defeats
+dead-code elimination in both SharpTS modes and the JS engines.
+
+If a runtime produces no `BENCH:` line (crash, parse error, missing API),
+`run-benchmarks.ps1` warns loudly and echoes the tail of its output rather than
+silently leaving a blank cell.
+
+## CI
+
+`.github/workflows/benchmarks.yml` runs this suite on `workflow_dispatch` and
+publishes the formatted table to the job summary, with the raw `results.txt`
+uploaded as an artifact. It is **not** run on every push (timing on shared CI
+runners is noisy and the full sweep is slow).
+
+## Why two benchmark suites?
+
+| | `benchmarks/` (this suite) | `SharpTS.Microbenchmarks/` |
+|---|---|---|
+| **Question** | Are we as fast as Node/Bun? | How close are we to the C# ceiling, and where's the overhead? |
+| **Compares against** | Node.js, Bun | Idiomatic C# (native types) + "equivalent" C# (`object?`/boxing) |
+| **Tool** | PowerShell + shared `bench.ts` | BenchmarkDotNet |
+| **Scope** | Whole-program, process-level | In-process, per-function (delegate-invoked) |
+| **Profiling** | Wall-clock mean/min/stdev | + allocations/GC (`MemoryDiagnoser`) |
+
+They can't be merged: BenchmarkDotNet must run in-process against managed code
+(it can't drive the `node`/`bun` executables), and the cross-runtime comparison
+must be black-box at the process boundary. Keeping them separate is intentional;
+the shared `scripts/lib/algorithms.ts` ensures both measure identical source.