Skip to content

feat: configurable high-iteration benchmarks with env var overrides#76

Open
Copilot wants to merge 2 commits into
mainfrom
copilot/run-benchmark-high-iteration
Open

feat: configurable high-iteration benchmarks with env var overrides#76
Copilot wants to merge 2 commits into
mainfrom
copilot/run-benchmark-high-iteration

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 6, 2026

Benchmark iteration counts were hardcoded, making it impossible to run at higher fidelity without editing source. This adds runtime control via env vars and a dedicated high-iteration npm script.

Changes

  • benchmarks/logger.bench.ts — replaces hardcoded BenchmarkConfig constants with makeConfig() that reads ZARIO_BENCH_ITERATIONS and ZARIO_BENCH_SAMPLES at runtime; warmup auto-scales to 20% of iterations
  • package.json — adds bench and bench:high scripts; adds tsx to devDependencies
  • benchmarks/RESULTS.md — captures results from the 500k-iteration run (8 samples)

Usage

# Standard defaults (50k–200k iter/sample, 6 samples)
npm run bench

# High-fidelity preset (500k iter/sample, 8 samples)
npm run bench:high

# Fully custom
ZARIO_BENCH_ITERATIONS=1000000 ZARIO_BENCH_SAMPLES=10 npm run bench

High-iteration results snapshot (500k/sample, 8 samples)

Benchmark Median ops/sec Median ns/op
Simple message (sync) 11,552,893 87
With metadata (sync) 10,572,753 95
Deep metadata (sync) 6,687,858 150
Filtered (early exit) ~1,007,000,000 1
Simple JSON log 11,568,692 86
Async enqueue 5,079,084 197
Child logger 10,453,676 96

Copilot AI changed the title [WIP] Run benchmark at very high iteration feat: configurable high-iteration benchmarks with env var overrides Apr 6, 2026
Copilot AI requested a review from dev-dami April 6, 2026 18:37
@dev-dami dev-dami marked this pull request as ready for review April 6, 2026 18:41
Copilot AI review requested due to automatic review settings April 6, 2026 18:41
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds runtime configurability to the logger benchmark suite so benchmark fidelity can be increased without editing source, and provides a high-iteration preset script for repeatable runs.

Changes:

  • Introduces env-var driven benchmark configs (ZARIO_BENCH_ITERATIONS, ZARIO_BENCH_SAMPLES) with a shared makeConfig() helper and warmup scaling.
  • Adds bench and bench:high npm scripts and includes tsx to run the TypeScript benchmark file directly.
  • Adds a benchmarks/RESULTS.md snapshot for the high-iteration preset run.

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 2 comments.

File Description
package.json Adds benchmark scripts and tsx dev dependency to run TS benchmarks via Node.
package-lock.json Updates lockfile to include tsx and its dependency tree.
benchmarks/logger.bench.ts Replaces hardcoded configs with env-var override support and warmup scaling logic.
benchmarks/RESULTS.md Documents a high-iteration benchmark run output and how to run benchmarks.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +33 to +39
const BENCH_ITERATIONS = Number(process.env["ZARIO_BENCH_ITERATIONS"] ?? 0);
const BENCH_SAMPLES = Number(process.env["ZARIO_BENCH_SAMPLES"] ?? 0);

function makeConfig(baseIterations: number, baseWarmup: number): BenchmarkConfig {
const iterations = BENCH_ITERATIONS > 0 ? BENCH_ITERATIONS : baseIterations;
const warmupIterations = BENCH_ITERATIONS > 0 ? Math.floor(iterations * 0.2) : baseWarmup;
const samples = BENCH_SAMPLES > 0 ? BENCH_SAMPLES : 6;
Copy link

Copilot AI Apr 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The benchmark config reads iteration/sample counts directly via Number(...) and only checks > 0. This allows non-integers (e.g. 0.5), Infinity (e.g. 1e309/Infinity), or very large values, which can lead to misleading metrics (division by non-integer) or effectively infinite loops/hangs. Consider parsing as an integer and validating with Number.isFinite + Number.isInteger, and either clamping or throwing a clear error when invalid/out-of-range values are provided.

Suggested change
const BENCH_ITERATIONS = Number(process.env["ZARIO_BENCH_ITERATIONS"] ?? 0);
const BENCH_SAMPLES = Number(process.env["ZARIO_BENCH_SAMPLES"] ?? 0);
function makeConfig(baseIterations: number, baseWarmup: number): BenchmarkConfig {
const iterations = BENCH_ITERATIONS > 0 ? BENCH_ITERATIONS : baseIterations;
const warmupIterations = BENCH_ITERATIONS > 0 ? Math.floor(iterations * 0.2) : baseWarmup;
const samples = BENCH_SAMPLES > 0 ? BENCH_SAMPLES : 6;
function parsePositiveIntegerEnv(name: string): number | undefined {
const rawValue = process.env[name];
if (rawValue == null || rawValue.trim() === "") {
return undefined;
}
const parsedValue = Number(rawValue);
if (!Number.isFinite(parsedValue) || !Number.isInteger(parsedValue) || parsedValue <= 0) {
throw new Error(
`Invalid ${name}: expected a finite positive integer, received "${rawValue}".`,
);
}
return parsedValue;
}
const BENCH_ITERATIONS = parsePositiveIntegerEnv("ZARIO_BENCH_ITERATIONS");
const BENCH_SAMPLES = parsePositiveIntegerEnv("ZARIO_BENCH_SAMPLES");
function makeConfig(baseIterations: number, baseWarmup: number): BenchmarkConfig {
const iterations = BENCH_ITERATIONS ?? baseIterations;
const warmupIterations = BENCH_ITERATIONS != null ? Math.floor(iterations * 0.2) : baseWarmup;
const samples = BENCH_SAMPLES ?? 6;

Copilot uses AI. Check for mistakes.
Comment thread package.json
Comment on lines +35 to +36
"bench": "node --expose-gc --import tsx/esm benchmarks/logger.bench.ts",
"bench:high": "ZARIO_BENCH_ITERATIONS=500000 ZARIO_BENCH_SAMPLES=8 node --expose-gc --import tsx/esm benchmarks/logger.bench.ts",
Copy link

Copilot AI Apr 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The bench:high script sets env vars using POSIX inline assignment (ZARIO_BENCH_ITERATIONS=...). This won’t work in Windows cmd.exe (and differs in PowerShell), so the benchmark scripts become shell-dependent. If cross-platform support is desired, use a cross-platform env setter (e.g., cross-env) or move the preset logic into the benchmark script itself (e.g., via CLI args).

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants