Feature Request: Compare current benchmark results against a previous baseline

[tabuna]

Problem

When optimizing code, I often want to know if a change made performance better or worse compared to a previous version (e.g., the last release). Currently, I have to manually run benchmarks before and after and eyeball the console output, which is error-prone and not automatable in CI.

Proposed Solution

Introduce baseline support to persist benchmark results and compare future runs against them.

Saving results

A ->baseline(string $path)->save() method that stores benchmark results (DTOs) in a serializable format (e.g. JSON):

bench()
    ->compare([
        'algo' => fn () => /* ... */,
    ])
    ->save('./benchmarks/baseline.json');

Comparing against a previous run

When a baseline is provided, existing output methods (e.g. ->toConsole()) can automatically display a comparison:

bench()
    ->baseline('./benchmarks/baseline.json')
    ->compare([
        'algo' => fn () => /* current implementation */,
    ])
    ->toConsole();

The output could include:

• previous vs current values
• delta (time, memory)
• clear indicators (improved / regressed / unchanged)

CI / regression check

A simple assertion-style method for automation:

bench()
    ->baseline('./benchmarks/baseline.json')
    ->assert(maxRegression: 5) // %
    ->compare([
        'algo' => fn () => /* ... */,
    ]);

• throws if regression exceeds threshold
• suitable for CI pipelines

Use Cases & Summary

• CI pipelines: Fail builds on performance regressions
• Developer workflow: Quickly validate optimizations locally
• Release comparison: Track performance changes between versions

This introduces a simple workflow:

run → baseline → compare → assert

without significantly increasing API surface, while enabling reproducible and automatable performance checks.

[andrey-helldar]

This is interesting

[andrey-helldar]

@tabuna Maybe this one?

bench()
    ->cacheDirectory('./.benchmarks') // by default, `./.benchmarks`
    ->compare(...)
    ->toBeRegressionTime(max: 5) // %
    ->toBeRegressionMemory(max: 5); // %

[andrey-helldar]

Released with 4.6.0.

Thanks for issue!