Skip to content

Benchmark versioning for comparing perf results #65

Description

@eellison

We need versioned benchmark results so perf comparisons are reproducible and diffable across torch branches / dates / hardware:

  • A dated/tagged results artifact (e.g. results/<suite>_<hw>_<date>.json) rather than scattered sweep files.
  • A compare tool: baseline-vs-branch, per-repro and per-model deltas.
  • Pin the torch commit + hardware + corpus tag in the result metadata.
    This is the foundation for the perf-prioritization pipeline (re-bench baseline vs tmp_work, aggregate model-level % improvement).

Metadata

Metadata

Assignees

No one assigned

    Labels

    infra-improvementBenchmark infra / tooling / framework improvement

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions