omnibench

Torch-Sim-driven benchmark suite for evaluating machine-learned interatomic potentials (MLIPs). It is developed within SevenNet-Omni paper and provides a unified CLI to run standardized benchmarks covering molecules, organometallic reactions, molecular crystal cohesions, and heterogeneous surface reactions.

Overview

Relative-MAE overview (normalized to SevenNet-Omni performance).

Design Principles

• Torch-Sim as the default relaxation engine
• D3 dispersion correction Two backends are available:
  1. SevenNet's batched CUDA kernel via D3Builder (--d3-backend sevennet)
  2. TorchSim native D3 + SumModel (--d3-backend torchsim)

Installation

For building MLIP provider-specific environment, use the helper scripts in scripts/:

./scripts/build-venv-sevenn.sh
./scripts/build-venv-nequip.sh
./scripts/build-venv-fairchem.sh
./scripts/build-venv-orb.sh

MACE + SevenNet D3 setup

MACE cannot be installed with a working SevenNet D3 path by default. After creating and activating .venv-mace, use this exact sequence if you need the SevenNet D3 backend in MACE runs:

uv venv .venv-mace
source .venv-mace/bin/activate
uv pip install -e ".[mace]"
uv pip install --no-deps "sevenn @ git+https://github.com/MDIL-SNU/SevenNet.git@7010cf440fd37a6ebfb915a2317ae9a7b7f8a3c6"
uv pip install "torch-geometric>=2.5.0" braceexpand
uv pip install "e3nn==0.4.4"

Then patch sevenn/__init__.py in that environment to disable the e3nn version guard that raises on import. This is required because the MACE stack expects e3nn==0.4.4.

Quick Start

# SevenNet
./scripts/build-venv-sevenn.sh
sbatch run-sevenn.sh

# NequIP
./scripts/build-venv-nequip.sh
sbatch run-nequip.sh

# UMA / FairChem
./scripts/build-venv-fairchem.sh
sbatch run-uma.sh

# ORB
./scripts/build-venv-orb.sh
sbatch run-orb.sh

# MACE
sbatch run-mace.sh

The recommended workflow is:

1. Build the provider venv once.
2. Submit the matching job script.

The job scripts cd to the repository root automatically and reuse an existing venv if it is already present.

Benchmarks

Name	Description	Structures	Metrics	Reference DFT Level of Theory
torsion-pbe-d3	Torsion barrier energy	188 molecules	MAE	PBE-D3
torsion-wb97m-d3	Torsion barrier energy	188 molecules	MAE	wB97M-D3
molecular-crystals-pbe-d3	Molecular crystal volume	86 crystals	MAPE	PBE-D3
bmcos1-r2scan-d3	Molecular crystal volume	67 crystals	MAPE	r2SCAN-D3
organometallic-pbe-d3	Organometallic reaction energy	53 reactions	MAE	PBE-D3
hoip-pbe-d3	Hybrid organic-inorganic perovskite formation energy	100 reactions	MAE	PBE-D3
ads41-r2scan	Metal surface adsorption	41 reactions	MAE	r2SCAN
metal-adsorption-pbe	Metal surface adsorption	120 reactions	MAE	PBE

CLI

omnibench run CHECKPOINT --bench BENCH_NAME [OPTIONS]
omnibench run CALCULATOR CHECKPOINT --bench BENCH_NAME [OPTIONS]

Options:
  --dispersion/--no-dispersion  Apply D3 dispersion correction
  --d3-backend [sevennet|torchsim]  D3 backend (default: sevennet)
  --output-dir PATH          Output directory (default: ./omnibench_results/)
  --calculator TEXT           Calculator provider: sevenn (default), nequip, mace, fairchem
  --calc-opt KEY=VALUE        Calculator-specific option (repeatable)
  --batch-size INTEGER        GPU batch capacity (0 = auto-estimate, 1 = sequential)

omnibench list               List available benchmarks

If --d3-backend is supplied while dispersion is off, omnibench prints a stdout warning and ignores the backend selection.

Calculator-specific options example (--calc-opt)

Calculator	Option	Description
sevenn	modal=mpa	Multi-fidelity modality
sevenn	enable_oeq=true	Enable OpenEquivariance
mace	head=omat_pbe	Required for mace-mh-1; selects the multi-head target such as omat_pbe, matpes_r2scan, or omol
fairchem	task=omat	UMA task name used to infer the D3 functional

Output

Each run produces results in {output_dir}/{bench_name}/{model_id}/.

omnibench_results/ads41-r2scan/7net-omni_matpes_r2scan/
  *.extxyz                 # Relaxed structures
  indiv_results.csv        # Individual energies
  rxn_energies.csv         # Reaction energies
  rxn_plot_data.csv        # Saved parity-plot table (when applicable)
  bench_meta.yaml          # Run metadata: dispersion, backend, calculator, etc.
  rxn_energies.yaml        # Evaluation metrics
  rxn_energies.png         # Parity plot (DFT vs predicted)

Citation

If you use this benchmark suite, please cite:

@article{kim_optimizing_2025,
	title = {Optimizing Cross-Domain Transfer for Universal Machine Learning Interatomic Potentials},
	doi = {10.48550/arxiv.2510.11241},
	journal = {arXiv},
	author = {Kim, Jaesun and You, Jinmu and Park, Yutack and Lim, Yunsung and Kang, Yujin and Kim, Jisu and Jeon, Haekwan and Ju, Suyeon and Hong, Deokgi and Lee, Seung Yul and Choi, Saerom and Kim, Yongdeok and Lee, Jae W and Han, Seungwu},
	year = {2025},
}

D3 Dispersion Backends

SevenNet D3 (default backend)

Uses sevenn.torchsim_d3.SevenNetD3Model — a composite model with batched CUDA D3 kernel. Supports any functional via SevenNet's internal parameter table. Only BJ damping (damp_bj) is used.

TorchSim Native D3

Uses torch_sim.models.dispersion.D3DispersionModel + torch_sim.models.interface.SumModel from Torch-Sim.

License

TBD