Pre-register decisions. Score them against chance.
Open-source null-first decision lab: freeze choices before outcomes, settle against equal-cost chance portfolios, keep a sealed local ledger, and report descriptive evidence — not predictions.
Not a lottery predictor. No betting integration.
M1 detects inconsistent local edits. A full consistent rewrite of every seal (adversary A5) remains a residual risk until M4. See CLAIM_POLICY and THREAT_MODEL.
People over-claim strategies. nullbench makes the honest experiment cheap:
- Pre-register (freeze)
- Score vs equal-cost chance (settle)
- Refuse silent backfill when seals disagree
pip install nullbench
# contributors / M1 gate from source:
pip install -e ".[dev]"Requires Python 3.11+.
nullbench doctor
nullbench demo --name try1
nullbench report --study try1 --open
nullbench next --study try1nullbench init my-study -d demo649
nullbench strategy add random --study my-study --tickets 5 --seed 1
nullbench freeze --study my-study --latest
nullbench settle --study my-study
nullbench report --study my-study --open| Level | Status |
|---|---|
| M0 Lab / PyPI | done |
| M1 Local seals + adversarial tests | done |
| M2 PRD / threat model / API / claims frozen | frozen |
| M3 OIDC publish + SBOM + plugin allowlist | done |
| M4 External vault notary (A5 control) | done |
nullbench maturity
nullbench maturity --check-m1
nullbench maturity --check-m4nullbench vault init
nullbench seal notarize --study try1
nullbench seal verify --study try1from nullbench import init_study, add_strategy, freeze_period, settle_period, build_reportStable surface: docs/PUBLIC_API.md.
| Doc | Purpose |
|---|---|
| PRD | Product requirements (frozen) |
| PUBLIC_API | Stable imports / CLI jobs |
| INTEGRITY | Seal model + plugin allowlist |
| THREAT_MODEL | Adversaries A1–A5 |
| CLAIM_POLICY | What you may say publicly |
| MATURITY | M0–M4 ladder |
| RUNBOOK | Release, Trusted Publisher, ops |
| ARCHITECTURE | Layers |
| CONTRIBUTING | How to hack |
| SECURITY | How to report issues |
| PUBLISH | Build / PyPI |
See CONTRIBUTING.md and CODE_OF_CONDUCT.md.
PRs that touch seals must keep pytest -m m1 green.
Pure simulation / evaluation. Do not market predicted numbers.
MIT — LICENSE.