Skip to content

[math-08] Enforce formula-to-code oracle coverage #200

Description

@krystophny

Stage: Formula-to-code verification integration
Source language: Wolfram Language plus the repository's implementation and test languages
Manuals to read first: #193-#199; verification/FORMULA_INDEX.md; repository CMake/CTest and golden-test documentation
Depends on: #193, #194, #195, #196, #197, #198, #199

Goal

Close the thesis-to-code audit by making every implemented physics formula consume a committed independent oracle or a behavioral identity test. Publish an errata list for every thesis/code discrepancy and prevent unverified formulas from entering KIM, KiLCA, or QL-Balance.

Files to edit

  • verification/run_all.wls: NEW deterministic runner for every Mathematica script and formula-index coverage gate.
  • verification/ERRATA.md: NEW discrepancy record with source equation, code path, correction, and evidence.
  • CMakeLists.txt: register generated-oracle readers and the formula-coverage test without requiring Mathematica in normal CI.

Behavior to implement

Run scripts in dependency order, require named PASS checks and nonzero failure, regenerate oracle files deterministically at declared precision, and compare clean regeneration byte-for-byte or numerically under a recorded serialization rule. Add Fortran tests that read each oracle and fail on semantic mutations. Audit FORMULA_INDEX.md: no code formula may remain missing, unclassified, or unresolved when this issue closes. Definitions must still pass units/convention checks; approximations must carry a domain and error bound; cited premises must link to a stable public source.

Scaffold

scripts = FileNames["*.wl", FileNameJoin[{root,"mathematica"}]];
results = RunProcess[{"math","-script",#}]& /@ scripts;
If[AnyTrue[results, #ExitCode != 0 &] || formulaCoverage[] != 1, Exit[1]];

Positive fixtures to add

Negative fixtures to add

  • Delete an index row, change a formula factor, stale an oracle, or leave an approximation unbounded: the appropriate local and CI checks fail.

Makefile target

Add verify-mathematica for licensed developer machines and verify-formula-oracles for ordinary CI.

Success criteria

math -script verification/run_all.wls
cmake -S . -B build -G Ninja
cmake --build build -j$(nproc)
ctest --test-dir build -R 'formula|oracle|kernel|suscept|ql_balance' --output-on-failure

FORMULA_INDEX.md must report 100% coverage and zero unresolved code formulas. Every discrepancy is corrected or recorded in verification/ERRATA.md with a linked follow-up issue.

Non-goals

  • Do not require a Mathematica license on ordinary GitHub Actions runners.
  • Do not accept generated fixtures without their symbolic source and provenance.

Verification

math -script verification/run_all.wls && ctest --test-dir build --output-on-failure

Metadata

Metadata

Assignees

Labels

P1Urgent: ship this sprintdocumentationImprovements or additions to documentation

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions