test(golden): floor near-zero noise in gr_numcompare (fixes rho false-fail)#169
Merged
Conversation
gr_numcompare judged every element by relative diff with only a 1e-12 absolute floor. Oscillatory fields have zero crossings that come out as O(1e-12) floating-point noise -- e.g. KIM's charge density rho is O(1e2) physically but ~1e-12 at its crossings. A last-bit difference between two builds there (fortnum vs GSL, or even two GSL versions) is a negligible absolute diff but tens of percent relative, so the KIM electromagnetic golden failed on rho alone while every physical quantity matched to ~1e-14. Reproduced locally: rho agrees to 5.9e-13 absolute (13 digits); pass/fail flips purely on which GSL version builds the baseline. Skip elements where both reference and current value are at or below a near-zero floor (default 1e-9, arg 5) -- 3 orders below the smallest real value in the golden set (~1e-6), so it never masks a physical divergence. Adds two unit tests (noise skipped; real divergence still fails). Refs #164.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The
goldenjob fails on the fortnum-migration PRs (first seen on #140) with exactly one quantity over tolerance:Every other quantity — all fields, all kernels, all of QL-Balance and KiLCA — passes, most at
0.000e+00.Root cause — a comparator artifact, not a code regression
gr_numcompare.pyjudged every array element by relative diff with only a1e-12absolute floor.rho(charge density) is an oscillatorym=6radial profile:max|rho| ≈ 96, but at its zero crossings the values areO(1e-12)floating-point noise. A last-bit difference between two builds there (fortnum vs GSL — or even two GSL versions) is a negligible absolute diff but tens of percent relative.Reproduced locally by building both baseline (GSL) and the fortnum branch, running the KIM electromagnetic case, and diffing
rho.datelement-by-element:Max absolute diff across all 128 elements:
5.87e-13— ~13 digits of agreement. The physicalrhoprofile is identical; only noise-level zero-crossings diverge, and only in relative terms. Whether a given zero-element trips the bar is a coin flip on the reference build's last bits — GSL-2.8 vs fortnum passes locally, CI's GSL-2.4 vs fortnum fails. That is the signature of a tolerance artifact, not a regression. Same physics as #164, on a quantitygr_numcomparecannot exclude.Fix
Skip elements where both the reference and current value are at or below a near-zero floor (default
1e-9, overridable via arg 5). The floor sits 3 orders below the smallest physically meaningful value in the golden set (1e-6) and 3 orders above the noise (1e-12), so it floors only genuine noise and never masks a real divergence on a physical value.rho.dat(and the many-zero kernel files) now pass on real output with a(N near-zero skipped)annotation.Refs #164. Unblocks the fortnum-migration golden gate (#140–#150).