Skip to content

Linearize additive systs around param model; add --forceLinear#135

Merged
davidwalter2 merged 4 commits into
WMass:mainfrom
davidwalter2:260513_forceLinear
May 19, 2026
Merged

Linearize additive systs around param model; add --forceLinear#135
davidwalter2 merged 4 commits into
WMass:mainfrom
davidwalter2:260513_forceLinear

Conversation

@davidwalter2
Copy link
Copy Markdown
Collaborator

Summary

  • Scale additive systematics by the param-model value at the linearization point. For systematic_type == \"normal\" with a non-trivial param model (e.g. ABCD / SmoothABCD whose POIs typically sit far from their default), the linearized variation rnorm(poi) * norm + theta * logk did not naturally scale with rnorm, so a ±20% effect defined at the MC nominal became a different relative effect once rnorm moved away from 1. Now logk is pre-multiplied by rnorm_init = param_model.compute(xparamdefault) once at fitter construction. The scaling is a constant so the hot path stays strictly linear in theta (no theta·poi bilinearity). log_normal already had this property, so its tensors are just aliased — no copy.
  • Add --forceLinear to force self.is_linear = True regardless of the structural checks. minimize() then takes a single Cholesky / Hessian-CG step from the current x (a Gaussian approximation around that point). Intended for iterative linearized unfolding, where the outer loop re-anchors the linearization point.
  • Minor refactor in parsing.py: pull VALID_BIN_BY_BIN_STAT_TYPES directly from rabbit.bbstat.bbstat instead of importing the full fitter module.
  • Black formatting on the three tutorial notebooks.

Test plan

  • Existing test_tensor.hdf5 (log_normal) and test_tensor_linearized.hdf5 (normal) fits still converge to EDM ≈ 1e-16 with no flag.
  • Inline sanity check confirms self.logk scales process 0 by 2.5 and leaves other processes untouched when a test param model with xparamdefault=[2.5] is plugged in.
  • --forceLinear emits the info message and takes the Cholesky path on test_tensor_linearized.hdf5.
  • Validate end-to-end inside the iterative linearized unfolding loop on a real W/Z analysis input.

🤖 Generated with Claude Code

davidwalter2 and others added 4 commits May 13, 2026 09:38
For systematic_type == "normal", the linearized variation
  rnorm(poi) * norm + theta * logk
does not scale with the param-model factor rnorm. A +/-20% variation
defined at the MC nominal therefore becomes a different relative effect
once rnorm moves away from 1 (e.g. for data-driven background
parametrisations like ABCD/SmoothABCD whose POIs are typically far from
their default value).

Pre-multiply logk by rnorm_init = param_model.compute(xparamdefault) once
at fitter construction so the relative size of an additive variation
matches the multiplicative case at the linearization point. The scaling
is a constant, so the hot path stays strictly linear in theta and no
theta*poi bilinearity is introduced.

For systematic_type == "log_normal" the existing rnorm * exp(theta*logk)
form already has this property, so self.logk / self.logk_csr just alias
the original indata tensors and no copy is made.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Force self.is_linear True regardless of the structural checks
(chisqFit/covarianceFit, param_model linearity, symmetric tensor,
systematic_type, bbstat linearity). The minimize() path then takes a
single Cholesky / Hessian-CG step from the current x, i.e. a Gaussian
approximation around that point. The user is responsible for ensuring
the result is meaningful, typically by wrapping the fit in an outer
iteration that re-anchors the linearization point.

Also drop the heavyweight 'from rabbit import fitter' import in
parsing.py in favor of pulling VALID_BIN_BY_BIN_STAT_TYPES directly
from rabbit.bbstat.bbstat, so parsing no longer needs to import the
full fitter module.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@davidwalter2 davidwalter2 merged commit 8a5aef2 into WMass:main May 19, 2026
32 of 40 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant