Skip to content

feat(diagnostics): VAR-aware convergence and stability report - #176

Open
thomaspinder wants to merge 6 commits into
mainfrom
feat/142-convergence-report
Open

feat(diagnostics): VAR-aware convergence and stability report#176
thomaspinder wants to merge 6 commits into
mainfrom
feat/142-convergence-report

Conversation

@thomaspinder

Copy link
Copy Markdown
Owner

Summary

One report answering "can I trust this posterior?" with the two things a generic az.summary cannot give a VAR user: per-parameter-block aggregation with worst-coordinate retention, and the posterior distribution of the companion-matrix spectral radius with the probability of explosive dynamics.

  • FittedVAR.convergence_report() / IdentifiedVAR.convergence_report(), delegating to a new public impulso.diagnostics.convergence_report(idata, *, n_lags, ...). Blocks: coefficient / intercept / exog / covariance / volatility / identification / other, resolved by a static name map → an ^v\d+_ SV-latent regex → an optional posterior_var_names() capability on volatility adapters → an other fallback (unknown names never raise). Each block reports worst rank-normalised split-R̂, min bulk/tail ESS, each with its coordinate label (B[y2, L1.y1]). Divergences are reported globally — a divergence is a trajectory property, not a parameter property.
  • Stability: new shared src/impulso/_stability.py (companion_matrix, companion_eigenvalues, spectral_radius — chunked, batch-agnostic, exported as primitives) built to be reused directly by the upcoming ecology stability metrics (feat: add ecology stability metrics with an Ives (2003) reproduction #149). The report carries the radius posterior (median, HDI, p_explosive, the raw read-only (chain, draw) array), with deterministic stride thinning for large systems.
  • Status semantics (ADR-0008): failed is reserved for sampler pathology (R̂ > 1.05, ESS < 100, divergence rate ≥ 1%); explosive draws warn but never fail — near-unit-root mass is a legitimate posterior on level data under a random-walk prior mean. Thresholds are a frozen, configurable, echoed ConvergenceThresholds (defaults from Vehtari et al. 2021 / Betancourt 2017).
  • The two documented VAR failure modes fire as coded messages: rhat_without_divergences (R̂ can hit 3.0 with zero divergences in VARs; message quotes the copy-pasteable low_rank_modified_mass_matrix nutpie remedy) and explosive_draws (spells out the IRF/forecast/FEVD/HD consequences). The report never calls warnings.warn — the object is the channel.
  • Conjugate fits supported honestly: stability fully computed; R̂ None with a single_chain message; missing sampler stats reported at info severity. Works with both pymc and nutpie sample-stat layouts (only diverging is assumed common; warmup groups ignored).
  • Docs: docs/reference/diagnostics.md, ADR-0008, primitives page additions, CONTEXT.md terms. No tutorial/how-to here — Add tutorial notebook: diagnosing fit in an Impulso VAR #57 owns the prose narrative and gets a comment with this API surface.

Closes #142

Review

Planned by a dedicated planning agent that measured the posterior layouts, arviz NaN edges, sampler-stat names, and eigenvalue costs before design. Independently reviewed: the reviewer re-derived the companion-matrix orientation by hand against spec.py's X_lag @ B.T construction, reproduced the exact fixture pathologies (R̂ 2.845, p_explosive 0.15), and endorsed all seven implementation deviations. One P2 (threshold-boundary prose contradicting the code) fixed in a follow-up commit.

Tests

+103 tests (20 stability, 83 diagnostics incl. one slow real-NUTS fit that runs in ~5s), plus a make_var_posterior conftest factory that builds deterministic pathologies: chain-offset injection for exact R̂ ≈ 2.845 at a known coordinate, noiseless two-point mixtures for p_explosive exactly 0.15, dual pymc/nutpie-shaped sample stats with a poisoned warmup group. Coverage: _stability.py 100%, diagnostics.py 99%.

ruff / ty / fast suite: all green (634 passed, 30 deselected). Docs build (warnings-as-errors) clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Egjd7ToFeb9TQqFnfRQZxV

@codecov-commenter

codecov-commenter commented Jul 29, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.01980% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 95.2%. Comparing base (866ca8f) to head (388ec52).

Files with missing lines Patch % Lines
src/impulso/diagnostics.py 98.0% 2 Missing and 3 partials ⚠️
src/impulso/__init__.py 0.0% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##            main    #176     +/-   ##
=======================================
+ Coverage   95.0%   95.2%   +0.2%     
=======================================
  Files         45      47      +2     
  Lines       3099    3402    +303     
  Branches     380     420     +40     
=======================================
+ Hits        2945    3242    +297     
- Misses       111     114      +3     
- Partials      43      46      +3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

thomaspinder and others added 5 commits July 29, 2026 23:05
Add `_stability.py` as the single source of truth for the VAR companion
form. `B` already stores lag blocks concatenated in lag order along its
trailing axis — exactly the companion matrix's top block row — so the
coefficients are copied in verbatim with no slicing.

`companion_eigenvalues` returns the raw complex roots rather than only
their moduli, because downstream reactivity and return-rate measures
need the imaginary parts. The batch is processed in chunks: cost scales
as O(N * (n * p)^3), so a large posterior over a large system allocates
heavily without a bound on peak memory.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Egjd7ToFeb9TQqFnfRQZxV
`convergence_report(idata, n_lags=...)` answers what a generic
`arviz.summary` cannot: is this VAR posterior usable? It reports R-hat
and both effective sample sizes *per parameter block* — coefficient,
intercept, exog, covariance, volatility, identification, other — each
with the coordinate attaining the worst value, so a mixing problem is
attributed to a part of the model rather than to the model as a whole.
Divergences are counted once, globally: a divergent transition is a
property of a trajectory, not of any single parameter.

Two failure modes get named messages with remedies. `rhat_without_
divergences` is the characteristic VAR pathology — near-collinear lag
regressors make the posterior ill-conditioned, diagonal mass-matrix
adaptation mixes badly across it, and NUTS never has to reject a
trajectory to do so, which is why zero divergences is not reassurance
here. `explosive_draws` reports posterior mass on non-stationary
parameter draws and what it breaks downstream.

Explosive draws warn but never fail: mass near a unit root is a
legitimate posterior statement about level data under a random-walk
prior mean, not evidence the sampler misbehaved. "failed" is reserved
for sampler pathology.

Block resolution is three-tiered — a static map of Impulso's own
variable names, the `v{i}_` prefix on stochastic-volatility latents,
then the new optional `posterior_var_names()` capability on
`VolatilityProcess` (documented as optional, in the
`_samples_rotations` mould, not a protocol requirement). Anything
unrecognised lands in `other`; an unknown variable is never an error.

Conjugate fits are supported with honest gaps: single chain, so R-hat
is None throughout and the report says why; no sampler statistics, so
divergences are None at info severity; stability computed in full. The
report never calls `warnings.warn` — the returned object is the channel.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Egjd7ToFeb9TQqFnfRQZxV
Add the `diagnostics` reference page to the API toctree, list
`companion_matrix` / `spectral_radius` under primitives, and record the
two load-bearing judgements in ADR-0008: why metrics are reported per
parameter block (a single worst-R-hat over a VAR posterior tells the
user nothing about what to change), and why explosive draws warn but
never fail (explosiveness is a property of the model, not the sampler —
failing there would train users to ignore "failed").

CONTEXT.md gains the four terms, two relationship bullets, and two
flagged ambiguities: "companion" is overloaded against ADPRR's
calibrated companion `q_cal`, so always write "companion matrix" in
full; and `StabilitySummary` here is not the `StabilityResult` planned
for the eigenvalue-spectrum work.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Egjd7ToFeb9TQqFnfRQZxV
R-hat/ESS comparisons are strict, but the divergence-rate and explosive-
fraction thresholds trigger at the boundary; the docstring and ADR-0008
claimed blanket strictness, contradicting the tested behaviour.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Egjd7ToFeb9TQqFnfRQZxV
@thomaspinder
thomaspinder force-pushed the feat/142-convergence-report branch from 7d32bea to daf27e5 Compare July 29, 2026 21:06
…base conflict

The rebase conflict resolution merged the new bibliography entry into the
blanchardQuah1989 entry, dropping its closing brace. sphinxcontrib-bibtex
then failed to parse references.bib, breaking build-docs and docs-linkcheck.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: add a VAR-aware convergence and stability report

2 participants