Mixture-proposal importance sampling for Flow estimator#109
Merged
Conversation
Replace the single-flow proposal in Flow._importance_sample with a multiple-importance-sampling mixture of the conditional and marginal flows, weighted by the balance heuristic (a defensive mixture in Hesterberg's sense). A new `proposal_mixture_beta` knob controls the fraction of proposals drawn from the conditional flow; the rest come from the marginal flow, which provides the broad/defensive coverage of the tails the (tempered) conditional under-covers. Both densities are evaluated on the full pooled set and the mixture density is used as the weight denominator, which bounds the worst-case weight and raises the effective sample size. `proposal_mixture_beta=1.0` exactly reproduces the previous single-proposal behaviour for both posterior and proposal modes; the default of 0.5 enables an even defensive mixture. Also add a log_prefix to the FlowDensity online-norm logger so its metrics are namespaced. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #109 +/- ##
==========================================
- Coverage 10.38% 10.33% -0.05%
==========================================
Files 30 30
Lines 3910 3927 +17
==========================================
Hits 406 406
- Misses 3504 3521 +17
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Generalises
Flow._importance_sampleto draw proposals from a mixture of the conditional and marginal flows instead of a single flow, weighted by the balance heuristic (multiple importance sampling). This is a defensive mixture in Hesterberg's sense: the marginal flow is the broad component that covers the tails the (tempered) conditional under-covers, while the conditional flow gives efficiency near the mode. The mixture density is used as the weight denominator, which bounds the worst-case importance weight and raises the effective sample size (importance_sample:n_eff_min).What changed
proposal_mixture_betaparameter (default0.5): fraction of thenum_proposalsdrawn from the conditional flow; the rest are drawn from the marginal flow. Total proposal count is unchanged, so per-call cost is the same.log_prob_condandlog_prob_margare evaluated on the full pooled set (required for the balance heuristic, regardless of which flow drew each sample).Σ_k (n_k / N) · g_k(θ), computed from the actual sampled counts (exact under rounding).c^{γ/(1+γ)}for proposal mode, importance-corrected posteriorc/mfor posterior mode.log_prefixto theFlowDensityonline-norm logger so its metrics are namespaced.Behaviour / compatibility
proposal_mixture_beta=1.0exactly reproduces the previous single-proposal behaviour in both modes (verified by reduction:log_g_mix → log_prob_cond, so the proposal weight collapses to−1/(1+γ)·log cand the posterior weight to−log m). Use it as a revert switch.0.5changes sampling results vs.main(this is the intended mixture). Watchimportance_sample:n_eff_min— it should increase.Refs
Test plan
python -m py_compile falcon/estimators/flow.pyn_eff_minis ≥ thebeta=1.0baseline.🤖 Generated with Claude Code