feat: StochasticWfn trial wavefunction - #128
Open
beevus77 wants to merge 157 commits into
Open
Conversation
Design and verification checklist for Phases 1a–3b from the develop branch. Overhaul port status to be added when the nda transplant lands. Co-authored-by: Cursor <cursoragent@cursor.com>
Add full_g_estimators and energy_fullG/vbias_fullG dispatch for dynamic stochastic trials when the cross Green's function is NMO×NMO. Co-authored-by: Cursor <cursoragent@cursor.com>
Port StochasticWfn to nda/const_shared_array APIs and expose protected NOMSD hooks for energy and vbias contractions on full and compact G. Co-authored-by: Cursor <cursoragent@cursor.com>
Register StochasticWfn in the Wavefunction variant, parse stochastic input keys, and build the inner stack via WavefunctionFactory. Co-authored-by: Cursor <cursoragent@cursor.com>
Initialize inner walkers after wavefunction build and call begin_inner_step at the top of each propagator step for stochastic trials. Co-authored-by: Cursor <cursoragent@cursor.com>
Ensure nix Boost include dirs are visible when compiling safire_lib so GCC does not pick up stale system headers under /usr/include/boost. Co-authored-by: Cursor <cursoragent@cursor.com>
Add overhaul port status, API mapping, and qualified verification notes now that Phases 1a–3b are ported and safire_lib builds on overhaul. Co-authored-by: Cursor <cursoragent@cursor.com>
…rization Remove the duplicate vbias_fullG HamiltonianOperations seam and route full NMO×NMO G through Real3IndexFactorization::vbias instead. Expand energy_fullG explicit instantiations and drop the NOMSD vbias_fullG friend seam accordingly. Co-authored-by: Cursor <cursoragent@cursor.com>
…ropagator At inner_nwalkers=1 and inner_nsteps=0, route Log_Overlap, Energy, and vbias through nomsd_ instead of the inner cross-reduction path. Do not build the inner Propagator until inner_nsteps>0; move inner_propagator helpers to the .cpp translation unit to avoid circular includes. Co-authored-by: Cursor <cursoragent@cursor.com>
Port the delegate-limit NOMSD parity keystone into test_wfn_factory with stepwise bisection REQUIREs for the active segfault hunt. Document the Ne_cc-pvdz overhaul fixture, investigation notes, and explicit BLOCKED status in StochasticDevelopment.md. Co-authored-by: Cursor <cursoragent@cursor.com>
Isolate StochasticWfn construction and inner-walker init from outer reductions and the plain-NOMSD path so segfaults can be classified against wfn_factory:sdet. Co-authored-by: Cursor <cursoragent@cursor.com>
The cross-reduction accumulates linear effective overlap; convert with std::log before writing OVLP to match NOMSD::Log_Overlap convention. Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Reshape flat hij/G arrays for tensor::contract rank matching; slice Lankf rows (not columns) in the FairDivide gemm; use non-conjugating dot for EXX/EJ to match energy_impl. Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Add overlap, energy, and vbias delegate-limit + inner_nwalkers invariance tests through the public Wavefunction API, with shared perturb_stochastic_walkers and linear_overlap helpers. Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Add full-G vs compact parity, dynamic ensemble smoke, and end-to-end outer propagator step tests (CLOSED/CPU only). Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Document CPU-verified [stochastic_wfn] suite, three port bugs fixed, Ne_cc-pvdz canonical fixture, Log_Overlap naming, and remaining gates (infrastructure tests, GPU, multi-rank). Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Document main as the active overhaul port, stochastic-wfn-develop as the frozen develop-line reference, and upstream/overhaul as the periodic rebase target. Co-authored-by: Cursor <cursoragent@cursor.com>
Expose a non-asserting check for whether a Hamiltonian input block has been registered, so WavefunctionFactory can guard inner_hamiltonian pushes. Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
…tonian key Overload the factory constructor to hold a HamiltonianFactory reference for on-demand Variational Hamiltonian builds. Register inner_hamiltonian as a factory-level stochastic key in interpret_inputs without forwarding it to the wavefunction ptree. Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
…mOps Build or fetch the Variational Hamiltonian from inner_hamiltonian in fromHDF5 and pass it to buildStochasticNomsdWavefunction so the inner stack uses h_var while the outer nomsd_ continues to score the True Ham. Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
…var) Verify that naming inner_hamiltonian with the same integral file reproduces the pre-3b-var clone path on the dynamic hot path and that the factory registers the namespaced inner Hamiltonian block. Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Wire WfnFac(InfoMap, HamFac) so production input decks can use the inner_hamiltonian key without requiring the two-argument test constructor. Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Mark inner_hamiltonian as implemented, add the Phase 3b-var section and test catalog, update assertion counts, and note AFQMCFactory production wiring. Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Document the VAFQMC export workflow and SAFIRE driver runs that validated inner_hamiltonian with distinct ham_var.h5, and mark Phase 3c as the active development branch. Co-authored-by: Cursor <cursoragent@cursor.com>
…r sampling Add a single conditioned field-sampling step: seed auxiliary fields from an external per-walker bias, run assemble_X -> vHS -> apply_propagators, and skip inner overlap/energy/weight updates. Expose via the Propagator variant. Co-authored-by: James Larsen <jameslucky7larsen@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
… 3c-i) Thread inner_conditioning through the factory and StochasticWfn input parsing. When enabled, build the inner propagator in importance-sampling mode, compute the conditioning bias x_bar(phi_w) via inner_nomsd MixedDensityMatrix + vbias on the outer walkers, grow a slot-major nw*P inner ensemble, and extend Log_Overlap / reduce_inner_cross_dm for block-structured pairing. Default inner_conditioning=false preserves Phase 3b behavior. Co-authored-by: James Larsen <jameslucky7larsen@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Run a real outer AFQMCBasePropagator::Propagate over a conditioned dynamic trial (inner_conditioning=true, inner_nsteps=1) and assert finite weights/energies/overlaps over 3 steps. CLOSED/CPU gate; finiteness only. Co-authored-by: James Larsen <jameslucky7larsen@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Decompose Phase 3c into 3c-i (walker-conditioned sampling, complete) and 3c-ii (leapfrog, remaining). Document inner_conditioning, implementation checklist, and full-tag verification (10 cases / 5699 assertions on Ne_cc-pvdz). Co-authored-by: James Larsen <jameslucky7larsen@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Thread the outer WalkerSet through Wavefunction::begin_inner_step and AFQMCBasePropagator::Propagate so StochasticWfn can refresh leapfrog state against the current walkers at the top of each outer step. Co-authored-by: James Larsen <jameslucky7larsen@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
…3c-ii) Add inner_leapfrog input key and importance-reweighted Log_Overlap so the hybrid step ratio new/old is Eq. 25-exact when enabled. Resample conditioned on the old walker and refresh OVLP at begin_inner_step; store per-sample conditioning magnitudes in inner_cond_mag_ for the reweighting. Co-authored-by: James Larsen <jameslucky7larsen@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Replace the interdependent inner_conditioning / inner_leapfrog / inner_persistence booleans with a single InnerMode enum (static, free, conditioned), shared resolve_inner_mode for factory and StochasticWfn, and drop the removed inner_pool_burn_in path. Legacy boolean inputs still translate where they describe a surviving mode. Co-authored-by: James Larsen <jblarsen@umich.edu> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Scope dynamic-inner smokes to RHF|UHF|NOMSD|MOLECULES and skip walker types the path rejects, so unsupported solids/lattices/NONCOLLINEAR no longer drown the suite. Also fix a missed CurandRandomGenerator seed construction in the estimator BP smoke. Co-authored-by: James Larsen <jblarsen@umich.edu> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Switch stochastic decks from the removed boolean knobs to inner_mode, collapse free/conditioned propagator survival into one helper, add legacy-translation and removed-option rejection coverage, and apply the remaining DYNAMIC_INNER call-site filters on the heavy suites. Co-authored-by: James Larsen <jblarsen@umich.edu> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
…ullG_vbias Key vbias on whether the caller handed a half-rotated or full G, not on nci, so StochasticWfn's full mean-field G is no longer quietly misread on Hubbard. Plumb has_fullG_vbias through HamOps and the wavefunction variants so callers can refuse unsupported operators up front. Co-authored-by: James Larsen <jblarsen@umich.edu> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Refuse off-anchor force-bias and mean-field contractions when the Hamiltonian operator cannot take a full un-rotated G, so solids abort with a named capability gap instead of a size mismatch and Hubbard no longer returns a quietly wrong P-dependent vMF. Update the mean-field parity test to assert that refusal on unsupported fixtures. Co-authored-by: James Larsen <jblarsen@umich.edu> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
…dvances the pool Remove inner_condition_on_new and inner_measure_restore (reject if present), delete end_inner_step and the propagator hook, and advance the persistent pool on every measurement including nm == 1 so nm only controls averaging. Match hafqmc's feed-forward measurement seam. Co-authored-by: James Larsen <jblarsen@umich.edu> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Replace inner_equil_steps and inner_measure_stride with one MH-sweep count driving both the propagation-side re-tether and the measurement advance, matching hafqmc's sample_update_steps. Legacy keys translate when they agree and are rejected when they disagree or mix with inner_sweeps. Co-authored-by: James Larsen <jblarsen@umich.edu> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Rename the sampling enum to SamplingTarget (static/gaussian/walker_overlap) and the input keys to hafqmc names (inner_n_samples, inner_sample_update_steps, inner_sampler, inner_n_measure_samples). Drop legacy boolean/mode translation and inner_log_aggregate; unknown inner_* keys now hard-abort. Update the suite and delete the obsolete log-aggregate smoke. Co-authored-by: James Larsen <jblarsen@umich.edu> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Reintroduce a one-time MH burn-in at chain prime (hafqmc burn_in, default 100), separate from the per-advance inner_sample_update_steps sweeps. Co-authored-by: James Larsen <jblarsen@umich.edu> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Read the trained B_T timestep from Hamiltonian/inner_timestep on the variational Hamiltonian file instead of accepting it from input with a silent default. Fail closed when the attribute is missing; stamp fixtures in the wfn_factory coverage of that contract. Co-authored-by: James Larsen <jblarsen@umich.edu> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
…P helpers Remove unused field-chain snapshot/restore paths left after measure-always-advances, and strip getenv-gated debug dumps that were never part of the production sampler. Co-authored-by: James Larsen <jblarsen@umich.edu> Co-authored-by: Cursor <cursoragent@cursor.com>
…ned() Replace the three identical WalkerOverlap accessors and the redundant leapfrog/persistence compound branches with a single predicate. Co-authored-by: James Larsen <jblarsen@umich.edu> Co-authored-by: Cursor <cursoragent@cursor.com>
Add visit_stochastic / visit_stochastic_or so StochasticWfn-only seams stop repeating the if-constexpr type gate. Keep the popControl ordering contract on permute_inner_blocks_after_pop explicit in the docstring. Co-authored-by: James Larsen <jblarsen@umich.edu> Co-authored-by: Cursor <cursoragent@cursor.com>
…tree The stamp was written into the already-consumed interpret_inputs input (pt_in), a dead store, so dynamic trials never received the trained dt. Write into pt, refuse a hand-set inner_propagator.timestep alongside inner_hamiltonian, and pin the contract in the factory tests. Co-authored-by: James Larsen <jblarsen@umich.edu> Co-authored-by: Cursor <cursoragent@cursor.com>
Move the long narrative into StochasticDevelopment/StochasticWfnDesign.md (outside this repo) and keep short self-contained API contracts in the production sources, including call-site constraints the signatures cannot express. Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
…rial surface Document StochasticWfn, the Wavefunction visitor helpers, the factory/inner-stack build path, and the full-G energy kernels with @brief/@param/@details. Keep contracts on the dangerous seams; leave design essays out of the headers. Co-authored-by: James Larsen <jblarsen@umich.edu> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Extract mark/perturb/BP-RDM helpers into test_stochastic_common.hpp, drop redundant factory/static-BP smokes covered elsewhere, soft-skip multi-det delegate parity, and rename stochastic cases to component: short descriptor. Co-authored-by: James Larsen <jblarsen@umich.edu> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
…factors Port StochasticWfn onto the typed JSON parameter schema so the branch builds against overhaul's ptree removal: extend WavefunctionParameters with stochastic fields, take upstream factories as the base, and rewire tests to construct *Parameters instead of ptrees. Co-authored-by: Cursor <cursoragent@cursor.com>
Hand-built Wavefunction/PropagatorParameters in unit tests bypass resolve_defaults; apply ham-type defaults explicitly. Update the propagator survival smoke to index walkers after iterator removal. Co-authored-by: James Larsen <jblarsen@umich.edu> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Add a short features.rst entry for the stochastic trial and its supported combinations, and drop local IDE/CMake scratch ignores from the tracked .gitignore so they stay out of the upstream PR. Co-authored-by: James Larsen <jblarsen@umich.edu> Co-authored-by: Cursor <cursoragent@cursor.com>
Merge stochastic-refactors: overhaul sync + StochasticWfn input-surface cleanup
Bring in compile-time-opt (SFQMC#126) and follow-up docs/test commits. Resolve conflicts by keeping overhaul's .cpp split for Wavefunction/HamOps/NOMSD while retaining StochasticWfn, energy_fullG, and has_fullG_vbias. Co-authored-by: Cursor <cursoragent@cursor.com>
Pin the full-G energy path to contiguous nda::C_layout operands so GPU cuTENSOR and host tblis see the same layout contract. Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
…ion.cpp Move begin_inner_step, measure_energy, and permute_inner_blocks_after_pop out of the header to match overhaul's Wavefunction dispatch split. Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Move NOMSD friend Hamiltonian helpers into NOMSD.cpp, keep inner anchor guesses on the host, pin WalkerSet resize to HOST guess views, and drop redundant post-DeviceFor synchronizes in stochastic CUDA kernels. Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Split MixedDensityMatrix/Log_Overlap calls, use buffered_array for getReferences, MEM-space buffers for GPU measure_energy tests, and std::span for branch(). Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Fully polarized COLLINEAR (ndown==0) is a real layout; zero-extent beta ops trap on GPU (nda assign / cuTENSOR) while host builds succeed. Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Replace open WlkSet/Mat templates with WalkerSet and array_view on the hot paths and relocate those definitions out of the .icc for faster builds. Relocated overlap paths also gate beta via has_beta(). Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
…tor cap" This reverts commit 6612b95.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a first-class StochasticWfn trial to the overhaul tree: a NOMSD-based trial whose overlap and local energy are estimated from an auxiliary-field inner ensemble, with a closed
inner_*input surface (illegal sampler combinations are unrepresentable; unknown keys abort).This is the production surface that worked in the N₂ / Fe study work — not a grab-bag of experimental knobs.
Rough size vs current
overhaul: ~57 files, +8.5k / −0.1k (mostlyStochasticWfn.*,full_g_estimators.hpp, CUDA reduction kernels, andtests/test_stochastic_wfn.cpp).What it adds
Trial + driver wiring
StochasticWfnvariant alternative, factory / HDF5 type detection (stochasticwfn/Wavefunction/StochasticWfn)begin_inner_step; pop-control realignment viaSLOT_LINEAGE+permute_inner_blocks_after_popTRIAL_FIELDS;Propagate_conditioned/Propagate_free/Propagate_given_fieldsmeasure_energy(replica-averaged wheninner_n_measure_samples > 1)full_g_estimators.hpp) for CLOSED and COLLINEAR; device reduction kernels for GPUinner_hamiltonian) for the inner propagatorClosed input surface (10 keys)
Schema-owned in
WavefunctionParameters(unknowninner_*abort via the JSON schema):inner_n_samplesinner_nsteps0⇒ static anchorinner_sampling_targetstatic|gaussian|walker_overlapinner_sample_update_stepsinner_burn_ininner_n_measure_samplesinner_sampler/inner_sampler_steppcn/gaussian)inner_seedinner_propagator/inner_hamiltonianinner_hamiltonianinner_sampling_target: walker_overlapturns on conditioning, leapfrog reweight, and persistent chains together. Combinations that used to be expressible and wrong (e.g. leapfrog without persistence) are no longer representable.inner_timesteptravels with the variational operator:exportstampsHamiltonian/inner_timesteponham_var.h5; the factory reads it; a missing stamp aborts; settinginner_propagator.timestepalongsideinner_hamiltonianaborts (no silent0.01default).Docs + tests
features.rstentry (type, Dense Cholesky × CLOSED/COLLINEAR, unsupported list)tests/test_stochastic_wfn.cpp(~20 cases): delegate-limit NOMSD parity, mean-field / BP, persistent pool + pop-control, measurement replicas, closed input surface — green on allocation (CPU + CUDA as validated on the fork)Design choices worth calling out in review
inner_burn_indefault 0 → 100 is not energy-neutral. N₂-equilibrium A/B (production settings → shipped defaults), 48 pairs: +4.327 ± 1.390 mHa (3.11 σ). Bundles burn-in with “measure always advances the pool”; both are deliberate.inner_burn_in: 0remains legal.SLOT_LINEAGE,TRIAL_FIELDS) lives inwalker_bufferso branch / load-balance copy fields for free; both sit outside thewalkerSizeIO()checkpoint window.ndown == 0) — after upstream Replace FULLYPOLARIZED by COLLINEAR[ndown=0] #123, fully polarized systems are ordinary COLLINEAR with an empty beta block. Zero-extent beta ops trap on GPU (ndahost→device assign / cuTENSOR) while host/tblis accepts them. This PR gates those paths (has_beta(),NOMSD::accumulate_estimators, walker resize, StochasticWfn overlap). Theaccumulate_estimatorsguard is an upstream hole too (their polarized consistency test covers Energy/MixedDM, not 1-body estimators); happy to peel it to a standalone overhaul PR if preferred.WalkerSet/array_viewsignatures and live in.cpp, matching overhaul Compile time optimization #126’s direction.atomcentered/generalizedFock/n2r/realspace/sk) takeint NMO— finishes upstream The extrication of AFQMCInfo #118 for headers that are still commented out of the liveObservablevariant. Not required for StochasticWfn; included as hygiene.cmax: default 4 → 20 and hard-error if Cholesky exits on the vector cap before tol (silent truncation on large active spaces, e.g. Fe₂S₂). Python-only; orthogonal — happy to peel to a follow-up if preferred.Known limits (intentional for this PR)
-np > 1) consistency for the dynamic ensemble is deferred (including a knowndeno_realdefect under multi-rank reduction)Out of scope / follow-ups
deno_real)mol.pycmax, orphaned-observable NMO ctors, and/or theNOMSD::accumulate_estimatorsempty-spin guard into standalone PRs