Split natural-gradients maths into a new Getting-started notebook - #765
Open
thomaspinder wants to merge 1 commit into
Open
Split natural-gradients maths into a new Getting-started notebook#765thomaspinder wants to merge 1 commit into
thomaspinder wants to merge 1 commit into
Conversation
natgrads.py and dual_svgp.py mixed heavy derivation with GPJax API demonstration, which made them hard to use as a pure usage reference. docs/examples/natural_gradients.py now carries the full maths (exponential- family view, Fisher = Jacobian, mirror descent, the dual/site derivation, both cone-safety proofs, the M-step claims table); the two existing notebooks keep their worked demos and plots in full but assume that notebook read first instead of re-deriving it inline. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UmdHyjMN5UdYLMD2JzE6s
|
📖 Docs preview: https://pr-765--endearing-crepe-c2d5fe.netlify.app Smoke render — the expensive notebooks run with reduced budgets, so |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Checklist
uv run poe formatbefore committing.Description
docs/examples/natgrads.pyanddocs/examples/dual_svgp.pymixed heavy mathematical derivation with GPJax API demonstration, which made them hard to use as a pure "how do I call this" reference despite being genuinely excellent as standalone maths writeups.This PR splits them:
docs/examples/natural_gradients.py(Getting started, afterpoisson) — the full maths: the exponential-family view of q(u), the Fisher-information-is-a-Jacobian identity, mirror descent, the "one step is enough" conjugate theorem (demonstrated once for both the moment-storage and dual/site-storage branches on a shared toy problem), the dual/site-parameterisation derivation (EP connection, tied update, the two silent storage-convention traps), cone-safety proofs for both branches, and thedual_elbo-vs-elboM-step claims table.docs/examples/natgrads.py(Accelerating Gaussian processes, unchanged position) — trimmed from 1152 to 966 lines. Keeps both worked demos in full (the conjugate 1-step-vs-Adam race, the non-conjugate mini-batched banana benchmark) and the failure-mode/backoff demo, now assuming the reader has readnatural_gradients.pyinstead of re-deriving the theory inline.docs/examples/dual_svgp.py— trimmed from 1576 to 1093 lines, same treatment: keeps its conjugate demo, the ρ=γ equivalence check, the 3-way banana benchmark, and the M-step-in-practice VEM loop, assuming the new notebook for the derivations.docs/index.md— new toctree entry forexamples/natural_gradientsunder "Getting started".All three notebooks were executed end-to-end during this work; every printed numeric claim in the prose is a real number from an actual run, not carried over from the old notebooks. One real bug was caught and fixed in the process (a missing
jitterkwarg indual_svgp.py's ported conjugate-model setup that was throwing its Titsias-optimum comparison off by several orders of magnitude), and one previously-dangling cross-reference (a numerical demo two notebooks each pointed at the other to find) was restored intonatural_gradients.py.Issue Number: N/A