sync: upstream cherry-picks — virtual experts, Apollo engine, Gemma 4 MoE#16
Open
mikeumus wants to merge 11 commits into
Open
sync: upstream cherry-picks — virtual experts, Apollo engine, Gemma 4 MoE#16mikeumus wants to merge 11 commits into
mikeumus wants to merge 11 commits into
Conversation
…rovements) Upstream commit ee611ac from chrishayuk/larql: - walk_ffn.rs: improved FFN walking with better expert selection - infer_patched.rs: inference pipeline fixes - mod.rs/lib.rs: corresponding module updates - infer.rs: improved query inference execution Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Upstream commit 741225e from chrishayuk/larql. Adds the larql-experts crate: 20+ pluggable virtual expert modules (arithmetic, date, dijkstra, finance, geometry, graph, hash, sql, statistics, string_ops, logic, etc.) with an expert registry and caller infrastructure in larql-inference. Clean cherry-pick. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…dispatch Upstream commit 754090d from chrishayuk/larql. Adds: Apollo KV cache engine with NPY store + routing, unlimited context with checkpoint store and token archive, remote expert backend (ShardConfig + MoeRouterWeights), grid generation, trie dispatch, and expert endpoint in larql-server. Conflict resolution: merged HEAD re-exports (ablating/injecting) with upstream re-exports (remote_expert types). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Use upstream's renamed types: RemoteMoeBackend/RemoteMoeError/moe_remote instead of the old RemoteExpertBackend/RemoteExpertError/remote_expert names. Keep our fork's ablating/injecting re-exports alongside the new MoE names. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…generate) Upstream commit 3a5b479 from chrishayuk/larql. Adds: experts/mask.rs (MoE routing mask), extended q4k_forward.rs for MoE, layer_graph/generate.rs expansion, run_cmd MoE flags, and updated ROADMAPs. Reinstates test_arch_golden.rs (deleted in our fork, upstream extended it with Gemma 4 MoE golden case). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…patch docs Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
session.rs: add Dispatcher trait + FilteredDispatcher, updated prompt
builder (compact op{args} format), add mock_tests module with 12 tests.
mod.rs: export Dispatcher + FilteredDispatcher.
test_run_experts.rs: add experts_chat_mode_dispatches_via_stdin test.
All upstream (c9209ad) content was the correct resolution — HEAD sections
were stubs added by our fork's empty initializations.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Selective cherry-picks from
chrishayuk/larqlupstream to bring our fork current through the virtual-experts branch merge (PR chrishayuk#32 + chrishayuk#33).What's included
ee611acwalk_ffn+infer_patchedimprovements741225elarql-expertscrate — 20 pluggable virtual expert modules (arithmetic, date, dijkstra, finance, geometry, graph, hash, logic, sql, statistics, trig, unit, …) with registry + caller infrastructure754090dShardConfig,MoeRouterWeights); trie dispatch; expert HTTP endpoint inlarql-server674b6164751d25expert/forward/math), Metaldecode/moe_combine, prompt module,remote_expert.rs→moe_remote.rsrenamec9209adlarql-modelscleanup +Dispatchertrait +FilteredDispatcher, mock test suite for session dispatch, Markov residual engine specaaf0324patch_down_projexample, virtual-experts dispatch doc3a5b479experts/mask.rs,q4k_forwardMoE path,generateexpansion,run_cmdMoE CLI flags6f55d96Conflict resolutions (all conservative — keep best of both)
huggingface.rs(d2ee0d2): kept our etag-based cache validation (stronger than upstream's size-only check; detects remote content changes)ffn/mod.rs+lib.rs(754090d → 4751d25): merged ourLastPositionAblatingFfn/LastPositionInjectingFfnre-exports with upstream's renamed MoE types (RemoteMoeBackend/RemoteMoeError)session.rs(c9209ad): took full upstreamDispatchertrait +FilteredDispatcher+ mock tests (HEAD sections were empty stubs)qrank_sweep.py(833c124): kept our.float()cast on the QK matrix (correct for fp8/bf16 models like Kimi-K2)What was intentionally skipped
7de5208/8193f02/1f75d52— residual stream + binary format commits below Architecture B; their substance is already in ourbdf7e88sync74bfe1d— intermediate benchmark work commit superseded by4751d25+3a5b4793de34bd/082634d— PR merge commits (their content is in the cherry-picks above)Our fork additions preserved
All Divinci-AI-specific additions (fact-editing
ablating/injectingbackends, etag cache validation,LARQL_API_KEYenv var, regression test forrebuild_overrides) are intact.🤖 Generated with Claude Code