Add xToken-aligned cross-tokenizer OPD - #13
Draft
artem-osmosis wants to merge 132 commits into
Draft
Conversation
…eight when colocate, sync lora+base for disaggregate mode at first time then only sync lora after that
Sync osmosis/dev with upstream radixark/miles main (254091f, 2026-07-06). 51 upstream commits; 212 files (+10066/-1205). 8 content conflicts resolved. Conflict resolutions (strategy: preserve osmosis behavior, port to upstream radixark#1329 API): - session/sessions.py: took upstream thin-adapter refactor (radixark#1510/radixark#1518/radixark#1563/radixark#1569); osmosis custom features (routing/indexer replay, R3 stripping) already live in core.py. - ray/rollout/train_data_conversion.py: kept BOTH osmosis adapter_slots + upstream opd_reverse_kl. - update_weight/common.py: dropped duplicate _check_weight_sync_results + orphaned post_process_weights tail (removed upstream by radixark#1329). - update_weight/update_weight_from_tensor.py: adopted begin/end_weight_update import; ported update_multi_lora_weights post_process_weights() -> end_weight_update(). - update_weight/update_weight_from_distributed/{broadcast,mixin,p2p}.py: kept osmosis multi-LoRA, never-sync-base, NCCL contiguity fix, and p2p LoRA support. - tests/.../test_lora_weight_sync_validation.py: took upstream signature (mock_common_ray for the new common.ray patch). Local static checks: py_compile + ruff (update_weight/ + session/) pass; all common imports resolve. Dynamic pytest tests/fast requires the miles container (torch/ray) — run in CI.
Pre-existing black (24.3.0, line-length 119) violations inherited from dev, surfaced by `pre-commit run --all-files` on the PR. Not touched by the upstream merge — formatting only, no logic change: - miles/ray/rollout/metrics.py: add blank line before log_rollout_data - miles/rollout/multi_lora_data_source.py: collapse _create_adapter_source signature
Sync upstream (radixark/miles main) into dev
* fp8: block-fp8 storage of the frozen LoRA base with per-layer free * gated canonical lora: gate-aware q sizing, TP-safe qkv interleave, no MTP adapters * example: Qwen3.5-35B-A3B MoE-LoRA colocate with the fp8 frozen-base store * ci: LoRA e2e test with --fp8-frozen-base-store enabled --------- Co-authored-by: MuuSeoTia <tiahi.m@northeastern.edu>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
sglang-milesAPI preflightValidation
sglang-miles-v0.5.13-finalatf8cfad35bdfeeed09b259857e738bf833aeb70e0/begin_weight_updateand/end_weight_updatetransactions returned 200; zero begin 404s and zero legacy/post_process_weightscalls0.36809193340.2980055362-0.07008639720.35629763750.3052670628-0.0089484361 / steptrue149 passed, 1 skipped60 passedruff check,git diff --check, and launcherbash -npass