Enable expert-parallel language modules in MegatronMIMO

[liding-nv]

Summary

This PR enables MegatronMIMO to describe and build non-colocated language MoE modules with expert parallelism while keeping encoder modules dense.

Key changes:

• Adds language-side expert_model_parallel_size / expert_tensor_parallel_size handling to MegatronMIMO parallelism config.
• Separates dense rank accounting from expert rank views, so EP/ETP refactor the language rank span instead of multiplying the module world size.
• Validates heterogeneous module rank ranges tile the distributed world exactly, and keeps encoder modules dense for this MoE path.
• Rebuilds MegatronMIMO HyperCommGrid setup around a dense base view plus an expert view:
  • dense: tp, cp, dp, pp
  • expert: expt_tp, ep, expt_dp, pp
• Wires the resulting expert groups into ProcessGroupCollection and MCore compatibility globals.
• Seeds per-module RNG with real TP/PP/EP/ETP ranks so expert-parallel language ranks do not silently share identical expert seeds.
• Passes language TP/CP groups into MCore MimoModel construction for sequence-parallel partitioning paths.
• Initializes the MCore global memory buffer for MegatronMIMO custom parallel setup path.
• Threads ep through the MegatronMIMO conversion CLI component parser.
• Extends unit coverage for config validation, grid construction, provider wiring, RNG seeding, global expert-group bridging, checkpoint/optimizer setup, and conversion CLI parsing.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[liding-nv]

/ok to test d3cfc95

[liding-nv]

/ok to test 6f1b8af

[claude]

Review

Finding: CLI help text missing ep=N

File: examples/conversion/convert_megatron_mimo.py, line 265

The ep=N key was added to _COMPONENT_KEY_TO_FIELD (line 66) and to the docstring/error message in _parse_component_spec (lines 75, 79), but the --component help text in _add_common_args still reads:

name=tp=N[,pp=N,dp=N,cp=N,etp=N,rank_offset=N]

It should include ep=N:

name=tp=N[,pp=N,dp=N,cp=N,ep=N,etp=N,rank_offset=N]

---

Everything else looks solid. The rank algebra (dense_model_parallel_size excluding EP/ETP), the expert factorization validation, the tiling validation upgrade (gaps + world_size coverage), the Phase 1 guards at both the config and builder layers, and the provider sync are all consistent and well-tested. The test coverage is thorough across all changed code paths.

Suggested test cases: No perf tests impacted.