Metal backend: Enable Voxtral Realtime

[manuelcandales]

This pull request adds Metal backend support for the Voxtral Realtime model (offline mode, streaming support is coming next)

Voxtral Realtime Metal backend support:

• Added Metal backend support to export_voxtral_rt.py, including custom decompositions and dynamic shape handling for Metal/AOTI compatibility. Also added validation for Metal-specific quantization (fpa4w).
• Updated export_model_artifact.sh and test_model_e2e.sh scripts to support a new mode parameter. This is used in the Voxtral Realtime model for selecting between streaming/offline export modes.

CI/CD and workflow updates:

• Added quantized Voxtral-Mini-4B-Realtime-2602 to Metal workflow matrix.
• Added new CMake presets for building and testing the Voxtral Realtime runner with Metal backend.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/17536

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 5 New Failures, 14 Pending

As of commit ab8aaa2 with merge base 119a099 ():

NEW FAILURES - The following jobs have failed:

• pull / android / run-emulator (gh)
  The process '/usr/bin/sh' failed with exit code 1
• pull / test-openvino-linux / linux-job (gh)
  RuntimeError: Command docker exec -t bad5a62a029ec9aba3f621ff8da168d809e6a29a071ac11c958748e07860bac6 /exec failed with exit code 1
• pull / unittest-nxp-neutron / linux-job (gh)
  RuntimeError: Command docker exec -t c06130f5d6379bfc3aff56e5fedb77b2310ff1c4d92121735b6c49ae9c614313 /exec failed with exit code 1
• trunk / test-torchao-huggingface-checkpoints (lfm2_5_1_2b, linux.2xlarge, executorch-ubuntu-22.04-clang12... / linux-job (gh)
  RuntimeError: Command docker exec -t 994382b9479a62ba26aa93bf6277af1f17866d4d2eb770c7cadedac20839077f /exec failed with exit code 1
• trunk / test-torchao-huggingface-checkpoints (lfm2_5_1_2b, linux.arm64.2xlarge, executorch-ubuntu-22.04-g... / linux-job (gh)
  RuntimeError: Command docker exec -t b6d0d5b666ba98b88e83ac6ae103be6ac2623ea33991c17f49c4fdda6196e8c6 /exec failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[mergennachin]

I wonder if we can just use this for xnnpack?

[mergennachin]

Also update the README.md and model.md file (you can just use your coding agent to update the model.md file)

[Copilot]

Pull request overview

This pull request adds Metal backend support for the Voxtral Realtime model, enabling it to run on Apple Silicon GPUs. The implementation automatically switches between custom SDPA operations (for XNNPACK) and standard PyTorch operations (for Metal/AOTI) based on the target backend, while maintaining support for both streaming and offline modes.

Changes:

• Implemented Metal-compatible attention mechanism using standard PyTorch SDPA and StaticKVCache with index_copy_ operations
• Added backend auto-detection and configuration in export and test scripts with support for vr-streaming and vr-offline modes
• Extended CI/CD workflows to test Voxtral Realtime on Metal backend with quantized-int4-metal configuration

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
examples/models/voxtral_realtime/model.py	Added use_standard_attention config flag, implemented StaticKVCache and StandardSDPA classes for Metal backend compatibility, updated LMAttention to switch between custom and standard attention based on backend
examples/models/voxtral_realtime/export_voxtral_rt.py	Added Metal backend support in export functions with Dim.AUTO for dynamic shapes, implemented linear bias decomposition for Metal, added Metal partitioner configuration, and updated documentation
examples/models/voxtral_realtime/CMakePresets.json	Added Metal-specific CMake presets for building the Voxtral Realtime runner with Metal backend support on Darwin platforms
.github/workflows/metal.yml	Added Voxtral-Mini-4B-Realtime-2602 to Metal CI test matrix, excluded non-quantized variant due to size constraints
.ci/scripts/test_model_e2e.sh	Added mode parameter support for vr-streaming/vr-offline modes with auto-detection (XNNPACK defaults to streaming, others to offline) and validation logic
.ci/scripts/export_model_artifact.sh	Added mode parameter support with auto-detection and validation, configured preprocessor arguments based on streaming mode, added fpa4w quantization support for Metal

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.