Zonos2 support

[stephen-dwq]

This PR adds the ZONOS2 TTS model as a served model.

Components

ZONOS2 has three nodes: LLM, dac_decoder, and speaker_encoder.

node	engine	partition	walks	what it does
LLM	KV_CACHE	LLM	prefill, prefill_clone, decode	Embeds, runs the transformer, and samples a full frame [cb0..cb8, text] per step. Maintains per-request repetition history and performs delayed-EOS detection.
dac_decoder	STATELESS (audio_codec)	DAC	dac_chunk	Collects streamed frames from LLM, undoes codebook shear, and decodes incrementally through DAC with a 4-frame crossfade overlap. Emits int16 PCM.
speaker_encoder	STATELESS (audio_codec)	LLM	prefill_clone	Serves voice cloning. Projects a reference clip to one 2048-D embedding. The LLM writes it into he hidden state of a reserved prompt frame. Runs once per request.

Each component is small, and thus can be placed on the same node; if we do not colocate we prefer to have dac_decoder and LLM on the same node. These two have higher traffic than speaker_encoder and LLM. The latter two sharing a node gives a shorter TTFT, but dac_decoder and LLM on the same node gives a more consistent ITL.

Graph Capturing

We only capture the decode loop of the LLM, including the multi-codebook sampler. This covers embedding, 28 layers of (QK-norm attention with per-head temperature and headwise gating, and stateful MoE), computation of per-codebook logits, multi-codebook sampling, and repetition penalization.

We place a ring buffer for repetition penalty, temperature, top-k, min-p, and per-request RNG state in static slot-indexed buffers for CUDA-capturability of batched sampling.

EOS detection is not captured.

Performance

ZONOS2 ships with a Mini-SGLang server.

We compare batches of 1-8 by steps of x2 with voice cloning:

     run   system       ok TTFTp50  E2Ep50  E2Ep95 ITLmean    SV RTFmean   req/s  aud-s/s audlen
clone/c1    mstar  100/100   0.209   1.399   2.426   0.061 0.934   0.445   0.690    2.330   3.38
clone/c1   zonos2  100/100   0.390   2.085   3.210   0.095 1.000   0.635   0.478    1.624   3.39
clone/c2    mstar  100/100   0.217   1.651   2.616   0.082 0.932   0.520   1.176    3.933   3.35
clone/c2   zonos2  100/100   0.398   2.217   3.701   0.107 0.953   0.700   0.827    2.971   3.59
clone/c4    mstar  100/100   0.246   2.192   3.765   0.111 0.845   0.694   1.703    5.802   3.41
clone/c4   zonos2  100/100   0.455   2.607   4.144   0.132 0.861   0.815   1.478    4.975   3.37
clone/c8    mstar  100/100   0.290   2.952   4.706   0.149 0.662   0.918   2.563    8.597   3.35
clone/c8   zonos2  100/100   0.537   3.145   5.251   0.168 0.714   1.003   2.334    7.880   3.38

The above numbers are obtained with the shipped Mini-SGLang server running the speaker_encoder on CPU as-shipped. For fairness, we also run the shipped Mini-SGLang server with speaker_encoder on GPU. The below table has M* speed-up values against the shipped Mini-SGLang server in the form (as-shipped / speaker_encoder on GPU):

c	TTFT p50	RTF	req/s	audio-s/s
1	1.87x / 1.68x	1.42x / 1.40x	1.44x / 1.38x	1.44x / 1.41x
2	1.83x / 1.57x	1.35x / 1.28x	1.42x / 1.32x	1.32x / 1.29x
4	1.85x / 1.62x	1.17x / 1.15x	1.15x / 1.16x	1.17x / 1.15x
8	1.85x / 1.50x	1.09x / 1.07x	1.10x / 1.07x	1.09x / 1.07x