4 bit models on hopper support

[erictanjn]

We would like to run the GLM 5.2 model in int4/mxfp4 on a single node with 8×H20 GPUs. Could you let us know whether Tilert currently has plans to support 4-bit inference on the Hopper architecture?

[mzmssg]

Thanks for asking. Hopper don't support 4-bit MMA and cvt. On top of that, int4, MXFP4, NVFP4 and their variants each need their own kernel and quantization work, so covering them would add a significant maintenance and optimization burden for little gain on Hopper. It's therefore not in our near-term roadmap.