In brief
- Xiaomi and inference partner TileRT have broken 1,000 tokens per second on a 1-trillion-parameter model, a first at that scale, using a standard 8-GPU commodity node—not custom chips.
- The speed comes from FP4 quantization on the model’s expert layers and DFlash speculative decoding,…
Read Full Article at Source