1

SGLang Speeds Kimi K3 Inference by 20% on NVIDIA Vera Rubin

huggingnews.com

SGLang achieved up to 20% faster FP8 MLA at batch 1 with a 128K context for the Kimi K3 model on NVIDIA Vera Rubin hardware. Other gains include 20% faster KDA verification and a 16% speedup for batch 16.

Sign in to suggest edits

Key sources

  1. SOURCEmarketbrief.now
Markdown