---
format: "aidr-story-markdown/v1"
id: "094b8d34300c83e3a387af2ec9e27671ad234c70f43579824e764c69272cc01b"
canonical_url: "https://aidr.today/094b8d34?lang=en"
title: "NVIDIA Hits 423 Tokens Per Second Per User on Kimi K3"
lang: "en"
requested_lang: "en"
available_langs: ["en","vi"]
translation_fallback: null
fallback_fields: []
published_at: "2026-10-18T01:40:05.000Z"
category: "Frameworks"
topics: ["nvidia","inference","framework","open-source"]
source_urls: ["https://huggingnews.com/ai/update-nvidia-hits-423-tokens-per-second-per-user-on-kimi-k3-e91b43e6","https://x.com/SemiAnalysis_/status/2111627094878790023","https://x.com/ArtificialAnlys/status/2110903261897388256","https://x.com/zwljohnny/status/2109679465609470216","https://x.com/tri_dao/status/2109691017570201965","https://x.com/_albertgu/status/2109744837432095077","https://marketbrief.now/ai/update-nvidia-hits-423-tokens-per-second-per-user-on-kimi-k3-e91b43e6","https://x.com/vllm_project/status/2111637219647840611"]
summary: "NVIDIA engineers developed a decode-specialized kernel in Cute DSL that increased the inference throughput of the 2.8T parameter Kimi K3 model by 1.68x, reaching a peak speed of 423 tokens per second per user. The optimization, implemented as a 12,000 line of code Megakernel and enabled through aggressive Megakernel fusion as an upstream update to vLLM by the CentML team, surpasses the previous leaderboard high of 255 tokens per second. NVIDIA is currently extending these optimizations to other models, including DeepSeekv4.1."
---

# NVIDIA Hits 423 Tokens Per Second Per User on Kimi K3

> [Open the canonical story](<https://aidr.today/094b8d34?lang=en>)

**Published:** 2026-10-18T01:40:05.000Z
**Category:** Frameworks
**Topics:** nvidia, inference, framework, open\-source

## Summary

NVIDIA engineers developed a decode\-specialized kernel in Cute DSL that increased the inference throughput of the 2\.8T parameter Kimi K3 model by 1\.68x, reaching a peak speed of 423 tokens per second per user\. The optimization, implemented as a 12,000 line of code Megakernel and enabled through aggressive Megakernel fusion as an upstream update to vLLM by the CentML team, surpasses the previous leaderboard high of 255 tokens per second\. NVIDIA is currently extending these optimizations to other models, including DeepSeekv4\.1\.

## Sources

- [Story source](<https://huggingnews.com/ai/update-nvidia-hits-423-tokens-per-second-per-user-on-kimi-k3-e91b43e6>)
- [Story source](<https://x.com/SemiAnalysis_/status/2111627094878790023>)
- [Supporting source](<https://x.com/ArtificialAnlys/status/2110903261897388256>)
- [Story source](<https://x.com/zwljohnny/status/2109679465609470216>)
- [Supporting source](<https://x.com/tri_dao/status/2109691017570201965>)
- [Supporting source](<https://x.com/_albertgu/status/2109744837432095077>)
- [Story source](<https://marketbrief.now/ai/update-nvidia-hits-423-tokens-per-second-per-user-on-kimi-k3-e91b43e6>)
- [Story source](<https://x.com/vllm_project/status/2111637219647840611>)