---
format: "aidr-story-markdown/v1"
id: "62ac7754db38e9e3777d92073f26c9125d941415e0f83f309b3928b228d54c10"
canonical_url: "https://aidr.today/62ac7754?lang=en"
title: "Nvidia Tops 1,000 Tokens a Second per User on 700B+ Models With Agent Swarm Code"
lang: "en"
requested_lang: "en"
available_langs: ["en","vi"]
translation_fallback: null
fallback_fields: []
published_at: "2026-11-01T18:06:12.000Z"
category: "Frameworks"
topics: ["nvidia","inference","framework"]
source_urls: ["https://marketbrief.now/ai/nvidia-tops-1000-tokens-a-second-per-user-on-700b-models-with-agent-swar-17485e8c","https://x.com/SemiAnalysis_/status/2116950855718998288","https://x.com/SemiAnalysis_/status/2115821309137465536","https://x.com/sgl_project/status/2115913588741468629","https://x.com/sgl_project/status/2116353629057962217","https://x.com/SemiAnalysis_/status/2114779348838338719","https://huggingnews.com/ai/nvidia-tops-1000-tokens-a-second-per-user-on-700b-models-with-agent-swar-17485e8c"]
summary: "A new Nvidia submission to the SemiAnalysis AgentX benchmark uses agent swarm software to exceed 1,000 tokens per second per user on models with more than 700B parameters. Running on Nvidia's Rubin NVL72 system, the submission delivers 20.9 times the performance per total cost of ownership compared to a GB300 NVL72 rack at 325 tokens per second per user. Open source inference framework SGLang reached 943 tokens per second per user at a concurrency of 1 using MegaKernel on a GB300 NVL72 rack."
---

# Nvidia Tops 1,000 Tokens a Second per User on 700B\+ Models With Agent Swarm Code

> [Open the canonical story](<https://aidr.today/62ac7754?lang=en>)

**Published:** 2026-11-01T18:06:12.000Z
**Category:** Frameworks
**Topics:** nvidia, inference, framework

## Summary

A new Nvidia submission to the SemiAnalysis AgentX benchmark uses agent swarm software to exceed 1,000 tokens per second per user on models with more than 700B parameters\. Running on Nvidia's Rubin NVL72 system, the submission delivers 20\.9 times the performance per total cost of ownership compared to a GB300 NVL72 rack at 325 tokens per second per user\. Open source inference framework SGLang reached 943 tokens per second per user at a concurrency of 1 using MegaKernel on a GB300 NVL72 rack\.

## Sources

- [Story source](<https://marketbrief.now/ai/nvidia-tops-1000-tokens-a-second-per-user-on-700b-models-with-agent-swar-17485e8c>)
- [Story source](<https://x.com/SemiAnalysis_/status/2116950855718998288>)
- [Supporting source](<https://x.com/SemiAnalysis_/status/2115821309137465536>)
- [Supporting source](<https://x.com/sgl_project/status/2115913588741468629>)
- [Story source](<https://x.com/sgl_project/status/2116353629057962217>)
- [Supporting source](<https://x.com/SemiAnalysis_/status/2114779348838338719>)
- [Story source](<https://huggingnews.com/ai/nvidia-tops-1000-tokens-a-second-per-user-on-700b-models-with-agent-swar-17485e8c>)