---
format: "aidr-story-markdown/v1"
id: "72a147bca7dea9cbfdd2be878f5125b4f2aa716868945179bac3a08b7535bdcf"
canonical_url: "https://aidr.today/72a147bc?lang=en"
title: "Introducing Amazon SageMaker HyperPod Inference Gateway"
lang: "en"
requested_lang: "en"
available_langs: ["en","vi"]
translation_fallback: null
fallback_fields: []
published_at: "2026-09-18T13:08:34.000Z"
category: "Models"
topics: ["anthropic","claude","multi-agent","open-source","amazon","inference","chips","infra"]
source_urls: ["https://aws.amazon.com/blogs/machine-learning/introducing-amazon-sagemaker-hyperpod-inference-gateway/","https://aws.amazon.com/blogs/machine-learning/amazon-sagemaker-inference-2026-year-to-date-launches-in-review/"]
summary: "Amazon SageMaker HyperPod Inference Gateway is a Kubernetes-native, GPU-aware routing add-on for Amazon EKS. It uses real-time GPU signals to send each inference request to the best-suited pod, cutting first-token latency by up to 82% with no changes to your model servers or client applications."
---

# Introducing Amazon SageMaker HyperPod Inference Gateway

> [Open the canonical story](<https://aidr.today/72a147bc?lang=en>)

**Published:** 2026-09-18T13:08:34.000Z
**Category:** Models
**Topics:** anthropic, claude, multi\-agent, open\-source, amazon, inference, chips, infra

## Summary

Amazon SageMaker HyperPod Inference Gateway is a Kubernetes\-native, GPU\-aware routing add\-on for Amazon EKS\. It uses real\-time GPU signals to send each inference request to the best\-suited pod, cutting first\-token latency by up to 82% with no changes to your model servers or client applications\.

## Sources

- [Story source](<https://aws.amazon.com/blogs/machine-learning/introducing-amazon-sagemaker-hyperpod-inference-gateway/>)
- [Story source](<https://aws.amazon.com/blogs/machine-learning/amazon-sagemaker-inference-2026-year-to-date-launches-in-review/>)