---
format: "aidr-story-markdown/v1"
id: "91acaa948be0ef776517a5556250910b71b6f18d03a7c2e95ae4fc5fa82aafd6"
canonical_url: "https://aidr.today/91acaa94?lang=en"
title: "Amazon SageMaker AI real-time endpoints now support KV cache aware routing"
lang: "en"
requested_lang: "en"
available_langs: ["en","vi"]
translation_fallback: null
fallback_fields: []
published_at: "2026-11-17T21:11:51.000Z"
category: "Tools"
topics: ["amazon","inference","devtools"]
source_urls: ["https://aws.amazon.com/blogs/machine-learning/amazon-sagemaker-ai-real-time-endpoints-now-support-kv-cache-aware-routing/"]
summary: "Amazon SageMaker AI real-time endpoints now support KV cache-aware routing, a new strategy that routes each request to the instance holding the longest cached match for its prompt. Learn how it works, when to use it compared with prefix-aware routing, and how to enable, invoke, and verify it on single model and inference component endpoints."
---

# Amazon SageMaker AI real\-time endpoints now support KV cache aware routing

> [Open the canonical story](<https://aidr.today/91acaa94?lang=en>)

**Published:** 2026-11-17T21:11:51.000Z
**Category:** Tools
**Topics:** amazon, inference, devtools

## Summary

Amazon SageMaker AI real\-time endpoints now support KV cache\-aware routing, a new strategy that routes each request to the instance holding the longest cached match for its prompt\. Learn how it works, when to use it compared with prefix\-aware routing, and how to enable, invoke, and verify it on single model and inference component endpoints\.

## Sources

- [Story source](<https://aws.amazon.com/blogs/machine-learning/amazon-sagemaker-ai-real-time-endpoints-now-support-kv-cache-aware-routing/>)

