---
format: "aidr-story-markdown/v1"
id: "27312a297198317008c047ca64e40a105cca73c58203d4b658bb6b5b7d905659"
canonical_url: "https://aidr.today/27312a29?lang=en"
title: "Push-based KV transfer to improve TTFT and TPOT for disaggregated inference in vLLM"
lang: "en"
requested_lang: "en"
available_langs: ["en","vi"]
translation_fallback: null
fallback_fields: []
published_at: "2026-10-19T16:05:17.000Z"
category: "Frameworks"
topics: ["inference","framework","amazon"]
source_urls: ["https://aws.amazon.com/blogs/machine-learning/push-based-kv-transfer-to-improve-ttft-and-tpot-for-disaggregated-inference-in-vllm/","https://aws.amazon.com/blogs/machine-learning/improve-ttft-for-multi-turn-agentic-workloads-with-bi-directional-kv-transfer-in-vllm/"]
summary: "vLLM introduced a push-based NIXL connector that moves KV cache transfer coordination off the critical path in prefill-decode disaggregation, delivering up to 3x better TTFT and 30% better TPOT in benchmarks on Amazon EC2 P5en instances with AWS EFA."
---

# Push\-based KV transfer to improve TTFT and TPOT for disaggregated inference in vLLM

> [Open the canonical story](<https://aidr.today/27312a29?lang=en>)

**Published:** 2026-10-19T16:05:17.000Z
**Category:** Frameworks
**Topics:** inference, framework, amazon

## Summary

vLLM introduced a push\-based NIXL connector that moves KV cache transfer coordination off the critical path in prefill\-decode disaggregation, delivering up to 3x better TTFT and 30% better TPOT in benchmarks on Amazon EC2 P5en instances with AWS EFA\.

## Sources

- [Story source](<https://aws.amazon.com/blogs/machine-learning/push-based-kv-transfer-to-improve-ttft-and-tpot-for-disaggregated-inference-in-vllm/>)
- [Story source](<https://aws.amazon.com/blogs/machine-learning/improve-ttft-for-multi-turn-agentic-workloads-with-bi-directional-kv-transfer-in-vllm/>)