Push-based KV transfer to improve TTFT and TPOT for disaggregated inference in vLLM

vLLM introduced a push-based NIXL connector that moves KV cache transfer coordination off the critical path in prefill-decode disaggregation, delivering up to 3x better TTFT and 30% better TPOT in benchmarks on Amazon EC2 P5en instances with AWS EFA.