---
format: "aidr-story-markdown/v1"
id: "f871eb5cbc09ced7b30e35cca245be858633646a020c4021089b92a4b7ab0030"
canonical_url: "https://aidr.today/f871eb5c?lang=en"
title: "Announcing Long-Context Training on TPUs with MaxText Context Parallelism"
lang: "en"
requested_lang: "en"
available_langs: ["en","vi"]
translation_fallback: null
fallback_fields: []
published_at: "2026-11-09T20:58:54.000Z"
category: "Frameworks"
topics: ["google","llm","framework"]
source_urls: ["https://developers.googleblog.com/announcing-long-context-training-on-tpus-with-maxtext-context-parallelism/"]
summary: "Google Cloud's MaxText framework now supports long-context training up to 14 million tokens on Cloud TPUs by implementing four context-parallel strategies: All-gather, A2A, P2P, and USP. These strategies use the Tokamax Splash Attention kernel to manage attention compute and memory bottlenecks without materializing full key-value sequences in high-bandwidth memory. Developers can adopt these capabilities in MaxText v0.2.4 to achieve high Model FLOPs Utilization (MFU) across large-scale model training workflows."
---

# Announcing Long\-Context Training on TPUs with MaxText Context Parallelism

> [Open the canonical story](<https://aidr.today/f871eb5c?lang=en>)

**Published:** 2026-11-09T20:58:54.000Z
**Category:** Frameworks
**Topics:** google, llm, framework

## Summary

Google Cloud's MaxText framework now supports long\-context training up to 14 million tokens on Cloud TPUs by implementing four context\-parallel strategies: All\-gather, A2A, P2P, and USP\. These strategies use the Tokamax Splash Attention kernel to manage attention compute and memory bottlenecks without materializing full key\-value sequences in high\-bandwidth memory\. Developers can adopt these capabilities in MaxText v0\.2\.4 to achieve high Model FLOPs Utilization \(MFU\) across large\-scale model training workflows\.

## Sources

- [Story source](<https://developers.googleblog.com/announcing-long-context-training-on-tpus-with-maxtext-context-parallelism/>)