Prefix Sliding Speeds AI Models 3x for Reasoning Traces Over 100,000 Tokens

Researchers developed a technique to discard intermediate reasoning tokens that lose importance as a model continues to process, allowing existing systems to run 3x faster without training. The approach, called Prefix Sliding, avoids the memory failures associated with vanilla full attention and the loss of detail common in compaction methods.

Using reinforcement learning with Prefix Sliding enables models to scale to reasoning traces exceeding 100,000 tokens. The system maintains performance by retaining only the initial prefix and a sliding window of the most recent few thousand tokens.