Researchers from Stanford and UC Berkeley Introduce TokenGate: Token-Gated Attention Makes Long-Context Prefill Up to 11.3x Faster

Researchers from Stanford and UC Berkeley introduced TokenGate, a token-gated attention module that accelerates long-context prefill by up to 11.3x and reduces peak memory by up to 3.4x without accuracy loss.