New LSA Method Speeds AI's Full Attention Processing Up to 72.4x

Researchers published a new framework called Latent Sparse Attention. The LSA method provides a processing boost of up to 72.4x for full attention. When handling 10M tokens, the search for relevant tokens accounts for 98% of an attention layer's time.