The Recurrent Looped Transformer (RLT) allows a model to reuse its decoder state across every prompt and response token to extend internal computation. By feeding the final hidden state of one token back into the next, the architecture creates latent computation paths that grow with sequence length without increasing physical parameter layers. For instance, a 48 layer decoder processing 100 tokens produces a computation path 4,800 layers deep using the same network weights at different time steps.

A causal encoder supports the system by providing reusable global KV memory, while a technical report for the design is available on GitHub. Princeton researcher Yifan Zhang, previously with Nvidia and ByteDance, developed the RLT to solve more complex problems under the same compute budget. The repository currently lacks training code, inference implementations, and model weights, with Zhang calling for a measured pace of AI progress until safe superintelligence is achieved.

Sign in to suggest edits

Key sources

  1. SOURCE@maxforai“来自普林斯顿的研究者 Yifan Zhang,今天发布了新一代循环回路Transformer架构:RLT(Recurrent Looped Transformer)”x.com
  2. SUPPORT@askalphaxiv“Longer sequences then create deeper latent computation paths without adding more physical layers”x.com
  3. SOURCEmarketbrief.now
Markdown