Next-Concept Modeling Generates Text 31x Faster Than Diffusion Models

Researchers have developed Next-Concept Modeling (NCM), a parallel text production method that is up to 31x faster than diffusion language models of similar parameter size for downstream tasks like machine translation and summarization. NCM is 10-21x faster for unconditional generation across benchmarks including OpenWebText, LM1B, and TinyStories, requires 3-5x fewer training tokens, and outperforms autoregressive and diffusion baselines on reasoning tasks like Sudoku. The project was developed by Kumbong Hermann, Ishan Khare, Iddah Mlauzi, Ellen Kellbuch, Simran Arora, and Azalia Mirh, in collaboration with Hazy Research.