Radical Numerics' Halcyon Models Cut Pretraining Compute 275x

Radical Numerics' Halcyon models, ranging from 0.1B to 4B parameters, can outperform late-fusion text-to-image baselines without using frozen language models or text-only pretraining. The 1.6B Halcyon model beats the JiT-T2I-XXL/8++ baseline while using 275x less pretraining compute, and the 0.52B version outperforms LlamaGen-XL/8 with 18.2x less compute. Radical Numerics has made the models and their training recipes open.