ModelsNov 17, 2026 · 07:002 sources

K2-Horizon: A Frontier Fleet of Open Source Language Models

From huggingface.co

Image: huggingface.co
AI;DR

We present K2-Horizon, a fleet of six fully open language models, from a 0.9B dense model to a 375B-A23B MoE, built for general-purpose reasoning and agentic work across math and science, software engineering, deep research, office automation, and beyond. The models lead or are competitive with strong open models of their size on agentic and coding benchmarks. On the modeling side, to strengthen reasoning, we introduce Mixture-of-Value Attention (MoVA), applying MoE to attention value projections, and softplus attention gating, a more flexible, theoretically motivated gate. Our 36B-A4B model uses both and nearly matches a 32B dense model trained on the same data, with one-seventh the active parameters. We also introduce Uno, lossless diffusion adapters that speed up decoding by up to 2.2x without a separate draft model.

Read the sourceSign in to suggest

Sources

One ranked edition, every morning

The stories that matter today. Free, cancel anytime.

Telegram @aidr_today