ResearchAug 22, 2026 · 06:495 sources

Marin Starts Training 535B AI Model on 18.75T Tokens in Biggest Open Run

From huggingnews.com

Image: huggingnews.com
AI;DR

The Marin team has begun training its 535B-A23B large language model using 11 GB200 NVL72 clusters on a 18.75T token dataset. The training involves pretraining on 80% and midtraining on 20% of the dataset over approximately three months, with a total compute requirement of 2.7e24 FLOPs. Public access to the training mixture, sampled documents, and live training loss is available through WandB and GitHub.

Read the sourceSign in to suggest

Sources

+ 4 more
  1. SUPPORT@percyliang“we can also forecast the loss of intermediate checkpoints, so we can see if we’re on track over the voyage”x.com
  2. SUPPORT@percyliang“Assembling this hero run really required data, architecture, infra, kernels to all come together and was a huge joint effort”x.com
  3. SUPPORT@eliebakouch“you can visualize the exact pre training mixture per domain, look at sampled documents, follow the live training loss”x.com
  4. SUPPORT@natolambert“I'm excited to get to work with the community post-training this model!”x.com
#llm#training#chipsMarkdown
Content history (2)
  • 2026-10-16 · Summary · vi · Replaced stored text
    Tuần này, đội ngũ Marin đã bắt đầu huấn luyện mô hình LLM 535B-A23B bằng các cụm 11 GB200…
    Đội ngũ Marin đã bắt đầu huấn luyện mô hình ngôn ngữ lớn 535B-A23B của họ vào tuần này bằ…
  • 2026-10-16 · Title · vi · Replaced stored text
    Marin bắt đầu huấn luyện mô hình AI 535B trên 18,75 nghìn tỷ token trong đợt chạy mã nguồ…
    Marin bắt đầu huấn luyện mô hình AI 535B trên 18,75T token trong vòng chạy mở lớn nhất

One ranked edition, every morning

The stories that matter today. Free, cancel anytime.

Telegram @aidr_today