Scaling Indic Pretraining: 2.8 Trillion Tokens and an Open Data Generation Pipeline

NVIDIA published a blog post on Hugging Face describing a 2.8 trillion token Indic pretraining effort and an open data generation pipeline.