ResearchOct 1, 2026 · 22:423 sources
AI Web Text Hits 31% and Harms Language Model Training
From marketbrief.now
A study analyzing the pretraining of 800 models found that incorporating AI-generated tokens into training sets can increase loss on held-out human text, degrading overall performance. Pangram identified 31.1% of FineWeb-filtered web tokens as AI-generated in August 2026, up from 27.5% in June 2026 and 10% in June 2024. For data-starved models, AI tokens initially improve accuracy before reversing into harm, while models with high human text budgets experience immediate degradation. The researchers' new scaling law predicts these effects for models up to 3.6x larger with 41% lower error than previous estimates.
Read the sourceSign in to suggest
Sources
#llm#googleMarkdown
Content history (1)
- 2026-10-04 · Title · vi · Wording fixVăn Bản Web Do AI Tạo Ra Chiếm 31% Và Gây Hại Cho Huấn Luyện Mô Hình Ngôn NgữVăn bản web do AI tạo ra chiếm 31% và gây hại cho huấn luyện mô hình ngôn ngữ
One ranked edition, every morning
The stories that matter today. Free, cancel anytime.
Telegram @aidr_today