The inventor of the bitter lesson has a bitter lesson for LLM scaling enthusiasts

Turing Award winner Richard Sutton reaffirms that raw compute alone is not enough for intelligence and that fundamentally new learning methods are needed. Former OpenAI researcher Tworek agrees, stating that blind transformer scaling has reward hacked the brains of a generation of scientists.