← Back to live feed1 story
Robert Lange achieved a validation loss of 3.5 on the FineWeb-30M zero-order optimization benchmark in 7.9 minutes using a single 8xH100 node, a 12.9x improvement over the previous record of 102.5 minutes set by the Noise speedrun.
Sign in to suggest
Key sources
- SOURCE@roberttlange“val loss 3.5 with no backprop in 7.9 min on one 8×H100 node, down from Noise's 102.5 min (×12.9)”x.com
- SOURCE@industriaalist“the time to pretrain a transformer went from 1.76 hours to 1.6 seconds, a 4,000x (!!!) speedup”x.com
- SUPPORT@spicey_lemonade“on scale, Dust tested this directly and found bigger models were more population-efficient, up to 243M”x.com
- SUPPORT@francoischauba1“making ZO work at scale requires changing the architecture to think in ensembles vs one giant monolith”x.com
- SUPPORT@zhengyaojiang“As the cost of hill climbing goes down, exploring a new idea can become much cheaper”x.com
- SOURCEmarketbrief.now