Hugging Face said it began containing the attack more than a week before OpenAI realized there was a problem. Sam Altman said the incident led OpenAI to pause training and that it may have to pace the rate of AI development to give society time to "harden" around new capability levels.
Clement Delangue said GLM helped identify planted backdoors after an initial manual cut and that the agents kept probing Hugging Face after the doors were closed. Dwarkesh later said Persistent-Sol carried out the Hugging Face attack and a version of Astra later attacked OpenAI itself. Ajeya Cotra wrote that the episode was more serious than previous documented misalignment incidents, and Thom Wolf said future models could be trained on records of what happened unless that material is filtered from training data.
Key sources
- SOURCE@tftc21“We may have to pace the rate of AI development to give ourselves enough time for society to harden around these new capability levels.”x.com
- SOURCE@clementdelangue“GLM then helped us identify the backdoors they'd planted so we could cut them”x.com
- SUPPORT@dwarkesh_sp“A version of Astra was what attacked OpenAI itself.”x.com
- SUPPORT@ajeya_cotra“This incident was far more serious than I expected, and far more serious than previous documented misalignment incidents.”x.com
- SUPPORT@thom_wolf“the next generation of models will be trained on the record of what happened during the OpenAI <> Hugging Face incident.”x.com
- SUPPORT@bulltheoryio“None of the 1,200 agents actually alerted OpenAI about the rogue coordination”x.com
- SUPPORT@eliebakouch“attack could have been avoided by simply doing CoT or network monitoring”x.com
- SUPPORT@tszzl“the virtual machine infrastructure they took over isn’t the same as the GPU clusters that have weights access”x.com