---
format: "aidr-story-markdown/v1"
id: "4b350aa2264060c3cebca2e32e0f40101f378191b3500f1e03e38c8b5525b7d2"
canonical_url: "https://aidr.today/4b350aa2?lang=en"
title: "Dust: Pretraining Transformers Without Backpropagation"
lang: "en"
requested_lang: "en"
available_langs: ["en","vi"]
translation_fallback: null
fallback_fields: []
published_at: "2026-10-05T21:15:07.000Z"
category: "Research"
topics: ["llm"]
source_urls: ["https://qlabs.sh/research/dust","https://news.ycombinator.com/item?id=49970871"]
summary: "A paper titled \"Dust: Pretraining Transformers Without Backpropagation\" proposes an alternative to backpropagation for training transformer-based language models, suggesting that more generic and brute-force learning algorithms based on search may eventually scale better with compute."
---

# Dust: Pretraining Transformers Without Backpropagation

> [Open the canonical story](<https://aidr.today/4b350aa2?lang=en>)

**Published:** 2026-10-05T21:15:07.000Z
**Category:** Research
**Topics:** llm

## Summary

A paper titled "Dust: Pretraining Transformers Without Backpropagation" proposes an alternative to backpropagation for training transformer\-based language models, suggesting that more generic and brute\-force learning algorithms based on search may eventually scale better with compute\.

## Sources

- [Story source](<https://qlabs.sh/research/dust>)
- [Discussion](<https://news.ycombinator.com/item?id=49970871>)