Yhn first seen 10 h ago, last 7 min ago, peak #12
Dust: Pretraining Transformers Without Backpropagation
Researchers at QLabs have published Dust, a method for pretraining transformer models without using backpropagation, the algorithm that underpins almost all modern AI training. The work, described on the company's research page, is drawing attention from machine learning practitioners debating whether such approaches could reduce the heavy memory and compute costs of training large models.
Why now: Removing backpropagation from transformer training would challenge a foundational assumption of deep learning and could cut training costs, making it highly debateable among AI researchers.
QLabsDusttransformersbackpropagation
Rank over time, top of the chart is #1. 8 snapshots from 1 h ago to 7 min ago.
Evidence
- Dust: Pretraining Transformers Without Backpropagation · E-Reverance · 165
API: https://socialmediatrends-api.osmike.com/v1/trends/1158303