virtual memory

Researchers at QLabs have published work on Dust, a method for pretraining transformer models without using backpropagation, the algorithm that underpins virtually all modern deep learning. The paper is drawing attention among machine learning practitioners, who are weighing whether alternatives to backprop could reduce the memory costs and computational limits of training large models.