Q Labs Introduces 'Dust': Gradient-Free Pretraining for Transformers

Another Homage to "The Bitter Lesson"?

Can we train large models without backpropagation?

People have been pondering this for a long time. However, backpropagation has been so successful that it has remained the sole mainstream algorithm for training neural networks in recent years. Today, all our architectures, optimizers, and even GPU hardware are, in a sense, built around it.

Yet some remain skeptical: backpropagation requires the entire network to be differentiable everywhere, which is actually a very strong constraint. Non-differentiable modules, discrete computations, and external programs simply cannot enter the training loop.

So, if compute becomes sufficiently cheap, could we stop "calculating" gradients altogether and switch to the most primitive approach—randomly perturbing the network, observing whether performance improves or degrades, keeping the directions that improve it, and discarding those that worsen it? This method is brute-force, but its advantage lies in being agnostic: regardless of what the internal structure of the network looks like or whether it is differentiable, it can still be applied.

A small research lab called Q Labs recently attempted a similar method, naming it "Dust."

They claim this is the first zeroth-order optimization method capable of competing head-to-head with backpropagation on Transformer pre-training tasks.

The general idea is this: instead of touching the weights, add Gaussian noise to the activation values of each layer's output, independently for every token. This makes each token effectively a "virtual experimental individual," allowing thousands of individuals to be evaluated in parallel during a single forward pass. This is what they call a "virtual population." If a specific noise perturbation reduces the loss, it gets credit; by taking the weighted average of these credited noises, one obtains an estimate of the gradient.

Q Labs claims that Dust approximates backpropagation well at large population sizes (i.e., significantly higher computational cost) and even surpasses it in various settings. They suggest, "This implies that under conditions of abundant computational resources, we might be able to move beyond backpropagation."

However, the bottleneck is obvious: scale. The authors themselves state frankly that Dust is currently far more expensive than backpropagation and needs to become several orders of magnitude cheaper before it qualifies as a potential "replacement." The experiments in the paper only reached up to 243M parameters and a training budget of 20M tokens, which is vastly different from industrial-scale pre-training involving hundreds of billions of tokens.

Third-party reproduction efforts by SOTAAZ Lab provide a more sober perspective on Dust: under the single-round SGD comparison specified in the paper, Dust's results are largely reproducible, but the gap between it and backpropagation is small, failing to show the significant advantage emphasized in the paper. Once backpropagation is paired with a more suitable optimizer or allowed to train for multiple epochs on the same data, it clearly leads, with a computational cost far lower than Dust. In other words, the significance of Dust lies more in proving that zeroth-order methods can be used for Transformer pre-training, rather than replacing backpropagation in the current landscape.

Why Challenge Backpropagation?

In their technical blog post, the team outlined their research motivation, claiming it is essentially another homage to "The Bitter Lesson": as the amount of available compute in the world increases, methods relying on clever inductive biases (such as differentiability, backpropagation, and high-order gradient approximations) often eventually lose out to "general-purpose, compute-hungry brute-force search."

They cite AlphaGo Zero as an example: versions guided by human game records learned faster initially, but pure self-play versions surpassed them given sufficient compute. The authors' analogy is that differentiability and backpropagation might be "good biases for the low-compute era": they make learning efficient, but in the high-compute era, they restrict the space of architectures that can be trained.

They further speculate that this might explain why current neural networks require massive amounts of data to generalize: gradient-based methods do not fully explore the loss landscape, whereas a search-based, more flexible credit assignment algorithm could be a key step toward better generalization.

Thus, Dust's goal is quite "ambitious": to replace backpropagation with an algorithm that relies more on brute-force computation and less on analytical structure.

How Does Dust Bypass Backpropagation?

The core idea of Dust is to shift the "weight perturbation" found in traditional Evolution Strategies (ES) into the "activation space" of Transformers, leveraging the token dimension for massive parallelism.

When training models using traditional ES, multiple slightly different sets of model weights must be generated. Each set performs a separate forward pass, and based on changes in loss, the system determines which perturbations are worth keeping.

Although this method does not rely on backpropagation, it comes at a high cost. Every member of the population requires a separate forward computation; the larger the population size, the higher the computational overhead.

Previous work, EGGROLL, attempted to reduce costs through low-rank perturbations but still required assigning an independent sequence position to each population member.

Dust takes a different approach: instead of directly modifying model weights, it adds random noise to the output of each linear layer and generates perturbations independently for each token.

Specifically, for a linear layer $y_t = Wx_t$, Dust modifies the output as follows:

$$y_t \rightarrow y_t + \sigma a_t$$

where $\sigma$ controls the perturbation magnitude, and $a_t$ is random noise drawn from a standard normal distribution.

In this way, each token effectively experiences slightly different model parameters, without requiring any actual copying or modification of the weights. The research team refers to this concept as a "Virtual Population."

The advantage is that a single forward pass can simultaneously evaluate thousands of virtual population members along the token dimension. In contrast, traditional weight-space evolution strategies typically require separate forward passes for each member.

This is also key to how Dust scales up population size while reducing sampling costs.

Without Backpropagation, How Do We Know Where to Update Parameters?

Once perturbations are introduced, the next step is to determine which ones actually improved model performance.

Dust’s method involves observing how the loss for each token changes after adding noise. If a perturbation in a certain direction reduces the loss, it receives a higher reward; otherwise, its reward is lowered.

However, tokens in Transformers have dependencies. Especially under causal attention mechanisms, changes to an earlier token can affect the predictions of multiple subsequent tokens.

Therefore, Dust introduces a credit assignment mechanism with a decay factor:

Here, $r_t$ represents the perturbation reward for the $t$-th token, $c_s$ indicates the loss improvement for the $s$-th token, and $\gamma$ controls the contribution weight of future tokens.

Intuitively, if perturbing a specific token makes subsequent predictions more accurate, it earns an additional reward.

Subsequently, Dust computes a weighted average of the random noises obtained from multiple samples and their corresponding rewards to estimate the error signal at the output of the linear layer. It then performs an outer product with the input to that layer to obtain a gradient estimate for updating the weights:

where $\hat{g}_t$ is the estimated output-side error signal derived from perturbation rewards, and $x_t$ is the input to the linear layer.

A crucial difference between this process and traditional backpropagation is:

Backpropagation calculates gradients layer by layer using the chain rule, whereas Dust estimates update directions during the forward pass based on loss changes caused by perturbations.

Thus, Dust can obtain approximate gradients for training Transformers without executing standard backpropagation.

With So Much Noise Added Simultaneously, How Is Interference Avoided?

Large-scale parallelism introduces new challenges.

If multiple layers and tokens are perturbed simultaneously, the loss change for a given token may be contaminated by other perturbations, making it difficult to attribute improvements to specific perturbations.

To address this, Dust incorporates several targeted optimizations.

First, perturbation calculations are performed separately for different types of layers, and unperturbed intermediate results are cached to reduce redundant forward passes.

Second, for components within attention modules such as Query, Key, Value, and Gate, Dust does not rely solely on token loss to judge perturbation effects. Instead, it assigns rewards based on changes in attention outputs caused by perturbations and previously estimated output error signals.

Finally, for the language model output head, Dust applies perturbations directly to cached logits and only recomputes the relevant cross-entropy loss. This allows for scaling up the population size at a lower cost.

The research team also used hyperparameter search to tune perturbation strength, sampling ratios, and credit assignment ranges across different layers.

Ultimately, Dust transforms the expensive model weight sampling inherent in traditional evolution strategies into more easily parallelizable token-level activation perturbations, enabling Transformers to perform parameter updates using reward signals from forward computations.

Its key value lies in providing a training path for large model pretraining that does not depend on standard backpropagation.

However, whether this approach yields computational efficiency advantages in larger models, longer training cycles, and real-world hardware environments still requires further validation through experimental results.

What Level of Model Can Dust Train Without Backpropagation?

While the method sounds clever, the critical question remains: Can gradient estimation via random perturbations compete head-to-head with mature backpropagation?

The research team trained a series of Transformers on the FineWeb dataset and compared Dust against traditional backpropagation and the evolution strategy EGGROLL.

Experimental results show that as population size increases, Dust’s training performance continues to improve, even surpassing backpropagation under certain training budgets.

Compared to EGGROLL, Dust’s advantage is even more pronounced. Even when EGGROLL uses 16,384 population members, its performance falls short of Dust, which uses only 64 perturbation samples—a 256-fold difference.

This demonstrates that shifting perturbations from weight space to activation space indeed improves sampling efficiency.

More interestingly, Dust remains effective even when the model parameter count expands approximately 120 times, growing from 2 million to 243 million.

With larger population sizes, the test loss for models with 38 million and 243 million parameters was actually lower than that of the smallest 2-million-parameter model. Furthermore, larger models benefit more significantly from increased sampling counts compared to smaller models.

This challenges the conventional belief that zero-order optimization struggles to scale to large neural networks, indicating that increasing parameter size does not necessarily cause random perturbation methods to fail.

So, how reliable are the update directions obtained by Dust through random sampling?

The research team further compared the gradients estimated by Dust with those from backpropagation and found that as the sample size increased, the two directions converged increasingly closely.

This trend persisted even when models had undergone prolonged training, indicating that Dust can continuously obtain useful gradient signals by scaling up the population size.

However, these results still have limitations.

Currently, Dust has only been validated on models with up to 243 million parameters, with a maximum pre-training data budget of 20 million tokens. Moreover, some results outperforming backpropagation were derived from curve-fitting extrapolations and have not yet been directly achieved in actual training runs.

Therefore, while Dust has demonstrated the feasibility of training Transformers without relying on standard backpropagation, whether it can achieve dual advantages in performance and computational efficiency at larger scales remains to be verified.

What is Q Labs?

Q Labs’ research follows a clear central theme: If we stop giving models more data and instead give them more compute, can they learn better? The lab’s website lists data efficiency and generalization capabilities as core focus areas and discloses that its investors include Jeff Dean.

Dust emerged along this trajectory. Previously, the team introduced NanoGPT Slowrun, which fixed the data budget to challenge researchers to compete on who could utilize limited data most effectively. Subsequently, they proposed q0, enabling multiple models to explore the same dataset, learn from each other, and combine their predictions. Under the Slowrun setting, q0 reported approximately 12.9x higher data efficiency relative to the reference baseline. Researchers from Princeton University also contributed to this work.

With Dust, they pushed the problem one step further: If the training method itself is changed, can additional compute yield new benefits?

The backgrounds of the two co-founders offer insight into the team. Samip Dahal co-authored the launch article for Replit’s Ghostwriter and participated in early AI coding tool development. Akshay Vegesna studied at Caltech and previously worked on reconstructing tracking systems using neural networks within Nuro’s perception team (an autonomous driving company). Both have experience applying machine learning to real-world systems, and they have now shifted their research focus to learning algorithms themselves.

The team’s approach to advancing research is also noteworthy: papers are followed immediately by code releases and public competitions. After releasing Dust, they launched the Noise speedrun, which prohibits backpropagation, inviting the community to continue improving algorithms around the same benchmark. Bishwas Mandal and Serdar Gülbahar, the other two authors of Dust, are also listed as core contributors to this project.

From fixing data budgets, to exploring model populations, to challenging backpropagation, Q Labs has consistently asked the same question: Is our current use of compute far from optimal?

Next Steps: How Far Can It Go?

When Dust was released, Samip acknowledged that it remained significantly more expensive than backpropagation. However, he believes that as compute increases, search-dependent zeroth-order methods may continue to improve and potentially eventually replace backpropagation.

Subsequent community optimizations indeed progressed rapidly. Recently, Samip announced that the training time for the Noise speedrun dropped from approximately 1.76 hours to 1.6 seconds within ten days—a speedup of roughly 4000x.

This achievement came with specific thresholds: under the then-current 10 million token data setting, reaching a validation loss of 4.2. The actual loss recorded for the 1.6-second run was 4.186; this metric represents the time taken to reach the specified score, rather than fully training the model to convergence. The comparison baseline was also the early Dust + Adam setup, not backpropagation.

Samip simultaneously noted that certain acceleration techniques do not scale to larger sizes. Once a small-scale benchmark is conquered, the true test begins.

As models grow larger, training becomes more thorough, and target losses decrease, will the update directions derived from random probing still be effective? And how much compute will be required to find them?

Dust has already produced results worth pursuing further. Upcoming experiments will reveal just how wide this path truly is.

References:

https://qlabs.sh/research/dust/

https://sotaaz.com/post/dust-backprop-free-rerun-en

© THE END

Please contact this official account for authorization to repost.

For submissions or media inquiries: [email protected]