SLIM-RL: Risk-Budgeted Random-Masking RL for Diffusion LLMs Without Trajectory Slicing
The training of diffusion large language models (dLLMs) using reinforcement learning has traditionally faced a steep computational bottleneck. Prior state-of-the-art frameworks like TraceRL assumed that random masking was fundamentally mismatched with a dLLM's inference trajectory. To resolve this, they relied on trajectory slicing—reconstructing the inference path by slicing rollouts into numerous trajectory-aligned training samples—a process that scales poorly as block sizes grow. To eliminate this computational overhead, researchers Ruikang Zhao, Zhenting Wang, Han Gao, and Ligong Han introduced SLIM-RL in a paper published on arXiv (cs.AI). This novel framework demonstrates that the trajectory-masking mismatch can be mitigated without trajectory reconstruction, offering a highly efficient, trace-free random-masking reinforcement learning alternative for dLLM optimization.
The technical foundation of SLIM-RL relies on two primary mechanisms. First, it introduces a tau-budget decoder that bounds the commit risk of each rollout step, directly minimizing the aggregate commit risk within the training dataset. This decoder is highly adaptable and transfers training-free across existing dLLM architectures such as LLaDA, Dream, and SDAR. Second, SLIM-RL employs a trace-free random-masking objective during optimization. This objective integrates advanced variance-reduction techniques, including sequence-level importance sampling and deterministic quadrature over masking levels, governed by a custom mean-preserving, monotonically decreasing per-block mask schedule.
Empirical evaluations on the SDAR-4B model show that SLIM-RL achieves parity with TraceRL’s peak MATH500 accuracy while utilizing only 46 percent of the training samples at a block size of 16. Under matched dynamic sampling, it outperforms TraceRL by 6.32 percent on MATH500 and 11.05 percent on GSM8K. At a smaller block size of 4, the 4B SLIM-RL model surpasses larger dLLMs, exceeding LLaDA-8B by 10.76 percent on math benchmarks and showing significant gains in code generation benchmarks like MBPP and HumanEval.
For machine learning engineers and researchers designing generative diffusion models, SLIM-RL proves that efficient random-masking training can bypass sequential trajectory simulation. This opens up scalable pathways for training highly capable dLLMs on constrained compute budgets, potentially bridging the performance gap between diffusion models and traditional autoregressive architectures. Please note that this analysis is based on the published abstract and metadata of the research paper.