MeanFlowNFT: Bringing Forward-Process RL to Average-Velocity Generators
Advances RL alignment for efficient few-step generative models with theoretical guarantees and empirical improvements
AI Summary
Researchers introduce MeanFlowNFT, a novel reinforcement learning framework that extends forward-process RL techniques to average-velocity generators like MeanFlow models. The method bridges the gap between instantaneous velocity optimization in DiffusionNFT and average velocity prediction in MeanFlow, preserving fast few-step sampling while improving performance. Experimental results show MeanFlowNFT outperforms prior state-of-the-art RL-tuned generators on most metrics across image and video generation tasks.
Excerpt
MeanFlow generators achieve fast few-step sampling by predicting average velocities over time intervals, making them attractive for efficient generation. Reinforcement learning (RL) has become a powerful way to align diffusion and flow models with human preferences and task-specific objectives. In particular, DiffusionNFT offers an efficient forward-process RL framework that does not require reverse-process trajectories or likelihood estimation. However, applying such RL methods to MeanFlow rema