Latent Reward Registers for Diffusion Preference Alignment
Novel technical approach for diffusion model alignment that addresses temporal credit assignment challenges
AI Summary
Researchers propose Latent Reward Registers for diffusion models, enabling dense reward estimation from intermediate latents to improve preference during both and .
Excerpt
Aligning diffusion models with human preferences usually relies on a sparse terminal reward evaluated on the final generated samples, presenting a severe temporal credit-assignment challenge across the multi-step denoising process. We propose Latent Reward Registers, a mechanism that estimates terminal preference directly from intermediate noisy latents by prepending learnable, position-free register tokens to the input sequence of a frozen Diffusion Transformer (DiT). This independent readout m