AV-GRPO: Modality-Anchored Decoupling Diffusion Reinforcement Learning for Joint Audio-Video Generation

arXiv · Zhiyu Xu, Weilong Yan, Yufei Shi, Shiyang Li, Yihao Liu, Kin-Man Lam, Yuewen Cao · 2026-09-27

This item is no longer on the current shelf. Its page remains available, and editorial corrections may be made. See what is featured now.

AV-GRPO reframes joint audio-video generation as decoupled unimodal RL, boosting fidelity and synchronization.

What it does

The paper proposes AV-GRPO, a modality-anchored online diffusion RL framework that separates audio and video reward streams. It introduces three modules: modality-anchored rollouts for disentangled learning signals, trajectory-locked frozen-tower optimization to lower compute and clarify credit assignment, and adaptive objectives that tailor perturbation strength to each modality’s dynamics. A new 5DAV dataset, decoupled across five dimensions, supports systematic training. Experiments on JavisBench and VABench show AV-GRPO surpasses LTX-2.3 in generation quality, semantic alignment, and cross-modal synchronization under both LoRA and full fine-tuning.

Why it matters

Joint audio-video generation suffers from tangled reward signals that make credit assignment noisy and training expensive. By turning the problem into two unimodal RL sub-tasks, AV-GRPO stabilizes learning and reduces compute while improving synchronization—key for realistic media synthesis.

How it applies

Practitioners can plug the modality-anchored rollout and frozen-tower modules into existing diffusion-based audio-video generators to gain higher fidelity with less GPU budget. The 5DAV dataset can be used as a benchmark for evaluating synchronization and modality-specific performance.

The short version

Synchronizing sound and sight just got a clean RL makeover.

My take

The modular RL design feels like the next logical step for scaling multimodal diffusion models. By freezing one tower during rollout, you keep the cost in check while still letting the other modality benefit from fresh gradients. It’s a pattern we’ll likely see repeated as more creators demand high-quality, tightly synced audio-visual content.

How it connects

• Echoes the broader trend of decomposing complex multimodal tasks into simpler, well-defined sub-tasks.
• Shows that reinforcement learning can still add value on top of diffusion models when reward design is carefully engineered.
• Provides a reusable dataset (5DAV) that can benchmark future audio-video sync methods.

Bottom line

Adopt the modality-anchored RL modules to get higher-fidelity, synchronized audio-video generation without a proportional increase in compute.

Takeaways

• Modality-anchored rollouts disentangle reward signals, stabilizing joint generation training.
• Trajectory-locked frozen-tower optimization cuts computational cost while preserving credit clarity.
• Adaptive objectives let each modality’s diffusion dynamics be tuned independently.
• The 5DAV dataset provides a systematic, difficulty-controlled benchmark for audio-video synthesis.
• AV-GRPO outperforms the prior LTX-2.3 baseline across quality, alignment, and synchronization metrics.

Topics: audio-visual generation, diffusion models, reinforcement learning, multimodal alignment, synchronization