DuoMatching Boosts Low-Step Video Quality via Dual Distribution Matching

Adding Image Supervision to Few-Step Video Generation

Submitted by the DuoMatching Team

QbitAI | WeChat Official Account: QbitAI

In the field of video generation, few-step video models have achieved the ability to generate continuous frames with only a small number of sampling steps.

However, while speed has improved, issues arise regarding detail fidelity, material representation, and adherence to prompt requirements, which may suffer in quality.

How can we further improve visual details and prompt alignment while maintaining real-time generation capabilities?

A research team from MMLab at The Chinese University of Hong Kong (CUHK), ByteDance, and The Hong Kong University of Science and Technology (HKUST) proposed DuoMatching. This method allows video generators to simultaneously receive joint distribution supervision and frame-level marginal distribution supervision, incorporating complementary visual and semantic priors into few-step video generation.

This approach introduces image priors during the training phase without increasing inference computational costs. In the paper's frame-wise 2-step setting, the overall VBench score improved from 80.67 to 83.53. Compared to four baselines in human evaluation, the overall preference rate exceeded 80%.

△ Four case studies—cat holding a sign, person, coastline, and ceramic cat—demonstrate the visual differences between Causal Forcing++ and DuoMatching via sliding comparison.

Room for Quality Improvement After Few-Step Generation

Streaming video generation requires generating new frames while maintaining consistency with existing content. Distribution Matching Distillation (DMD) enables few-step student models to learn the video distribution approximated by pre-trained video teachers, effectively mitigating the gradual degradation of visual quality often seen in autoregressive generation processes.

However, reduced degradation does not guarantee that every frame is optimal. The research team observed that distilled videos still exhibit gaps in visual quality, and some prompt requirements are not fully realized. For instance, hair and material details may lack richness, or text-specified appearance features might be omitted. Even after slowing down quality degradation, there remains a need to further enhance the visuals themselves.

DuoMatching addresses this by organizing supervision at two levels: focusing on the joint distribution of the complete video and directly addressing the single-frame distribution extracted from the video. The visual and semantic priors accumulated by image generation models provide supplementary support for the latter objective.

Joint Guidance for Complete Videos and Single Frames

Joint distribution matching can be understood as "viewing the entire video," encompassing both content and inter-frame relationships. Frame-level marginal distribution matching corresponds to "viewing sampled frames," providing direct guidance on single-frame visual details and text alignment. While joint matching inherently constrains single-frame distributions, optimizing single-frame quality and inter-frame relationships under shared parameters requires additional frame-level supervision to provide complementary signals.

Specifically, DuoMatching retains the original joint DMD and adds marginal DMD provided by an image teacher, using a weight to control the strength of the latter. Both objectives jointly update the video generator. The image teacher participates in distribution supervision during training; it does not require generating a "standard image" first and then forcing every video frame to conform to it.

△ DuoMatching trains the video generator using both joint and marginal distribution matching. LatentBridge and LVS facilitate the integration of frame-level supervision into the video latent space.

Integrating Image Priors into Compressed Video Latent Spaces

Once the objectives are defined, a practical obstacle remains: image models and video models use different internal representations. Modern video VAEs compress along the temporal axis, potentially packing multi-frame information into a single latent slice. Directly feeding such slices to an image teacher would force it to interpret mixed temporal information as a static image, misaligning the supervision target.

LatentBridge: Mapping Video Representations to Single-Frame Representations

The most straightforward approach is to decode video latents into images and re-encode them using an image VAE, but the full decode-encode pipeline incurs high training overhead. DuoMatching uses a lightweight LatentBridge to learn this mapping step: it takes the current latent slice, the previous slice, and local frame indices as input, outputting single-frame latents usable by the image teacher.

After pre-training, the LatentBridge parameters are frozen, yet the video generator still receives gradients from frame-level supervision through it. In ablation studies, directly supervising compressed latents resulted in a dynamic degree score of 76.94; using LatentBridge increased this to 93.61. This demonstrates that aligning the representation acted upon by supervision is crucial for effectively leveraging image priors.

LVS: Distributing Limited Supervision Across Different Time Segments

Adjacent frames in a video are often highly similar. If sampled frames cluster within the same time segment, the image teacher may repeatedly inspect nearly identical content. Latent Variation Sampling (LVS) divides time segments based on the variation between adjacent latents and samples from each segment. This ensures that limited frame-level supervision covers more diverse content states, reducing redundancy.

LatentBridge solves "how the image teacher interprets video representations," while LVS solves "where to apply supervision." Both serve the training process; during inference, the original video generator is used, eliminating the need to deploy the image teacher or bridging modules in the generation pipeline.

Motion Dynamics Largely Preserved After Quality Enhancement

The research team validated DuoMatching across three generation settings: bidirectional, frame-wise autoregressive, and chunk-wise autoregressive. Taking the frame-wise 2-step setting as an example, compared to Causal Forcing++, the overall VBench score improved by 2.86 points, with gains in semantic, aesthetic, and imaging quality scores.

△ Different methods should be compared under identical generation modes and step counts.

Regarding motion, the dynamic degree changed slightly from 94.44 to 93.61, and motion smoothness shifted from 98.20 to 98.39. Combining quantitative metrics with visual comparisons, the paper’s overall conclusion is that visual quality, composition, and semantic alignment are enhanced while motion dynamics are largely preserved. This answers a key question arising from introducing image supervision: when visuals look better, do videos retain their dynamic nature?

Human evaluations further verified whether these improvements were perceptible to viewers. Twenty-three participants each compared 40 pairs of videos generated from the same prompts using a binary choice format. Relative to CausVid, Causal Forcing++, One-Forcing, and Reward Forcing, DuoMatching achieved overall preference rates of 91.30%, 80.87%, 94.35%, and 83.91%, respectively. These percentages represent the proportion of pairwise comparisons where DuoMatching was selected.

△ Each screen displays two sets of cases: the left side shows the baseline with visual annotations, and the right side shows DuoMatching. The sequence presents comparisons against Causal Forcing++, Reward Forcing, One-Forcing, and CausVid.

For video generation systems that require continuous output and real-time feedback, DuoMatching offers a path to improve quality by starting with training supervision: accumulating the priors learned by image models for use in video generation while maintaining the efficiency of few-step inference.

The paper is now available on arXiv, and both the code and model weights have been released. The project homepage provides additional generation results, prompts, and side-by-side comparisons, facilitating further observation of details, semantics, and dynamic performance.

Paper Title: DuoMatching: Joint-Marginal Distribution Matching for Few-Step Video Generation Paper: https://arxiv.org/abs/2610.03543 Code: https://github.com/JohnZhan2023/DuoMatching Model: https://huggingface.co/JohnZhan/DuoMatching

Like, Share, and Favorite!

We welcome your thoughts in the comments section!

— End —

[Academic Submissions] Please send emails to [email protected] on weekdays. Include [Submission] in the subject line and tell us: who you are, where you are from, the submission content, along with project/homepage links and contact information.

🎓 We will (try our best to) reply promptly :)

🌟 Star Us 🌟

Daily updates on cutting-edge technology progress