NVIDIA and MIT Open-Source H3 Video Enhancement with Native 4K Regeneration

NVIDIA and MIT Team Release Model Weights for Video Enhancement Project Open-H3-Regenerate-2K on Hugging Face

Recently, the Efficient-Large-Model team, composed of researchers from NVIDIA and MIT, released the model weights for the video enhancement project Open-H3-Regenerate-2K on Hugging Face. The accompanying inference code is available in the NVlabs/Sana repository. Based on MiniMax H3, this solution regenerates low-resolution video drafts into high-resolution videos, supporting 2K output at 2560×1440 and native 4K regeneration up to 3840×2160.

The complete MiniMax H3 system includes multimodal context processing, base audio-video generation, and high-resolution regeneration modules. Among these, the H3-Base model, whose weights have been released, generates 768P audio-video content, while the official H3-Regenerate-2K module is provided via API. This newly released project leverages the generative capabilities of the H3 base model to provide a self-deployable, open-source implementation for the high-resolution regeneration stage.

During the training phase of the refinement model, the research team kept the parameters of the H3 Transformer backbone and the video VAE frozen, training only the newly added refinement LoRA. At inference time, the refinement LoRA is merged with community adapters such as LightX2V FL2VA Turbo and LMS Sharp. Through three-step denoising, visual details are regenerated at the target resolution, using scene, layout, and motion information from the video draft as the foundation.

To improve inference efficiency, the project employs a community-developed latent space amplifier that directly scales the video's latent representation, reducing the overhead of repeated decoding and encoding at high resolutions. Additionally, the solution integrates SoL-Engine’s sparse attention, low-precision computation, and fast VAE decoder to accelerate both the refinement and decoding stages.

Tests published by the project team show that under warm-up conditions on a single NVIDIA B200 GPU, the total time for latent space amplification, three-step refinement, and 2K decoding for an approximately 5-second video dropped from 87.26 seconds (in the unoptimized implementation) to 22.45 seconds—a speedup of roughly 3.9x. Notably, the fast decoder reduced video decoding time from about 15 seconds to approximately 1.6 seconds.

In terms of image quality, the team conducted comparative evaluations on RefinerBench-150. Under identical input and 2K output conditions, Open-H3-Regenerate-2K achieved an average score of 60.0 on the UniPercept perceptual quality benchmark, compared to 53.4 for the official H3-Regenerate-2K API. On VBench, the proposed solution scored higher in aesthetic quality, imaging quality, and subject consistency, while the official API retained a slight edge in motion smoothness and dynamic degree.

MiniMax H3 opened its weights in August this year, sparking rapid growth in fine-tuning, distillation, and tool adaptation efforts around H3. It attracted participation from research teams and community developers worldwide, surpassing 24 million downloads within one month and generating over 300 public derivative models. H3 has become one of the most downloaded models globally in 2026.

Among these developments, the joint NVIDIA-MIT work on H3 covers various directions. Beyond this video enhancement solution, the team also released StreamRender-H3, a streaming video rendering model built on H3 Ref2VA, and Sol-H3-VAE, a fast decoder distilled from the H3 video VAE, targeting interactive video rendering and decoding efficiency optimization respectively.

Previously, LightX2V introduced MiniMax-H3 Turbo, offering distilled versions with four and eight steps. The community fine-tuned model Singularity focused on optimizing action performance, facial details, and visual style. Overseas teams FastVideo and Pika released the FastH3 four-step preview version and the six-step sampling Pika H3 Flex, respectively. Meanwhile, ComfyUI and Unsloth brought H3 into more local creative workflows through adapted weights and quantized versions.

This article was provided by MiniMax and reprinted with permission from QbitAI. Views expressed belong to the original authors.