Horizon Robotics Releases LoFT Video Geometry Foundation Model

Original title: 地平线发布 LoFT 原生视频几何基础模型,多项指标达到 SoTA
78AI Score

Horizon Robotics' research lab released LoFT, a native video geometry foundation model built on a dual-loop Transformer that jointly predicts camera pose, intrinsics and per-frame depth from RGB video to recover 3D scene structure. Trained on roughly 2.97 million sequences and 962 million frames, it reports state-of-the-art results across SpatialBench, ClassicBench and VRBench, targeting robotic spatial perception and embodied learning.

Full Report

IT Home News, November 19: On November 18, Horizon Robotics' Robot Lab released LoFT (Looped Foundation Transformer for Video Geometry), a native video geometry foundation model. According to the announcement, the model can jointly predict camera pose, intrinsics, and per-frame depth from RGB video, recovering the 3D structure of a scene and providing a geometric foundation for robotic spatial perception, environment understanding, and embodied learning.

According to the official description, LoFT is natively designed for continuous video modeling and adopts a dual-loop Transformer architecture that combines cross-frame visual information aggregation with iterative optimization of camera state, mining temporal correlations and multi-view geometric information in video.

Horizon Robotics releases LoFT, a native video geometry foundation model, achieving SoTA on multiple metrics

According to information released by the company, IT Home has learned that the first loop aggregates cross-frame information through local and global temporal attention, gradually expanding the temporal receptive range and exploiting the continuity of adjacent frames and geometric correlations across views; the second loop explicitly updates camera pose and intrinsics and feeds the camera state back into the next round of feature computation.

The model combines dual-loop features to predict per-frame depth and its confidence, recovering the visible 3D structure of the scene.

Horizon Robotics releases LoFT, a native video geometry foundation model, achieving SoTA on multiple metrics

▲ Native dual-loop Transformer: LB1 extracts general geometric features, and LB2 performs Coarse2Fine geometry and pose estimation

According to the official description, to support large-scale training, the team built a unified 3D reconstruction data processing and training platform with approximately 2.97 million valid training sequences and 962 million frame image records, covering diverse scenarios such as simulation and real-world, dynamic and static, and providing a data foundation through unified data cleaning, quality filtering, and sampling strategies.

Image

▲ Precise spatial perception in high-frequency dynamic scenes

Image

▲ Achieving text-level fine-grained geometric reconstruction

Image

▲ Robust visual reconstruction of dynamic vehicles in driving scenarios

Image

▲ High-precision reconstruction of complex everyday scenes

In terms of evaluation, the research team compared LoFT with representative models such as VGGT-Ω and DA3-Giant across three evaluation suites: SpatialBench, ClassicBench, and VRBench. SpatialBench covers 12 datasets and 98 continuous video scenes; ClassicBench integrates 7 classic video reconstruction benchmarks; and VRBench covers robotics, autonomous driving, and general video scenarios. According to officially released information, LoFT achieves SoTA on multiple depth estimation metrics.

Image

▲ General 3D spatial reconstruction evaluation SpatialBench, achieving SoTA on multiple metrics

Image

▲ Achieving SoTA results on the classic reconstruction benchmark ClassicBench and the continuous video reconstruction benchmark VRBench

On the application side, according to the official description, LoFT can leverage temporal and multi-view geometric information in continuous video to recover the 3D structure of a scene and camera motion trajectories, covering scenarios such as robot first-person video, dynamic streets, indoor manipulation, and open-environment exploration.

Reference

Advertising statement: External redirect links contained in this article (including but not limited to hyperlinks, QR codes, passcodes, and other forms) are used to convey more information and save selection time. The results are for reference only. All articles on IT Home that contain external links include this statement.