Robbyant 开源 LingBot-Touch 触觉模型

原标题:Looks like a brilliant open source release for robotics.
71AI 研判分

Robbyant 以 Apache 2.0 协议开源 LingBot-Touch,将指尖触觉传感器读数转化为机器人控制软件可用的信号,补上摄像头无法感知夹持力度与滑移的短板。团队用同一批触觉数据训练了三个同规模编码器(各 5.46M 参数),在 4000 小时数据下,在潜空间预测掩码部分的 JEPA 版本在触觉到视频检索任务上取得 49.3% top-5 准确率,MAE 低于 9%,而 VAE 版本仅 3.1%。作者认为触觉数据稀疏且信噪比低,是 JEPA 胜出的关键原因。

报道正文

Rohan Paul (@rohanpaul_ai) · 2026-11-10 15

Looks like a brilliant open source release for robotics.

A great problem of robotics is that a camera can’t tell a robot how hard it’s squeezing or whether a grip is slipping, but a fingertip sensor can.

Robbyant released LingBot-Touch, an open-source model (Apache 2.0) that gives robots a sense of touch by turning fingertip sensor readings into signals their control software can use.

Robbyant trained three same-size encoders (5.46M parameters each) on the same touch data for LingBot-Touch. At 4,000 hours, the JEPA version, which predicts the masked parts in latent space, got 49.3% top-5 on tactile→video retrieval. MAE stayed under 9%. The VAE got 3.1%, worse than it did at 1,000 hours.

I think the reason is the data itself. A finger only presses a few spots, so most of every frame is close to zero, and the readings that aren't zero are noisy. Redrawing that means copying zeros and noise nobody can predict, and a JEPA only predicts the representation, so it can leave both out. The paper names the sparsity and the low signal-to-noise ratio as the reasons they went with JEPA.

The sparsity shaped training too. With 80% of taxels masked from step one, a lot of clips had no contact left to see and the gradients blew up, so they ramp the mask from 40% to 80% over the first 300 steps. Targets come from an EMA encoder at four depths, following V-JEPA 2.1.

To isolate the effect of multi-source data, they hold each sensor's own data fixed and add only the other modalities. With the target sensor's data unchanged, the multi-source model improves on seven of the eight sensor-task pairs, indicating that data from heterogeneous sensors directly benefits each individual sensor.

推文配图

推文配图