Touch-R1 Becomes First Touch Reasoning Model Accepted at NeurIPS 2026

When vision and touch conflict, what will an intelligent agent do?

It looks like hard plastic, but feels very soft when pressed — when vision and touch conflict, humans tend to rely on touch to correct visual misjudgments; however, existing multimodal models often trust vision more, thereby misjudging the true physical properties of objects.

Figure 1 Research overview: Touch-R1 outperforms other representative multimodal large language models on the tactile reasoning benchmark TouchReason-Bench (left); Touch-R1 uses tactile evidence to overturn visual misjudgment and identifies the material as rubber (right)

The Daimon Robotics team, together with Xiamen University, Nanjing University, Great Bay University, and Fudan University, proposes Touch-R1, which applies reinforcement learning to cross-sensor tactile reasoning. The paper has been accepted by NeurIPS 2026. The team also constructed the TouchReason-1M dataset and the TouchReason-Bench benchmark to support training and evaluation.

• Paper title: Touch-R1: Reinforcing Touch Reasoning in MLLMs
• Paper address: https://arxiv.org/abs/2605.27154
• Code: https://github.com/laiyingxin2/Touch-R1
• Model: https://huggingface.co/laiyingxin/Touch-R1
• Dataset: https://huggingface.co/datasets/laiyingxin/TouchReason

As the first touch-anchored physical R1 model, this research is the first to systematically introduce reinforcement learning into tactile reasoning models, enabling the model to learn to perform cross-sensor physical reasoning based on the essence of touch, transforming tactile physical cognition from "what it looks like" to "what it actually is," and providing a new path for building more reliable tactile reasoning.

01｜The first touch-anchored physical reasoning model, establishing physical common sense for embodied intelligence

Humans usually judge the physical properties of objects by first looking, then confirming or revising the judgment through touch. This "perception—verification—correction" loop is an important way in which we interact with the physical world.

However, current large models still lean toward vision. Vision can recognize objects, but it is difficult to perceive hardness, roughness, friction, deformation, and force, and it is even more prone to failure when facing transparent, reflective, or occluded objects. Tests show that when visual and tactile information conflict, models such as GPT-4o and Gemini-2.5-Pro often trust "what it looks like" more than "what it feels like." Such misjudgments may lead to robot grasping failures, damage to objects, and even safety risks.

There are three major difficulties in enabling models to understand touch: first, tactile data is scarce, and tactile physical reasoning data is even scarcer; second, physical properties such as hardness and roughness are often ordinal rather than categorical; third, understanding signals from different sensors so that judgments remain consistent. Touch-R1 addresses these three difficulties by enabling the model to learn to reason based on tactile evidence and correct visual misjudgments. Touch thus ceases to be merely auxiliary information and becomes physical evidence capable of changing judgments.

02｜Creating the first tactile data system for embodied models, establishing complete infrastructure for tactile reasoning

The research team first constructed TouchReason-1M, a large-scale, cross-material, cross-sensor tactile reasoning dataset. This is the first tactile data system created for embodied models, integrating multi-sensor visuotactile data, ordinal attribute labels, and verifiable reasoning question-answering. The dataset covers more than 1,000 everyday objects and 9 major material categories, and brings together four visuotactile sensors: DM-Tac X, GelSight Mini, Xense, and Tac3D. Unlike previous tactile datasets, the data for all objects was collected under strictly controlled standard actions and standard forces, enabling horizontal comparison across different objects.

TouchReason-1M focuses on visuotactile perception, a technology that can provide sufficiently high-resolution tactile data for large-model tactile reasoning. However, different sensors have different elastomers, markers, optical paths, and structures, so contact with the same object naturally produces different image representations. TouchReason-1M does not simply concatenate existing data; instead, it unifies the contact method: pressing from initial contact to 20N, then completing six exploration actions with standard force, while synchronously recording raw tactile images, deformation, tangential changes, depth changes, and visual images of object contact. TouchReason-1M contains a total of 14,700 valid tactile sequences and 1.323 million synchronized data pairs, establishing a common physical reference for the model's cross-sensor consistency learning.

Figure 3 Tactile reasoning dataset TouchReason-1M

At the model level, Touch-R1 adopts a three-stage training process: first, it learns the dynamic patterns of contact by predicting future tactile frames; then it performs supervised fine-tuning through structured question answering to form a "perception-comparison-conclusion" reasoning chain; finally, it uses GRPO reinforcement learning designed for touch.

Touch-R1 further advances cross-sensor tactile perception from "physical representation alignment" to "physical property reasoning consistency": the model parses physical properties from tactile images of different sensors and makes judgments and comparisons, and constrains the judgment results through a cross-sensor consistency reward, thereby resolving differences among heterogeneous sensors and obtaining the true and stable essence of physical properties. At the same time, the model also introduces ordinal rewards to help it understand the degree relationships of physical properties such as hardness and roughness; structured format rewards and tactile input perturbations ensure that the model reasons based on tactile evidence, reducing shortcuts such as guessing answers from vision alone or "answering correctly without truly understanding through touch."

Figure 4: The three-stage training process of Touch-R1

03 | Using touch as reasoning evidence, Touch-R1 leads by a large margin on nine performance metrics

To validate the model's capabilities, the team simultaneously built the tactile reasoning benchmark TouchReason-Bench, using 4,800 question-answer pairs to examine the model's physical property recognition, ordinal reasoning, comparative reasoning, cross-sensor consistency, and visual-tactile conflict handling, and using 200 objects unseen during training to examine the model's generalization performance.

Figure 5: Overview of the construction process of TouchReason question-answer pairs

On this benchmark, Touch-R1-7B achieved the best results on 9 core metrics, including physical property recognition, ordinal reasoning, and cross-sensor consistency, with an overall score of 60.1, 18.4 points higher than the touch-specific model Octopi-13B, about 24.7 points higher than GPT-4o, and 12.6 points higher than the previous strongest tactile model SToLa.

Figure 6: Touch-R1-7B leads comprehensively on multiple core metrics

Touch-R1 also demonstrates how touch can correct visual misjudgments: for an object that looks like plastic, the model can combine tactile deformation and force-depth curves to identify the object's compliance, thereby overturning the visual prior and correcting the material judgment to rubber. This reflects Touch-R1's most important capability: the model does not merely receive vision and touch simultaneously; when evidence conflicts, it judges which source of information is more trustworthy.

Figure 7: Example question answering with Touch-R1-7B

Ablation experiments further confirm the contribution of each part of the Touch-R1 architecture. Cold-start supervised fine-tuning, standard GRPO, ordinal-aware rewards, cross-sensor physical consistency constraints, and structured format control all brought varying degrees of improvement to model performance. At the same time, as model scale increased, multiple metrics improved steadily, demonstrating Touch-R1's generalization potential.

The value of Touch-R1 goes far beyond endowing models with tactile reasoning ability. Through the complete infrastructure of datasets, models, and evaluation benchmarks, Touch-R1 promotes touch from auxiliary perception to a core modality of embodied foundation models. At the same time, based on cross-sensor consistent physical cognition, Touch-R1 lays the foundation for cross-hardware migration of tactile data and model capabilities, and provides underlying capabilities for real physical interaction scenarios such as material recognition, quality inspection, precision assembly, fragile object grasping, and deformable object manipulation.

About the authors

The paper authors are from Xiamen University, Nanjing University, Great Bay University, Fudan University, and Daimon Robotics. The first author is Lai Yingxin, currently a PhD student at Xiamen University and an intern at Daimon Robotics; the co-first author is Zhou Yafei, currently a master's student at Great Bay University and an intern at Daimon Robotics; the corresponding author is Yuan Weihao, currently an associate professor at Nanjing University and Chief AI Scientist at Daimon Robotics.

© THE END

For reprinting, please contact this official account for authorization.

Submission or reporting: [email protected]

Open in WeChat