Heterogeneous Tactile Transformer (HTT)
A self-supervised tactile backbone that learns shared representations across heterogeneous tactile sensors, boosting perception and contact-rich manipulation—even with sensors unseen during pretraining.
Touch Perception for Robots
Collaborative Learning and Adaptive Robots (CLeAR) Lab.
The ability of humans to navigate our environment is heavily dependent on the quality and speed of sensing. Tactile sensing, or the sensation of touch and pressure, is especially important for both coarse and fine object manipulation tasks, such as handling parcels and writing. Humans utilize tactile sensing data in real time for muscle path planning, fast tactile perception, tacto-object recognition, and overall human mobility. Similarly, robots tasked with service activities that require interacting with humans or objects require fast and reliable tactile feedback. At CLeAR, we are particularly interested in how perception with next-generation skin can be used to enhance physical human-robot interaction for trustworthy collaboration.
Octopi: Object Property Reasoning with Large Tactile-Language Models
We introduce PhysiCLeaR, an annotated dataset of everyday objects and tactile readings collected from a Gelsight Mini sensor, as well as Octopi, a system that leverages both tactile representation learning and large vision-language models to perform physical reasoning and inference, given tactile videos of multiple objects.
Tasbolat and team win IROS 2021 Best Paper Award.
We have won the Best Paper Award at the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2021 in October. IROS is a premier flagship academic conference in robotics and on...
Extended Tactile Perception
We show that robots can extend their perception through grasped tools/objects via dynamic tactile sensing.
Event-Driven Visual-Tactile Sensing and Learning
Accurate, Fast, and Low-powered Multi-Sensory Perception via Neuromorphic Sensing and Learning
TactileSGNet: A Spiking Graph Neural Network for Event-based Tactile Object Recognition
We propose a Spiking Graph Neural Network to take into account taxel geometry.
Fast texture classification using tactile neural coding and spiking neural network
We show that robots achieve fast classification of textures through Neural Encoding and Spiking Neural Network.
Towards Effective Tactile Identification of Textures using a Hybrid Touch Approach
We show that iCub robot classifies the surface textures with both sliding and touch movements under loose constraints with high accuracy.