Design and Benchmarking of A Multi-Modality Sensor for
Robotic Manipulation with GAN-Based Cross-Modality Interpretation
​​
Accepted by IEEE Transactions on Robotics
In this paper, we present the design and benchmark of an innovative sensor, ViTacTip, which fulfills the demand for advanced multi-modal sensing in a compact design. A notable feature of ViTacTip is its transparent skin, which incorporates a `see-through-skin' mechanism. This mechanism aims at capturing detailed object features upon contact, significantly improving both vision-based and proximity perception capabilities. In parallel, the biomimetic tips embedded in the sensor's skin are designed to amplify contact details, thus substantially augmenting tactile and derived force perception abilities. To demonstrate the multi-modal capabilities of ViTacTip, we developed a multi-task learning model that enables simultaneous recognition of hardness, material, and textures.
To assess the functionality and validate the versatility of ViTacTip, we conducted extensive benchmarking experiments, including object recognition, contact point detection, pose regression, and grating identification. To facilitate seamless switching between various sensing modalities, we employed a Generative Adversarial Network (GAN)-based approach. This method enhances the applicability of the ViTacTip sensor across diverse environments by enabling cross-modality interpretation.
Poster Presentation at 2024 ICRA
MagicTac: A Novel High-Resolution 3D Multi-layer Grid-Based Tactile Sensor
Wen Fan, Haoran Li, Dandan Zhang
​
Paper Link:
https://ieeexplore.ieee.org/document/10610615
Accepted by 2024 ICRA
Accurate robotic control over interactions with the environment is fundamentally grounded in understanding tactile contacts. In this paper, we introduce MagicTac, a novel high-resolution grid-based tactile sensor. This sensor employs a 3D multi-layer grid-based design, inspired by the Magic Cube structure. This structure can help increase the spatial resolution of MagicTac to perceive external interaction contacts. Moreover, the sensor is produced using the multi-material additive manufacturing technique, which simplifies the manufacturing process while ensuring repeatability of production. Compared to traditional vision-based tactile sensors, it offers the advantages of i) high spatial resolution, ii) significant affordability, and iii) fabrication-friendly construction that requires minimal assembly skills. We evaluated the proposed MagicTac in the tactile reconstruction task using the deformation field and optical flow. Results indicated that MagicTac could capture fine textures and is sensitive to dynamic contact information. Through the grid-based multi-material additive manufacturing technique, the affordability and productivity of MagicTac can be enhanced with a minimum manufacturing cost of £4.76 and a minimum manufacturing time of 24.6 minutes.
​
Tac-VGNN: A Voronoi Graph Neural Network for Pose-Based Tactile Servoing
Wen Fan, Max Yang, Yifan Xing, Nathan F. Lepora, Dandan Zhang*
​
Paper Link: https://arxiv-export3.library.cornell.edu/abs/2303.02708
Accepted by 2023 ICRA
Tactile pose estimation and tactile servoing are fundamental capabilities of robot touch. Reliable and precise pose estimation can be provided by applying deep learning models to high-resolution optical tactile sensors. Given the recent successes of Graph Neural Network (GNN) and the effectiveness of Voronoi features, we developed a Tactile Voronoi Graph Neural Network (Tac-VGNN) to achieve reliable pose-based tactile servoing relying on a biomimetic optical tactile sensor (TacTip). The GNN is well suited to modeling the distribution relationship between shear motions of the tactile markers, while the Voronoi diagram supplements this with area-based tactile features related to contact depth. The experiment results showed that the Tac-VGNN model can help enhance data interpretability during graph generation and model training efficiency significantly than CNN-based methods. It also improved pose estimation accuracy along vertical depth by 28.57% over vanilla GNN without Voronoi features and achieved better performance on the real surface following tasks with smoother robot control trajectories.
For more project details, please view our website: https://sites.google.com/view/tac-vgnn/home
Tactile Robotic Telemedicine
​Tactile robotic telemedicine enables medical professionals to remotely examine patients with precision and accuracy.
​
Robot-Assisted Telediagnostics with Tactile Perception
Pulse Diagnosis:
Pulse diagnosis involves the physician palpating the patient’s radial artery at various locations and depths to assess the overall condition of the body. This method is a fundamental component of the diagnostic process in Traditional Chinese Medicine. Through this technique, the physician evaluates several pulse characteristics, including speed, rhythm, strength, and shape, to aid in forming a diagnosis.
​
Palpation Diagnosis:
Palpation diagnosis entails the examination of the body through touch and applied pressure. Physicians utilise their hands to identify areas of warmth, coolness, resistance, or discomfort. This method is commonly employed in the abdominal region to detect organ enlargement, masses, or fluid accumulation.
​
​