Robot learning for Dexterous Manipulation Based on Visual-Tactile Fusion

For sensing techniques used for robotic dexterous manipulation, there are two primary modalities: vision and tactile. The robot can be guided to approach the target objects based on visual feedback, while tactile feedback can provide useful information of interaction between objects and the environment. These two modalities are complementary to each other for contact-rich tasks. To this end, this project aims to explore multimodal representation learning, which could be a useful tool for developing a robust and task-related representation to facilitate reinforcement learning with high efficiency.