Kavli Affiliate: Xiang Zhang
| First 5 Authors: Shuqi Zhao, Xinghao Zhu, Yuxin Chen, Chenran Li, Xiang Zhang
| Summary:
Dexterous manipulation is a critical aspect of human capability, enabling
interaction with a wide variety of objects. Recent advancements in learning
from human demonstrations and teleoperation have enabled progress for robots in
such ability. However, these approaches either require complex data collection
such as costly human effort for eye-robot contact, or suffer from poor
generalization when faced with novel scenarios. To solve both challenges, we
propose a framework, DexH2R, that combines human hand motion retargeting with a
task-oriented residual action policy, improving task performance by bridging
the embodiment gap between human and robotic dexterous hands. Specifically,
DexH2R learns the residual policy directly from retargeted primitive actions
and task-oriented rewards, eliminating the need for labor-intensive
teleoperation systems. Moreover, we incorporate test-time guidance for novel
scenarios by taking in desired trajectories of human hands and objects,
allowing the dexterous hand to acquire new skills with high generalizability.
Extensive experiments in both simulation and real-world environments
demonstrate the effectiveness of our work, outperforming prior
state-of-the-arts by 40% across various settings.
| Search Query: ArXiv Query: search_query=au:”Xiang Zhang”&id_list=&start=0&max_results=3