AI 译本 · Equalang
新上架PDF · 2026年11月翻译 · 公式优化
加载中…
加载中…
arXiv · 2026
DONIC：从人类演示中学习通用灵巧双手跟踪控制
Mingze Sun, Ziang Cao, Yuan Fei, Qiankuan Tao et al.
Despite recent progress in dexterous reference tracking, existing models focus primarily on single-hand manipulation and a limited range of interactions, leaving the bimanual dexterous behaviors common in real life unaddressed. The scarcity of contact-rich bimanual manipulation data and the increased complexity of coordinating contacts make developing a generalist bimanual tracker particularly challenging. Therefore, we present 1) DONIC, the first generalist bimanual tracker jointly trained across operation categories and human-motion sources, together with 2) BiHOI, a multi-view dataset of 1,575 sequences spanning 229 objects and emphasizing sustained bimanual interaction, where reconstructed interactions are retargeted into robot references. DONIC combines proximal policy optimization with behavioral cloning in a feedback loop that recycles verified successful executions into training demonstrations. An object-centered per-hand reward maintains object-motion feedback while supervising each hand according to its reference contact role. Clip-Centered Advantages remove reference-dependent advantage offsets while preserving within-trajectory differences. Together, these designs enable one single policy to learn coordinated manipulation across heterogeneous references. Without finetuning, a single checkpoint achieves 86.79% success on unseen trajectories of training objects and 57.32% on held-out objects, surpassing the state-of-the-art bimanual tracking baselines by 77.36 and 45.73 percentage points.
尽管灵巧参考跟踪领域近来取得了进展，现有模型仍主要聚焦于单手操作和有限类型的交互，因此未能解决现实生活中常见的灵巧双手行为。接触丰富的双手操作数据十分匮乏，而协调接触又增加了复杂性，这使得开发通用双手跟踪器尤具挑战性。因此，我们提出：1）DONIC，这是首个在不同操作类别和人类动作来源的数据上联合训练的通用双手跟踪器；2）BiHOI，这是一个多视角数据集，包含1,575段序列，涉及229种物体，并着重呈现持续的双手交互；其中重建出的交互会被重定向为机器人参考动作。DONIC将近端策略优化与行为克隆结合在一个反馈循环中，把经过验证的成功执行结果重新用于训练演示。以物体为中心的逐手奖励机制在提供物体运动反馈的同时，根据每只手的参考接触角色对其进行监督。以片段为中心的优势估计消除了依赖参考动作的优势偏移，同时保留轨迹内部的差异。这些设计共同使单一策略能够在异构参考动作上学习协同操作。无需微调，单个模型检查点在训练物体的未见轨迹上取得86.79%的成功率，在留出物体上取得57.32%的成功率，分别比当前最先进的双手跟踪基线高出77.36和45.73个百分点。
PDF · 2026年11月翻译 · 公式优化