AI 译本 · Equalang
新上架PDF · 2026年11月翻译
加载中…
加载中…
arXiv · 2026
Harness-S：从同级执行框架中学习，实现可泛化的智能体训练
Kangning Zhang, Wenxiang Jiao, Bowen Fang, Shuai Shao et al.
Harness evolution from a common base exhibits Locality, Complementarity, and Nonstationarity: variants repair different failures, and their utility changes as the policy learns. Rather than seeking a universally superior harness, we introduce Harness-S to selectively rescue failures of a base execution system through sibling harnesses and transfer the resulting capabilities into a single policy that remains effective under the original harness. Directional Harness Evolution develops specialized sibling harnesses along distinct repair directions, forming a reusable Sibling Harness Pool that remains fixed during policy training. Complementary Harness Rescue Training invokes this pool only when all initial attempts under the base harness fail. The same policy generates fresh rollouts under selected auxiliary harnesses, and these trajectories replace the failed group for reinforcement learning. An Adaptive Harness Router selects the Top-k variants using Harness Behavior Profiles and voting across multiple LLMs. Policy-aware profile re-anchoring refreshes this evidence as the policy learns. Across AppWorld, SWE-Gym, and DBBench, the trained policy substantially outperforms policy-only training when returned to the original base harness. Behavioral analyses and held-out cases support the transfer of harness-elicited behaviors into the policy itself. Reusing the router for Rank-1 inference provides additional gains at the same executor rollout count.
从共同基础框架演进出的不同执行框架，呈现出局部性、互补性和非平稳性：不同变体会修复不同的问题，而它们的效用也会随着策略学习而变化。我们不追求一个普遍更优的执行框架，而是提出 Harness-S：通过同级执行框架有选择地弥补基础执行系统中的失败，并将由此获得的能力迁移到单一策略中，使其在原始执行框架下仍然有效。方向性执行框架演进会沿不同的修复方向构建专门的同级执行框架，形成可复用的同级执行框架池，并在策略训练期间保持不变。互补式执行框架救援训练仅在基础执行框架下的所有初始尝试均失败时，才调用该框架池。同一策略会在选定的辅助执行框架下生成新的轨迹，并以这些轨迹替换失败的轨迹组，用于强化学习。自适应执行框架路由器会利用执行框架行为档案，并通过多个大语言模型投票，选出 Top-k 个变体。随着策略学习，策略感知的档案重新锚定机制会更新这些证据。在 AppWorld、SWE-Gym 和 DBBench 上，训练后的策略切换回原始基础执行框架时，表现显著优于仅训练策略的方法。行为分析和留出案例支持这样一种结论：由执行框架引出的行为能够迁移到策略本身。在 Rank-1 推理中复用该路由器，还能在执行器轨迹数量不变的情况下进一步提升性能。
PDF · 2026年11月翻译