AI 译本 · Equalang
新上架PDF · 2026年11月翻译
加载中…
加载中…
arXiv · 2026
对抗性推测推理提升长时程智能体的能力
Hongyin Luo, Wei Fang, Lijie Yang, Owen Stepan et al.
Speculative decoding has been treated purely as a systems optimization that speeds up large language model decoding without changing the model behavior. Under this view, the growing effort to train better draft models has been focused solely on acceptance rate. We argue that this view does not hold true for long-horizon agent reasoning. Measuring frontier open-weight models on complex agentic tasks with different speculative decoding approaches, we found that numerical noise grows within verification windows, and the draft acceptance results of the thinking process can change the output distribution of the model after 30 steps, thus noticeably influencing the reasoning accuracy of long-horizon agents in random directions. In this work, we improve the reasoning capability of target models by controlling such noise. While we cannot change the overall token distribution verified by the target model, we can encode critical reasoning with highest precision by training the draft model to be rejected. We identify these points in agent traces and train the draft model to favor adversarial preference. At inference the trained draft and the target part ways precisely at these decision points, verification restarts, so that critical thinking tokens are sampled from the target model and encoded under its least noisy conditions. This approach makes critical reasoning better indexed and more durable over a long horizon. On complex agentic coding benchmarks, our approach raises end-to-end success rate with no change to the target model or its output distribution. Our results recast the speculative decoding mechanism from a pure accelerator into an active allocator for context precision that improves the accuracy of long-horizon agents by training 0.06% to 0.28% of hosted parameters.
推测解码一直被视为一种纯粹的系统优化手段，能够加快大型语言模型的解码速度，同时不改变模型行为。按照这一观点，人们投入越来越多精力训练更好的草稿模型，关注的也只有接受率。我们认为，这一观点并不适用于长时程智能体推理。我们使用不同的推测解码方法，在复杂智能体任务上评测前沿开放权重模型，发现验证窗口内的数值噪声会不断累积；经过 30 步后，思维过程中的草稿接受结果可能改变模型的输出分布，进而以随机的方向显著影响长时程智能体的推理准确率。在这项工作中，我们通过控制这类噪声来提升目标模型的推理能力。虽然我们无法改变目标模型验证后的整体词元分布，但可以训练草稿模型，使其被拒绝，从而以最高精度编码关键推理内容。我们在智能体轨迹中找出这些关键点，并训练草稿模型倾向于做出对抗性偏好。在推理时，训练好的草稿模型会恰好在这些决策点与目标模型分道扬镳，并重新开始验证，因此关键思考词元会由目标模型采样，并在噪声最低的条件下编码。这种方法能让关键推理内容在长时程过程中得到更清晰的标记，并更持久地保留下来。在复杂智能体编程基准测试中，我们的方法无需改变目标模型或其输出分布，就能提高端到端成功率。我们的结果将推测解码机制重新定位为一种上下文精度的主动分配器，而非单纯的加速器：只需训练托管参数的 0.06% 至 0.28%，就能提升长时程智能体的准确率。
PDF · 2026年11月翻译