中文 Contact us Try online

Research · Frontier AI Deep Dive

The Real Bottleneck for Long-Horizon Agents: Persistence, Error Recovery and Oversight (Deep Dive)

October 27, 2026Full text in Chinese, about 40 minDeep dive

long-horizon tasksdense progress evaluationerror recoveryagent verificationbehavior oversightmemory writing

This article is published in Chinese. Below are an English translation of its standfirst, an outline of its sections and its references; the full text, all figures and the algorithm walk-throughs are on the Chinese page.

Read the full article in Chinese

Summary

Over the past year, the tasks in agent demos have gone from “book a flight” to “reproduce a research codebase” and “turn a due-diligence pack into a report”. Task durations have grown from minutes to hours, and tool calls from dozens to hundreds. OSWorld 2.0 puts it plainly: its 108 real workflows need 318 tool calls on average, against about 30 for the previous generation of benchmarks; with a 500-step budget, even the best-performing configurations complete only 20.6% of tasks [1].

As tasks get longer, failure looks different. On short tasks, an agent usually fails because it “can’t do it”; on long tasks, other kinds of failure are more common:

  • Stopping early: declaring “done” when the work is not finished.
  • Blind to its own mistakes: an error planted dozens of steps earlier propagates all the way to the present, unnoticed.
  • Wrong together: several independent runs reach the same answer, and the answer itself is wrong.
  • Hard to follow: after hundreds of steps, people cannot tell what it changed along the way, or why.
  • Unreliable memory: state is not carried across sessions, or is remembered but not retrieved when needed.

In October 2026, a batch of papers turned these failure modes into things that can be measured and trained. This article selects 10 of them (plus 2 background papers) and organizes them around five pillars — persistence, error recovery, verification, oversight, and memory and collaboration. For each work it sets out the problem, mechanism, experiments and limitations, with 5 algorithm walk-throughs along the way, and closes with our assessment, an outlook, and practical implications for financial risk control.

Cover: the real bottleneck for long-horizon agents is not whether they can do the task, but persistence, error recovery and oversight
Cover: the real bottleneck for long-horizon agents is not whether they can do the task, but persistence, error recovery and oversight. Figure text is in Chinese.

What the article covers

  1. Background: why “able to finish” is not the same as “reliable”
  2. Persistence: measuring “sustained progress” with a three-hour yardstick
  3. Error recovery: learning to “turn back” from one’s own failures
  4. Verification: who judges right and wrong when there is no reference answer
  5. Oversight: helping people see what the agent actually did
  6. Memory and collaboration: remembering accurately, updating correctly, working well together
  7. Side-by-side comparison and how the field is evolving
  8. Our view
  9. Outlook
  10. Practical implications
  11. Closing remarks

References

  1. [1] Yuan, M., Zhou, Z., Xiong, X., et al. OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks. arXiv:2606.29537, 2026-06-28 (background). https://arxiv.org/abs/2606.29537
  2. [2] Li, Z., Li, Z., Shi, Y., et al. (incl. Mi, H.). Long-Horizon-Terminal-Bench 2.0: Automating Diverse Task Construction and Measuring Sustained Agent Progress. arXiv:2610.14146, 2026-10-09. https://arxiv.org/abs/2610.14146
  3. [3] Hu, X., Li, H., Wang, Y., et al. CUA-Recovery: Benchmarking Error Recovery in Long-Horizon Computer-Use Agents. arXiv:2610.27645, 2026-10-20. https://arxiv.org/abs/2610.27645
  4. [4] Liu, Y., Wang, P., Wang, B., et al. RealCLI: Toward a Unified Evaluation of Human-Agent Collaboration for CLI Agents. arXiv:2610.22507, 2026-10-15 (based on the abstract). https://arxiv.org/abs/2610.22507
  5. [5] Sun, Z., Yan, J., Zhao, H., et al. What Did the Agent Actually Do? Evidence-Grounded Oversight for Long-Horizon Agents. arXiv:2610.06406, 2026-10-05. https://arxiv.org/abs/2610.06406
  6. [6] Zhang, C., Han, R., Wang, Z., CuiZhu, Z., Collier, N., Pfister, T., Lee, C.-Y. VeriHarness: Scaling Agentic Verification for Long-Horizon Tasks. arXiv:2610.00972, 2026-10-01. https://arxiv.org/abs/2610.00972
  7. [7] Yin, J., Chen, Z., Wang, H., Sun, Y., Jin, Z., Song, D., Zhang, D. VerifierBench: Benchmarking the Generation of Executable Verifiers for LLM Agents. arXiv:2610.28679, 2026-10-19 (based on the abstract). https://arxiv.org/abs/2610.28679
  8. [8] Fei, T., Kang, J., Zheng, M., He, X., Yu, X. Hy-Memory-30B-A3B: Attributed Reinforcement Learning for Long-Term Memory Writing. arXiv:2610.22196, 2026-10-15. https://arxiv.org/abs/2610.22196
  9. [9] Gao, S., Li, Y., Chen, H., et al. MemDerive: A Diagnostic Benchmark for Agent Memory over Evolving Data. arXiv:2610.25373, 2026-10-18 (based on the abstract). https://arxiv.org/abs/2610.25373
  10. [10] Tao, Y., Zhong, X., Zhao, H. H., et al. DyadMem: A Long-Term Memory Benchmark of How Agents Work with Users. arXiv:2610.03020, 2026-10-02 (based on the abstract). https://arxiv.org/abs/2610.03020
  11. [11] Shao, Z., Wang, P., Zhu, Q., et al. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models (origin of GRPO). arXiv:2402.03300, 2024-02-05 (background). https://arxiv.org/abs/2402.03300
  12. [12] Lightman, H., Kosaraju, V., Burda, Y., et al. Let's Verify Step by Step (process supervision). arXiv:2305.20050, 2023-05-31 (background). https://arxiv.org/abs/2305.20050

HX Research · Frontier AI Deep Dive · Great Unicorn Intelligence

Let’s talk

Want to apply these methods to your business?

Talk to our modeling and agent team about your use case — from reasoning budgets and evidence chains to auditable decision flows, we can assess the path to production together.

Talk to us Back to Research