中文 Contact us Try online

Research · Frontier AI Deep Dive

The Main Battleground for Multimodal AI Is Shifting to Evidence: From Native Multimodal Foundation Models to Visual Evidence in Long Documents (Deep Dive)

October 2026Full text in Chinese, about 35 minDeep dive

This article is published in Chinese. Below are an English translation of its standfirst, an outline of its sections and its references; the full text, all figures and the algorithm walk-throughs are on the Chinese page.

Read the full article in Chinese

Summary

In October 2026, Meta and NVIDIA each released native multimodal MoE foundation models built for agentic workflows [1][2][3]; whether a model “can understand images” is no longer news. Meanwhile, a batch of benchmark and method papers pushed the problem further: on 370,000 pages of real-world PDFs, even when the correct page is handed straight to the model, accuracy tops out at 81.7%; on table questions from financial statements, 9 models average only 28.9% [4]; and on continually revised enterprise records, the model with the best overall accuracy gets only 65.5% right on questions where the evidence is insufficient and the answer should be declined [7].

This article treats a multimodal system as an evidence chain — finding the evidence, seeing it clearly, reading it correctly and judging it accurately — reviews October’s representative work link by link, breaks down 4 core algorithms, and discusses what they mean for financial use cases such as credit document review and financial-statement analysis.

Cover: the main battleground for multimodal AI is shifting to evidence — find it, see it, read it, judge it
Cover: the main battleground for multimodal AI is shifting to evidence — find it, see it, read it, judge it. Figure text is in Chinese.

What the article covers

  1. Background: foundation models are converging; the hard part is evidence
  2. Where it breaks: a diagnosis from three benchmarks
  3. Finding it: multi-vector retrieval and “pay-as-you-go” indexes
  4. Seeing it clearly: focusing within a high-resolution page image
  5. Reading and judging correctly: from parsing to declining to answer
  6. Side by side: technical approaches along the evidence chain
  7. Our view
  8. Outlook
  9. Practical implications
  10. Closing remarks

References

  1. [1] Meta AI Research. Introducing Muse Spark Open. Meta official blog, 2026-10-15. https://research.meta.ai/blog/muse-spark-open
  2. [2] Pedro Cuenca (Hugging Face). Welcome Muse Spark Open: open-weight frontier multimodal mixture-of-experts model from Meta. Hugging Face Blog, 2026-10-15. https://huggingface.co/blog/meta-muse-spark-open
  3. [3] Chris Alexiuk, Chintan Patel (NVIDIA). NVIDIA Nemotron 3.5 Super VL Powers Specialized Agents with Multimodal Reasoning. NVIDIA Technical Blog, 2026-10-15. https://developer.nvidia.com/blog/nvidia-nemotron-3-5-super-vl-powers-specialized-agents-with-multimodal-reasoning/
  4. [4] Zhivar Sourati, Marianne Menglin Liu, Rongguang Wang, et al. VELD-QA: A Large-Scale Dataset for Visual Evidence Grounding and Question Answering in Long Documents. 2026-10-14. https://arxiv.org/abs/2610.20964
  5. [5] Simon Ott, Yufan Chen, Ruiping Liu, et al. VL-DocIR: A Benchmark for Vision-Based Long Document Information Retrieval. NeurIPS 2026, 2026-10-12. (abstract only) https://arxiv.org/abs/2610.17057
  6. [6] Ruochi Li, Jianzhe Lin, Haoxuan Zhang, et al. TopoGraphRAG-Bench: Evaluating Multimodal GraphRAG on Layout-Grounded Evidence Reasoning. NeurIPS 2026, 2026-10-07. (abstract only) https://arxiv.org/abs/2610.09360
  7. [7] Seohyun Lee, Jongha Kim, Dohwan Ko, et al. MMEnterprise-Bench: Evaluating Enterprise Agents over Evolving Multimodal Records. 2026-10-20. https://arxiv.org/abs/2610.27911
  8. [8] Zifei Wang, Wei Wen, Qiang Ji, Qian-Wen Zhang, Ruizhi Qiao, Xing Sun. EVIE: Evidence-Vector-Informed Embeddings for Visual Document Retrieval. Tencent IMA Product Center / Youtu Lab, 2026-10-08. https://arxiv.org/abs/2610.11553
  9. [9] Haowen Xiang, Yibo Yan, Xin Zou, et al. FlexVDR: Learning to Allocate Matryoshka Representation Budgets for Visual Document Retrieval. HKUST (Guangzhou) / Alibaba et al., 2026-10-19. https://arxiv.org/abs/2610.30177
  10. [10] Mihir Agarwal, Sabyasachi Basu, Viraj Agrawal. Simple Linear Corrections for Compression in Late Interaction. 2026-10-09. https://arxiv.org/abs/2610.14196
  11. [11] Zerui Chen, Changrui Chen, Fei Ni, Jiankang Deng. Look Before You Reason: Implicit Visual Thinking for Efficient Multimodal Reasoning. Imperial College London, 2026-10-19. https://arxiv.org/abs/2610.27341
  12. [12] Kai Liu, Anqi Li, Junxian Li, et al. TwinPrune: Density-Aware Two-Phase Token Pruning for Vision-Language Models. Shanghai Jiao Tong University / Huawei et al., 2026-10-13. https://arxiv.org/abs/2610.17530
  13. [13] Rahul Chowdhury, Timothy A. Rupprecht, Xuan Shen, et al. From Patching to Pruning Visual Computation in Vision–Language Models. 2026-10-02. (abstract only) https://arxiv.org/abs/2610.03389
  14. [14] Yuheng Chen, Yufan Chen, Zhuojun Cai, et al. Read, Parse, Describe: Unified Document Parsing with Visual Element Description Generation. NeurIPS 2026, 2026-10-14. (abstract only) https://arxiv.org/abs/2610.20335
  15. [15] Manuel Faysse, Hugues Sibille, Tony Wu, et al. ColPali: Efficient Document Retrieval with Vision Language Models. ICLR 2025, 2024-06-27. (background) https://arxiv.org/abs/2407.01449

HX Research · Frontier AI Deep Dive · Great Unicorn Intelligence

Let’s talk

Want to apply these methods to your business?

Talk to our modeling and agent team about your use case — from reasoning budgets and evidence chains to auditable decision flows, we can assess the path to production together.

Talk to us Back to Research