AdaptMMBench: Benchmarking Adaptive Multimodal Reasoning for Mode Selection and Reasoning Process

Abstract
Adaptive multimodal reasoning has emerged as a promising frontier in Vision-Language Models (VLMs), aiming to dynamically modulate between tool-augmented visual reasoning and text reasoning to enhance both effectiveness and efficiency. However, existing evaluations often rely on final-answer accuracy. These metrics do not explicitly assess reasoning mode selection, namely, whether a model selects between text-only and tool-augmented visual reasoning according to its own capability boundary, nor do they reveal the quality of the subsequent reasoning and tool-execution process In this paper, we propose AdaptMMBench, a comprehensive benchmark for adaptive multimodal reasoning across five domains: natural scene, OCR, GUI, knowledge, and math, encompassing both direct perception and complex reasoning tasks. AdaptMMBench utilizes a Matthews Correlation Coefficient (MCC) metric to evaluate the quality of reasoning mode selection, isolating this meta-cognition ability by dynamically identifying task difficulties based on models’ capability boundaries. Moreover, AdaptMMBench facilitates multi-dimensional reasoning process evaluation across key step coverage, tool effectiveness, and computational efficiency. Our evaluation shows that strong reasoning mode selection is helpful, but final adaptive gains and efficiency also depend on overall model capability. Process metrics further diagnose bottlenecks in reasoning coherence, tool use, and cost for improving adaptive performance and efficiency. Code and a subset of the data are available at https://github.com/xtong-zhang/AdaptMMBench.
Authors
XintongZhang∗, XiaowenZhang∗, JingrongWu∗, ZhiGao†, ShilinYan, ZhenxinDiao, KunpengGao, XuanyanChen, YuweiWu†, YundeJia, QingLi†
Publication Year
2026
http://eng.bigai.ai/wp-content/uploads/sites/7/2026/10/AdaptMMBench-Benchmarking-Adaptive-Multimodal-Reasoning.pdf
Publication Venue
Findings of EMNLP
Scroll to Top