What they found
The authors study "thinking with images," where a vision-language model crops and zooms into parts of a picture to see small details. Their analysis shows that higher accuracy does not mean faithful tool use. The models often call the zoom tool but do not rely on the zoomed region when they answer. The authors trace this to training data: when a question can be answered without zooming, the model can get it right while ignoring the evidence. They fix this by adapting the training data to the model's current ability, using questions where correct zoomed evidence can still help. They also add a data pipeline that finds small objects and builds checkable questions about their details, plus a reward that favors answering without tools when zooming is not needed. They report better faithfulness and state-of-the-art results on visual search benchmarks (+11.6% on V*, +7.9% on HR-Bench 4K, +9.4% on HR-Bench 8K). They also report that the model beats the compared baselines on multimodal hallucination benchmarks.
Why it matters
Tool use that looks like reasoning but does not affect the answer can mislead people who read the steps to trust the result. This work suggests the fix may lie in the training data, not in extra supervision of each reasoning step. The abstract says faithfulness and accuracy were improved together, so one did not have to be traded for the other.
How it changes the race
If the result holds, builders of vision-language SI could gain both accuracy and more trustworthy tool use by curating data against the model's changing skill, which is a cheaper route than supervising reasoning directly. The abstract gives no cost figures, so the size of that benefit is unclear.
Safety and control
The paper bears on faithfulness: it shows that visible tool calls can fail to reflect what the model actually relied on. Better faithfulness makes region-grounded reasoning easier for humans to check, but the abstract does not say how well this was verified beyond the reported benchmarks.
The abstract, by the authors
Thinking with images (TwI) empowers vision-language models (VLMs) to overcome perceptual bottlenecks by actively cropping and zooming into fine-grained visual details, promising both higher accuracy and interpretable, region-grounded reasoning. However, our analysis shows that the improved accuracy of TwI models does not guarantee faithful tool use; they frequently invoke tools without actually relying on the zoomed region, undermining the reliability of tool-assisted reasoning. We trace this unfaithful behaviour to the training data: when questions are answerable without zooming, the model can answer correctly while ignoring the returned evidence, and this problem is exacerbated during training as the model becomes better at answering without tools. To keep zooming useful throughout training, we adapt the training data to the model's evolving capability and train on questions where correct zoomed evidence can still help the model answer correctly. This selection is supported by a data synthesis pipeline that localises small objects and constructs verifiable questions about their fine details, and complemented by an adaptiveness reward that favours tool-free answers when zooming is unnecessary. The experimental results show that our method substantially improves reasoning faithfulness and achieves state-of-the-art performance on visual search benchmarks (+11.6% on V*, +7.9% on HR-Bench 4K, and +9.4% on HR-Bench 8K). On multimodal hallucination benchmarks, our model also outperforms the compared TwI baselines. These results suggest that improving TwI faithfulness does not sacrifice accuracy and that both can be learnt from the right training data without directly supervising the reasoning process.
Quoted from arXiv:2611.26199. The research and its words belong to its authors.
