Rethinking
Learn on the model’s
own latent rollout.
Regenerate one autoregressive latent span with the current model. Keep gradients through the recurrence, before any answer tokens are supplied.
Rethinking Latent Visual Reasoning: Grounding Latent Reasoning in Visual Evidence
“Reasoning like humans. Grounded in what you see.”
1 University of Alabama at Birmingham2 Amazon AGI
* Work done during an internship at Amazon AGI.
Abstract
Latent visual reasoning (LVR) enables multimodal large language models (MLLMs) to perform intermediate computation in continuous latent tokens rather than expressing every reasoning step in words. However, unlike textual CoT, latent reasoning is not directly observable, making it difficult to supervise what latent tokens learn. In this work, we first conduct a thorough analysis of latent-token behavior and identify a latent evidence-credit gap: latent tokens respond only weakly to image perturbations that alter the correct answer. We hypothesize that this issue stems from the lack of explicit supervision during GRPO training. These findings suggest that a final-answer reward provides too little guidance on what visual evidence to preserve or how credit should be assigned across latent tokens. To bridge this gap, we propose ReaLVR, which brings visual-evidence supervision to the model's own free-running latent trajectories. ReaLVR contrasts correct and model-generated wrong answers to determine where stronger supervision is needed, and relevant and mismatched visual evidence to specify what to preserve. Across three model families, ReaLVR consistently outperforms evaluated LVR baselines, achieving the highest five-task average of 63.7% on Qwen2.5-VL-7B. Crucially, we are the first to scale visual reasoning in latent space, showing that our framework continues to deliver robust improvements at frontier model scales up to 235B. Further analyses show more question-sensitive latent-token positions, stronger alignment with relevant visual regions, and greater fixed-context dependence on the most attended latent tokens.
Author preprint PDF on arXiv · Full-text HTML on arXiv · arXiv:2609.34563v1, 2026/09/28
THE IDEA, IN MOTION
From what we see to how we reason.
Watch the inputs become tokens, follow the latent rollout, and see how visual evidence guides learning.
Extra supervision is training only · Original inference unchanged · Token highlights are illustrative.
01 OVERVIEW
Reasoning in latent space.
Grounded in the image.
Continuous latent tokens offer a compact way for multimodal models to reason. But a correct final answer provides little guidance on which visual details those tokens should preserve.
We identify a latent evidence-credit gap: latent tokens respond only weakly to image perturbations that change the correct answer. ReaLVR brings explicit visual supervision to the model’s own free-running latent trajectory, with no change to its architecture or inference procedure.
63.7%Five-task mean · Qwen2.5-VL-7B
+3.3 ppCompared with LVR-RL · same 7B backbone
6 backbonesThree model families · up to 235B total parameters
02 METHOD
One trajectory.
Two complementary contrasts.
A training objective that makes visual evidence matter inside latent reasoning.
Learn on the model’s
Preserve relevant
visual evidence.
A cosine margin aligns the trajectory with a relevant visual prototype and separates it from the hardest mismatched prototype. Use a region annotation when available; otherwise, pool the image.
Assign credit through
answer contrast.
Read the same latent span with correct and model-generated wrong answers. Their positive attention difference, plus a uniform baseline, allocates detached supervision weights.
03 RESULTS
Better visual reasoning.
Across models and scales.
ReaLVR improves the average performance across the evaluated latent reasoning baselines.
FIVE-TASK AVERAGE · QWEN2.5-VL-7B
Accuracy (%)
LVR-RL60.4
Monet-RL62.2
ILVR-Stage262.9
ReaLVR63.7
0100%
Mean ± standard deviation for ReaLVR: 63.7 ± 0.2%, across three seeds. Bars show the unweighted five-task mean.
SCALING TO 235BVisual supervision
Visual supervision
at frontier scale.
On Qwen3-VL-235B-A22B, ReaLVR improves all three evaluated benchmarks over LVR-SFT.
- MMVP
- 80.9 → 81.9
- BLINK
- 74.2 → 75.4
- MME-RealWorld
- 70.4 → 71.0
235B total / 22B active parameters. This evaluation covers three tasks; no five-task mean is reported.
A closer look at the five benchmarks
Qwen2.5-VL-7B · accuracy (%) · three-seed mean
| Method | MMVP | BLINK | HRBench-4K | HRBench-8K | MME-RealWorld | Mean |
|---|---|---|---|---|---|---|
| LVR-RL | 64.2 | 53.6 | 69.6 | 64.4 | 50.1 | 60.4 |
| Monet-RL | 69.9 | 52.4 | 71.3 | 66.0 | 51.5 | 62.2 |
| ILVR-Stage2 | 69.4 | 56.8 | 71.0 | 66.9 | 50.3 | 62.9 |
| ReaLVR | 72.0 | 55.8 | 71.8 | 66.6 | 52.2 | 63.7 |
Selected latent reasoning baselines from Table 1. Bold indicates the best value in each column among the displayed methods.
04 LOOKING INSIDE
Do the latent tokens
carry visual evidence?
Accuracy is only part of the story. We also probe whether latent representations respond to answer-changing visual evidence and whether answers depend on the selected tokens.
In a fixed-context diagnostic, replacing the eight most attended latent tokens reduces correct-answer probability by 4 percentage points for LVR and 11 points for ReaLVR. This measures local dependence with the remaining states held fixed.
Scope & open questions
Fixed-context interventions measure local dependence, rather than a fully rerolled reasoning process or proof of faithful reasoning. Visual targets are less spatially specific without region annotations, and inference still uses a prescribed latent-token budget.
05 CITATION
Cite ReaLVR
@misc{xiao2026realvr,
title={Rethinking Latent Visual Reasoning: Grounding Latent Reasoning in Visual Evidence},
author={Xi Xiao and Tianchen Zhao and Youngeun Kim and Zhuowei Li and Linghan Xu and Jiaye Wu and Zheng Zhang and Xiang Xu and Xuanbai Chen and Farhan Tejani and Jakub Zablocki and Julia Xu and Yifan Xing},
year={2026},
eprint={2609.34563},
archivePrefix={arXiv},
primaryClass={cs.CV},
doi={10.48550/arXiv.2609.34563},
url={https://arxiv.org/abs/2609.34563},
note={Preprint}
}