Reviso: Not All Visual Evidence Deserves Equal Attention in Video Captioning

Reviso is an agentic video captioning framework that learns a Context-Focus-Points observation plan and uses verification-guided SFT plus GRPO rewards to gather prioritized visual evidence with tools.

Multimodal large language models can generate detailed video captions, but they may overemphasize minor details while overlooking essential context and key events. The importance of visual information depends on each video's content, and important information may not be resolved from a coarse overview. Learning an observation strategy requires deciding what to observe and how to observe it with different tools.

Reviso first samples timestamped frames at a low frame rate and builds a hierarchical Context-Focus-Points observation plan identifying context, prioritizing aspects, and defining targets and tools. Guided by the plan, it performs multiple rounds of tool-assisted active perception, revisiting temporal segments at higher frame rates and using image cropping and OCR. It interprets observations with global context and prior evidence, identifies resolved and unresolved targets, updates the plan, and integrates global and local information into the final caption. Training uses a verification-guided pipeline to construct trajectories through factual verification and teacher-guided refinement, then SFT followed by GRPO. The GRPO reward combines Key-Fact Reward, Plan Alignment Reward, and Unproductive Tool-Call Penalty.

• Reviso achieves the highest precision on DREAM-1K and improves F1 by 4.4 percentage points over its Qwen3.5-9B backbone, with precision 42.3, recall 37.0, and F1 39.5. §\subsectionfont Main Results
• Reviso reaches precision 65.7, recall 56.8, and F1 60.9 on MCTS-VCB, improving precision and recall by 8.1 and 19.1 points and F1 by 15.4 points. §\subsectionfont Main Results
• Reviso achieves action F1 34.3 and object F1 33.9 on CaReBench, outperforming its backbone by 9.1 and 6.7 points. §\subsectionfont Main Results
• Reviso achieves TP score 2.95 and TP accuracy 0.568 on VDC Detailed, described as best among evaluated open-source models on both metrics. §\subsectionfont Main Results
• Reviso achieves LVBench 52.0, MMVU MCQ 81.7, MMVU Open 57.9, and MMVU Overall 72.7, improving over its backbone by 4.9 points on LVBench and 2.1 points on MMVU Overall. §Caption-To-QA Evaluation
• Full Reviso with KFR, PAR, and UTCP reaches DREAM-1K F1 39.5, CaReBench action F1 34.3, and object F1 33.9; removing PAR causes the largest reward-ablation reduction, including a 4.2-point drop in object F1. §\subsectionfont Ablation Study

• The paper has no dedicated limitations section; all evaluations cap models at 128 input frames, so behavior with larger frame budgets is not established.
• Tool-assisted observation gains are relatively limited for shorter videos, which the paper attributes potentially to dense coverage already provided by the frame budget.
• Closed-source models are gray reference rows excluded from ranking, and GPT-6 Astra paired with Codex achieves a higher F1 than Reviso in the strategy comparison.
• Evaluation covers four video captioning benchmarks and two downstream QA benchmarks; generalization to other video domains or tasks is not shown.

• The paper argues that not all visual evidence deserves equal attention because importance depends on video content and context. §\sectionfont Introduction
• Reviso constructs a hierarchical Context-Focus-Points observation plan from a low-frame-rate global preview; Context identifies setting, Focus prioritizes aspects, and Points define targets and tools. §\subsectionfont Priority-Guided Active Observation
• Guided by the CFP plan, Reviso performs multiple rounds of active perception, revisiting segments at higher frame rates and using image cropping and OCR; it updates the plan as targets are resolved. §\subsectionfont Priority-Guided Active Observation
• Training trajectories are built through six steps: caption collection and human correction, CFP planning, tool-assisted observation, atomic-fact verification, teacher-guided recovery, and retention of verified trajectories. §\subsectionfont Data Construction
• SFT initializes Reviso from Qwen3.5-9B on verified teacher trajectories with token-level supervision; environment responses are masked from the loss. §Supervised Fine-Tuning
• GRPO uses KFR for caption fact F1, PAR for alignment of initial plan targets with reference observation targets, and UTCP to penalize costly observations when group caption quality reaches a threshold. §GRPO for Priority-Guided Observation
• Reviso is compared with 15 MLLMs and evaluated on four captioning benchmarks plus LVBench and MMVU; all models are limited to a maximum of 128 input frames. §\subsectionfont Experimental Setup
• Tool-assisted observation yields larger object-F1 gains for longer videos, reaching about 10 percentage points for clips lasting 24–27 seconds, with limited additional gains for shorter videos. §Benefits of Tool-Assisted Observation