Look Only When Needed: Certificate-Guided Visual Routing for Efficient Diffusion VLMs

LOWN uses a cheap TEXT-ONLY forward to certify the latest image-conditioned predictions and runs the image-conditioned FULL forward only when the certified prefix is too short, retaining 99.6% of LLaDA-V's score at 35.3% of decoding FLOPs.

Diffusion vision-language models process the entire input at every denoising step, and visual tokens make up 89–97% of LLaDA-V's input on the studied benchmarks, so most decoding compute repeatedly processes an unchanged image. Visual token pruning removes some visual tokens but loses their information and still processes the remaining image at every step. Existing caching and fewer-step methods do not ask whether the current step needs the image.

LOWN stores the top-1 predictions and confidences from the latest image-conditioned FULL forward as an anchor. At each step it runs a cheap TEXT-ONLY forward, orders masked positions by decreasing anchor confidence, and certifies a candidate when the TEXT-ONLY top-1 agrees with the anchor. The longest confidence-ordered certified prefix is U_text; when its length is at least k, the step skips the FULL forward and unmasks the whole prefix to anchor predictions. When the prefix is shorter than k, LOWN runs a FULL forward, replaces the anchor, and unmasks k positions in disagreement-first order. The method uses no training or task-specific threshold and changes neither model parameters nor attention pattern.

• Without training or a tuned threshold, LOWN retains most of LLaDA-V's score on ten benchmarks at a fraction of the decoding FLOPs. §Abstract
• Paired FULL and TEXT-ONLY forwards agree on most unmasked decisions and on most final tokens before unmasking, while the image changes an unmasked decision on an average minority of steps. §3.2
• The share of image-dependent steps varies widely across benchmarks, from MMBench to DocVQA. §3.2
• On LLaDA-V, standalone LOWN keeps 99.6% average normalized score at 0.353 times the baseline FLOPs, and LOWN with RedVTP at 50% retention keeps 98.3% at 0.223 times the baseline FLOPs. §5.2
• Against fixed and random image-access schedules, LOWN scores higher on MathVista at comparable FLOPs: LOWN gets 56.00 at 0.325, Period 4 gets 52.00 at 0.321, and Random gets 53.40 at 0.349. §5.3
• The same rule transfers to LaViDa, where LOWN retains 99.7% of the baseline average at 0.174 times the FLOPs. §E.3

• On a DocVQA table question, LOWN runs a FULL forward on 14 of 15 steps and costs 0.90 times the baseline FLOPs, versus 0.45 times over the whole benchmark; the paper presents this as its least-saving case.
• Video evaluation uses a controlled protocol with 128-token responses, 128 baseline diffusion steps, block length 32, and 32 frames, which differs from VidLaDA's released 64-frame, two-token evaluation.
• FLOPs accounting counts matrix multiplications only and excludes the vision encoder, which would add 0.16–1.89% of the baseline decoding cost on the ten benchmarks, and it also excludes non-matmul operations.
• TEXT-ONLY acceptance is not perfectly reliable: a fresh FULL forward overturns 2.2–4.7% of the tokens LOWN unmasks on TEXT-ONLY steps.

• Visual tokens are 89–97% of LLaDA-V's input across the benchmarks, while a forward without them costs 3–10% as much. §1
• The measurement runs TEXT-ONLY alongside FULL on the same partially masked response, keeps position IDs of non-visual tokens, and lets only the FULL forward determine the trajectory. §A.1
• On average 83.4% of unmasked decisions agree between FULL and TEXT-ONLY, 85.0% of final tokens were already top-1 before unmasking, and image-dependent steps average 22.8%. §3.2
• Image-dependent steps cluster early on eight of the ten benchmarks but are not universal and vary by task, motivating adaptive rather than fixed image access. §3.2
• LOWN anchors to the latest FULL top-1 predictions and confidences, certifies candidates by TEXT-ONLY agreement, and uses the longest certified prefix to set acceptance length. §4.1
• If the certified prefix has at least k tokens, LOWN skips the FULL forward and unmasks the whole prefix; otherwise it runs FULL and unmasks k positions in disagreement-first order. §4.2
• The cost model is R = f + sigma rho: the certificate lowers FULL-forward frequency and whole-prefix acceptance can lower the number of steps, while visual pruning changes only the FULL-forward term. §4.3
• LLaDA-V results show 99.6% average normalized score at 0.353 times FLOPs, and LOWN with RedVTP at 50% retention gives 98.3% at 0.223 times FLOPs. §5.2