Latent Reasoning in Continuous Space for Unified Multimodal Models

LARC performs reasoning in continuous latent space for unified multimodal models, using curriculum SFT and self-evolving RL to improve image generation and enable test-time scaling.

Existing reasoning-augmented UMMs fail to inherit the test-time scaling and fine-grained controllability observed in LLMs. The paper attributes this failure to a modality mismatch: reasoning is performed in discrete token space, whereas image generation operates in continuous space. Discrete chains struggle to encode dense, spatially grounded information, so reasoning collapses into surface-level prompt manipulation. LARC aims to align text and image reasoning in a continuous latent space.

LARC represents a reasoning trajectory as alternating text spans and continuous latent steps produced by the UMM backbone; in latent mode, the last hidden state becomes the next input embedding. Stage 1 curriculum SFT first trains a text-only trajectory, then activates the first k latent steps in phases up to N, combining next-token prediction over text with flow-matching loss on the target image. Stage 2 uses GRPO with an information-gain reward from a frozen Stage-1 reference, estimated as paired conditional flow-matching losses under shared noise. A diversity reward penalizes cosine similarity among latent steps, and the total reward is information gain plus lambda_div times diversity.

• LARC improves GenEval text-to-image generation over BAGEL, with the largest stated GenEval gain on Attribute Binding. §3.1
• LARC improves DPG-Bench prompt-following fidelity over BAGEL. §3.1
• LARC improves OneIG-Bench overall score over BAGEL. §3.1
• LARC improves CompBench image editing over BAGEL across instruction adherence and pixel-level metrics. §3.1
• LARC improves MIRA visual reasoning over BAGEL and Uni-CoT. §3.2
• Latent test-time scaling adds only marginal clock-time overhead as latent steps increase. §D.1

• Paper states LARC is instantiated on UMMs whose image module is a flow-matching head; extending to discrete space generation or other generative heads is left to future work.
• Paper states interleaved latent reasoning introduces additional inference computation and latency overhead versus the base UMM; efficient decoding, distillation, or adaptive latent allocation are left open.
• The paper's main experiments train LARC on BAGEL only; generalization to other UMM backbones is not evaluated.

• The paper identifies a discrete-continuous modality mismatch as the root cause of limited gains and absent test-time scaling in reasoning-augmented UMMs. §1
• Reasoning trajectories alternate between language-mode text spans and latent-mode continuous hidden states. §2.2
• In latent mode, the next-position embedding is the last hidden state rather than a token embedding. §2.2
• Stage 1 curriculum SFT activates latent steps progressively from 0 to N over N+1 phases. §A.1
• The information-gain reward measures how much a latent trace raises the reference model's likelihood of the target image. §2.4
• The information-gain reward is approximated with paired flow-matching losses under shared noise. §C.2
• A diversity reward prevents all latent steps within a trajectory from converging to identical vectors. §D.2
• LARC is presented as the first reasoning-augmented UMM to exhibit clear test-time scaling. §3.3