Wizard: Manipulate Any World in Real Time via Agentic Control

Wizard is an agentic game world model that decouples explicit game-state control from video generation to enable real-time, zero-shot control of scenes, characters, and motions.

Existing game world models restrict control flexibility, generalize poorly to unseen environments and characters, and offer limited real-time interactivity. Action-conditioned models restrict players to predefined action sets, while prompt-conditioned models depend on motion and skill patterns seen during training. Agentic frameworks that use structured proxies mainly demonstrate basic movement such as vehicle driving and character locomotion, with limited support for articulated character actions.

Wizard represents scenes with geometric proxies encoding depth and height and characters with articulated skeletons, forming a unified visual proxy. An agentic framework updates scene geometry, animates skeletons, retrieves and retargets motions from a motion library, and maintains combat states. The video generator is built on Wan2.2-TI2V-5B and conditions on proxy sequence, initial scene image, and character reference images. It uses channel concatenation, zero-initialized condition embedding, and character-aware adaptive attention masking to bind references to character regions. Causal adaptation and few-step distillation enable real-time autoregressive inference.

• The constructed dataset contains 50.3 hours of aligned proxy-video pairs covering 750 scenes and 178 characters. §3
• Wizard supports real-time interaction at 24 FPS using agentic control with geometric scene proxies and articulated character skeletons. §1
• Wizard demonstrates zero-shot generalization to novel scenes, characters, and motions without additional training in qualitative combat rollouts. §5.2
• The agentic framework enforces a combat rule that when either character's health reaches zero, the controller resolves death, ends combat, and displays the outcome. §5.2
• Players can transform characters during gameplay with a reference image and manipulate scenes by inverting the environment or introducing new scene elements through agent-controlled visual proxies. §5.3

• Reliance on a motion library makes it difficult to find compatible motions for characters with uncommon body structures.
• Consistency between scene proxies and the generated video may decline over longer rollouts.
• Experiments report qualitative visualizations in Sections 5.2 and 5.3 without quantitative metrics or baselines, leaving quantitative evaluation open.

• Wizard decouples scene construction, character animation, and gameplay logic from visual generation through structured spatial conditioning. §1
• The dataset contains 50.3 hours of aligned proxy-video pairs, 750 scenes, and 178 characters from real gameplay, synthetic data, Unreal Engine simulations, and public datasets. §3
• Scene proxies encode depth and height, while character skeletons and geometry form unified visual proxies; channel coding distinguishes character instances and camera-relative orientations. §1
• A vision-language model retrieves compatible skeletons and animations from a motion library based on character morphology and intended motion style, then retargets them. §1
• Adaptive masking restricts reference-image attention to the corresponding character regions to strengthen appearance alignment and preserve identity. §4.2
• The agent maintains explicit combat states including health, relative distance, and orientation to enforce gameplay rules and adapt boss behavior. §1
• Causal adaptation and few-step distillation are applied to enable real-time autoregressive video generation. §4.2