YANAMI: A Large-Scale Multimodal Dataset and Benchmark for Multiturn Stateful Interaction

YANAMI introduces a 400K-dialogue multimodal dataset and a 400-dialogue benchmark for multiturn stateful interaction, where models must track, update, and visually realize evolving dialogue states.

Existing unified multimodal datasets and benchmarks are dominated by single-turn tasks. Many recent multi-turn datasets contain turns that can be solved independently without tracking information across turns. This provides limited supervision and evaluation for multimodal stateful interaction, which requires understanding, maintaining, updating, and faithfully realizing evolving dialogue states.

The authors construct YANAMI-400K, a dataset of 400K multi-turn multimodal dialogues with more than 2M images and 2.6M dialogue turns across four scenarios: Multi-Image Fusion, Image Editing, World Modeling, and Visual Reasoning. Each dialogue incorporates six state-management capabilities, including reference resolution, context retrieval, consistency maintenance, constraint keeping, and correction-based state updating. The data pipeline uses model-assisted filtering, dialogue construction, automatic verification, and human verification. YANAMI-Bench contains 400 multi-turn dialogues and evaluates Dialogue State Understanding and Visual Realization using a VLM-as-a-Judge framework. The authors fine-tune BAGEL on YANAMI-400K and evaluate it on generation, editing, understanding, and stateful interaction benchmarks.

• YANAMI-400K contains 400K multi-turn multimodal dialogues, more than 2M images, and 2.6M dialogue turns. §Abstract
• Fine-tuning BAGEL on YANAMI-400K improves image editing, generation, and multimodal stateful interaction by the stated percentages while preserving general visual understanding. §Abstract
• On YANAMI-Bench, the best average scores of existing UMMs and open-source image editing models are 67.41 and 67.30, while GPT-Image-2 and Nano Banana 2 achieve 83.05 and 80.93. §4.1
• Fine-tuned BAGEL reaches an average DSU score of 95.69, compared with 81.44 for BAGEL and 67.94 for Show-o2. §4.1
• Performance drops by 2%, 6%, and 6% for BAGEL, FLUX.2, and InternVL-U, respectively, between the first and last image-generation turns under GT-History. §4.1
• Correlations between Qwen-3.8-Flash-Next and human ratings consistently exceed 0.80. §4.3

• The paper states that YANAMI currently focuses on four representative interaction scenarios and six core state-management capabilities, and that future extensions may incorporate additional application domains, languages, modalities, and longer interaction horizons.
• The paper states that it would be valuable to complement the controlled evaluation with broader interactive studies.
• The paper evaluates training benefits on BAGEL and states that examining the benefits of YANAMI-400K across additional unified multimodal model architectures would be valuable.

• YANAMI consists of YANAMI-400K for training and YANAMI-Bench for evaluation of multimodal stateful interaction. §1
• The four interaction scenarios are Multi-Image Fusion, Image Editing, World Modeling, and Visual Reasoning. §3.1
• The six state-management capabilities are reference resolution, context retrieval, consistency maintenance, spatial and physical consistency, constraint keeping, and correction-based state updating. §1
• YANAMI-Bench evaluates Dialogue State Understanding across Reference Resolution, Context Retrieval, Constraint Tracking, and Correction Updating, and evaluates Visual Realization through Visual Quality and State Consistency. §3.2
• The benchmark is constructed with LLM assistance followed by human verification, and SHA-256 hashes confirm no exact image duplicates between YANAMI-Bench and YANAMI-400K. §3.1
• Scores use a 4:4:2 weighting over dialogue-state understanding, State Consistency, and Visual Quality for models with understanding and generation, and a 6:4 weighting for models without understanding. §B.3
• Main evaluation uses GT-History, while Self-History uses model-generated images from previous turns as subsequent visual context. §4.1
• Every dialogue in YANAMI-400K involves at least four of the six capabilities, requiring multiple cross-turn dependencies to be exercised jointly. §A.3