From Fragmentation to Fusion: A Survey and Roadmap for Omnimodal Embeddings

The paper argues that embedding research across modalities is converging on shared principles and proposes a roadmap from transformation and alignment to joint omnimodal embedding, graded by coverage, comparability, and compositionality.

Embedding research is split by modality, with separate communities, surveys, and evaluation practices. The paper argues this fragmentation obscures a deeper convergence driven by contrastive learning, Transformers, and a granularity shift. It proposes a cross-cutting framework and roadmap for omnimodal embeddings rather than a modality-specific survey.

The paper is a position paper and survey. It defines a modality-agnostic anatomy of five components and distinguishes input data units from output representational objects. It applies an intersection, difference, and reuse lens to contrastive, reconstructive, and generative-hybrid objectives; Transformer and alternative architectures; and granularity shifts. It organizes unification into transformation, alignment, and joint embedding, graded by coverage, comparability, and compositionality. It then discusses modality-specific properties, cross-pollination, and unified evaluation.

• The three paths satisfy the 3C capabilities in turn: transformation broadens coverage, alignment enables comparability, and joint embedding aims at compositionality. §1.4
• LETS-C serializes each series for a general text embedding model and fuses that embedding with the preprocessed numerical series by element-wise addition, outperforming 27 baselines on a ten-dataset UEA benchmark. §6.2
• Glyph renders long contexts into images whose visual tokens are 3–4 × fewer than the original text tokens. §6.1
• DeepSeek-OCR pushes optical compression past 10 × at usable OCR accuracy. §6.1

• Appendix B.3 states that phrase-based arXiv queries exclude neighboring terminologies, so counts are lower bounds for the family of interest.
• Appendix B.3 states that title-level matching biases the sample toward work that signals its subject in the title, while methodological work that uses embeddings without naming them is invisible.
• Appendix B.3 states that the counts measure where attention sits, how siloed venues remain, and which corner is growing fastest, not whether embeddings preserve complementary information, model cross-modal interactions, or improve downstream reasoning.
• Section 6.2 leaves open whether LETS-C's text embedding itself, rather than the fused raw series, matches purpose-built architectures.

• The paper argues that fragmentation appears in research, surveys, and evaluation. §1
• Four convergent trends are contrastive learning, Transformers as universal encoders, the pretrain–finetune paradigm, and the granularity shift. §1.3
• The 3C lens grades systems by coverage, comparability, and compositionality. §1.4
• An embedding model can be decomposed into five recurring components defined by role rather than implementation. §2.1
• The data unit is the input atom, while the representational object is the output atom that downstream consumers use. §2.2
• Transformation provides cross-modal coverage by rendering, serialization, or discrete tokens but loses native structure. §6.3
• Alignment earns comparability but cannot earn compositionality, and aligned modalities still exhibit the modality gap. §7.3
• Joint embedding is argued to be a general framework that unifies and surpasses alignment by letting modalities interact inside the model. §8.2