MEGA Hub

MedVTok: A General-Purpose Medical Visual Tokenizer

Authors

Do you know Chenglong Ma?You can claim authorship or link another user.Do you know Yuanfeng Ji?You can claim authorship or link another user.Do you know Junzhi Ning?You can claim authorship or link another user.Do you know Jiyao Liu?You can claim authorship or link another user.Do you know Yujie Wei?You can claim authorship or link another user.Do you know Shaohao Rui?You can claim authorship or link another user.Do you know Chaoyang Zhang?You can claim authorship or link another user.Do you know Wenjie Li?You can claim authorship or link another user.Do you know Wenhao Tang?You can claim authorship or link another user.Do you know Cheng Tang?You can claim authorship or link another user.Do you know Wei Li?You can claim authorship or link another user.Do you know Shujian Gao?You can claim authorship or link another user.Do you know Yujie Zhang?You can claim authorship or link another user.Do you know Ying Chen?You can claim authorship or link another user.Do you know Zilong Li?You can claim authorship or link another user.Do you know Zongyuan Ge?You can claim authorship or link another user.Do you know Diping Song?You can claim authorship or link another user.Do you know Jin Ye?You can claim authorship or link another user.Do you know Ming Hu?You can claim authorship or link another user.Do you know Junjun He?You can claim authorship or link another user.Do you know Hongming Shan?You can claim authorship or link another user.

Abstract

Multimodal medical AI requires a common visual interface for encoding heterogeneous imaging evidence for patient condition modeling. However, existing medical visual tokenizers are often tied to a specific dimension and imaging modality, forcing multimodal systems to rely on fragmented and poorly reusable representations. We introduce MedVTok, a general-purpose visual tokenizer that maps 2D images and 3D volumes from diverse imaging modalities into a unified token space. Building such a tokenizer poses two key challenges: preserving anatomical structure across dimensions and capturing clinical semantics across modalities. To this end, MedVTok introduces (i) slice differential regularization, which explicitly models inter-slice anatomical coherence missing in previous per-pixel/per-voxel supervision, and (ii) multi-expert representation alignment, which integrates knowledge from multiple medical expert encoders while retaining modality-specific complementary knowledge. Trained on 28M images and 70K volumes across 9 imaging modalities, MedVTok supports classification, retrieval, segmentation, synthesis, visual question answering, and medical report generation through a single tokenizer. Across 24 benchmarks, MedVTok achieves state-of-the-art performance, demonstrating a scalable token interface that unifies medical image dimension, modality, and task usage.

Community

00

Publication notes

Author note
NeurIPS 2026