Kavli Affiliate: Feng Yuan
| First 5 Authors: Yifan Gao, Yifan Gao, , ,
| Summary:
Foundation models pre-trained on large-scale natural image datasets offer a
powerful paradigm for medical image segmentation. However, effectively
transferring their learned representations for precise clinical applications
remains a challenge. In this work, we propose Dino U-Net, a novel
encoder-decoder architecture designed to exploit the high-fidelity dense
features of the DINOv3 vision foundation model. Our architecture introduces an
encoder built upon a frozen DINOv3 backbone, which employs a specialized
adapter to fuse the model’s rich semantic features with low-level spatial
details. To preserve the quality of these representations during dimensionality
reduction, we design a new fidelity-aware projection module (FAPM) that
effectively refines and projects the features for the decoder. We conducted
extensive experiments on seven diverse public medical image segmentation
datasets. Our results show that Dino U-Net achieves state-of-the-art
performance, consistently outperforming previous methods across various imaging
modalities. Our framework proves to be highly scalable, with segmentation
accuracy consistently improving as the backbone model size increases up to the
7-billion-parameter variant. The findings demonstrate that leveraging the
superior, dense-pretrained features from a general-purpose foundation model
provides a highly effective and parameter-efficient approach to advance the
accuracy of medical image segmentation. The code is available at
https://github.com/yifangao112/DinoUNet.
| Search Query: ArXiv Query: search_query=au:”Feng Yuan”&id_list=&start=0&max_results=3