Kavli Affiliate: Feng Wang
| First 5 Authors: Yuwen Xiong, Zhiqi Li, Yuntao Chen, Feng Wang, Xizhou Zhu
| Summary:
We introduce Deformable Convolution v4 (DCNv4), a highly efficient and
effective operator designed for a broad spectrum of vision applications. DCNv4
addresses the limitations of its predecessor, DCNv3, with two key enhancements:
1. removing softmax normalization in spatial aggregation to enhance its dynamic
property and expressive power and 2. optimizing memory access to minimize
redundant operations for speedup. These improvements result in a significantly
faster convergence compared to DCNv3 and a substantial increase in processing
speed, with DCNv4 achieving more than three times the forward speed. DCNv4
demonstrates exceptional performance across various tasks, including image
classification, instance and semantic segmentation, and notably, image
generation. When integrated into generative models like U-Net in the latent
diffusion model, DCNv4 outperforms its baseline, underscoring its possibility
to enhance generative models. In practical applications, replacing DCNv3 with
DCNv4 in the InternImage model to create FlashInternImage results in up to 80%
speed increase and further performance improvement without further
modifications. The advancements in speed and efficiency of DCNv4, combined with
its robust performance across diverse vision tasks, show its potential as a
foundational building block for future vision models.
| Search Query: ArXiv Query: search_query=au:”Feng Wang”&id_list=&start=0&max_results=3