Kavli Affiliate: Xiang Zhang
| First 5 Authors: Xiang Zhang, Bingxin Ke, Hayko Riemenschneider, Nando Metzger, Anton Obukhov
| Summary:
By training over large-scale datasets, zero-shot monocular depth estimation
(MDE) methods show robust performance in the wild but often suffer from
insufficiently precise details. Although recent diffusion-based MDE approaches
exhibit appealing detail extraction ability, they still struggle in
geometrically challenging scenes due to the difficulty of gaining robust
geometric priors from diverse datasets. To leverage the complementary merits of
both worlds, we propose BetterDepth to efficiently achieve geometrically
correct affine-invariant MDE performance while capturing fine-grained details.
Specifically, BetterDepth is a conditional diffusion-based refiner that takes
the prediction from pre-trained MDE models as depth conditioning, in which the
global depth context is well-captured, and iteratively refines details based on
the input image. For the training of such a refiner, we propose global
pre-alignment and local patch masking methods to ensure the faithfulness of
BetterDepth to depth conditioning while learning to capture fine-grained scene
details. By efficient training on small-scale synthetic datasets, BetterDepth
achieves state-of-the-art zero-shot MDE performance on diverse public datasets
and in-the-wild scenes. Moreover, BetterDepth can improve the performance of
other MDE models in a plug-and-play manner without additional re-training.
| Search Query: ArXiv Query: search_query=au:”Xiang Zhang”&id_list=&start=0&max_results=3