Kavli Affiliate: Cheng Peng
| First 5 Authors: Deming Li, Deming Li, , ,
| Summary:
In-the-wild photo collections often contain limited volumes of imagery and
exhibit multiple appearances, e.g., taken at different times of day or seasons,
posing significant challenges to scene reconstruction and novel view synthesis.
Although recent adaptations of Neural Radiance Field (NeRF) and 3D Gaussian
Splatting (3DGS) have improved in these areas, they tend to oversmooth and are
prone to overfitting. In this paper, we present MS-GS, a novel framework
designed with Multi-appearance capabilities in Sparse-view scenarios using
3DGS. To address the lack of support due to sparse initializations, our
approach is built on the geometric priors elicited from monocular depth
estimations. The key lies in extracting and utilizing local semantic regions
with a Structure-from-Motion (SfM) points anchored algorithm for reliable
alignment and geometry cues. Then, to introduce multi-view constraints, we
propose a series of geometry-guided supervision at virtual views in a
fine-grained and coarse scheme to encourage 3D consistency and reduce
overfitting. We also introduce a dataset and an in-the-wild experiment setting
to set up more realistic benchmarks. We demonstrate that MS-GS achieves
photorealistic renderings under various challenging sparse-view and
multi-appearance conditions and outperforms existing approaches significantly
across different datasets.
| Search Query: ArXiv Query: search_query=au:”Cheng Peng”&id_list=&start=0&max_results=3