Kavli Affiliate: Cheng Peng
| First 5 Authors: Cheng Peng, Yutao Tang, Yifan Zhou, Nengyu Wang, Xijun Liu
| Summary:
Recent efforts in using 3D Gaussians for scene reconstruction and novel view
synthesis can achieve impressive results on curated benchmarks; however, images
captured in real life are often blurry. In this work, we analyze the robustness
of Gaussian-Splatting-based methods against various image blur, such as motion
blur, defocus blur, downscaling blur, etc. Under these degradations,
Gaussian-Splatting-based methods tend to overfit and produce worse results than
Neural-Radiance-Field-based methods. To address this issue, we propose Blur
Agnostic Gaussian Splatting (BAGS). BAGS introduces additional 2D modeling
capacities such that a 3D-consistent and high quality scene can be
reconstructed despite image-wise blur. Specifically, we model blur by
estimating per-pixel convolution kernels from a Blur Proposal Network (BPN).
BPN is designed to consider spatial, color, and depth variations of the scene
to maximize modeling capacity. Additionally, BPN also proposes a
quality-assessing mask, which indicates regions where blur occur. Finally, we
introduce a coarse-to-fine kernel optimization scheme; this optimization scheme
is fast and avoids sub-optimal solutions due to a sparse point cloud
initialization, which often occurs when we apply Structure-from-Motion on
blurry images. We demonstrate that BAGS achieves photorealistic renderings
under various challenging blur conditions and imaging geometry, while
significantly improving upon existing approaches.
| Search Query: ArXiv Query: search_query=au:”Cheng Peng”&id_list=&start=0&max_results=3