Kavli Affiliate: Zheng Zhu
| First 5 Authors: Yi Wei, Linqing Zhao, Wenzhao Zheng, Zheng Zhu, Jie Zhou
| Summary:
3D scene understanding plays a vital role in vision-based autonomous driving.
While most existing methods focus on 3D object detection, they have difficulty
describing real-world objects of arbitrary shapes and infinite classes. Towards
a more comprehensive perception of a 3D scene, in this paper, we propose a
SurroundOcc method to predict the 3D occupancy with multi-camera images. We
first extract multi-scale features for each image and adopt spatial 2D-3D
attention to lift them to the 3D volume space. Then we apply 3D convolutions to
progressively upsample the volume features and impose supervision on multiple
levels. To obtain dense occupancy prediction, we design a pipeline to generate
dense occupancy ground truth without expansive occupancy annotations.
Specifically, we fuse multi-frame LiDAR scans of dynamic objects and static
scenes separately. Then we adopt Poisson Reconstruction to fill the holes and
voxelize the mesh to get dense occupancy labels. Extensive experiments on
nuScenes and SemanticKITTI datasets demonstrate the superiority of our method.
Code and dataset are available at https://github.com/weiyithu/SurroundOcc
| Search Query: ArXiv Query: search_query=au:”Zheng Zhu”&id_list=&start=0&max_results=3