Kavli Affiliate: Jia Liu
| First 5 Authors: Haisong Liu, Tao Lu, Yihui Xu, Jia Liu, Wenjie Li
| Summary:
In this paper, we study the problem of jointly estimating the optical flow
and scene flow from synchronized 2D and 3D data. Previous methods either employ
a complex pipeline which splits the joint task into independent stages, or fuse
2D and 3D information in an “early-fusion” or “late-fusion” manner. Such
one-size-fits-all approaches suffer from a dilemma of failing to fully utilize
the characteristic of each modality or to maximize the inter-modality
complementarity. To address the problem, we propose a novel end-to-end
framework, called CamLiFlow. It consists of 2D and 3D branches with multiple
bidirectional connections between them in specific layers. Different from
previous work, we apply a point-based 3D branch to better extract the geometric
features and design a symmetric learnable operator to fuse dense image features
and sparse point features. We also propose a transformation for point clouds to
solve the non-linear issue of 3D-2D projection. Experiments show that CamLiFlow
achieves better performance with fewer parameters. Our method ranks 1st on the
KITTI Scene Flow benchmark, outperforming the previous art with 1/7 parameters.
Code will be made available.
| Search Query: ArXiv Query: search_query=au:”Jia Liu”&id_list=&start=0&max_results=10