Kavli Affiliate: Wei Gao
| First 5 Authors: [#item_custom_name[1]], [#item_custom_name[2]], [#item_custom_name[3]], [#item_custom_name[4]], [#item_custom_name[5]]
| Summary:
Occlusions between consecutive frames have long posed a significant challenge
in optical flow estimation. The inherent ambiguity introduced by occlusions
directly violates the brightness constancy constraint and considerably hinders
pixel-to-pixel matching. To address this issue, multi-frame optical flow
methods leverage adjacent frames to mitigate the local ambiguity. Nevertheless,
prior multi-frame methods predominantly adopt recursive flow estimation,
resulting in a considerable computational overlap. In contrast, we propose a
streamlined in-batch framework that eliminates the need for extensive redundant
recursive computations while concurrently developing effective spatio-temporal
modeling approaches under in-batch estimation constraints. Specifically, we
present a Streamlined In-batch Multi-frame (SIM) pipeline tailored to video
input, attaining a similar level of time efficiency to two-frame networks.
Furthermore, we introduce an efficient Integrative Spatio-temporal Coherence
(ISC) modeling method for effective spatio-temporal modeling during the
encoding phase, which introduces no additional parameter overhead.
Additionally, we devise a Global Temporal Regressor (GTR) that effectively
explores temporal relations during decoding. Benefiting from the efficient SIM
pipeline and effective modules, StreamFlow not only excels in terms of
performance on the challenging KITTI and Sintel datasets, with particular
improvement in occluded areas but also attains a remarkable $63.82%$
enhancement in speed compared with previous multi-frame methods. The code will
be available soon at https://github.com/littlespray/StreamFlow.
| Search Query: [#feed_custom_title]