Kavli Affiliate: Xiang Zhang
| First 5 Authors: Chi Zhang, Xiang Zhang, Mingyuan Lin, Cheng Li, Chu He
| Summary:
Even though the collaboration between traditional and neuromorphic event
cameras brings prosperity to frame-event based vision applications, the
performance is still confined by the resolution gap crossing two modalities in
both spatial and temporal domains. This paper is devoted to bridging the gap by
increasing the temporal resolution for images, i.e., motion deblurring, and the
spatial resolution for events, i.e., event super-resolving, respectively. To
this end, we introduce CrossZoom, a novel unified neural Network (CZ-Net) to
jointly recover sharp latent sequences within the exposure period of a blurry
input and the corresponding High-Resolution (HR) events. Specifically, we
present a multi-scale blur-event fusion architecture that leverages the
scale-variant properties and effectively fuses cross-modality information to
achieve cross-enhancement. Attention-based adaptive enhancement and
cross-interaction prediction modules are devised to alleviate the distortions
inherent in Low-Resolution (LR) events and enhance the final results through
the prior blur-event complementary information. Furthermore, we propose a new
dataset containing HR sharp-blurry images and the corresponding HR-LR event
streams to facilitate future research. Extensive qualitative and quantitative
experiments on synthetic and real-world datasets demonstrate the effectiveness
and robustness of the proposed method. Codes and datasets are released at
https://bestrivenzc.github.io/CZ-Net/.
| Search Query: ArXiv Query: search_query=au:”Xiang Zhang”&id_list=&start=0&max_results=3