Kavli Affiliate: Yi Zhou
| First 5 Authors: Luchuan Song, Luchuan Song, , ,
| Summary:
We propose StreamME, a method focuses on fast 3D avatar reconstruction. The
StreamME synchronously records and reconstructs a head avatar from live video
streams without any pre-cached data, enabling seamless integration of the
reconstructed appearance into downstream applications. This exceptionally fast
training strategy, which we refer to as on-the-fly training, is central to our
approach. Our method is built upon 3D Gaussian Splatting (3DGS), eliminating
the reliance on MLPs in deformable 3DGS and relying solely on geometry, which
significantly improves the adaptation speed to facial expression. To further
ensure high efficiency in on-the-fly training, we introduced a simplification
strategy based on primary points, which distributes the point clouds more
sparsely across the facial surface, optimizing points number while maintaining
rendering quality. Leveraging the on-the-fly training capabilities, our method
protects the facial privacy and reduces communication bandwidth in VR system or
online conference. Additionally, it can be directly applied to downstream
application such as animation, toonify, and relighting. Please refer to our
project page for more details: https://songluchuan.github.io/StreamME/.
| Search Query: ArXiv Query: search_query=au:”Yi Zhou”&id_list=&start=0&max_results=3