Kavli Affiliate: Matthew Fisher
| First 5 Authors: Tobias Hinz, Matthew Fisher, Oliver Wang, Eli Shechtman, Stefan Wermter
| Summary:
We introduce CharacterGAN, a generative model that can be trained on only a
few samples (8 – 15) of a given character. Our model generates novel poses
based on keypoint locations, which can be modified in real time while providing
interactive feedback, allowing for intuitive reposing and animation. Since we
only have very limited training samples, one of the key challenges lies in how
to address (dis)occlusions, e.g. when a hand moves behind or in front of a
body. To address this, we introduce a novel layering approach which explicitly
splits the input keypoints into different layers which are processed
independently. These layers represent different parts of the character and
provide a strong implicit bias that helps to obtain realistic results even with
strong (dis)occlusions. To combine the features of individual layers we use an
adaptive scaling approach conditioned on all keypoints. Finally, we introduce a
mask connectivity constraint to reduce distortion artifacts that occur with
extreme out-of-distribution poses at test time. We show that our approach
outperforms recent baselines and creates realistic animations for diverse
characters. We also show that our model can handle discrete state changes, for
example a profile facing left or right, that the different layers do indeed
learn features specific for the respective keypoints in those layers, and that
our model scales to larger datasets when more data is available.
| Search Query: ArXiv Query: search_query=au:”Matthew Fisher”&id_list=&start=0&max_results=10