Kavli Affiliate: Michael Wimmer
| First 5 Authors: Joao Liborio Cardoso, Francesco Banterle, Paolo Cignoni, Michael Wimmer,
| Summary:
We introduce context-aware translation, a novel method that combines the
benefits of inpainting and image-to-image translation, respecting
simultaneously the original input and contextual relevance — where existing
methods fall short. By doing so, our method opens new avenues for the
controllable use of AI within artistic creation, from animation to digital art.
As an use case, we apply our method to redraw any hand-drawn animated
character eyes based on any design specifications – eyes serve as a focal point
that captures viewer attention and conveys a range of emotions, however, the
labor-intensive nature of traditional animation often leads to compromises in
the complexity and consistency of eye design. Furthermore, we remove the need
for production data for training and introduce a new character recognition
method that surpasses existing work by not requiring fine-tuning to specific
productions. This proposed use case could help maintain consistency throughout
production and unlock bolder and more detailed design choices without the
production cost drawbacks. A user study shows context-aware translation is
preferred over existing work 95.16% of the time.
| Search Query: ArXiv Query: search_query=au:”Michael Wimmer”&id_list=&start=0&max_results=3