Kavli Affiliate: Yi Zhou
| First 5 Authors: Tingting Liao, Tingting Liao, , ,
| Summary:
Imagine Mr. Bean stepping into Tom and Jerry–can we generate videos where
characters interact naturally across different worlds? We study inter-character
interaction in text-to-video generation, where the key challenge is to preserve
each character’s identity and behaviors while enabling coherent cross-context
interaction. This is difficult because characters may never have coexisted and
because mixing styles often causes style delusion, where realistic characters
appear cartoonish or vice versa. We introduce a framework that tackles these
issues with Cross-Character Embedding (CCE), which learns identity and
behavioral logic across multimodal sources, and Cross-Character Augmentation
(CCA), which enriches training with synthetic co-existence and mixed-style
data. Together, these techniques allow natural interactions between previously
uncoexistent characters without losing stylistic fidelity. Experiments on a
curated benchmark of cartoons and live-action series with 10 characters show
clear improvements in identity preservation, interaction quality, and
robustness to style delusion, enabling new forms of generative
storytelling.Additional results and videos are available on our project page:
https://tingtingliao.github.io/mimix/.
| Search Query: ArXiv Query: search_query=au:”Yi Zhou”&id_list=&start=0&max_results=3