Kavli Affiliate: Yi Zhou
| First 5 Authors: Ting-Hsuan Liao, Yi Zhou, Yu Shen, Chun-Hao Paul Huang, Saayan Mitra
| Summary:
We explore how body shapes influence human motion synthesis, an aspect often
overlooked in existing text-to-motion generation methods due to the ease of
learning a homogenized, canonical body shape. However, this homogenization can
distort the natural correlations between different body shapes and their motion
dynamics. Our method addresses this gap by generating body-shape-aware human
motions from natural language prompts. We utilize a finite scalar
quantization-based variational autoencoder (FSQ-VAE) to quantize motion into
discrete tokens and then leverage continuous body shape information to
de-quantize these tokens back into continuous, detailed motion. Additionally,
we harness the capabilities of a pretrained language model to predict both
continuous shape parameters and motion tokens, facilitating the synthesis of
text-aligned motions and decoding them into shape-aware motions. We evaluate
our method quantitatively and qualitatively, and also conduct a comprehensive
perceptual study to demonstrate its efficacy in generating shape-aware motions.
| Search Query: ArXiv Query: search_query=au:”Yi Zhou”&id_list=&start=0&max_results=3