Kavli Affiliate: Yi Zhou
| First 5 Authors: Bingzhi Liu, Yin Cao, Haohe Liu, Yi Zhou,
| Summary:
Diffusion models have demonstrated promising results in text-to-audio
generation tasks. However, their practical usability is hindered by slow
sampling speeds, limiting their applicability in high-throughput scenarios. To
address this challenge, progressive distillation methods have been effective in
producing more compact and efficient models. Nevertheless, these methods
encounter issues with unbalanced weights at both high and low noise levels,
potentially impacting the quality of generated samples. In this paper, we
propose the adaptation of the progressive distillation method to text-to-audio
generation tasks and introduce the Balanced SNR-Aware~(BSA) method, an enhanced
loss-weighting mechanism for diffusion distillation. The BSA method employs a
balanced approach to weight the loss for both high and low noise levels. We
evaluate our proposed method on the AudioCaps dataset and report experimental
results showing superior performance during the reverse diffusion process
compared to previous distillation methods with the same number of sampling
steps. Furthermore, the BSA method allows for a significant reduction in
sampling steps from 200 to 25, with minimal performance degradation when
compared to the original teacher models.
| Search Query: ArXiv Query: search_query=au:”Yi Zhou”&id_list=&start=0&max_results=3