Kavli Affiliate: Zeeshan Ahmed
| First 5 Authors: Jinzheng Zhao, Niko Moritz, Egor Lakomkin, Ruiming Xie, Zhiping Xiu
| Summary:
Cascaded speech-to-speech translation systems often suffer from the error
accumulation problem and high latency, which is a result of cascaded modules
whose inference delays accumulate. In this paper, we propose a transducer-based
speech translation model that outputs discrete speech tokens in a low-latency
streaming fashion. This approach eliminates the need for generating text output
first, followed by machine translation (MT) and text-to-speech (TTS) systems.
The produced speech tokens can be directly used to generate a speech signal
with low latency by utilizing an acoustic language model (LM) to obtain
acoustic tokens and an audio codec model to retrieve the waveform. Experimental
results show that the proposed method outperforms other existing approaches and
achieves state-of-the-art results for streaming translation in terms of BLEU,
average latency, and BLASER 2.0 scores for multiple language pairs using the
CVSS-C dataset as a benchmark.
| Search Query: ArXiv Query: search_query=au:”Zeeshan Ahmed”&id_list=&start=0&max_results=3