Kavli Affiliate: Zeeshan Ahmed
| First 5 Authors: Chuanneng Sun, Zeeshan Ahmed, Yingyi Ma, Zhe Liu, Lucas Kabela
| Summary:
This paper studies contextual biasing with Large Language Models (LLMs),
where during second-pass rescoring additional contextual information is
provided to a LLM to boost Automatic Speech Recognition (ASR) performance. We
propose to leverage prompts for a LLM without fine tuning during rescoring
which incorporate a biasing list and few-shot examples to serve as additional
information when calculating the score for the hypothesis. In addition to
few-shot prompt learning, we propose multi-task training of the LLM to predict
both the entity class and the next token. To improve the efficiency for
contextual biasing and to avoid exceeding LLMs’ maximum sequence lengths, we
propose dynamic prompting, where we select the most likely class using the
class tag prediction, and only use entities in this class as contexts for next
token prediction. Word Error Rate (WER) evaluation is performed on i) an
internal calling, messaging, and dictation dataset, and ii) the SLUE-Voxpopuli
dataset. Results indicate that biasing lists and few-shot examples can achieve
17.8% and 9.6% relative improvement compared to first pass ASR, and that
multi-task training and dynamic prompting can achieve 20.0% and 11.3% relative
WER improvement, respectively.
| Search Query: ArXiv Query: search_query=au:”Zeeshan Ahmed”&id_list=&start=0&max_results=3