Kavli Affiliate: Wei Gao
| First 5 Authors: Qiyao Xue, Qiyao Xue, , ,
| Summary:
Hate speech detection on Chinese social networks presents distinct
challenges, particularly due to the widespread use of cloaking techniques
designed to evade conventional text-based detection systems. Although large
language models (LLMs) have recently improved hate speech detection
capabilities, the majority of existing work has concentrated on English
datasets, with limited attention given to multimodal strategies in the Chinese
context. In this study, we propose MMBERT, a novel BERT-based multimodal
framework that integrates textual, speech, and visual modalities through a
Mixture-of-Experts (MoE) architecture. To address the instability associated
with directly integrating MoE into BERT-based models, we develop a progressive
three-stage training paradigm. MMBERT incorporates modality-specific experts, a
shared self-attention mechanism, and a router-based expert allocation strategy
to enhance robustness against adversarial perturbations. Empirical results in
several Chinese hate speech datasets show that MMBERT significantly surpasses
fine-tuned BERT-based encoder models, fine-tuned LLMs, and LLMs utilizing
in-context learning approaches.
| Search Query: ArXiv Query: search_query=au:”Wei Gao”&id_list=&start=0&max_results=3