Improving ASR with Synthetic Intra-Sentential Code-Switched Speech Generated Using Linguistically-Constrained LLMs and Multilingual TTS

Code-switching poses significant challenges for automatic speech recognition (ASR), particularly for low-resource language pairs where annotated bilingual speech data is scarce. In this work, we propose a framework for generating synthetic intra-sentential code-switched speech using large language models and multilingual text-to-speech synthesis. Code-switched text is generated from parallel corpora using a linguistically guided approach that combines Matrix Language Frame theory with a phrase-level extension of the Equivalence Constraint Theory. The generated text is converted into speech using MMS multilingual TTS and normalized with OpenVoice voice cloning to ensure consistent speaker identity. Experiments on Hausa–Yoruba and Hausa–English show improved ASR performance and more natural switching patterns.