Developing AI-powered tools for real-time dialect and accent recognition in multilingual communities

Thesis Abstract

Multilingual communities are characterized by a rich diversity of dialects and accents, which pose significant challenges for effective communication, public service delivery, and language-based technologies. Despite advancements in speech recognition technologies, accurate real-time identification of dialectal and accent variation remains limited, primarily due to the complexity of linguistic features and variability in speech patterns across different languages and regions. This study aims to develop an artificial intelligence (AI)-powered tool capable of real-time dialect and accent recognition in multilingual settings, thereby enhancing speech technology accessibility and linguistic inclusivity. The specific objectives include (1) to analyze the phonetic and prosodic features distinguishing various dialects and accents within selected multilingual communities; (2) to design a machine learning-based framework utilizing acoustic and linguistic input features; (3) to implement and train a deep learning model—specifically, a convolutional neural network (CNN) integrated with recurrent neural networks (RNN)—for dialect and accent classification; and (4) to evaluate the system’s performance against existing speech recognition benchmarks across diverse linguistic datasets. The study adopts a pragmatic mixed-methods research design, leveraging both quantitative and qualitative approaches to ensure a comprehensive understanding of speech variability and AI model efficacy. The population of the study comprises recorded speech samples from 1,200 native speakers across three multilingual communities in South Asia, each representing different language groups with distinct dialectal and accentual features. Stratified random sampling was employed to select participants, ensuring balanced representation across age, gender, and regional subgroups. Data collection instruments include standardized speech elicitation protocols, high-fidelity microphones, and existing linguistic corpora. Acoustic features such as Mel-frequency cepstral coefficients (MFCCs), pitch contours, and spectral features were extracted using Praat and Kaldi speech analysis tools. Linguistic features were annotated through manual transcription and phonological analysis, guided by the Optimality Theory framework to understand dialectal constraints and variations. Quantitative analysis involved training multiple supervised machine learning algorithms—including CNNs, RNNs, and support vector machines (SVMs)—using TensorFlow and Scikit-learn frameworks. Model performance was evaluated through metrics such as accuracy, precision, recall, F1-score, and confusion matrices, with cross-validation procedures applied to prevent overfitting. Additionally, feature importance analysis was conducted via SHapley Additive exPlanations (SHAP) to identify the most influential linguistic cues for dialect and accent discrimination. Qualitative insights from linguistic experts complemented the technical evaluation, validating the model's interpretability and practical applicability. Expected findings include the development of a highly accurate (anticipated accuracy exceeding 85%) AI tool capable of distinguishing dialects and accents in real time, outperforming existing speech recognition systems, especially in low-resource multilingual contexts. The results are anticipated to reveal key acoustic-phonetic markers unique to each dialect and accent, supporting the theoretical premise that phonetic variation underpins linguistic diversity. Furthermore, the study is expected to demonstrate that integrating phonological theory such as the Optimality Theory enhances model interpretability and linguistic accuracy. This research contributes to knowledge by advancing the theoretical understanding of dialectal and accentual features in computational models, as well as practical implications for deploying speech recognition technology in multilingual and multicultural environments. It provides a scalable, low-latency AI solution that can be integrated into virtual assistants, transcription services, and language learning platforms, fostering greater inclusivity. The study concludes with recommendations for further development of multilingual speech systems, emphasizing the importance of linguistic diversity awareness and ethical considerations in deploying AI-driven language technologies. It advocates for ongoing collaboration between linguists and AI specialists to refine dialect-sensitive speech recognition systems continually.