Design and evaluate a chatbot for language learning in multilingual contexts

Thesis Abstract

The increasing globalization and multicultural interactions have underscored the need for effective language learning tools adaptable to diverse linguistic backgrounds, posing significant challenges for educators and learners in multilingual settings. This study aims to design, develop, and evaluate a culturally and linguistically adaptable chatbot to facilitate language acquisition in multilingual contexts, addressing the paucity of technologically advanced, user-centered language learning solutions tailored to diverse learner populations. The specific objectives are to identify critical linguistic features necessary for effective chatbot interactions, develop a prototype based on these features, assess the usability and engagement level of the chatbot among learners representing three distinct multilingual communities, and evaluate its impact on learners’ language proficiency over time. Employing a mixed-methods research design, the study combines qualitative design-based research (DBR) approaches for chatbot development with a quantitative quasi-experimental framework for evaluation. The population comprises 150 adult language learners from three multilingual urban communities, with stratified random sampling employed to select 50 participants from each community, ensuring representative diversity in language backgrounds and proficiency levels. Data collection instruments include structured questionnaires for baseline and post-intervention assessments, usability surveys adapted from the System Usability Scale (SUS), linguistic proficiency tests aligned with the Common European Framework of Reference for Languages (CEFR), and semi-structured interview protocols to capture learners’ perceptions and engagement experiences. The chatbot prototype is designed based on sociocultural learning theories and the Interaction Hypothesis, with natural language processing (NLP) techniques integral to enabling dynamic, context-aware interactions across multiple languages. Usability and engagement data will be analyzed using descriptive statistics, ANOVA to compare proficiency improvements across groups, and thematic analysis for qualitative interview data. The linguistic proficiency outcomes will be measured through pre- and post-intervention CEFR-aligned assessments, with regression analysis applied to identify predictors of language gains attributable to chatbot interaction. Additionally, usability and learner engagement metrics will be correlated with proficiency improvements to determine the chatbot’s role in promoting language learning. Anticipated findings suggest that the culturally and linguistically adaptive chatbot will significantly enhance learner engagement and motivation, resulting in measurable improvements in language proficiency, particularly in oral and functional language skills. The study is expected to demonstrate that chatbot-mediated interaction offers a scalable, personalized, and culturally sensitive supplement to traditional learning modes in multilingual environments. The findings will contribute to knowledge by elucidating effective design principles for multilingual conversational agents grounded in sociocultural and interactionist theories, expanding the understanding of technology-enhanced language learning in diverse contexts. The study concludes that carefully designed, adaptable chatbots can serve as effective, autonomous language learning companions in multilingual settings by fostering sustained interaction and motivation. Recommendations include integrating such chatbots into mainstream language curricula, further refining NLP capabilities for nuanced cultural expressions, and conducting longitudinal studies to explore long-term impacts on language retention. The research advances theoretical understanding of learner-centered design in multilingual AI language tools and provides practical guidance for developers and educators aiming to leverage conversational agents to address linguistic diversity challenges in contemporary language education.