Evaluating the Effectiveness of AI-Powered Chatbots in Customer Service Satisfaction

Thesis Abstract

In the evolving landscape of digital customer interaction, AI-powered chatbots have become integral to service delivery, yet their actual efficacy in enhancing customer satisfaction remains insufficiently empirically validated. This study investigates the effectiveness of AI-driven chatbots in improving customer service satisfaction within the telecommunications sector, aiming to provide evidence-based insights into their impact and operational efficacy. The primary objective is to evaluate customer perceptions and satisfaction levels consequent to interactions with chatbots, identify the key factors influencing satisfaction, and ascertain the extent to which chatbot performance correlates with overall customer loyalty. Specifically, the study seeks to answer whether chatbot responsiveness, accuracy, and empathy significantly affect customer satisfaction ratings, and to what degree these factors influence customer loyalty and retention. Employing a mixed-methods research design, the study combines quantitative surveys with qualitative interviews to generate comprehensive insights. The population comprises 1,200 customers who have interacted with AI-powered chatbots on the customer service platforms of three major telecommunication companies over the past six months. A stratified random sampling technique selected 300 participants for survey participation, ensuring demographic and usage-pattern representation. Data collection instruments include a standardized customer satisfaction questionnaire, adapted from the SERVQUAL model to measure responsiveness, reliability, and empathy, alongside semi-structured interview guides to explore subjective experiences. The reliability of the survey instrument was assessed through Cronbach’s alpha, achieving a coefficient of 0.88, indicating high internal consistency. Validity was established via expert review and pilot testing. Quantitative data are analyzed through multiple regression analysis to test the hypothesized relationships between chatbot performance factors and customer satisfaction levels. Descriptive statistics summarize customer demographics and satisfaction scores, while hypothesis testing involves Analysis of Variance (ANOVA) to compare satisfaction across different customer segments. Qualitative data from interviews undergo thematic analysis to identify recurring themes related to perceived chatbot effectiveness, personalization, and emotional rapport. These findings are integrated with the quantitative results to produce a holistic understanding of chatbot impacts. It is anticipated that the study will reveal a significant positive correlation between chatbot responsiveness and customer satisfaction, with factors such as accuracy and empathetic interactions mediating overall perceptions. The research expects to identify key gaps in chatbot performance, particularly around personalized service and emotional intelligence, which influence satisfaction and loyalty. The findings are expected to contribute to existing theoretical frameworks, notably the Technology Acceptance Model (TAM) and Service-Dominant Logic, by extending their applicability to AI chatbot interactions in customer service contexts. The study’s contributions to knowledge include empirical validation of specific chatbot attributes that strongly influence customer satisfaction and a nuanced understanding of customer perceptions in a digital service environment. It also offers practical insights for service managers seeking to optimize chatbot design and deployment strategies to enhance customer experience. The main conclusion emphasizes that while AI chatbots can significantly improve operational efficiency, their success in fostering customer satisfaction depends critically on their ability to deliver accurate, empathetic, and personalized responses. Based on these findings, the study recommends targeted enhancements in chatbot AI algorithms, increased investment in training models to recognize emotional cues, and routine performance evaluations aligned with customer feedback metrics. To further advance understanding, future research should explore longitudinal impacts of chatbot integration over extended periods and examine industry-specific dynamics influencing customer preferences and satisfaction levels.