Multimodal Pragmatics in a Global Tech Customer Support Center

Thesis Abstract

In the increasingly globalized technology sector, customer support centers operate across diverse linguistic and cultural contexts, where multimodal cues in spoken interactions, chat transcripts, and video-based assistance jointly influence perceived clarity, trust, and problem resolution outcomes. This study investigates how multimodal pragmatics shapes communication effectiveness in a Global Tech Customer Support Center, addressing the problem that conventional text- or speech-focused analyses overlook the integrated role of gaze, gesture, prosody, layout, and platform affordances in customer-agent interactions. The aim is to elucidate how multimodal resources contribute to negotiation of meaning and alignment across culturally diverse customers and agents, with specific objectives (1) to identify multimodal patterns that correlate with successful issue resolution and customer satisfaction; (2) to examine how discourse strategies are modulated by language background, service tier, and interaction modality (voice, chat, video); (3) to test the applicability of established theoretical frameworks to cross-modal pragmatic inferences in real-world support settings; and (4) to develop an evidence-based model linking multimodal cues to outcome variables for training and interface design. The study adopts a mixed-methods design grounded in Speech Act Theory and Multimodal Interactional Analysis, supplemented by Gricean pragmatics for cooperative communication and Streeck, Kaltenbacher, and Goodwin’s frameworks on social action across modalities. The population comprises 1,200 recorded customer interactions spanning six months, drawn from three global regional hubs (North America, EMEA, APAC) within a leading tech enterprise. A stratified random sample yields 300 interactions for quantitative analysis and 60 for qualitative, with 150 paired voice-and-chat transcripts and corresponding video-augmented sessions where available. Data collection instruments include a multimodal annotation schema adapted from the MUMIN framework to capture gaze, gesture, prosody, facial expression, screen-sharing dynamics, and interface affordances; a standardized customer satisfaction survey; and transactional metadata (resolution time, escalation rate, session length). Validity and reliability are enhanced through intercoder reliability checks (Cohen’s Kappa > .80 for key multimodal categories), pilot testing, and triangulation across transcripts, video recordings, and objective outcomes. Analytical procedures integrate quantitative and qualitative strands. Descriptive statistics will outline frequency and distribution of multimodal cues; regression analyses will test the predictive power of multimodal features on customer satisfaction scores (NPS) and issue resolution within the first contact. Multilevel modeling will account for nested data (interactions within agents and regional centers). Thematic analysis will be conducted on a subset of 60 interactions to extract pragmatic strategies related to politeness, stance-taking, and repair sequences, informing a cross-modal coding scheme. A structural equation model will be specified to examine indirect effects of visual and acoustic cues on perceived agent competence and trust, mediated by perceived clarity of information. The study will also perform cross-cultural comparisons to identify region-specific multimodal norms and their impact on outcomes. Key expected findings include (a) multimodal synchrony between agent gaze, gestures, and prosodic emphasis that significantly predicts shorter problem-resolution time and higher customer satisfaction; (b) domain-specific visual affordances (screen-sharing clarity, annotation use) positively linked to successful escalations and reduced need for follow-up contacts; (c) interaction modality moderates the weight of certain cues, with video sessions privileging non-verbal alignment over textual cues in high-complexity inquiries; and (d) cultural background influences interpretation of pragmatic cues, necessitating dynamic adaptation of communication strategies by agents. The anticipated contribution to knowledge lies in bridging multimodal pragmatics with practical performance metrics in global customer support, offering a validated model that integrates discourse pragmatics, multimodal resources, and service outcomes. The study will inform training programs emphasizing multimodal alignment and culturally sensitive communication, as well as interface and workflow design to optimize visual and interaction affordances. Practical recommendations will include guidelines for agent training modules, real-time assistive cues, and evaluation metrics that incorporate multimodal indicators alongside traditional satisfaction and resolution metrics. The main conclusion is that multimodal resources substantially shape pragmatic meaning negotiation and service outcomes in global tech support, and that deliberate cultivation of cross-modal alignment yields measurable improvements in efficiency, satisfaction, and perceived service quality.