Design, implementation and evaluation of a morphological analyzer for Uyghur texts using neural sequence tagging

Thesis Abstract

This study addresses the persistent scarcity of robust computational tools for Uyghur morphology by developing a neural sequence tagging-based morphological analyzer tailored to Uyghur agglutinative morphology and rich derivational processes. The aim is to design, implement, and evaluate a scalable analyzer capable of segmenting, lemmatizing, and labeling morphemes within continuous text, thereby improving downstream natural language processing tasks such as parsing, machine translation, and information retrieval. Specific objectives are to (i) curate a diverse Uyghur morphologically annotated corpus, (ii) design a neural architecture combining BiLSTM-CRF with character-level CNN encoders and attention mechanisms to capture rich affixal patterns, (iii) implement a modular pipeline for tokenization, segmentation, POS/morphological tagging, and lemmatization, (iv) evaluate the model against baseline deterministic and data-driven approaches, and (v) analyze error patterns to inform model refinements and resource development. The study adopts a design-based research approach within a quasi-experimental framework, integrating iterative prototyping and evaluation cycles. The research design is cross-sectional and corpus-based, drawing on a compiled Uyghur corpus consisting of 1.2 million tokens drawn from news outlets, literary texts, and conversational transcripts, with a stratified sampling of 200,000 tokens for the annotated gold standard. Data collection instruments include a hybrid annotation schema informed by existing Uyghur morphological grammars and a panel of three linguists for inter-annotator reliability, achieving a Cohen’s kappa of at least 0.82 on a 10,000-token calibration set. The neural model employs a multi-task learning objective that jointly optimizes segmentation, morpheme labeling, and lemmatization, integrating a BiLSTM-CRF backbone, character-level convolutional encoders to capture root and affix patterns, and an attention layer to align morpheme boundaries with context. Training uses a 70/15/15 train/validation/test split, with early stopping on validation perplexity and a dropout rate of 0.3 to mitigate overfitting. Regularization and hyperparameter tuning follow a grid search over learning rate (1e-4 to 5e-4), hidden state sizes (256–512), and CNN kernel sizes (3–5). Analytical techniques include sequence tagging evaluation using precision, recall, and F1 at the morpheme and word levels, with ablation studies to quantify the contribution of submodules. Comparative analyses employ Likert-scale usability assessments by NLP practitioners to gauge integration feasibility in downstream tasks. Statistical significance of improvements over baselines (e.g., rule-based morphological analyzers and vanilla BiLSTM-CRF models) is tested using paired bootstrap resampling (10,000 iterations) and McNemar tests for categorical labeling decisions. Error analysis follows a thematic categorization of missegmentation, incorrect POS/morphological labels, and lemma mismatches, with a subsequent refinement cycle to address high-impact error classes such as multi-derivational morphemes and loanword adaptations. Expected findings indicate substantial improvements in morpheme boundary detection (F1 increment of 8–12 points over baselines) and labeling accuracy (lemma accuracy gains of 6–9 points) especially in derivational morphology and complex agglutination sequences. The attention mechanism is anticipated to improve long-distance morpheme-lemma alignments, while character-level encoders are expected to enhance handling of rare affixes. The study anticipates identifying persistent error domains—such as ambiguous homographs and irregular stem changes—guiding targeted data augmentation and rule-informed constraints. Contributions to knowledge include a rigorously evaluated neural morphological analyzer for Uyghur that integrates morphological theory with neural sequence tagging, a publicly available annotated Uyghur morphological lexicon, and a transferable architectural blueprint for other Turkic languages exhibiting rich agglutinative morphology. The work advances theoretical understanding of neural approaches to morpho-syntactic analysis in under-resourced languages and provides practical tools to improve downstream NLP applications, including machine translation and information retrieval. The conclusion posits that a hybrid neural architecture with explicit morpho-orthographic constraints, validated by robust evaluation, yields reliable morphological analyses and supports reproducible, scalable deployment. Recommendations emphasize expanding cross-dialectal corpora, incorporating unsupervised pretraining on Uyghur texts, and integrating the analyzer with open-source NLP pipelines to promote broader accessibility and impact.