Comparative Analysis of Edge AI Frameworks on Embedded Platforms

Thesis Abstract

The rapid deployment of AI-enabled applications at the edge has exposed a gap in understanding how different Edge AI frameworks perform on diverse embedded platforms under real-world constraints such as limited memory, heterogeneous accelerators, and varying power envelopes. This study addresses the need for a rigorous, cross-framework evaluation to guide developers, researchers, and system integrators in selecting suitable frameworks for specific embedded deployments. The aim is to compare contemporary Edge AI frameworks—TensorFlow Lite Micro, PyTorch Mobile, and OpenVINO—across a representative set of embedded hardware platforms, with emphasis on inference efficiency, energy consumption, memory footprint, model versatility, and developer productivity. Specific objectives include (1) benchmarking end-to-end inference latency, throughput, and energy per inference for a suite of vision and sensor-based models (CNNs, MobileNets, and small transformer variants) on ARM Cortex-M and Cortex-A series devices; (2) assessing framework support for model quantization, pruning, and operator coverage, and their impact on accuracy and resource usage; (3) evaluating deployment-time and runtime overheads, including model conversion, runtime initialization, and multi-threading behavior; (4) analyzing developer usability and debugging experience through a mixed-methods approach combining objective task-based measurements with structured expert surveys; and (5) identifying best-fit scenarios and trade-offs for each framework under constrained power budgets and memory hierarchies. The methodology adopts a mixed-methods research design grounded in the Technology Acceptance Model and the Resource-Based View to assess technical performance and organizational practicality. The population consists of embedded development teams and representative edge devices, with a purposive sample of 30 developers and 12 embedded hardware platforms (including Raspberry Pi 4, Nvidia Jetson Nano, Jetson Xavier NX, and STM32H745-based boards) used for controlled experiments. Data collection employs quantitative instruments comprising standardized benchmarks (MLPerf Tiny, MINI-IMAGENET-Tiny, and a custom sensor fusion workload) and power measurement via high-precision shunt-based instrumentation, alongside qualitative data gathered through semi-structured interviews and think-aloud Protocols to capture developer experiences. Instrument validity and reliability are ensured through pilot testing, inter-rater reliability checks for qualitative coding (Cohen’s kappa > 0.75), and calibration of power measurement equipment. Data analysis utilizes repeated-measures ANOVA to compare framework performance across devices and workloads, multivariate regression to identify predictors of latency and energy consumption, and post-hoc Tukey tests for pairwise framework comparisons. Model-specific analyses include operator coverage assessments, quantization impact studies, and memory footprint profiling, complemented by a thematic analysis of developer feedback to elucidate usability factors and debugging efficiency. A conceptual model is developed to illustrate the relationships among framework features, hardware characteristics, workload types, and performance outcomes. Expected findings indicate that specialized lightweight frameworks with extensive quantization and operator suites (e.g., TensorFlow Lite Micro) provide superior memory efficiency and predictable latency on low-power microcontrollers, while OpenVINO demonstrates competitive performance on heterogeneous Intel- and ARM-based platforms with richer accelerator support, and PyTorch Mobile offers flexible development workflows at the expense of higher runtime overhead on constrained devices. The study contributes to knowledge by providing a comprehensive, empirically grounded framework-performance map for edge deployments, identifying critical bottlenecks in operator coverage and runtime initialization, and offering a decision guide aligned with device capabilities and application requirements. Implications for practice include recommendations for framework selection based on target hardware, model characteristics, and energy budgets, as well as guidelines for optimizing model compression and deployment pipelines. The main conclusion emphasizes that no single framework universally outperforms others; instead, optimal choices hinge on device class, workload type, and acceptable trade-offs between latency, energy, and development productivity. Practical recommendations encompass targeted optimizations in quantization strategies, operator fusion techniques, and tooling improvements to streamline model conversion and diagnostics, along with avenues for future research such as adaptive runtime scaling and context-aware framework selection for dynamic edge workloads.