Comparative Analysis of Edge AI Frameworks on Embedded MCUs

Thesis Abstract

Edge AI has emerged as a pivotal approach for deploying intelligent processing directly on resource-constrained embedded microcontrollers (MCUs), yet the heterogeneity of frameworks and hardware constraints complicates selection for real-world deployments. This study investigates the comparative performance of leading edge AI frameworks—TensorFlow Lite for Microcontrollers (TFLM), PyTorch Mobile (via TorchScript on MCU-adjacent runtimes), and OpenVINO for microcontrollers—across representative embedded platforms (ARM Cortex-M55, RISC-V Microchip PolarFire, and ESP32-S3). The aim is to provide a rigorous, generalizable basis for framework selection under constrained compute, memory, and power budgets. The specific objectives are (i) to evaluate model accuracy, inference latency, throughput, and energy efficiency for a standardized vision and sensor fusion workload; (ii) to analyze framework-supported quantization, pruning, and compiler optimizations and their impact on performance and memory footprint; (iii) to examine portability and developer productivity through a structured usability assessment; and (iv) to model the trade-offs between accuracy and resource use using a multi-criteria decision framework. The research adopts a mixed-methods design anchored in empirical benchmarking complemented by expert elicitation. A cross-sectional population comprises embedded MCU development boards representative of contemporary edge devices (n=3 MCU families). A purposive sample of 12 pre-trained networks, including lightweight CNNs and small transformer variants, is deployed per framework, with three quantization schemes (float32, int8, int4) and two pruning levels to create 108 experimental configurations. Data collection instruments include a hardware-in-the-loop benchmarking suite capturing latency (ms), frames per second (FPS), memory footprint (KB), and energy per inference (µJ) under controlled ambient conditions. Additionally, a semi-structured interview protocol with 6 embedded-systems engineers assesses usability, integration effort, and developer productivity, while a standardized questionnaire evaluates perceived framework maturity and reliability across tasks. Validity and reliability are established through triangulation, pilot testing (n=6 benchmarks), and inter-rater reliability for qualitative coding (Cohen’s kappa = 0.82). Statistical analyses employ repeated-measures ANOVA to compare frameworks across hardware configurations, with post hoc Tukey tests to identify pairwise differences. Regression analysis investigates the relationship between model size, quantization level, and energy consumption, while a multi-criteria decision analysis (MCDA) using Analytic Hierarchy Process (AHP) synthesizes accuracy, latency, energy, and ease of use into a framework recommendation. Key expected findings include (i) quantization and pruning magnitudes differentially impact frameworks, with TFLM showing the most stable performance across quantization levels on Cortex-M55, whereas OpenVINO demonstrates stronger memory efficiency on ESP32-S3 with int8 models; (ii) inference latency and energy scales are more sensitive to memory bandwidth constraints than to raw compute, highlighting the critical role of memory optimization; (iii) UI and tooling maturity significantly influence developer productivity, with qualitative insights suggesting a preference for frameworks offering integrated model converters and debugging aids; and (iv) a quantifiable trade-off curve between accuracy and energy for each framework enabling a decision-maker to select configurations aligned with application-level requirements. The study contributes to knowledge by providing a rigorous, platform-agnostic benchmarking framework for edge-AI deployment on MCUs, clarifying how framework-specific optimizations interact with hardware constraints, and offering an evidence-based decision model for practitioners. The anticipated conclusion emphasizes that no single framework dominates across all devices and workloads; instead, performance is contingent on target hardware and workload characteristics, with a recommended pairing of TFLM on Cortex-M55 for compact vision tasks and OpenVINO-enabled configurations for sensor-fusion workloads on ESP32-S3 when energy efficiency is prioritized. Practical recommendations include guidelines for selecting quantization and pruning strategies per framework, best practices for model conversion pipelines, and a decision-support checklist to balance accuracy, latency, energy, and development effort in edge AI deployments.