Comparative Analysis of Edge AI Frameworks on Latency and Energy Use

Thesis Abstract

Edge AI has emerged as a critical paradigm for deploying intelligent services with low latency and reduced cloud dependency. The study addresses the practical challenge of selecting appropriate edge AI frameworks by systematically comparing their performance in terms of latency and energy consumption under varying workload and hardware conditions. The aim is to provide evidence-based guidance for developers and researchers on framework choice, optimization opportunities, and deployment strategies for energy-constrained edge devices. Specific objectives are (1) to quantify and compare end-to-end latency across four widely adopted edge AI frameworks (TensorFlow Lite, PyTorch Mobile, ONNX Runtime, and Edge Impulse) on heterogeneous edge platforms; (2) to measure per-inference energy consumption and overall runtime energy profiles under CPU-only and heterogeneous accelerator configurations; (3) to analyze the sensitivity of latency and energy metrics to workload characteristics (model size, input resolution, batch size); (4) to identify framework-level and hardware-level optimization opportunities that yield the best latency-energy trade-offs; and (5) to develop a decision-support model integrating empirical results with a theoretical lens grounded in the resource-constrained mobile computing theory and the technology acceptance model for practitioner adoption. The methodology follows a mixed-methods research design combining experimental benchmarking and explanatory data analysis. The population comprises representative edge devices running contemporary autonomous inference workloads Raspberry Pi 4B with a 4-core ARM Cortex?A72, NVIDIA Jetson Nano, and Qualcomm Snapdragon 8c-based development board. A stratified sampling approach selects 20 configurations per framework, yielding a sample size of 80 experimental runs for strict comparability. Data collection instruments include calibrated power meters (WattsUp Pro) for precise per-inference energy measurements, high-resolution timers for latency capture (NTP-synchronized system clocks with 1 ms precision), and a standardized benchmark suite comprising image classification and object detection tasks using MobileNetV2, EfficientNet-lite, and tiny-YOLOv3. The study employs regression analysis and ANOVA to assess the effects of framework, hardware platform, model type, and input size on latency and energy, complemented by repeated-measures analysis where applicable. Validity and reliability are enhanced through pre-registered protocols, cross-validation of energy measurements, and inter-tool triangulation across hardware-instrument readings. Theoretical framing integrates the Shortest Path Theory of Edge Computing for latency modeling and the Energy-Delay Product (EDP) as a holistic efficiency metric, with reference to established theories such as the Resource-Based View for strategic framework selection and the Technology Acceptance Model to interpret practitioner adoption implications. Expected findings indicate that latency variability is dominated by framework overhead and hardware acceleration support, with ONNX Runtime and Edge Impulse showing favorable latency profiles on CPU-restricted devices, while TensorFlow Lite and PyTorch Mobile leverage hardware accelerators more effectively on Jetson Nano. Energy analyses are anticipated to reveal that tiny model configurations substantially reduce absolute energy per inference, yet larger models exhibit diminishing returns due to accelerator underutilization or suboptimal memory bandwidth, leading to non-linear energy-latency trade-offs. The interaction effects among framework, model, and hardware are expected to identify a subset of configurations that optimize the latency-energy envelope, informing a practical decision-support model for framework selection under given workload commitments and device constraints. The study contributes to knowledge by providing a rigorous, comparative, and context-aware evaluation of edge AI frameworks, integrating empirical benchmarking with theory-driven interpretation to elucidate how software frameworks interact with device heterogeneity to shape latency and energy outcomes. It offers actionable patterns and a decision-support framework that practitioners can employ to tailor edge inference deployments to specific application requirements, workloads, and energy budgets. The main conclusion envisaged is that no single framework universally outperforms others across all conditions; instead, optimal choices are contingent on the device platform, model complexity, and workload characteristics, underscoring the necessity of scenario-specific benchmarking. Recommendations include a framework-selection checklist for edge developers, guidelines for model optimization aligned with hardware accelerators, and avenues for future research to explore adaptive framework switching and dynamic energy-aware scheduling to sustain performance within tight power envelopes.