Comparative Analysis of Edge AI Frameworks for IoT Inference Performance

Thesis Abstract

This study addresses the escalating demand for efficient on-device inference in Internet of Things (IoT) environments by systematically comparing contemporary edge AI frameworks in terms of inference latency, energy efficiency, and accuracy under constrained resources. The aim is to identify framework selections and configurations that best balance performance and resource utilization across diverse edge devices. Specific objectives include (1) benchmarking inference latency, memory footprint, and energy consumption of TensorFlow Lite, PyTorch Mobile, ONNX Runtime, and Edge Impulse on representative devices (Raspberry Pi 4, NVIDIA Jetson Nano, and Google Coral) across three model classes (image classification, anomaly detection, and voice command recognition); (2) evaluating robustness to intermittent connectivity and battery discharge; (3) examining the impact of model quantization, pruning, and hardware acceleration on accuracy and throughput; (4) assessing developer productivity and ecosystem maturity through a structured usability assessment guided by the Technology Acceptance Model (TAM). The study employs a mixed-methods design underpinned by the Resource-Based View and the Diffusion of Innovations theory to interpret framework adoption dynamics in constrained environments. A cross-sectional laboratory experiment will be conducted with a total sample of 120 device–framework configurations, using a factorial arrangement four frameworks × three devices × three model classes, with five repetitions per configuration to ensure statistical reliability. Data collection instruments include calibrated bench-top workloads for each model class, a high-precision power measurement setup (Watts Up Pro), and a standardized evaluation suite that records latency, frames-per-second, memory usage, and energy per inference. Validity and reliability will be established through pilot testing (n=12) and test–retest procedures, with instrument calibration traceable to IEEE power and performance standards. Data analysis will integrate frequentist and Bayesian approaches repeated-measures ANOVA to detect performance differences across frameworks and device types, multivariate regression to model energy–latency trade-offs, and non-parametric tests where normality is violated. Post-hoc analyses (Tukey HSD) will identify pairwise distinctions, while effect sizes (Cohen’s d) will quantify practical significance. Thematic analysis of developer surveys and interview notes will accompany quantitative results to illuminate usability and integration challenges, guided by grounded theory coding to distill emergent factors. Expected findings anticipate that modified runtimes with quantization and hardware acceleration will substantially reduce energy per inference while incurring modest accuracy losses (expecting <2% average drop for image classification with quantization to int8). It is anticipated that Edge TPU-accelerated devices will outperform CPU-bound configurations in latency and energy efficiency, whereas frameworks with broader operator coverage may yield higher baseline accuracy but at increased resource cost. The study aims to contribute to knowledge by providing a rigorous, device- and workload-specific comparative map of edge AI frameworks, highlighting trade-offs and actionable guidelines for practitioners in IoT deployment. The theoretical contribution includes an integrated framework combining Resource-Based View with Innovation Diffusion to explain framework selection under resource constraints, augmented by a practical model of energy–accuracy–latency trade-offs. The principal conclusion is expected to designate a prioritized framework–device–model class combination that optimizes throughput and energy efficiency for real-time IoT inference, with configurable strategies for quantization, pruning, and hardware acceleration. Recommendations will address framework selection criteria for varied industrial contexts, best practices for model optimization on resource-limited devices, and avenues for future work such as adaptive runtime optimization and broader ecosystem interoperability studies.