Comparative Analysis of Embedded AI Accelerators for Edge Devices

Thesis Abstract

This study addresses the growing demand for energy-efficient, low-latency AI processing on edge devices by comparing embedded AI accelerators across representative workloads in computer vision and sensor analytics. The research aims to identify how accelerator architecture, memory hierarchy, and precision strategies influence throughput, latency, and energy efficiency under real-world edge scenarios. Specific objectives include (1) benchmarking inference latency, peak throughput, energy per inference, and thermal behavior across four embedded accelerators (TensorRT-based NPU, Qualcomm Hexagon, Movidius Myriad X, and Nvidia Jetson Xavier NX) using standardized CV and sensor fusion tasks; (2) evaluating accuracy- and precision- management techniques (INT8, FP16, and mixed precision) and their impact on model performance; (3) examining the robustness of accelerators under varying ambient temperatures and constrained power budgets; (4) developing a comparative multi-criteria decision framework to guide hardware-software co-design for edge AI; and (5) deriving practical recommendations for developers and hardware vendors. The study integrates theoretical perspectives from the Technology Acceptance Model and Resource-Based View to interpret performance trade-offs in deployment contexts. A mixed-methods approach combines quantitative benchmarking with qualitative insights from hardware- and firmware-level documentation analysis. The population comprises edge AI deployments using convolutional neural networks and lightweight recurrent models suitable for real-time inference on resource-constrained devices. A statistically determined sample of 80 benchmark runs per accelerator, across four workloads and four power envelopes, ensures robust comparative inference, with replication across three ambient temperature conditions (25°C, 35°C, 45°C). Data collection employs instrumented benchmarking suites capturing latency, frames per second, energy per inference, memory bandwidth utilization, and peak thermal throttling events, supplemented by vendor-provided performance counters and thermal sensors. Validity and reliability are established through pre-registered benchmarking protocols, calibration runs, and inter-device reproducibility checks. Data analysis applies repeated-measures ANOVA to detect significant differences in performance metrics across accelerators and workloads, Tukey post hoc tests for pairwise comparisons, and multivariate regression to model energy efficiency as a function of model precision, batch size, and workload type. A Bayesian hierarchical model will characterize uncertainty across devices and operating conditions. The conceptual framework is anchored in the Priestley–Tsang theory of embodied cognition for hardware-software co-design and the Structured Innovation Theory to explain technology orchestration in edge ecosystems. Expected findings include (i) quantifiable performance gaps between accelerators in latency and energy per inference, magnified under high-resolution CV tasks and thermal throttling; (ii) consistent advantages for mixed-precision strategies in resource-limited regimes; (iii) trade-offs between model fidelity and hardware efficiency that inform task-specific accelerator selection; and (iv) a validated decision framework enabling practitioners to select accelerators aligned with application accuracy, latency, and energy constraints. The study contributes to knowledge by delivering a comprehensive, empirically grounded comparison of leading embedded AI accelerators, identifying architecture- and precision-driven determinants of edge AI efficiency, and proposing a replicable benchmarking methodology that can be adopted by researchers and practitioners. The main conclusion anticipates that no single accelerator dominates across all tasks; instead, performance is governed by workload characteristics, thermal environments, and precision choices, necessitating context-aware selection and co-design. Practical recommendations include guidelines for choosing accelerators based on application domain (real-time CV versus sensor analytics), recommended precision profiles to balance accuracy and energy consumption, and a framework for ongoing benchmarking to support hardware refresh cycles in edge deployments. Further studies are suggested to investigate long-term reliability under field conditions, broader workload families including transformer-based edge models, and the integration of dynamic power management strategies.