A Modular Energy-Efficient Edge AI Architecture Framework for IoT

Thesis Abstract

This study addresses the growing demand for intelligent, low-latency processing in Internet of Things (IoT) environments by proposing a modular energy-efficient edge AI architecture that enables dynamic deployment of machine learning workloads across heterogeneous edge nodes. The problem centers on the tension between limited power budgets, constrained bandwidth, and the need for real-time inference in ubiquitous IoT deployments, which often rely on cloud-centric strategies that incur latency and privacy risks. The aim is to develop a modular framework that (i) partitions AI workloads across micro edge, near-edge, and cloud layers, (ii) optimizes energy use through adaptive model compression, hardware-aware scheduling, and runtime reconfiguration, and (iii) provides a formalized methodology for evaluating trade-offs among latency, energy, and accuracy. Specific objectives include (1) to design a modular edge AI architecture with plug-and-play components for model partitioning, accelerators, and communication protocols; (2) to develop energy-aware scheduling algorithms and dynamic offloading policies validated on representative IoT scenarios; (3) to quantify the impact of hardware heterogeneity (CPU, GPU, ASIC, and FPGA accelerators) on end-to-end latency and energy consumption; (4) to propose a policy framework grounded in transfer learning and information theory to guide model selection and data routing; and (5) to validate the framework through empirical experiments and simulations that establish reproducible benchmarks. The methodology adopts a multi-method design anchored in a theoretical model of edge-AI co-design inspired by the Orbit Theory of distributed systems and Energy-Aware Scheduling (EAS) principles. A laboratory-based experimental testbed comprising 60 heterogeneous devices is used, including 20 resource-constrained micro edge devices (RISC-V and ARM Cortex-M), 20 near-edge servers with accelerators (NVIDIA Jetson modules), and 20 gateway/cloud nodes. A combination of synthetic and real-world data is collected, including 12 IoT workloads derived from smart industrial, smart city, and health-monitoring use cases, with a total dataset size of 1.2 million labeled instances. Data collection instruments include calibrated power meters, stack-level telemetry collectors, and open-source benchmarking suites (MLPerf Tiny for micro devices, and MLPerf Inference for edge servers). The data analysis employs a hierarchical mixed-methods approach (i) quantitative analyses using regression modeling, multivariate ANOVA, and non-parametric tests to evaluate latency, energy consumption, and accuracy across configurations; (ii) energy-proportionality and Pareto frontier assessments to identify optimal trade-offs; and (iii) qualitative observations using thematic analysis of system logs and operator notes to capture operational challenges. Model specification comprises an optimization framework for workload partitioning and scheduling, expressed as a mixed-integer linear program (MILP) with constraints for latency, energy, memory, and bandwidth, and augmented by a reinforcement learning (RL) policy that adapts to changing workloads. The theoretical contribution includes a formal modular architecture model with defined interfaces, a cost-utility function for energy-latency-accuracy trade-offs, and proofs of convergence for the proposed RL-based scheduler under standard ergodicity assumptions. Expected findings indicate that modularization with hardware-aware partitioning reduces total energy consumption by 28–45% relative to cloud-centric baselines, while maintaining or improving end-to-end latency within 100–250 ms for typical IoT inference tasks. The framework is anticipated to yield a scalable, reusable methodology that supports rapid deployment of edge AI across diverse IoT domains, with transferability of learned scheduling policies across similar hardware configurations. The study contributes to knowledge by operationalizing a theory-driven, modular edge AI architecture that harmonizes algorithmic efficiency, hardware heterogeneity, and network dynamics, and by delivering a validated methodology for evaluating energy-latency-accuracy trade-offs in real-world IoT ecosystems. Conclusions point to the efficacy of adaptive offloading and model compression strategies, with recommendations for standardizing benchmarking suites and extending the framework to include secure enclaves and privacy-preserving inference mechanisms. Further work suggested includes exploring federated learning across edge layers and integrating energy harvesting-aware scheduling to extend device lifetimes in remote deployments.