Empirical Evaluation of Edge AI for Real-Time IoT Networks

Thesis Abstract

The rapid proliferation of Internet of Things (IoT) devices and the demand for real-time processing have intensified the need for on-device intelligence, yet existing cloud-centric paradigms introduce latency, privacy, and bandwidth challenges that degrade performance in time-sensitive applications. This study addresses the gap by empirically evaluating Edge AI architectures for real-time IoT networks, focusing on deployment feasibility, inference latency, energy efficiency, and accuracy under heterogeneous workloads. The aim is to determine how edge-enabled AI inference can meet stringent latency requirements while preserving device energy budgets and ensuring robust performance across diverse IoT scenarios. The objectives are (i) to compare on-device and edge-assisted inference pipelines across multiple neural network models suitable for sensor fusion, activity recognition, and anomaly detection; (ii) to quantify latency, energy consumption, and throughput under varying network conditions and workload distributions; (iii) to assess model compression, quantization, and hardware accelerators on edge devices in terms of accuracy vs. resource trade-offs; (iv) to evaluate the resilience of Edge AI systems to intermittent connectivity and environmental noise; and (v) to develop a practical benchmarking framework and guidelines for deploying Edge AI in real-time IoT networks. The methodology adopts an empirical, field-based research design with a mixed-methods approach. The population comprises IoT deployments in a smart manufacturing and vulnerable-asset monitoring scenario, incorporating sensors, actuators, gateways, and edge servers built on ARM-based devices and NVIDIA Jetson platforms. A stratified sampling strategy yields a total of 60 IoT devices distributed across three sites, with 20 devices per site, ensuring representation of varying sensor modalities and data rates. Data collection instruments include standardized data suites for sensor streams (accelerometer, gyroscope, temperature, vibration), real-time annotation tools for ground-truth labels, and system-level telemetry collectors to capture latency, CPU/GPU utilization, memory footprint, and energy consumption. The study employs a two-phase data collection (i) baseline measurements using full-precision models executed in the cloud and on-device as controls, and (ii) edge-augmented pipelines with compressed/quantized models and hardware accelerators, tested under three workload regimes (low, moderate, high) and two connectivity conditions (stable local network and intermittent wireless link). Analytical techniques include descriptive statistics to summarize performance metrics, repeated-measures ANOVA to examine latency and energy differences across configurations, and regression analysis to model the relationship between model size, compression ratio, and inference accuracy. Model performance is evaluated using standard metrics—accuracy, F1-score, precision, recall—while latency is measured as end-to-end inference time. Energy efficiency is analyzed via Joules per inference and energy-delay product. A Bayesian hierarchical model is employed to account for site-level heterogeneity and device-specific effects. Theoretical grounding draws on the Edge Computing paradigm and the Federated Learning and Transfer Learning theories to interpret knowledge sharing and generalization across heterogeneous devices, supplemented by the Resource-Based View to explain capability-driven performance in constrained environments. A validation phase includes ablation studies on compression techniques (quantization levels, pruning) and the impact of hardware accelerators (CUDA-enabled GPUs, NPU units) on predictive performance. Expected findings indicate that edge-augmented inference reduces end-to-end latency by 40–70% compared with cloud-only configurations, with energy savings ranging from 25–60% depending on model complexity and device capability. Moderate compression coupled with hardware acceleration is anticipated to yield near-baseline accuracy (within 2–5 percentage points) while meeting real-time constraints (sub-100 ms latency) under most workloads. The study also expects that intermittent connectivity penalizes cloud-reliant strategies more severely than edge-based approaches, underscoring the resilience of Edge AI in real-world deployments. The contribution to knowledge lies in providing an empirically validated framework for evaluating Edge AI in real-time IoT networks, offering practical benchmarks, optimization guidelines, and an evidence-based model for predicting performance under diverse operational conditions. The study informs designers and operators about optimal model-precision trade-offs, accelerator choices, and deployment architectures, contributing to theory by integrating Edge Computing, Federated/Transfer Learning, and resource-based perspectives into a cohesive empirical assessment. It concludes with actionable recommendations for deployment planning, standardization of benchmarking protocols, and directions for future research on adaptive edge orchestration and privacy-preserving inference.