Design and implementation of an adaptive real-time anomaly detection system for IoT networks

Thesis Abstract

The rapid proliferation of Internet of Things (IoT) devices in diverse sectors such as healthcare, industrial automation, and smart cities has heightened concerns over the security and reliability of IoT networks, particularly due to the increasing sophistication and frequency of cyberattacks and operational anomalies. These challenges necessitate the development of robust, adaptive, and real-time anomaly detection systems capable of identifying deviations in network behavior promptly to mitigate potential security breaches and system failures. This study aims to design, implement, and evaluate a novel adaptive real-time anomaly detection framework specifically tailored for IoT networks, emphasizing both accuracy and computational efficiency. The primary objectives include (1) analyzing the characteristics of typical IoT network data to identify relevant features for anomaly detection, (2) developing an adaptive detection model based on unsupervised machine learning algorithms such as clustering and density-based methods, combined with statistical analysis derived from the Theory of Change and Anomaly Detection framework, (3) implementing the model within a simulated IoT environment, and (4) evaluating the system's performance against benchmark datasets and real-world traffic under various attack scenarios. The research adopts a mixed-methods approach organized into three sequential phases. The first phase involves a qualitative assessment of IoT network data, utilizing thematic analysis to extract key features that influence anomaly detection efficacy. This is complemented by quantitative modeling, where the operational data collected from a sample of 500 interconnected IoT devices, representing diverse domains such as smart home, healthcare, and industrial sensors, is used. Data collection instruments include network traffic analyzers, intrusion detection logs, and custom scripts embedded within simulated environments created using Docker containers. The reliability and validity of collected data are ensured through calibration of sensors and validation against established intrusion datasets such as NSL-KDD and UNSW-NB15. In the analytical phase, the study employs statistical techniques including Principal Component Analysis (PCA) for feature reduction, K-means clustering, and Density-Based Spatial Clustering of Applications with Noise (DBSCAN) to identify anomalies dynamically. The model’s performance is evaluated using metrics such as precision, recall, F1-score, and computational latency; furthermore, the effectiveness is statistically tested through analysis of variance (ANOVA) to compare detection rates across different attack types and network conditions. Expected findings indicate that the proposed adaptive framework can dynamically adjust to evolving network conditions, achieving higher detection accuracy with reduced false positives compared to traditional static models. The integration of unsupervised clustering with statistical anomaly scores enables early identification of both known and unknown threats, significantly improving response times. The study also anticipates demonstrating the viability of using lightweight machine learning models suitable for resource-constrained IoT devices, thereby addressing a critical challenge in deployment scenarios. Furthermore, the research will contribute to the theoretical understanding of adaptive anomaly detection mechanisms by validating the application of the Theory of Change as a guiding framework for continual system evolution in response to emerging threats. The study’s contribution to knowledge lies in developing a scalable, adaptive anomaly detection system that advances current IoT security paradigms by combining statistical and machine learning techniques in a novel manner. It provides a formalized methodology for real-time detection that can be extended to various IoT settings, emphasizing robustness and efficiency. The main conclusion underscores the importance of adaptive, data-driven approaches for maintaining security in increasingly complex IoT networks. Recommendations include further research on deploying decentralized anomaly detection architectures, enhancing real-time processing capabilities through edge computing, and exploring federated learning for distributed model training. Future studies should also examine the integration of blockchain technology to ensure data integrity and enhance trustworthiness. Overall, this thesis aims to serve as a foundation for the development of resilient IoT infrastructures capable of detecting anomalies proactively, thereby enhancing the security and operational stability of interconnected systems.