LARO-IDS: Family-leakage-aware robust multi-objective optimization for model selection in IoT intrusion detection
Machine learning-based intrusion detection systems (IDS) are commonly selected based on conventional validation or development metrics, although such criteria may not sufficiently reflect robustness against unseen attack families or suitability for resource-constrained Internet of Things and edge environments. This study proposes learned acquisition and reconstruction optimization (LARO)—IDS (LARO-IDS), a family-leakage-aware robust multi-objective optimization framework for model selection in Internet of Things intrusion detection. Instead of selecting the model that only maximizes conventional predictive performance, LARO-IDS jointly considers development macro-F1, mean cross-family robustness, worst-family behavior, robustness variability, and prediction latency in the candidate-selection objective, while training time and model size are retained as additional deployment-cost indicators for final comparison. Candidate models were evaluated using a model-selection evaluation subset and a leave-one-attack-family-out robustness protocol, then ranked using a weighted-sum scalarization of normalized objectives, with the results further supported by Pareto-efficiency analysis. Experiments on the CICIoT2023 dataset show that conventional score-based selection favors RF_03_regularized, which achieved the highest macro-F1. In contrast, LARO-IDS selects RF_01_fast, which preserves nearly identical predictive performance, with only a −0.0015 macro-F1 difference, while achieving slightly higher mean cross-family F1 scores across attack families. The LARO-selected model also reduces training time by 49.49%, prediction latency by 46.36%, and model size by 50.12% compared with the conventionally selected model. Sensitive analysis of objective weights further shows that RF_01_fast remains selected under balanced, performance-priority, robustness-priority, and edge-priority scenarios. These results demonstrate that robust IDS model selection should integrate family-leakage-aware robustness and latency-aware deployment cost rather than relying solely on conventional predictive performance.

- Ahmad Z, Shahid Khan A, Wai Shiang C, Abdullah J, Ahmad F. Network intrusion detection system: A systematic study of machine learning and deep learning approaches. Trans Emerg Telecommun Technol. 2020;32(1). https://doi.org/10.1002/ett.4150
- Gyamfi E, Jurcut A. Intrusion Detection in Internet of Things Systems: A Review on Design Approaches Leveraging Multi-Access Edge Computing, Machine Learning, and Datasets. Sensors. 2022;22(10):3744. https://doi.org/10.3390/s22103744
- Alsaedi A, Moustafa N, Tari Z, Mahmood A, Anwar A. TON_IoT Telemetry Dataset: A New Generation Dataset of IoT and IIoT for Data-Driven Intrusion Detection Systems. IEEE Access. 2020;8:165130-165150. https://doi.org/10.1109/ACCESS.2020.3022862
- Ferrag MA, Friha O, Hamouda D, Maglaras L, Janicke H. Edge-IIoTset: A New Comprehensive Realistic Cyber Security Dataset of IoT and IIoT Applications for Centralized and Federated Learning. IEEE Access. 2022;10:40281-40306. https://doi.org/10.1109/ACCESS.2022.3165809
- Neto ECP, Dadkhah S, Ferreira R, Zohourian A, Lu R, Ghorbani AA. CICIoT2023: A Real-Time Dataset and Benchmark for Large-Scale Attacks in IoT Environment. Sensors. 2023;23(13):5941. https://doi.org/10.3390/s23135941
- Sommer R, Paxson V. Outside the Closed World: On Using Machine Learning for Network Intrusion Detection. In: 2010 IEEE Symposium on Security and Privacy. IEEE; 2010:305-316. https://doi.org/10.1109/SP.2010.25
- Apruzzese G, Pajola L, Conti M. The Cross-Evaluation of Machine Learning-Based Network Intrusion Detection Systems. IEEE Trans Netw Serv Manag. 2022;19(4):5152-5169. https://doi.org/10.1109/TNSM.2022.3157344
- Cantone M, Marrocco C, Bria A. Machine Learning in Network Intrusion Detection: A Cross-Dataset Generalization Study. IEEE Access. 2024;12:144489-144508. https://doi.org/10.1109/ACCESS.2024.3472907
- Shi W, Cao J, Zhang Q, Li Y, Xu L. Edge Computing: Vision and Challenges. IEEE Internet Things J. 2016;3(5):637-646. https://doi.org/10.1109/JIOT.2016.2579198
- Deng S, Zhao H, Fang W, Yin J, Dustdar S, Zomaya AY. Edge Intelligence: The Confluence of Edge Computing and Artificial Intelligence. IEEE Internet Things J. 2020;7(8):7457-7469. https://doi.org/10.1109/JIOT.2020.2984887
- Xu D, Li T, Li Y, et al. Edge Intelligence: Architectures, Challenges, and Applications. arXiv. Published online 2020. https://doi.org/10.48550/ARXIV.2003.12172
- Deb K. Multi-objective Optimisation Using Evolutionary Algorithms: An Introduction. In: Multi-Objective Evolutionary Optimisation for Product Design and Manufacturing. Springer London; 2011:3-34. https://doi.org/10.1007/978-0-85729-652-8_1
- Deb K, Pratap A, Agarwal S, Meyarivan T. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans Evol Comput. 2002;6(2):182-197. https://doi.org/10.1109/4235.996017
- Karl F, Pielok T, Moosbauer J, et al. Multi-Objective Hyperparameter Optimization in Machine Learning—An Overview. ACM Trans Evol Learn Optim. 2023;3(4):1-50. https://doi.org/10.1145/3610536
- Morales-Hernández A, Van Nieuwenhuyse I, Rojas Gonzalez S. A survey on multi-objective hyperparameter optimization algorithms for machine learning. Artif Intell Rev. 2022;56(8):8043-8093. https://doi.org/10.1007/s10462-022-10359-2
- Emmerich MTM, Deutz AH. A tutorial on multiobjective optimization: fundamentals and evolutionary methods. Nat Comput. 2018;17(3):585-609. https://doi.org/10.1007/s11047-018-9685-y
- Jin Y, Wang H, Chugh T, Guo D, Miettinen K. Data-Driven Evolutionary Optimization: An Overview and Case Studies. IEEE Trans Evol Comput. 2019;23(3):442-458. https://doi.org/10.1109/TEVC.2018.2869001
- AbdulJawad M. Comparative Analysis of Machine Learning Algorithms for Intrusion Detection in IoT Networks. IJASCA. 2025. https://doi.org/10.15849/ijasca.251130.15
- Breiman L. Random Forests. Mach Learn. 2001;45(1):5-32. https://doi.org/10.1023/A:1010933404324
- Geurts P, Ernst D, Wehenkel L. Extremely randomized trees. Mach Learn. 2006;63(1):3-42. https://doi.org/10.1007/s10994-006-6226-1
- Chen T, Guestrin C. XGBoost. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM; 2016:785-794. https://doi.org/10.1145/2939672.2939785
- Ke G, Meng Q, Finley T, et al. LightGBM: A highly efficient gradient boosting decision tree. Adv Neural Inf Process Syst. 2017;30.
- Verma A, Ranga V. Machine Learning Based Intrusion Detection Systems for IoT Applications. Wireless Pers Commun. 2019;111(4):2287-2310. https://doi.org/10.1007/s11277-019-06986-8
- Rahman MM, Shakil SA, Mustakim MR. A survey on intrusion detection system in IoT networks. Cyber Secur Appl. 2025;3:100082. https://doi.org/10.1016/j.csa.2024.100082
- Ring M, Wunderlich S, Scheuring D, Landes D, Hotho A. A survey of network-based intrusion detection data sets. Comput Secur. 2019;86:147-167. https://doi.org/10.1016/j.cose.2019.06.005
- Koroniotis N, Moustafa N, Sitnikova E, Turnbull B. Towards the development of realistic botnet dataset in the Internet of Things for network forensic analytics: Bot-IoT dataset. Future Gener Comput Syst. 2019;100:779-796. https://doi.org/10.1016/j.future.2019.05.041
- Moustafa N, Slay J. UNSW-NB15: a comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set). In: 2015 Military Communications and Information Systems Conference (MilCIS). IEEE; 2015:1-6. https://doi.org/10.1109/MilCIS.2015.7348942
- Hnaif A, Jaber KM, Alia MA, Daghbosheh M. Parallel scalable approximate matching algorithm for network intrusion detection systems. Int Arab J Inf Technol. 2021;18(1):77-84. https://doi.org/10.34028/iajit/18/1/9
- Tiwari RS, Lakshmi D, Das TK, Tripathy AK, Li KC. A lightweight optimized intrusion detection system using machine learning for edge-based IIoT security. Telecommun Syst. 2024;87(3):605-624. https://doi.org/10.1007/s11235-024-01200-y
- Gao W, Wang M, Pei Y, Li F, Wang C. A Lightweight Multi-Classification Intrusion Detection Model for Edge IoT Networks. Electronics. 2026;15(5):938. https://doi.org/10.3390/electronics15050938
- Choudhary T, Mishra V, Goswami A, Sarangapani J. A comprehensive survey on model compression and acceleration. Artif Intell Rev. 2020;53(7):5113-5155. https://doi.org/10.1007/s10462-020-09816-7
- Di Mauro M, Galatro G, Fortino G, Liotta A. Supervised feature selection techniques in network intrusion detection: A critical review. Eng Appl Artif Intell. 2021;101:104216. https://doi.org/10.1016/j.engappai.2021.104216
- Bergstra J, Bengio Y. Random search for hyper-parameter optimization. J Mach Learn Res. 2012;13(10):281-305.
- Akiba T, Sano S, Yanase T, Ohta T, Koyama M. Optuna. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM; 2019:2623-2631. https://doi.org/10.1145/3292500.3330701
- Borchert O, Salinas D, Flunkert V, Januschowski T, Günnemann S. Multi-Objective Model Selection for Time Series Forecasting. arXiv. Published online 2022. https://doi.org/10.48550/ARXIV.2202.08485
- Williams P, Kendall W, Hooten M. Model Selection using Multi-Objective Optimization. arXiv. Published online 2018. https://doi.org/10.48550/arXiv.1810.10669
- Sezgin A, Ulaş M, Boyacı A. Multi-Objective Feature Selection for Intrusion Detection Systems: A Comparative Analysis of Bio-Inspired Optimization Algorithms. Sensors. 2025;25(19):6099. https://doi.org/10.3390/s25196099
- Marler RT, Arora JS. The weighted sum method for multi-objective optimization: new insights. Struct Multidisc Optim. 2009;41(6):853-862. https://doi.org/10.1007/s00158-009-0460-7
- Pedregosa F, Varoquaux G, Gramfort A, et al. Scikit-learn: Machine learning in Python. J Mach Learn Res. 2011;12:825-2830.
- Kamalov F, Moussa S, Zgheib R, Mashaal O. Feature selection for intrusion detection systems. In: 2020 13th International Symposium on Computational Intelligence and Design (ISCID). IEEE; 2020:265-269. https://doi.org/10.1109/ISCID51228.2020.00065
