AccScience Publishing / IJOCTA / Online First / DOI: 10.36922/IJOCTA026240119
Submit to IJOCTA
Cite this article
14
Download
214
Views
Related Info Links
Google Scholar
More by Authors Links
Mohab M. Eweda
Journal Browser
Volume | Year
Issue
Search
Forthcoming Issue
Current Issue
View All
News and Announcements
View All
RESEARCH ARTICLE

DDPG-based adaptive energy management of a PV–wind–BESS cold-ironing DC microgrid for sustainable port electrification

Mohab M. Eweda1* ,  Rana M. Ahmed1 ,  Mona I. Abdelkader1 ,  Ahmed K. Abdelsalam1
Show Less
1 Department of Electrical and Control Engineering, College of Engineering and Technology, Arab Academy for Science, Technology and Maritime Transport, Alexandria, Alexandria Governorate, Egypt
IJOCTA, 026240119 https://doi.org/10.36922/IJOCTA026240119
Received: 14 June 2026 | Revised: 29 July 2026 | Accepted: 3 August 2026 | Published online: 12 August 2026
© 2026 by the Author(s). This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution -Noncommercial 4.0 International License (CC-by the license) ( https://creativecommons.org/licenses/by-nc/4.0/ )
Download PDF
XML
Cite
Abstract

Cold ironing reduces ship emissions in ports, but reliable renewable-based shore power requires effective energy management. This paper presents a deep deterministic policy gradient (DDPG)-based energy management system (EMS) for a photovoltaic (PV)–wind–battery energy storage system (BESS) cold-ironing direct current (DC) microgrid designed to supply renewable shore power to berthed ships. The proposed system consists of 2 MW PV arrays, a 1 MW permanent magnet synchronous generator wind turbine, a 1 MWh BESS, a regulated 1,000 V DC bus, a shore-side power conversion unit, and a backup grid interface activated only in case of failure of the standalone renewable energy sources/BESS system. A complete dynamic model was developed in MATLAB/Simulink to evaluate system performance under normal operation, renewable-power shortage, excess-generation conditions, and realistic Alexandria Port meteorological data. A conventional rule-based EMS was first implemented as a baseline controller to coordinate renewable generation, battery charging/discharging, PV power limitation, and DC-bus regulation. Then, a DDPG-based EMS was developed to generate continuous battery-control actions and supervise PV and wind operating conditions under variable renewable generation. The results show that the proposed PV–wind–BESS system maintained the DC-bus voltage close to its 1,000 V reference while reliably supplying the cold-ironing load. Compared with the conventional EMS, the DDPG-based EMS provided smoother battery response, improved renewable-source coordination, and reduced voltage fluctuations during critical shortage and excess-generation cases. The Alexandria Port case study further validates the performance of the proposed DDPG-based EMS under realistic operating conditions for renewable-powered cold ironing in Mediterranean and North-African coastal ports.

Keywords
Cold ironing
Photovoltaic–wind–battery energy storage system
Deep deterministic policy gradient
Energy management system
Direct current microgrid
Battery energy storage system
Port electrification
Funding
None.
Conflict of interest
The authors declare that they have no conflict of interest.
References
  1. Canepa M, Ballini F, Dalaklis D, Frugone G, Sciutto D. Cold Ironing: Socio-Economic Analysis in the Port of Genoa. Logistics. 2023;7(2):28. doi: 10.3390/LOGISTICS7020028
  2. Spengler T, Tovar B. Potential of cold-ironing for the reduction of externalities from in-port shipping emissions: The state-owned Spanish port system case. J Environ Manage. 2021;279:111807. doi: 10.1016/J.JENVMAN.2020.111807
  3. Piccoli T, Fermeglia M, Bosich D, Bevilacqua P, Sulligoi G. Environmental Assessment and Regulatory Aspects of Cold Ironing Planning for a Maritime Route in the Adriatic Sea. Energies. 2021;14(18):5836. doi: 10.3390/EN14185836
  4. Yuksel O, Bayraktar M, Seyhan A. Environmental and economic analysis of cold ironing using renewable hybrid systems. Clean Technol Environ Policy. 2025;27(8):3489-3517. doi: 10.1007/S10098-024-03065-W
  5. Rolan A, Manteca P, Oktar R, Siano P. Integration of Cold Ironing and Renewable Sources in the Barcelona Smart Port. IEEE Trans Ind Appl. 2019;55(6):7198-7206. doi: 10.1109/TIA.2019.2910781
  6. Ghedamsi A, Mabrouk A, Bouaicha H, Belhadj J. PV power plant for Cold Ironing at the Goulette port. In: 2022 IEEE International Conference on Electrical Sciences and Technologies in Maghreb (CISTEM). New York, NY: IEEE; 2022:1-6. doi: 10.1109/CISTEM55808.2022.10044041
  7. Kumar J, Parthasarathy C, Västi M, Laaksonen H, Shafie-Khah M, Kauhaniemi K. Sizing and Allocation of Battery Energy Storage Systems in Åland Islands for Large-Scale Integration of Renewables and Electric Ferry Charging Stations. Energies. 2020;13(2):317. doi: 10.3390/en13020317
  8. Johnson SC, Papageorgiou DJ, Harper MR, Rhodes JD, Hanson K, Webber ME. The economic and reliability impacts of grid-scale storage in a high penetration renewable energy system. Adv Appl Energy. 2021;3:100052. doi: 10.1016/j.adapen.2021.100052
  9. Prenc R, Cuculić A, Baumgartner I. Advantages of using a DC power system on board ship. J Marit Transp Sci. 2016;52(1):83-97. doi: 10.18048/2016.52.05
  10. Alabdullah MH, Abido MA. Microgrid energy management using deep Q-network reinforcement learning. Alex Eng J. 2022;61(11):9069-9078. doi: 10.1016/J.AEJ.2022.02.042
  11. Ji Y, Wang J, Xu J, Fang X, Zhang H. Real-Time Energy Management of a Microgrid Using Deep Reinforcement Learning. Energies. 2019;12(12):2291. doi: 10.3390/EN12122291
  12. Guo C, Wang X, Zheng Y, Zhang F. Real-time optimal energy management of microgrid with uncertainties based on deep reinforcement learning. Energy. 2022;238:121873. doi: 10.1016/J.ENERGY.2021.121873
  13. Foruzan E, Soh LK, Asgarpoor S. Reinforcement Learning Approach for Optimal Distributed Energy Management in a Microgrid. IEEE Trans Power Syst. 2018;33(5):5749-5758. doi: 10.1109/TPWRS.2018.2823641
  14. Barbalho PIN, Moraes AL, Lacerda VA, Barra PHA, Fernandes RAS, Coury DV. Reinforcement Learning Solutions for Microgrid Control and Management: A Survey. IEEE Access. 2025;13:39782-39799. doi: 10.1109/ACCESS.2025.3546578
  15. Muriithi G, Chowdhury S. Optimal Energy Management of a Grid-Tied Solar PV-Battery Microgrid: A Reinforcement Learning Approach. Energies. 2021;14(9):2700. doi: 10.3390/EN14092700
  16. Du Y, Li F. Intelligent Multi-Microgrid Energy Management Based on Deep Neural Network and Model-Free Reinforcement Learning. IEEE Trans Smart Grid. 2020;11:1066-1076. doi: 10.1109/TSG.2019.2930299
  17. Hosseini E, Horrillo-Quintero P, Carrasco-Gonzalez D, et al. Reinforcement learning-based energy management system for lithium-ion battery storage in multilevel microgrid. J Energy Storage. 2025;109:115114. doi: 10.1016/J.EST.2024.115114
  18. Upadhyay S, Ahmed I, Mihet-Popa L. Energy Management System for an Industrial Microgrid Using Optimization Algorithms-Based Reinforcement Learning Technique. Energies. 2024;17(16):3898. doi: 10.3390/EN17163898
  19. Kumar K, Kwon S, Bae S. Deep reinforcement learning-based control strategy for integration of a hybrid energy storage system in microgrids. J Energy Storage. 2025;108:114936. doi: 10.1016/j.est.2024.114936
  20. Talab OA, Avci I. Energy Management in Microgrids Using Model-Free Deep Reinforcement Learning Approach. IEEE Access. 2025;13:5871-5891. doi: 10.1109/ACCESS.2025.3525843
  21. Villalva MG, Gazoli JR, Ruppert Filho E. Comprehensive approach to modeling and simulation of photovoltaic arrays. IEEE Trans Power Electron. 2009;24(5):1198-1208. doi: 10.1109/TPEL.2009.2013862
  22. Tian H, Mancilla-F David, Ellis K, Muljadi E, Jenkins P. A cell-to-module-to-array detailed model for photovoltaic panels. Sol Energy. 2012;86(9):2695-2706. doi: 10.1016/J.SOLENER.2012.06.004
  23. Qi C, Ming Z. Photovoltaic Module Simulink Model for a Stand-alone PV System. Phys Procedia. 2012;24:94-100. doi: 10.1016/J.PHPRO.2012.02.015
  24. Tsai HL, Tu CS, Su YJ. Development of Generalized Photovoltaic Model Using MATLAB/SIMULINK. In: Proceedings of the World Congress on Engineering and Computer Science 2008. Hong Kong: International Association of Engineers; 2008. Accessed May 16, 2026. https://www.iaeng.org/publication/WCECS2008/WCECS2008_pp846-851.pdf
  25. Qin L, Lu X. Matlab/Simulink-Based Research on Maximum Power Point Tracking of Photovoltaic Generation. Phys Procedia. 2012;24:10-18. doi: 10.1016/J.PHPRO.2012.02.003
  26. Hart DW. Power Electronics. New York, NY: The McGraw-Hill Companies, Inc; 2010.
  27. Ahmed R, Naaman A, M'Sirdi NK, Abdelsalam AK, Dessouky YG. Sensorless MPPT technique for PMSG micro wind turbines based on state-flow. In: 2014 International Conference on Renewable Energies for Developing Countries. New York, NY: IEEE; 2014:161-166. doi: 10.1109/REDEC.2014.7038550
  28. Al-Nhoud O, Al-Smairan M. Assessment of Wind Energy Potential as a Power Generation Source in the Azraq South, Northeast Badia, Jordan. Mod Mech Eng. 2015;5(3):87-96. doi: 10.4236/MME.2015.53008
  29. Koutroulis E, Kalaitzakis K. Design of a maximum power tracking system for wind-energy-conversion applications. IEEE Trans Ind Electron. 2006;53(2):486-494. doi: 10.1109/TIE.2006.870658
  30. Buticchi G, Lorenzani E, Immovilli F, Bianchini C. Active rectifier with integrated system control for microwind power systems. IEEE Trans Sustain Energy. 2015;6(1):60-69. doi: 10.1109/TSTE.2014.2356335
  31. MathWorks. Battery: Behavioral battery model. Simscape Battery Documentation. Accessed May 16, 2026. https://www.mathworks.com/help/simscape-battery/ref/battery.html
  32. MathWorks. Battery state of charge. MathWorks. Accessed May 16, 2026. https://www.mathworks.com/discovery/battery-state-of-charge.html
  33. Hauke B. Basic Calculation of a Boost Converter's Power Stage. Application report SLVA372D. Rev November 2022. Dallas, TX: Texas Instruments; 2022. Accessed May 16, 2026. https://www.ti.com/lit/an/slva372d/slva372d.pdf
  34. Lee J. Basic Calculation of a Buck Converter's Power Stage. Application Note AN041. December 2015. Taipei, Taiwan: Richtek Technology Corporation; 2015. Accessed May 16, 2026. https://www.richtek.com/Design%20Support/Technical%20Document/AN041
  35. International Electrotechnical Commission, Institute of Electrical and Electronics Engineers. IEC/IEEE 80005-1:2019. Utility connections in port–Part 1: High voltage shore connection (HVSC) systems—General requirements. Published March 2019. Accessed May 17, 2026. https://www.iso.org/standard/64717.html
  36. Bird J. Electrical Circuit Theory and Technology. 6th ed. Abingdon, UK: Routledge; 2017. doi: 10.4324/9781315561929
  37. Jamal S, Pasupuleti J, Ekanayake J. A rule-based energy management system for hybrid renewable energy sources with battery bank optimized by genetic algorithm optimization. Sci Rep. 2024;14(1):4865. doi: 10.1038/s41598-024-54333-0
  38. Ahmed R. Energy management and control for hybrid renewable energy sources in rural area. Dissertation. Aix-Marseille, France: Aix-Marseille University; 2015. doi: 10.70675/DA0CCD0BZEBC9Z4FECZ94A3Z6F292EA669F6
  39. Åström KJ, Murray RM. Feedback Systems: An Introduction for Scientists and Engineers. Princeton, NJ: Princeton University Press; 2010.
  40. Ogata K. Modern Control Engineering. 5th ed. Upper Saddle River, NJ: Prentice Hall; 2002.
  41. MathWorks. Rate Limiter: Limit rate of change of signal. Simulink Documentation. Accessed May 17, 2026. https://www.mathworks.com/help/simulink/slref/ratelimiter.html
  42. Sutton RS, Barto AG. Reinforcement Learning: An Introduction. 2nd ed. Cambridge, MA: Bradford Books; 2018.
  43. Silver D, Lever G, Heess N, Degris T, Wierstra D, Riedmiller M. Deterministic Policy Gradient Algorithms. Proceedings of the 31st International Conference on Machine Learning Research. 2014;32(1):387-395. https://proceedings.mlr.press/v32/silver14.html
  44. Lillicrap TP, Hunt JJ, Pritzel A, et al. Continuous control with deep reinforcement learning. Preprint posted online September 9, 2015. arXiv. doi: 10.48550/arXiv.1509.02971
  45. Wu H, Locment F, Sechilariu M. Experimental Implementation of a Flexible PV Power Control Mechanism in a DC Microgrid. Energies. 2019;12(7):1233. doi: 10.3390/EN12071233
  46. Goh HH, Huang Y, Lim CS, et al. An Assessment of Multistage Reward Function Design for Deep Reinforcement Learning-Based Microgrid Energy Management. IEEE Trans Smart Grid. 2022;13(6):4300-4311. doi: 10.1109/TSG.2022.3179567
  47. National Aeronautics and Space Administration. NASA POWER Data Access Viewer. NASA Prediction of Worldwide Energy Resource (POWER). Accessed August 7, 2026. https://power.larc.nasa.gov/data-access-viewer/
Share
Back to top
An International Journal of Optimization and Control: Theories & Applications, Electronic ISSN: 2146-5703 Print ISSN: 2146-0957, Published by AccScience Publishing
We use cookies on our website to ensure you get the best experience.
Read more about our cookies here
Accept