Submit to Algorithms Review for Algorithms Propose a Special Issue

Journal Browser

Reinforcement Learning and Its Applications in Modern Power and Energy Systems

Print Special Issue Flyer
Special Issue Editors
Special Issue Information
Keywords
Published Papers

A special issue of Algorithms (ISSN 1999-4893). This special issue belongs to the section "Evolutionary Algorithms and Machine Learning".

Deadline for manuscript submissions: closed (30 September 2023) | Viewed by 13880

Share This Special Issue

Special Issue Editors

Dr. Van-Hai Bui

E-Mail Website
Guest Editor

Department of Electrical Engineering, School of Engineering, State University of New York (SUNY), Maritime College, 6 Pennyfield Avenue, Throggs Neck, New York, NY 10465, USA
Interests: deep learning; deep reinforcement learning; distributed energy resource integration; energy management system; operation and control of microgrid; optimization
Special Issues, Collections and Topics in MDPI journals

Dr. Sina Zarrabian

E-Mail Website
Guest Editor

Department of Electrical Engineering, School of Engineering, State University of New York (SUNY), Maritime College, 6 Pennyfield Avenue, Throggs Neck, NY 10465, USA
Interests: wide-area monitoring and control of power systems; applications of artificial intelligence and machine learning in power systems modernization; smart grids; microgrids

Dr. Paul Kump

E-Mail Website
Guest Editor

Department of Electrical Engineering, School of Engineering, State University of New York (SUNY), Maritime College, 6 Pennyfield Avenue, Throggs Neck, NY 10465, USA
Interests: internet of things; machine learning; signal processing; alternative teaching strategies; electronic music

Special Issue Information

Dear Colleagues,

It is our pleasure to invite submissions to the Special Issue on “Reinforcement Learning and Its Applications in Modern Power and Energy Systems”.

Power and energy systems undergo major transitions to facilitate the large-scale penetration of distributed energy resources, such as photovoltaic, wind energy, and other emerging technologies. These transitions significantly increase the complexity and uncertainty in the operation of power and energy systems (PESs). This brings great challenges to optimally operating and controlling PESs using existing techniques based on physical models. With the rapid development of advanced sensors and smart meters, huge amounts of data can be collected, which brings opportunities for novel data-driven methods to deal with complicated operation and control issues in modern power and energy systems. Additionally, combining deep learning and reinforcement learning (RL) to form deep reinforcement learning (DRL) has overcome many inherent disadvantages of conventional RL algorithms. In recent years, DRL has been gaining considerable attention in many fields and has become one of the most widely promoted methods for control and optimization problems. In this Special Issue, we are looking for novel methods, algorithms, and technologies using reinforcement learning algorithms to enhance energy efficiency for the operation and control of power and energy systems. Review and survey articles on the following topics are encouraged for submission.

Topics of interest for publication include, but are not limited to, the following:

Applications of artificial intelligence (AI) in operation and control of power and energy systems;
Building energy management systems;
Charging stations with DRL;
Decentralized and distributed operation and control of power and energy systems;
Energy management systems;
Integration of renewable energy sources and mobile loads;
Operation and control power and energy systems;
Peer-to-Peer energy trading in power systems;
Novel RL/DRL algorithms and applications in power and energy systems;
Multi-energy system with combined cooling, heat, and power;
Multiagent deep reinforcement learning applications.

Dr. Van-Hai Bui
Dr. Sina Zarrabian
Dr. Paul M. Kump
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Algorithms is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 1600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

artificial intelligence
building energy management systems
combined cooling, heat, and power systems
deep reinforcement learning
deep learning
distributed energy resources
distributed operation and control
double auction
game theory
energy management systems
machine learning algorithms
multi-agent reinforcement learning
microgrids
muti-energy system
multiagent system
optimization
optimal energy trading
smart grid

Published Papers (6 papers)

Download All Papers

Research

Jump to: Review

27 pages, 9036 KiB

Open AccessArticle

A Novel Deep Reinforcement Learning (DRL) Algorithm to Apply Artificial Intelligence-Based Maintenance in Electrolysers

by Abiodun Abiola, Francisca Segura Manzano and José Manuel Andújar

Algorithms 2023, 16(12), 541; https://0-doi-org.brum.beds.ac.uk/10.3390/a16120541 - 27 Nov 2023

Viewed by 1660

Abstract

Hydrogen provides a clean source of energy that can be produced with the aid of electrolysers. For electrolysers to operate cost-effectively and safely, it is necessary to define an appropriate maintenance strategy. Predictive maintenance is one of such strategies but often relies on data from sensors which can also become faulty, resulting in false information. Consequently, maintenance will not be performed at the right time and failure will occur. To address this problem, the artificial intelligence concept is applied to make predictions on sensor readings based on data obtained from another instrument within the process. In this study, a novel algorithm is developed using Deep Reinforcement Learning (DRL) to select the best feature(s) among measured data of the electrolyser, which can best predict the target sensor data for predictive maintenance. The features are used as input into a type of deep neural network called long short-term memory (LSTM) to make predictions. The DLR developed has been compared with those found in literatures within the scope of this study. The results have been excellent and, in fact, have produced the best scores. Specifically, its correlation coefficient with the target variable was practically total (0.99). Likewise, the root-mean-square error (RMSE) between the experimental sensor data and the predicted variable was only 0.1351. Full article

(This article belongs to the Special Issue Reinforcement Learning and Its Applications in Modern Power and Energy Systems)

► Show Figures

Figure 1

22 pages, 395 KiB

Open AccessArticle

Optimal Siting and Sizing of FACTS in Distribution Networks Using the Black Widow Algorithm

by Nicolas Santamaria-Henao, Oscar Danilo Montoya and César Leonardo Trujillo-Rodríguez

Algorithms 2023, 16(5), 225; https://0-doi-org.brum.beds.ac.uk/10.3390/a16050225 - 27 Apr 2023

Cited by 5 | Viewed by 1443

Abstract

The problem regarding the optimal placement and sizing of different FACTS (flexible alternating current transmission systems) in electrical distribution networks is addressed in this research by applying a master–slave optimization approach. The FACTS analyzed correspond to the unified power flow controller (UPFC), the thyristor-controlled shunt compensator (TCSC, also known as the thyristor switched capacitor, or TSC), and the static var compensator (SVC). The master stage is entrusted with defining the location and size of each FACTS device using hybrid discrete-continuous codification through the application of the black widow optimization (BWO) approach. The slave stage corresponds to the successive approximations power flow method based on the admittance grid formulation, which allows determining the expected costs of the energy losses for a one-year operation period. The numerical results in the IEEE 33-, 69-, and 85-bus grids demonstrate that the best FACTS device for locating in distribution networks is the SVC, given that, when compared to the UPFC and the TCSC, it allows for the best possible reduction in the equivalent annual investment and operating cost. A comparative analysis with the General Algebraic Modeling System software, with the aim to solve the exact mixed-integer nonlinear programming model, demonstrated the proposed BWO approach’s effectiveness in determining the best location and size for the FACTS in radial distribution networks. Reductions of about

12.63 %

and

13.97 %

concerning the benchmark cases confirmed that the SVC is the best option for reactive power compensation in distribution grids. Full article

(This article belongs to the Special Issue Reinforcement Learning and Its Applications in Modern Power and Energy Systems)

► Show Figures

Figure 1

20 pages, 373 KiB

Open AccessArticle

Optimal Integration of D-STATCOMs in Radial and Meshed Distribution Networks Using a MATLAB-GAMS Interface

by German Francisco Barreto-Parra, Brandon Cortés-Caicedo and Oscar Danilo Montoya

Algorithms 2023, 16(3), 138; https://0-doi-org.brum.beds.ac.uk/10.3390/a16030138 - 4 Mar 2023

Cited by 5 | Viewed by 1323

Abstract

This paper proposes an interconnection of the MATLAB and GAMS software interfaces, which were designed based on a master-slave methodology, to solve the mixed-integer nonlinear programming (MINLP) model problem associated with the problem regarding the optimal location and sizing of static distribution compensators (D-STATCOMs) in meshed and radial distribution networks, considering the problem of optimal reactive power flow compensation and the fact that the networks have commercial, industrial, and residential loads for a daily operation scenario. The objective of this study is to reduce the annual investment and operating costs associated with energy losses and the installation costs of D-STATCOMs. This objective function is based on the classical energy budget and the capacity constraints of the device. In the master stage, MATLAB software is used to program a discrete version of the sine-cosine algorithm (DSCA), which determines the locations where the D-STATCOMs will be installed. In the slave stage, using the BONMIN solver of the GAMS software and the known locations of the D-STATCOMs, the MINLP model representing the problem under study is solved to find the value of the objective function and the nominal power of the D-STATCOMs. To validate the effectiveness of the proposed master-slave optimizer, the 33-node IEEE test system with both radial and meshed topologies is used. With this test system, numerical comparisons were made with the exact solution of the MINLP model, using different solvers in the GAMS software, the genetic-convex strategy, and the discrete-continuous versions of the Chu and Beasley genetic algorithm and the salp swarm optimization algorithm. The numerical results show that DSCA-BONMIN achieves a global solution to the problem under study, making the proposed method an effective tool for decision-making in distribution companies. Full article

(This article belongs to the Special Issue Reinforcement Learning and Its Applications in Modern Power and Energy Systems)

► Show Figures

Figure 1

26 pages, 3109 KiB

Open AccessArticle

Deep Reinforcement Learning-Based Dynamic Pricing for Parking Solutions

by Li Zhe Poh, Tee Connie, Thian Song Ong and Michael Kah Ong Goh

Algorithms 2023, 16(1), 32; https://0-doi-org.brum.beds.ac.uk/10.3390/a16010032 - 5 Jan 2023

Cited by 5 | Viewed by 3202

Abstract

The growth in the number of automobiles in metropolitan areas has drawn attention to the need for more efficient carpark control in public spaces such as healthcare, retail stores, and office blocks. In this research, dynamic pricing is integrated with real-time parking data to optimise parking utilisation and reduce traffic jams. Dynamic pricing is the practice of changing the price of a product or service in response to market trends. This approach has the potential to manage car traffic in the parking space during peak and off-peak hours. The dynamic pricing method can set the parking fee at a greater price during peak hours and a lower rate during off-peak times. A method called deep reinforcement learning-based dynamic pricing (DRL-DP) is proposed in this paper. Dynamic pricing is separated into episodes and shifted back and forth on an hourly basis. Parking utilisation rates and profits are viewed as incentives for pricing control. The simulation output illustrates that the proposed solution is credible and effective under circumstances where the parking market around the parking area is competitive among each parking provider. Full article

(This article belongs to the Special Issue Reinforcement Learning and Its Applications in Modern Power and Energy Systems)

► Show Figures

Figure 1

18 pages, 340 KiB

Open AccessArticle

Optimal Reactive Power Compensation via D-STATCOMs in Electrical Distribution Systems by Applying the Generalized Normal Distribution Optimizer

by Laura Patricia García-Pineda and Oscar Danilo Montoya

Algorithms 2023, 16(1), 29; https://0-doi-org.brum.beds.ac.uk/10.3390/a16010029 - 3 Jan 2023

Cited by 3 | Viewed by 2140

Abstract

This research deals with the problem regarding the optimal siting and sizing of distribution static compensators (D-STATCOMs) via the application of a master–slave optimization technique. The master stage determines the nodes where the D-STATCOMs must be located and their nominal rates by applying the generalized normal distribution optimizer (GNDO) with a discrete–continuous codification. In the slave stage, the successive approximations power flow method is implemented in order to establish the technical feasibility of the solution provided by the master stage, i.e., voltage regulation and device capabilities, among other features. The main goal of the proposed master–slave optimizer is to minimize the expected annual operating costs of the distribution grid, which includes the energy loss and investment costs of the D-STATCOMs. With the purpose of improving the effectiveness of reactive power compensation during the daily operation of the distribution grid, an optimal reactive power flow (ORPF) approach is used that considers the nodes where D-STATCOMs are located as inputs in order to obtain their daily expected dynamical behavior with regard to reactive power injection to obtain additional net profits. The GNDO approach and the power flow method are implemented in the MATLAB programming environment, and the ORPF approach is implemented in the GAMS software using a test feeder comprising 33 nodes with both radial and meshed configurations. A complete comparative analysis with the Salp Swarm Algorithm is presented in order to demonstrate the effectiveness of the proposed two-stage optimization approach in the fixed operation scenario regarding the final objective function values. In addition, different tests considering the possibility of hourly power injection using D-STATCOMs through the ORPF solution demonstrate that additional gains can be obtained in the expected annual operative costs of the grid. Full article

(This article belongs to the Special Issue Reinforcement Learning and Its Applications in Modern Power and Energy Systems)

► Show Figures

Figure 1

Review

Jump to: Research

24 pages, 972 KiB

Open AccessReview

Emerging 6G/B6G Wireless Communication for the Power Infrastructure in Smart Cities: Innovations, Challenges, and Future Perspectives

by Ahmed Al Amin, Junho Hong, Van-Hai Bui and Wencong Su

Algorithms 2023, 16(10), 474; https://0-doi-org.brum.beds.ac.uk/10.3390/a16100474 - 9 Oct 2023

Cited by 4 | Viewed by 1989

Abstract

A well-functioning smart grid is an essential part of an efficient and uninterrupted power supply for the key enablers of smart cities. To effectively manage the operations of a smart grid, there is an essential requirement for a seamless wireless communication system that provides high data rates, reliability, flexibility, massive connectivity, low latency, security, and adaptability to changing needs. A contemporary review of the utilization of emerging 6G wireless communication for the major applications of smart grids, especially in terms of massive connectivity and monitoring, secured communication for operation and resource management, and time-critical operations, are presented in this paper. This article starts with the key enablers of the smart city, along with the necessity of the smart grid for the key enablers of it. The fundamentals of the smart city, smart grid, and 6G wireless communication are also introduced in this paper. Moreover, the motivations to integrate 6G wireless communication with the smart grid system are expressed in this article as well. The relevant literature overview, along with the novelty of this paper, is depicted to bridge the gap of the current research works. We describe the novel technologies of 6G wireless communication to effectively perform the considered smart grid applications. Novel technologies of 6G wireless communication have significantly improved the key performance indicators compared to the prior generation of the wireless communication system. A significant part of this article is the contemporary survey of the considered major applications of a smart grid that is served by 6G. In addition, the anticipated challenges and interesting future research pathways are also discussed explicitly in this article. This article serves as a valuable resource for understanding the potential of 6G wireless communication in advancing smart grid applications and addressing emerging challenges. Full article

(This article belongs to the Special Issue Reinforcement Learning and Its Applications in Modern Power and Energy Systems)

► Show Figures