2.1. Theoretical Background
The main goal in any highway-related traffic management study is the seamless entrance and exit of vehicles in and out of the main stream of a highway, respectively, as well as the continuous traffic flow on it. In particular, an entrance ramp which includes the necessary auxiliary lane(s) enables the incoming vehicles to accelerate so as to attain the necessary speed to incorporate smoothly into the highway traffic flow without the need to stop the flow of either the incoming vehicles or the vehicles already moving on the highway [
8].
Ramp metering is a method for measuring and controlling traffic that is applied to entrance ramps on highways. The measurement of the number of vehicles is performed through sensors located at the entrances and exits of the highway. The purpose of this method is to control the entry of vehicles in the highway so to make the best use of the capacity of the highway [
6,
8],. Specifically, the ramp has a demand for traffic d
0 expressed in vehicles per hour (vh/h), while r
0 (vh/h) is the traffic on the ramp regulated by signs, q
c (vh/h) the highway capacity and q
in (vh/h) the flow of the main stream. When a traffic jam occurs, the maximum output current q
out (vh/h) is less than the normal main current flow q
c. The above phenomenon is called capacity drop and the goal is to avoid it as much as possible [
9]. The capacity drop phenomenon can be seen in flow—density chart in
Figure 1. In cases where ramp metering is not applied, capacity drops tend to occur quite often. Therefore, with the use of ramp metering and the continuous retrieval and analysis of traffic data, this phenomenon can be alleviated. Then, the flow of vehicles is smooth and the traffic times are reduced on the main lanes. On the other hand, there is a possibility of creating a long queue of vehicles on the entrance ramp and correspondingly long delays there. Thus, it is easily understood that a balance must be maintained between the traffic on the highway and the length of the queue at the entrance.
Controlled access through ramps is a key tool for achieving the required flow on highways and avoiding traffic congestion at the same time. The main goal, given that the capacity of a highway is obviously not unlimited, is to maintain as high as possible the level of the main flow. In this perspective, traffic lights are installed on the entrance ramps, in order to control the rate at which vehicles enter the main stream of the highway [
8]. Controlled access is widely used on entrance ramps to achieve the best possible traffic regulation. More specifically, it is possible to reduce the waiting times of vehicles entering the highway as well as the travel times on it. In addition, controlled access on ramps increases safety when vehicles enter the main flow, thus reducing the risk and number of accidents caused especially in situations of increased traffic. A typical depiction of an entrance ramp with traffic lights for control access is shown in
Figure 2.
2.2. Related Work and Purpose of the Current Study
Ramp metering is a one of the most studied and important methods for traffic control so to reduce highway congestion and improve safety [
7]. There are several studies through the years that utilize ramp metering by combining different models and computational methods so to achieve the best possible performance. Some of the first ramp metering methods that were suitable mostly for local implementation are the occupancy (COC) strategy and demand-capacity (DC) according to Masher et al. [
10]. These methods can be classified as fixed-time control methods which are generally simple and based on historical data in order to adjust the time and entrance rate through ramps [
11]. Local-traffic response methods based on the demand-capacity method and the occupancy method are developed in [
10,
12,
13] studies. The aforementioned methods cannot perform well in cases of rapid changes and fluctuations in traffic on a highway. Adaptive RM control methods which can be classified to local and coordinated traffic response can be considered as an evolution to these methods.
Papageorgiou et al. [
5] proposed the ALINEA method, which is also a local adaptive method, engaging a closed loop algorithm in order to define the entrance rate through ramps in a highway and simultaneously to retain the flow in a predefined value [
14]. ALINEA is one of the most celebrated methods considering the RM and thus is combined with various diverse technologies and tools. ALINEA was the first to introduce a local traffic coordinated method and due to its performance in the next years it was studied and expanded by many researchers in order to be adapted to different conditions and become more effective. Some of these modified ALINEA-based methods are the following: (i) flow-based ALINEA (FL-ALINEA) [
7]; (ii) upstream-occupancy ALINEA (UP-ALINEA) [
7]; (iii) upstream-flow-based ALINEA (UF-ALINEA) [
7]; (iv) ramp-queue control (X-ALINEA/Q) [
7]; (v) AD-ALINEA [
15]; (vi) ALINEA with mainline speed recovery [
16]; (vii) data-driven feedback approach for ALINEA fine-tuning [
17]; (viii) proportional integral ALINEA (PI-ALINEA) [
18]; (ix) feed-forward ALINEA (FF-ALINEA) [
19] and (xi) congestion-status ALINEA (CS-ALINEA) [
20].
In addition to the ALINEA method and its modifications, there are also some other well-known algorithms for ramp metering. METALINE is a well-known RM algorithm which uses a linearized version of a nonlinear macroscopic flow model as presented in [
21]. System-wide adaptive ramp metering (SWARM) is also a multivariate method which attempts to alleviate traffic congestion though controlled ramp entrances [
22]. Another interesting approach for RM is the usage of fuzzy logic. Taylor et al. [
23] proposed a method that envisions to improve robustness, prevent congestion and balance the conflict needs of a highway. ALINEA, METALINE, SWARM and fuzzy logic methods are some representative methods proposed in previous years. Because of ALINEA’s scientific impact the aforementioned methods in many cases compared their performance with it. One major common characteristic of the abovementioned studies is that they study the RM problem from a local perspective and this may not lead to the desired results when these methods are applied at a larger scale.
The algorithms mentioned till this point presented very promising results and establish a basis for efficiently solving the ramp metering problem. Every and each one of them has its own strengths and weaknesses. However, the advances in ICT, sensors and data technologies have created new standards and new needs for almost every data-driven application. Machine learning methods and especially neural networks is a quite attractive approach for problems that need to deal with fluctuating data values. The idea of applying neural networks to ramp metering is not new, but due to technological limitations it is only in the last few years that it has tended to gain more and more ground. Zhang et al. [
24] proposed the first approaches with artificial neural networks (ANNs) for RM in 1994 and 1997 [
25]. Their results were promising for local RM problems and since then many studies were published which tried to go one step further either by using different machine learning methods or by scaling up the problem and examining more complex highway systems. In addition to ANNs, reinforcement learning approaches are used in several studies considering the RM problem. Also, the technology evolution both for data and machine learning techniques nowadays has led to deep learning approaches for ramp metering. A brief classification of the methods mentioned above can be found in
Table 2.
Most of the aforementioned studies [
14,
15,
16,
18,
19,
20,
24,
25,
27,
31,
32,
37,
39,
44,
45,
49] studied and modelled the ramp metering problem by using only one on-ramp for entrance in the main flow of the highway. The addition of an extra ramp which shares a common ramp controller to adjust the red-light duration so to control the entrance rate in the main flow increases drastically the computational complexity. In this study, as presented, in the next section (
Section 3) the modelling of the highway traffic flow assumes two ramps which are coordinated so as to maintain the density of the highway at a predefined threshold.
Moreover, machine learning methods are becoming more and more popular for traffic control and RM. Thus, it is not an exaggeration to claim that almost every state-of-the-art method employs machine learning algorithms for RM and traffic regulation in general. This is underpinned by searching relevant literature for RM and traffic regulation studies conducted in the previous years. Based on the latest research processes, state-of-the-art algorithms in ramp metering studies, are discussed in the following references: [
12,
20,
44,
45,
49].
Ghanbartehrani et al. [
12] investigated the development of an algorithm for ramp signal control based on the incorporation of linear regression and clustering approaches in order to learn the traffic flow trends over time. The proposed algorithm is compared with the widely known and used as well, traffic-responsive algorithm ALINEA. The results of the current comparison confirm the effective maintenance of the traffic flow at reasonable levels using the proposed algorithm, when ALINEA demonstrated cycles of long red phases followed by overcompensation and brief breakdowns [
12].
Liu et al. [
20] proposed an improved ALINEA-based algorithm, named CS-ALINEA. During the development and the simulation of the current algorithm the traffic flow is used to replace occupancy as the control parameter and the control rate can be selected according to the congestion status reclassified in an adaptive way [
20]. The algorithm often ignores the impact of ramp overflow on ground road traffic in order to provide the traffic efficiency of the mainstream. The extracted results show that the proposed algorithm can optimize the ramp queuing length and reduce waiting time of vehicles while the efficiency of urban freeway can also be guaranteed an many cases [
20].
Zhou et al. [
44] proposed a reinforcement learning approach in order ‘to learn an optimal ramp metering policy controlling a downstream bottleneck that is far away from the metered ramp’. In fact, the problem of ramp metering for a distant downstream bottleneck in this study is modeled as a Q-learning problem supported by an artificial neural network. In this artificial network an intelligent ramp meter agent develops and trains a nonlinear optimal ramp metering model in such a way that the capacity of the distant downstream bottleneck can be fully utilized, but not as much so as to cause congestion due to the current density limit exceedance [
44]. The results of the current research show that the proposed metering policy approach that was developed, trained and evaluated can achieve satisfying traffic flow evolution over the entire freeway for a specific range of noise level(s) that is defined as vehicles per hour (veh/h) [
44].
Chai et al. [
45] demonstrated a reinforcement learning algorithm to optimize the on-ramp control regulation by combining the metering rate, the length of ramp queue, the throughput and the occupancy rate of the interweaving area and the volume of the road network. In comparison with widely used simulation methods as no-control and classical ALINEA, the proposed method performs intelligent learning and optimized function and achieves an improvement on the control effect of the road network performance, ramp queue length, weaving area occupancy rate and traffic volume [
45].
An interesting study regarding control strategy for ramp metering was made by Zhao et al. [
49]. The current study proposes a fuzzy self-adaptive proportional–integral–derivative (FSAPID) control strategy for RM control at distance downstream bottlenecks, where the specific FSAPID control is composed of proportional–integral–derivative control and fuzzy control as well. The results depict that the current algorithm approach provides fast convergence, strong predictive ability and high action precision, and achieves preferable performance especially when the bottleneck in the examined highway is located far downstream [
49].
Given the technological evolution, RM studies tend to adapt machine learning methods to address the problem as efficiently as possible. The critical phase of machine learning methods and especially for neural networks is the training phase. Most researchers try to train their models with as many as possible unbiased random data so as to prepare their model to be ready to handle even the rarest and most unexpected situations with the highest possible efficiency. In this light, the proposed solution in this study used a randomized dataset which was created by engaging different scenarios of traffic flow in a highway employing entrance ramps. The details for the data generation and the scenario implemented are provided in the next section.