Quantitative Precipitation Forecasting Using an Improved Probability-Matching Method and Its Application to a Typhoon Event

Abstract

This present study aims to explore how forecasters can quickly make accurate predictions by using various high-resolution model forecasts. Based on three high temporal-spatial resolution (3 km, hourly) numerical weather prediction models (CMA-MESO, CMA-GD, CMA-SH3) from the China Meteorological Administration (CMA), the hourly precipitation characteristics of three model within 24 h from March to September 2020 are discussed and integrated into a single, hourly, deterministic quantitative precipitation forecast (QPF) by making use of an improved weighted moving average probability-matching method (WPM). The results are as follows: (1) In non-rainstorm forecasts, CMA-MESO and CMA-GD have similar forecast abilities. However, in rainstorm forecasts, CMA-MESO has a notable advantage over the other two models. Thus, CMA-MESO is selected as a critical factor when participating in sensitivity experiments. (2) Compared with the traditional equal-weight probability-matching method (PM), the WPM improves the different grade QPF because it can effectively reduce rainfall pattern bias by making use of the weighted moving average (WMA). Additionally, the WPM threat score in rainstorm forecast similarly improved from 0.051 to 0.056, with a 9.8% increase relative to the PM. (3) The sensitivity experiments show that an optimal rainfall intensity score (WPM-best) can further improve the QPF and overcome all single models in both rainstorm and non-rainstorm forecasts, and the WPM-best has a rainstorm threat score skill of 0.062, with an increase of 21.6% compared with the PM. The performance of the WPM-best will be better if the precipitation intensity is stronger and the valid forecast periods is longer. It should be noted that there is no need to select models before using the WPM-best method, because WPM-best can give a very low weight to the less-skillful model in a more objective way. (4) The improved WPM method is also applied to investigate the heavy-rainfall case induced by typhoon Mekkhala (2020), where the improved WPM technique significantly improves rainstorm forecasting ability compared with a single model.

1. Introduction

High-resolution numerical weather prediction (NWP) models have been continuously improved with the rapid development of computers, and the models’ forecasting performance have steadily increased. However, weather forecasters may face some new challenges in operational applications. One crucial issue is that the precipitation products by almost all NWP models have considerable forecast errors and have trouble to accurately forecast the precipitation intensity [1]. Thus, how to eliminate these biases has yet to be addressed in the improvement of NWP skills. Many studies have demonstrated that the development of post-processing techniques can effectively reduce systematic bias [2,3]. For example, the model output statistics (MOS) technique can enhance quantitative precipitation forecast (QPF) skills [4,5], and quantile mapping algorithms are effective in removing historical biases relative to observations [6]. In recent years, Zhu and Luo [7] employed a frequency-matching method (FM) to produce a more realistic rainfall forecast based on frequency distributions of forecast and observations. Wu et al. [8] showed that the optimal threat score-based correction algorithm (OTS) is superior to all lead times, single models, and multi-model means.

Another critical issue to consider is how forecasters can develop a more scientific decision-making process by selecting the most appropriate prediction factors from several products of high-resolution models or ensemble forecasts. Various techniques have therefore been proposed to solve this problem [9,10,11]. For example, Primo et al. [12] showed that logistic regression is preferable to linear methods in more flexibly calibrating probabilistic forecasts. Messner and Mayr [13] demonstrated that the analog methods by which current forecasts can be corrected by utilizing past ensemble forecasts errors were assumed to form an improved forecast. Qi et al. [14] proposed that tropical cyclone track forecasts using the ensemble mean method, which selected prediction members based on errors at short lead times, were better than that using deterministic model predictions. In addition, several studies have demonstrated that combining different forecasts with more than one NWP model can effectively improve QPF [15,16]. For instance, Ebert [17] pointed out that the PM method could improve the rain pattern and the event hit rate. This can be utilized as an alternative to the traditional ensemble mean precipitation to retain the amplitude of the simulated model features [18]. Fang [19] developed a modified PM technique to adjust rainfall pattern with a large-size, low-resolution ensemble, and to adjust the rainfall frequency distribution with a small-size, high-resolution ensemble, to improve the landfall typhoon rainstorm forecast.

Further studies should be carried out to improve these methods, and two steps are included in this paper. Initially, hourly QPF characteristics of three high-resolution models are analyzed, and multi-model calibrated ensembles are further constructed based on the FM and OTS methods. In the second step, an improved WPM method is proposed to integrate multi-model calibrated ensembles into a single, hourly, deterministic QPF. The remainder of this paper is organized as follows: Section 2 describes the data and calibration method used in this study. Section 3 investigates the statistical characteristics of multi-model hourly QPF and compares the calibration results from equal-weight PM with that from WPM. In addition, a case study is presented to demonstrate the results of WPM. A brief summary is provided in Section 4.

2. Data and Methods

2.1. Data

The observation and reanalysis datasets used in this paper are as follows. (1) The hourly observation precipitation data were provided by the China Integrated Meteorological Information Sharing System (CIMISS). In this paper, datasets from 420 high-quality stations were selected based on Hunan Province (24° N—31° N, 108° E—115° E). (2) Reanalysis products (FNL) were provided by National Centers for Environmental Prediction (NCEP). (3) The best track datasets of tropical cyclones over the western North Pacific Ocean were provided by the Shanghai Typhoon Institute (STI) of the China Meteorological Administration (CMA).

Three high temporal-spatial resolution numerical weather prediction models from CMA were used in this paper, including CMA-MESO, CMA-GD, and CMA-SH3.

Table 1. Three high temporal-spatial resolution numerical weather prediction models.

Name	Output Resolution	Duration	Operation/UTC	Organization
CMA-MESO	3 km, 1 h	36 h	00/03/06/09/12/15/18/21	China Meteorological Administration
CMA-GD	3 km, 1 h	96 h	00/12	Guangdong Meteorological Service
CMA-SH3	3 km, 1 h	24 h	Per hour	Shanghai Meteorological Service

lists specific information about the models. The output products of three models had the same spatial and temporal resolutions of 3 km and 1 h, respectively, and valid forecast periods of 36, 96, and 24 h. This paper selected precipitation products from March to September 2020 as multi-model ensemble members, with uniform initial forecast times of 00:00 and 12:00 UTC and a valid forecast period of 0–24 h. Based on the specific motivation of the China meteorological service business, the hourly QPF was divided into light rain (≥0.1 mm), moderate rain (≥2 mm), heavy rain (≥4 mm), and rainstorm (≥8 mm). Multi-model grid products were the nearest neighbors, interpolated to 420 stations to ensure that the comparison was consistent with observations. Figure 1 shows the domains of the three models and marks the location of Hunan Province.

2.2. Method

2.2.1. Generating Multi-Model Ensemble Members

The FM technique [7] aims to eliminate the frequency deviation between QPF and the observations. The new QPF threshold has the same frequency as the observation (Figure 2a). The OTS technique [8] aims to maximize the threat score (TS). The new QPF threshold corresponds to the maximum TS (Figure 2b). Both FM and OTS have the ability to correct the precipitation intensity and are unable to correct the precipitation location bias. The calculation equation of calibrated precipitation is as follows:

$$y=\{\begin{array}{c}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}0, x<x_1\\ OB{S}_k+\left(OB{S}_{k+1}-OB{S}_k\right)\frac{x-x_k}{x_{k+1}-x_k}, x_k\le x<x_{k+1}\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\frac{x}{x_5}\times OB{S}_5, x\ge x_5\end{array}$$

(1)

where x denotes the original model precipitation; y denotes the calibrated precipitation; OBS_k is the precipitation grading that selects five grades, namely, 0.1, 2, 4, 8, and 20; and x_k is the new precipitation grading. For the FM method, x_k is the model threshold with the same frequency as that of the observed OBS_k. For OTS, x_k is the model precipitation, corresponding to the maximum of TS in each grade. The training window is the last 60 days.

Using FM and OTS, the multi-model QPF intensity was corrected to construct an ensemble prediction system encompassing nine members (

Table 2. List of multi-model ensemble members for hourly QPF.

Ensemble Members	Description	Training Window
CMA-MESO	Classic CMA-MESO QPF	None
CMA-MESO-FM	QPF magnitude adjusted based on FM	Past 60 days
CMA-MESO-OTS	QPF magnitude adjusted based on optimal TS	Past 60 days
CMA-GD	Classic CMA-GD QPF	None
CMA-GD-FM	QPF magnitude adjusted based on FM	Past 60 days
CMA-GD-OTS	QPF magnitude adjusted based on optimal TS	Past 60 days
CMA-SH3	Classic CMA-SH3 QPF	None
CMA-SH3-FM	QPF magnitude adjusted based on FM	Past 60 days
CMA-SH3-OTS	QPF magnitude adjusted based on optimal TS	Past 60 days

).

2.2.2. WPM Method and Sensitivity Experiments

The PM method can overcome the deficiencies of the ensemble mean and can provide a more realistic rainfall forecast than that from the ensemble mean. This method is an equal-weight, multi-model calibration technique, based on the precipitation of an optimal spatial distribution using the ensemble mean. The precipitation has a higher-accuracy frequency distribution due to the ensemble members [17,18,19]. Figure 3a shows the schematic process of PM.

The WPM technique improves the precipitation distribution by replacing the equal weight in the PM method with a weighted moving average (WMA); its calculation method is shown in Figure 3b and

Table 3. Indicators used in WPM. The sorted observations and ensemble member precipitation are denoted as O and F, respectively. The i and j denote the ith day and jth ensemble member, respectively. M and N represent the number of ensemble members and the number of valid samples, respectively.

Indicator	Expression
Spearman correlation coefficient (Ri)	$\frac{{{\displaystyle \sum }}_{i=1}^N\left(F_i-\overline{F}\right)\left(O_i-\overline{O}\right)}{\sqrt{{{\displaystyle \sum }}_{i=1}^N{\left(F_i-\overline{F}\right)}^2{{\displaystyle \sum }}_{i=1}^N{\left(O_i-\overline{O}\right)}^2}}$
Weight (Wj)	$\frac{{{\displaystyle \sum }}_{i=1}^N{R}_i}{{{\displaystyle \sum }}_{j=1}^M{{\displaystyle \sum }}_{i=1}^N{R}_i}$
New pattern (D)	${{\displaystyle \sum }}_{j=1}^M{F}_j{W}_j$

. First, the weights of the ensemble members can be calculated by the real-time Spearman correlation coefficient (R) at each starting time during the training period. For each member, its maximum R is assumed to appear N times. The sum of maximum R during N days, divided by the sum of R-max of the population from where the sample was selected, will then be weighted. After that, the improved pattern can be obtained through a weighted average, multiplied the forecast precipitation of each member by its associated weights, and then the results are added.

However, both PM and WPM have deficiencies; strong precipitation may be weakened by using the median of the ensemble forecast; thus, sensitivity experiments using different values are designed as follows.

Sensitivity experiments: A group of comparative experiments based on the PM method using different distribution fields and values was designed for the ensemble forecast. Figure 3 and

Table 4. Design of the sensitivity experiments.

Experiments	Pattern	Intensity	Training Window
PM	Ensemble mean	Ensemble members	None
WPM	WMA	Ensemble members	Past 60 days
WPM-best	WMA	The optimal member	Past 60 days

list the details of the experiments. Specifically, the PM experiment (Figure 3a) uses the ensemble mean as the pattern and the median as the intensity; the WPM experiment (Figure 3b) uses the WMA as the pattern and the median as the intensity; the WPM-best (Figure 3c) uses the WMA as the pattern and the precipitation intensity of the optimal model as the intensity.

Furthermore, the relative advantages of having extra post-processed members and multi-models are demonstrated by applying the WPM-best method to the two optimal models (six members) and a single model (three members).

2.2.3. Verification

Verification indicators used in this paper include the threat score (TS), clear-rainy TS, probability of detection (POD), and false alarm ratio (FAR). The calculating details are provided in

Table 5. Assessment indicators used in multi-model and sensitivity experiments. NA, NB, NC, and ND represent the number of hits, misses, false alarms, and correct negatives, respectively.

Indicator	Expression
Threat score (TS)	$\frac{NA}{NA+NB+NC}$
Clear-rainy TS	$\frac{NA+ND}{NA+NB+NC}$
Probability of detection (POD)	$\frac{NA}{NA+NB}$
False alarm ratio (FAR)	$\frac{NC}{NA+NC}$

. Here, the clear-rainy threshold is 0.1 mm, which is the smallest detectable amount of rain gauge in China.

3. Results

3.1. Analysis of Multi-Model Hourly QPF

Figure 4a compares the TS performance of the multi-model QPF. Generally, the CMA-MESO and CMA-GD models show almost the same scores, with a maximum difference of 0.007 between them, in non-rainstorm forecasts. However, in rainstorm forecasts, the CMA-MESO model shows a notable advantage with its TS reaching 0.058, followed by the CMA-GD model (0.052). The TS of the CMA-SH3 model across each grade is invariably lower than those of the CMA-MESO and CMA-GD. In addition, notably, as the intensity of precipitation increases, the scores of all models decline, and the relative differences between these models gradually increase.

Specifically, the CMA-MESO model shows a higher POD and lower FAR, whereas the CMA-SH3 has a lower POD and higher FAR, with those of CMA-GD in between (Figure 4b,c). In light rain forecasts, the CMA-GD model has the highest POD (0.643). In other grades of rainfall, the CMA-MESO model invariably has the highest POD. The POD and FAR for the rainstorm forecasts of the CMA-MESO model are 0.152 and 0.915, respectively. As the precipitation intensity increases, the relative differences among different models gradually shrink.

3.2. Analysis of Ensemble Members

Comparing the nine ensemble members in rainstorm forecasts (Figure 5e) reveals that the CMA-MESO model has the optimal performance among all members. In non-rainstorm forecasts (Figure 5a–d), three CMA-MESO members and three CMA-GD members all have good performances. Specifically, in clear-rainy forecasts, CMA-MESO-FM has the highest TS score of 0.853, followed by CMA-GD-FM (0.849). In light rain forecasts, CMA-MESO (0.356) and CMA-MESO-OTS (0.359) show the optimal performance, with no noticeable difference. The member with the lowest score is CMA-SH3-FM (0.263), which is 26.7% lower than the highest score. In moderate and heavy rain forecasts, CMA-GD-OTS has the highest TS scores of 0.164 and 0.104, followed by CMA-MESO, with scores of 0.157 and 0.099, respectively. Combined with the analysis results in Section 3.1 (Figure 4), CMA-MESO is used as a critical factor in the sensitivity test WPM-best due to its excellent performances for both rainstorm and non-rainstorm forecasts.

3.3. Results of Sensitivity Experiments

The sensitivity experiments showed that the WPM method with WMA can effectively improve precipitation forecasts across different intensities compared with the use of PM as an equal-weight calibration method (Figure 6a). Specifically, the TS increases from 0.051 to 0.056 for the rainstorm forecast using this method, representing a 9.8% growth. Based on WPM, the WPM-best with all members further improved precipitation forecasts, with the TS in rainstorm forecasts further increasing to 0.062. Compared with the PM, the WPM-best method improved TS by 2.8%, 7.7%, 10.3%, and 21.6%, from light rain to rainstorm forecasts, respectively. Compared with CMA-MESO, the WPM-best method improved TS by 1.7%, 7.0%, 8.1%, and 6.9%, respectively, from light rain to rainstorm forecasts. Thus, as the precipitation intensity increases, this method showed increasingly noticeable advantages. Notably, PM and WPM both increased the clear-rainy forecast but degrade the light rain and rainstorm forecast compared with CMA-MESO.

Specifically, the WPM-best method with all members considerably increased the POD of precipitation forecasts at different intensities, whereas the FAR was decreased (Figure 6b,c). Compared with the PM, the WPM-best increased the POD from light rain to rainstorm forecasts by 5.1%, 21.7%, 21.2%, and 28.6%, respectively, without causing a significant change in the FAR (0.4%, 0.4%, −0.4%, and −1.2%). Compared with CMA-MESO, the POD of the WPM-best from light rain to rainstorm forecasts increased by 1.3%, 5.9%, 7.5%, and 6.6% and the FAR decreased by 1.1%, 1.6%, 1.2%, and 0.7%, respectively. Compared with the WPM, the WPM-best furtherly increased the TS by increasing the POD.

Furthermore, the WPM-best experiment using the two optimal models (three CMA-MESO members and three CMA-GD members) showed the almost same performance with WPM-best (all members). This means that the weighting of the models can be done by the machine because the less-skillful model automatically receives very low performance weights. Therefore, it is not necessary to remove any less-skillful model before using WPM. The experiment using only one model (three CMA-MESO members) failed to improve the forecast compared to CMA-MESO, because both FM and OTS can only improve intensity, and not distribution. In conclusion, there is no need to select models before using the WPM-best method. WPM-best can effectively decrease the multi-model distribution bias, but it has no effect on a single model with extra post-processed FM and OTS members.

From the perspective of the 0–24 h valid forecast periods in non-rainstorm forecasts (Figure 7a–d), the WPM-best (all members) improved the valid periods by 90.3% compared with CMA-MESO. In rainstorm forecasts (Figure 7e), the WPM-best improved the valid periods by 70.8%. Additionally, the method exhibited a better correction effect in longer valid forecast periods.

3.4. Case Study of Typhoon Mekkhala (2020)

To evaluate the effectiveness of the WPM-best on multi-model typhoon rainfall, the WPM-best (all members) method was applied in the case study, to the typhoon “Mekkhala” (2020) rainstorm occurring on 11 August 11 2020 (Figure 8). Under the impact of the typhoon, the rainstorm was recorded by 118 stations (≥8 mm/h) in Hunan Province. From the weather chart, at the early stage, when severe precipitation occurred (00:00 on 11 August), typhoon “Mekkhala” (2020) landed on Fujian Province with the minimum atmospheric pressure of 975 hPa and a maximum wind speed of grade 13 (38 m/s). At the same time, a large area of cyclonic convergence appeared in southeast Hunan, with a divergence reaching −9 × 10⁻⁶ s⁻¹ on the 700 hPa weather chart and the maximum intensity of precipitation of 61.4 mm/h. From 06:00 to 12:00 (Figure 8b,c), with typhoon “Mekkhala” (2020) moving toward the inland, the convective precipitation in Hunan intensified with a maximum precipitation of 72.6 mm/h. At 18:00 (Figure 8d), with the typhoon weakening and vanishing, the southerly wind intensified, and the rainstorm was about to end.

The comparative results of the multi-model precipitation and WPM-best calibration forecasts (Figure 9) reveal that, compared with CMA-MESO, the WPM-best shows improved forecasts for moderate rain, heavy rain, and rainstorm by 7.6%, 10.4%, and 39.2%, respectively. However, it has a lower performance than CMA-MESO, by 0.3% and 3.8%, in clear-rainy and light rain forecasts, respectively. This is consistent with the statistical results shown in Section 3.3, demonstrating that WPM-best performance will be improved if the precipitation intensity becomes stronger.

4. Conclusions

Based on the hourly QPF of three high-resolution NWP models spanning from March to September 2020, FM and OTS calibration methods were used to construct a multi-model ensemble correction forecast and a group of comparative experiments were designed based on the multi-model ensemble method. Specifically, the WMA and model optimization methods were used to improve the precipitation pattern and intensity of the PM method. Finally, the improved correction method was applied in a typhoon rainstorm case. The following results were obtained:

(1)

In non-rainstorm forecasts, CMA-MESO and CMA-GD have similar forecast capabilities. In rainstorm forecasts, CMA-MESO has a notable advantage over CMA-GD and CMA-SH3, with the TS increasing to 0.052 (CMA-GD) from 0.058 (CMA-MESO), representing an 11.5% growth. Additionally, among the nine ensemble members, CMA-MESO showed the highest accuracy for the rainstorm forecast, with a reliable performance in the non-rainstorm forecast. Thus, this was selected as a key factor for the sensitivity experiment.

(2)

Compared with traditional equal-weight PM, the WPM improves the different grades of QPF, obtaining an optimal rainfall pattern using WMA, with a rainstorm threat score skill of 0.051 to 0.056, an increase of 9.8%. On this basis, the WPM-best method, which uses an optimal rainfall intensity than WPM, furtherly improves precipitation forecasts. The higher the precipitation grade, the more significant the improvement. The TS in the rainstorm forecast further increases to 0.062.

(3)

The sensitivity experiments show that there is no need to select models before using the WPM-best method, because WPM-best can give a very low weight to the less-skillful model in a more objective way. However, this method has no effect on a single model with extra post-processed FM and OTS members, because both FM and OTS can only improve intensity, not distribution. The performance of WPM-best improves with longer valid forecast periods.

(4)

The results of the case analysis of typhoon “Mekkhala” (2020) show that CMA-MESO has the highest forecast TS among the three high-resolution models, and the WPM-best method furtherly improves the rainstorm forecast by 39.2% compared with CMA-MESO.

In this study, the “optimal” model used for WPM-best was selected as CMA-MESO for all forecast times. The model performance varies as different times and geographical locations. Therefore, choosing the optimum model for each forecast time could be a constructive way to further improve the QPF. We are expecting to validate the method of dynamically selecting the optimal model in future studies.