Next Article in Journal
Using Mobile Monitoring and Atmospheric Dispersion Modeling for Capturing High Spatial Air Pollutant Variability in Cities
Previous Article in Journal
Effective Moisture Evolution since the Last Glacial Maximum Revealed by a Loess Record from the Westerlies-Dominated Ili Basin, NW China
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Atmosphere
Volume 13
Issue 11
10.3390/atmos13111932
atmosphere-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

The Impact of Stochastic Perturbations in Physics Variables for Predicting Surface Solar Irradiance

by
Ju-Hye Kim
1,*,
Pedro A. Jiménez
1,
Manajit Sengupta
2,
Jimy Dudhia
1,
Jaemo Yang
2 and
Stefano Alessandrini
1
1
National Center for Atmospheric Research, Boulder, CO 80307, USA
2
National Renewable Energy Laboratory, Golden, CO 80401, USA
*
Author to whom correspondence should be addressed.
Atmosphere 2022, 13(11), 1932; https://0-doi-org.brum.beds.ac.uk/10.3390/atmos13111932
Submission received: 19 October 2022 / Revised: 11 November 2022 / Accepted: 16 November 2022 / Published: 20 November 2022
(This article belongs to the Section Atmospheric Techniques, Instruments, and Modeling)
Browse Figures
Versions Notes

Abstract

:
We present a probabilistic framework tailored for solar energy applications referred to as the Weather Research and Forecasting-Solar ensemble prediction system (WRF-Solar EPS). WRF-Solar EPS has been developed by introducing stochastic perturbations into the most relevant physical variables for solar irradiance predictions. In this study, we comprehensively discuss the impact of the stochastic perturbations of WRF-Solar EPS on solar irradiance forecasting compared to a deterministic WRF-Solar prediction (WRF-Solar DET), a stochastic ensemble using the stochastic kinetic energy backscatter scheme (SKEBS), and a WRF-Solar multi-physics ensemble (WRF-Solar PHYS). The performances of the four forecasts are evaluated using irradiance retrievals from the National Solar Radiation Database (NSRDB) over the contiguous United States. We focus on the predictability of the day-ahead solar irradiance forecasts during the year of 2018. The results show that the ensemble forecasts improve the quality of the forecasts, compared to the deterministic prediction system, by accounting for the uncertainty derived by the ensemble members. However, the three ensemble systems are under-dispersive, producing unreliable and overconfident forecasts due to a lack of calibration. In particular, WRF-Solar EPS produces less optically thick clouds than the other forecasts, which explains the larger positive bias in WRF-Solar EPS (31.7 W/m2) than in the other models (22.7–23.6 W/m2). This study confirms that the WRF-Solar EPS reduced the forecast error by 7.5% in terms of the mean absolute error (MAE) compared to WRF-Solar DET, and provides in-depth comparisons of forecast abilities with the conventional scientific probabilistic approaches (i.e., SKEBS and a multi-physics ensemble). Guidelines for improving the performance of WRF-Solar EPS in the future are provided.

1. Introduction

The Weather Research and Forecasting (WRF)-Solar model was developed to meet the increasing demand for specialized numerical weather predictions for solar power applications [1]. WRF-Solar is a specific configuration and augmentation of a fully physically based WRF model [2], and it showed improvements in solar irradiance prediction compared to the standard WRF model by enhancing the representation of aerosol cloud radiation feedbacks [1,3].
Various efforts have followed to improve the predictability of surface irradiance in cloudy skies, and that has been a special focus on improving the representation of sub-grid scale clouds [3,4,5,6,7,8]. These efforts found that the impacts of shallow cumulus clouds on solar energy predictions should be considered; however, there are still challenges to addressing the uncertainty associated with cloud representation. Solving these challenges by improving the physics schemes alone remains a challenge due to the extensive feedback between the processes involving cloud formation and dissipation [3]. Also, there are limitations in the deterministic forecast itself in terms of capturing uncertainties in the initial and boundary conditions, the model’s spatial resolution, and the requirement for proper physics representation of that scale [9,10,11,12,13].
In numerical weather prediction, various ensemble approaches have been developed and evaluated to overcome the limitations of deterministic forecasting and to provide more accurate forecast information and confidence levels. In early studies, uncertainties in model forecasts were investigated by perturbing the initial condition [14,15,16,17], resulting in an under-dispersive ensemble that did not sufficiently account for the uncertainty in the short-range forecasts. The multi-physics approach was followed by the use various combinations of different physics schemes, and it increased the ensemble spread, providing diversity among the ensemble members [18,19,20]. However, it is not easy to arbitrarily compose a statistically independent ensemble because specific members share similar characteristics [21]. Also, the fact that different members have different biases is one reason why the multi-physics approach improves the spread [20,22]. Ref. [23] pointed out that this result conflicts with the fundamental purpose of forecast uncertainty, which aims to represent the random rather than the systematic component of the forecast error.
A stochastically perturbed ensemble approach has been proposed to represent unresolved sub-grid-scale processes [24] and has been successfully implemented and tested in many numerical weather prediction models [25,26,27]. The stochastic kinetic energy backscatter scheme (SKEBS) [19,20,25,28] is one of the most widely used stochastic parameterization methods, and it has been implemented in the WRF model. In SKEBS, kinetic energy from unresolved scales is made available for, or backscattered onto, the resolved scales via stochastic perturbations of the stream function and potential temperature at selected wavenumbers. Ref. [3] showed that the global horizontal irradiance (GHI) forecasting performance improved when considering the effect of sub-grid-scale clouds by running ensembles of SKEBS in the WRF-Solar.
To extend the capabilities of WRF-Solar beyond deterministic forecasts, we have developed the WRF-Solar ensemble prediction system (WRF-Solar EPS). Our strategy involves adding stochastic perturbations to the most relevant variables determining the solar irradiance predictions. These variables have been identified with tangent linear models of selected parameterizations [29]. This paper, as an extension of [29], explores how the stochastic perturbations implemented in WRF-Solar EPS act on solar irradiance forecasting during a sufficiently long period to have a statistically robust characterization of the model’s performance. With the aim of objective evaluation, the performance of WRF-Solar EPS is compared to deterministic WRF-Solar predictions (WRF-Solar DET), a SKEBS-based ensemble, and a WRF-Solar multi-physics ensemble (WRF-Solar PHYS) in terms of ensemble mean. Whether the ensemble spread of the WRF-Solar EPS forecast is reliable enough is also an important concern. Hence, we focus on evaluating both the accuracy of the predictions and the uncertainty quantification. With this aim, an in-depth analysis is performed using satellite retrievals of the GHI over the contiguous United States (CONUS). Recently, [30] investigated the performance of WRF-Solar. Both ground observations and the satellite-based National Solar Radiation Database (NSRDB) were used, and it was found that using the NSRDB is sufficient to assess predictability and to discuss spatial variability given the current WRF-Solar performance. Ref. [31] also compared the GHI predictability of the deterministic WRF-Solar and WRF-Solar EPS using the NSRDB and evaluated the forecasting performance of WRF-Solar EPS.
WRF-Solar EPS is the first numerical weather prediction model specifically designed to provide probabilistic surface solar irradiance forecasting. It has been recently developed, and herein we present the results of our first assessment. The first objective is to measure how much WRF-Solar EPS improves predictability compared to WRF-Solar DET. The second is to compare the performance of WRF-Solar EPS with forecast results of existing probabilistic approaches (i.e., SKEBS and a multi-physics ensemble), which have a long development history of rich evaluations and improvements. Finally, our last objective is to identify the potential of WRF-Solar EPS by analyzing the deficiencies and advantages in solar energy forecasting and to provide a scientific basis for future improvement.
The paper is structured as follows: Section 2 provides information on the selection and implementation methodology of key physical variables for developing WRF-Solar EPS. Section 3 explains the experiment design, including an introduction to the NSRDB observations. Forecast verification results are shown in Section 4, and the discussions are in Section 5.

2. Adding Stochastic Perturbations to Physics Schemes

A multiple-variable stochastic ensemble prediction system, WRF-Solar EPS, was developed by implementing the stochastic perturbations in variables in six parameterizations. Ref. [29] described the methodology of variable selection in detail, which can be summarized as follows. The variables selected for perturbation were identified with tangent linear models for six WRF-Solar modules responsible for radiation and cloud formation and dissipation. These include the Thompson microphysics scheme [32], the Mellor–Yamada–Nakanishi–Niino (MYNN) [33] planetary boundary layer (PBL), the Noah land surface model (Noah LSM) [34], the Deng shallow cumulus system [3,7,35], and the Fast All-sky Radiation Model (FARMS) radiation [36] schemes, along with the unresolved clouds module based on relative humidity (CLD3) [37].
The tangent linear models were applied for these six stand-alone physics modules and used to analyze uncertainties of the output variables, uncertainties in the input variables, and to select the most sensitive variables controlling radiative transfer and cloud processes. We identified 14 variables: surface albedo, aerosol optical depth (AOD), Ångström exponent, asymmetry factor, water vapor mixing ratio, cloud/ice/snow mixing ratios, ice number concentration, potential temperature, turbulent kinetic energy (TKE), soil moisture content, soil temperature, and vertical velocity. Table 1 shows a list of these variables (column 2) and their associated parameterizations (column 3).
The stochastic perturbations are introduced according to the method of [23], which itself is originally derived from the stochastically perturbed parametrization Tendencies (SPPT) scheme [38]. The pattern of each perturbation is fully characterized by the standard deviation of a Gaussian distribution (σ, column 4 in Table 1), the horizontal wavelength (λ, column 5), and the decorrelation time (τ, column 6) between consecutive perturbations. In [18], perturbations were implemented to physics parameters, whereas we implemented them to each 2-dimensional (ω = 0, column 7) or 3-dimensional (ω = 1, column 7) physics variable inside each parameterization at every model computation time step. The perturbation used in this study is summarized in the following format, shown in Equation 1:
X n ′ = [ 1 + f ( σ n , λ n , τ n ) ] X n ,
where X n ′ and X n mean the perturbed and unperturbed quantities for each variable number n, respectively. Here X n is the physical quantity of the selected variable. Note that the use of the multiplier form means that clouds are perturbed only where they already exist, and in proportion to their amount. This means that the magnitude of the perturbation is the magnitude of X n multiplied by f ( σ n , λ n , τ n ) . Additionally, the perturbations are added only to calculate tendencies, not to perturb the model states. This means that they are added just before calculating the physics process and subtracted immediately after calculating the tendency.
Figure 1 shows examples of distributions of stochastic perturbations of f ( σ n , λ n , τ n ) for the AOD and TKE on the WRF-Solar domain. The perturbation pattern is homogeneous with a horizontal length scale λn, a mean of zero, and a variance of σn2. The decorrelation time of the AOD is 1 h, whereas the decorrelation time of the TKE is 10 min, so the perturbation added to each variable accounts for each variable’s unique timescale. Depending on the defined stochastic parameters, in this way, the perturbations of the 14 variables evolve within the selected five parameterizations and the CLD3 module throughout the forecasts.
Figure 2 shows an example of GHI forecasts by implementing the multivariable stochastic perturbations in WRF-Solar. It compares the time series of GHI from the 48 h forecasts of WRF-Solar DET and WRF-Solar EPS with surface radiation budget network (SURFRAD) observations. Each ensemble member predicts a different GHI distribution in cloudy regions, whereas all ensemble members tend to exhibit similar GHI forecasts for clear-sky conditions.

3. Experiment Design

3.1. Numerical Simulations

Four forecast experiments were performed using WRF-Solar DET, WRF-Solar EPS, SKEBS [19,20,25,28], and WRF-Solar PHYS, based on WRF-Solar [1,30]. The experiment for WRF-Solar DET includes 363 runs (every day from 1 January 2018 to 29 December 2018), and each ensemble system runs 3630 forecasts (363 cases × 10 ensemble members). All experiments were configured to cover CONUS with 9 km grid spacing. The National Centers for Environmental Prediction (NCEP) Global Forecast System (0.25° × 0.25°; 3 h intervals) forecast was used for the initial and boundary conditions, and 48 h forecasts were conducted with initializations every 06 UTC.
The physics parameterizations used in the WRF-Solar DET, WRF-Solar EPS, and SKEBS experiments follows the WRF-Solar reference physics configuration described in [30], consisting of the MYNN PBL parameterization, the Noah LSM, the Deng shallow cumulus scheme, and the rapid radiative transfer model (RRTM) for global models for the shortwave and longwave radiation [39]. On the other hand, the Thompson aerosol awareness microphysics scheme [32,40] was used in the representation of cloud and precipitation processes, and the FARMS radiation scheme was also activated to provide cloud-sensitive surface shortwave irradiance at every model time step.
WRF-Solar EPS runs a stochastically generated ensemble with 10 members based on WRF-Solar DET. We activated stochastic perturbations using the default parameters shown in Table 1 in five schemes: FARMS, MYNN PBL, Thompson microphysics, Noah LSM, and Deng shallow cumulus. The Thompson sub-grid cloud fraction (CLD3) module was not activated because it is not compatible with the Deng shallow cumulus scheme that predicts cloud fraction. The SKEBS ensemble follows the default SKEBS configuration.
WRF-Solar PHYS required more attention. We first selected 20 variations of the reference WRF-Solar configuration to account for uncertainties in processes such as cloud microphysics, radiation, cumulus, land surface, and aerosol effects. Next, an automatic procedure based on self-organizing maps (SOMs) [41] was used to find 10 configurations that better represent the characteristics of the original 20 configurations. Table 2 summarizes the physics configurations of the 10 members. Using SOM to filter out ensembles with similar forecasting tendencies helped to efficiently maintain the spread of this WRF-Solar PHYS ensemble. The members combine the reference WRF-Solar configuration with the Grell–Freitas (GF) [42] cumulus scheme, the Kain and Fritsch (KF) [43] cumulus parameterization, Grell shallow cumulus, Tiedtke cumulus [44,45], MYNN shallow cumulus, the Noah LSM [46], Goddard microphysics [47,48], and Goddard radiation [49,50] parameterization. See Table 2 for specific details.

3.2. Satellite-Based Data Sets

The NSRDB [53] is a satellite-based solar irradiance observational analysis consisting of solar radiation data over the United States and the surrounding countries. In this study, the NSRDB data [30] were used to validate the GHI forecast results of four prediction systems; these data were obtained by regridding the 2 km NSRDB data to fit the configuration of the 9 km WRF-Solar domain. The NSRDB provides instantaneous solar irradiance by 30 min intervals [54], and WRF-Solar is designed to provide 15 min interval instantaneous forecast outputs. In this study, the validation of forecasts was performed using instantaneous data at 30 min time intervals for the 24 h forecast from 0600 UTC of the second day. [30] discussed the predictability of the day-ahead WRF-Solar forecast in 2018 using the NSRDB and ground observations, and the adequacy of the NSRDB as a reference data for the current WRF-Solar performance was verified in depth. The results showed that the NSRDB data are suitable for seasonal and annual timescale GHI evaluation, have sufficient performance compared to ground observations, and have the advantage of overcoming the limitation of the spatial coverage of ground observation.
Evaluations of predicted GHI and cloud optical depth (COD) in four forecasting systems were performed using these 2018 NSRDB data. Figure 3a shows the mean GHI distribution in the NSRDB for 2018, which shows high (low) GHI values in the southwest (northeast) part of the United States. Figure 3b shows the spatial distribution of the frequency of clouds for 2018. The frequency of clouds is analyzed using the difference between all-sky GHI and clear-sky GHI in the NSRDB observations. The methodology to determine whether there is a cloud (or not) is that if the difference between the two values is greater than (less than) 1 W/m2, then it is assumed that there is (is not) a cloud at that grid point at that time [54]. Clouds are present in more than 80% of observations in the northeast United States but less than 45% of observations in Southern California and the Desert Southwest. In addition, it can be confirmed that clouds occur relatively frequently in the Colorado and Wyoming Rocky Mountain areas among the western regions.
In this study, all evaluations are performed for land in the daytime only for observed/simulated GHI pairs on a reduced grid, using one sample of every 5 × 5 WRF grid points. We analyze the single-value forecast by taking the average of 10 ensemble members for the three ensemble systems at any forecast lead time of the second day.

4. Results

4.1. Diurnal and Annual Evaluation Errors

We begin the verification of the day-ahead forecasting performance by comparing diurnal and annual cycle errors from the four WRF-Solar experiments using the NSRDB dataset. Figure 4 shows the diurnal cycles of the bias, root-mean-square error (RMSE), and the spread as a function of forecast lead times in 2018 for the entire CONUS domain. The spread is calculated by the average of the standard deviations between 10 ensemble members. The four experiments show positive bias (up to 45 W/m2) along the forecast lead time. Although WRF-Solar DET, SKEBS, and WRF-Solar PHYS show similar values, WRF-Solar EPS exhibits a slightly higher bias (5–15 W/m2) than the others. The three ensemble forecasts show smaller RMSE than WRF-Solar DET along the forecast lead time; however, SKEBS and WRF-Solar PHYS show 4.5% and 3.5% smaller RMSE than WRF-Solar EPS, respectively. The mean ensemble spread in WRF-Solar EPS (18.6 W/m2) is 27.7% and 29.0% less than that of SKEBS (25.7 W/m2) and WRF-Solar PHYS (26.1 W/m2), respectively. Because the GHI value itself is at its maximum when the solar zenith angle is close to 0, the RMSE also tends to be largest in the middle of the daytime.
This analysis was also performed for clear sky conditions, and we found that the bias and RMSE were noticeably lower than in all sky conditions (not shown). For example, in clear skies, the biases obtained from the four models for the forecast lead time 36 were −15 to −0 W/m2, and the RMSE were 22 to 29 W/m2. These results suggest that cloud-related uncertainties are significant in GHI prediction.
The annual evolution of the metrics underlines the benefits of running ensemble forecasting systems. Figure 5 shows the annual cycles of the 3-day running average bias, mean absolute error (MAE), RMSE, and correlation for the four experiments. The positive bias tends to decrease in spring and increase in summer. In general, the model error is the smallest in winter and the largest in summer. The convective weather in summer can increase the uncertainty of the cloud forecasts, which also affects the solar irradiance forecasts. When comparing the three ensemble systems with the deterministic forecast, the ensemble forecasts show a lower RMSE and MAE but a higher correlation throughout the year. In addition, the correlation of the ensemble forecasts is higher than that of the deterministic forecast throughout the year, and they performed even better during the period of spring to summer. The evolution of the bias between WRF-Solar DET and SKEBS is essentially the same. This is to be expected because SKEBS introduces stochastic perturbations into the stream function that do not directly change the physical quantities associated with the GHI. This means that the physical quantities related to the cloud and radiation process are redistributed, but their total does not change. Surprisingly, WRF-Solar EPS shows a slightly larger bias than WRF-Solar DET. Considering that the method of generating stochastic perturbation in WRF-Solar EPS changes the quantity of a physics variable that directly affects the calculation of the GHI, this can cause a quantitative change in the GHI value itself together with the bias. In the case of the WRF-Solar PHYS ensemble, because all members have different physics configurations, the bias patterns of all 10 ensemble members are different. However, finally, the ensemble mean shows compensated results. Results of SKEBS and WRF-Solar PHYS show slightly better statistics than WRF-Solar EPS; however, the three ensembles are equivalent in terms of the correlation.

4.2. Evaluation of Spatial Distribution of Errors

The continuous coverage of the satellite retrievals from the NSRDB allows us to evaluate the spatial distribution of the errors across CONUS. Figure 6 compares the spatial distributions of the annual average bias, MAE, and correlation for the GHI predictions of the four experiments calculated for day-2 forecasts with 30 min intervals over a year on each grid point. All the WRF-Solar forecasts show an overall positive bias, and WRF-Solar DET, WRF-Solar EPS, and SKEBS show similar patterns of the bias with the largest positive bias being found in the southeastern United States. In particular, WRF-Solar DET and SKEBS show almost the same pattern and size of bias. On the other hand, WRF-Solar EPS shows a similar pattern of bias with WRF-Solar DET and SKEBS, but it clearly shows a slightly higher positive bias compared to the other two. In the northeastern portion of the domain, WRF-Solar EPS largely suppressed a negative bias. Compared to these three forecast results, WRF-Solar PHYS shows a different pattern of bias, with the largest bias in the central United States and a reduced bias in the southeastern areas.
Looking at the MAE, all forecasts show the largest MAE in the eastern United States. In particular, the WRF-Solar DET produces a larger error compared to the other ensemble forecasts. SKEBS and WRF-Solar PHYS show different patterns in bias, but in terms of MAE, two forecast results show very similar error distributions. The correlation coefficient generally shows high values (0.94–0.98) in the western United States and lower values (0.82–0.90) in the northeastern United States. This is the opposite pattern of the MAE. The three ensemble forecasts show a higher overall correlation than the deterministic forecast in most regions, especially the eastern region.
Figure 7 shows the spatial distribution of the percentage of forecast error reduction in terms of MAE for the three ensemble predictions compared to WRF-Solar DET. WRF-Solar EPS improved the overall predictability by 7.5% in most regions compared to WRF-Solar DET. SKEBS and WRF-Solar PHYS significantly improved the predictability for the eastern United States, showing an increased predictability by up to 20% on specific locations with an average error reduction of 11.5%. Comparing the spatial distribution of the error reduction of SKEBS and WRF-Solar PHYS, SKEBS showed a relatively large error reduction trend for the central United States region, and WRF-Solar PHYS showed a decreased error for the southern coastal regions in Florida, Louisiana, and Texas.

4.3. Uncertainty Quantification

A rank histogram [55,56] is one way to evaluate the statistical consistency of the ensemble. It requires computing the probability of the occurrence of observation in each set of (n + 1) forecast bins in increasing order. When an ensemble is statistically consistent, the probability of the occurrence of observations in each forecast bin should be equally likely, so that the rank histogram is flat and has a uniform rank probability of 1/(n + 1). If the rank histogram is flat, the ensemble is also referred to as reliable. A rank histogram can be presented, together with the missing rate error (MRE) [22], to provide further information on the spread of the ensemble. This is, the fraction of observations falling outside the highest/lowest ranked prediction, above or below the expected missing rate of 1/(n + 1). A larger positive (negative) MRE reveals a more under-dispersive (over-dispersive) ensemble.
Ref. [31] analyzed rank histogram and spread–skill diagrams to evaluate the reliability of WRF-Solar EPS and its calibrated ensembles. In this study, the same analysis was performed for the three raw ensemble forecasts: WRF-Solar EPS, SKEBS, and WRF-Solar PHYS. Figure 8 compares the rank histogram and MRE of the three ensemble forecast systems. All the ensembles show U-shape histograms, revealing under-dispersion in a very similar way. A closer look reveals that WRF-Solar EPS has the highest left bars, which means that small GHI values exist more frequently in the observation than in the prediction. It can be interpreted that the optically thick clouds are simulated less frequently in WRF-Solar EPS compared to SKEBS and WRF-Solar PHYS, and this can be supported by the result in part 4.4. This result is consistent with the higher positive bias signal shown by WRF-Solar EPS in the previous sections. The large positive MREs of the three ensembles provide a quantification of the under-dispersive nature of the ensembles due to the tendency of observations to fall outside the spread of the ensemble. Among the three ensembles, WRF-Solar PHYS has a relatively better dispersion relationship (MRE = 35.02%), followed by SKEBS (MRE = 40.34%), and WRF-Solar EPS (MRE = 46.25%).
In the spread–skill diagram in Figure 9, the ensemble spread is compared to the RMSE of the ensemble mean over each bin of intervals for the two metrics. Considering that the diagonal 1:1 line represents a perfect spread–skill agreement, the spread of the three ensembles underestimates the magnitude of the error quantified in terms of the RMSE. It shows that SKEBS and WRF-Solar PHYS are closer to the 1:1 line compared to WRF-Solar EPS for the 2018 forecast experiments, but that still all experiments show under-dispersive ensembles.

4.4. Cloud Detection Evaluations

The ability to predict the presence or absence of clouds is an essential part of accurate surface irradiance predictions. In this section, the frequency of the predicted clouds of the four experiments was analyzed and compared with the cloud frequency shown in the NSRDB. The difference between all-sky GHI and clear-sky GHI is used to determine whether (or not) there is a cloud at that grid point at that time. If the difference between the two values is greater than (less than) 1 W/m2, then it is assumed that there is (is not) a cloud. In the case of the three ensembles, if the difference between the two values is greater than 1 W/m2 in 50% or more of the ensemble members, it is considered that there is a cloud, which follows the methodology of [54]. We evaluated the predictability by classifying it into four categories, as shown in Table 3, according to whether or not a cloud was detected in the NSRDB and in each model forecast. For example, “CN” indicates that there is a cloud in the NSRDB but no cloud in the forecast, and “CC” means that clouds are detected in both the NSRDB and the forecast.
Figure 10 presents the overall frequencies of clouds classified into the four categories shown in Table 3 for the four forecast experiments. The probability of detection of clear skies, NN/(NC+NN) × 100, is well represented in the four experiments at 82–85%, whereas a large portion of missed clouds, CN/(CN + CC) × 100, is found under cloudy-sky conditions, by 29–35%. All experiments produced fewer cloud occurrences than the NSRDB; however, the three ensemble systems represent cloud masks slightly better than WRF-Solar DET. The rate of having clouds in both the NSRDB and the model forecast, CC/(CC + CN + NC + NN) × 100, is 45–49%, whereas the ratio in which the NSRDB has clouds but the model does not, CN/(CC + CN + NC + NN) × 100, is 20–24%, showing that the model frequently misses clouds. Another notable point is that WRF-Solar PHYS detects clouds more frequently in both the presence (CC) and absence (NC) of actual clouds compared to other predictions. It can be inferred that various types of clouds can be simulated ultiphysicsiphysics ensembles with various combinations of physical parameterizations, whether they simulated real clouds or fake clouds.
To evaluate the spatial distribution of the accuracy of clouds in the four forecasting systems, the Peirce skill score (PSS) [57,58] in Equation (2) was calculated for each grid point based on the contingency shown in Table 3:
P S S = C C · N N − C N · N C ( C C + C N ) · ( N C + M M ) × 100 ,
The PSS considers climatological relative frequencies to reduce the influence of different backgrounds representing clouds across CONUS. Figure 11 compares the cloud occurrence frequencies in terms of the PSS. The four forecasting systems tend to successfully capture the clouds in the eastern and northwestern regions of the United States, showing the PSS to be above 70%, whereas clouds are poorly collected in Kansas, Oklahoma, and northern Texas, resulting in a significant drop in the PSS to 50%. WRF-Solar EPS, SKEBS, and WRF-Solar PHYS show higher PSS than WRF-Solar DET, and the area-averaged PSS of WRF-Solar PHYS is approximately 1.4% and 2.8% higher than that of SKEBS and WRF-Solar EPS, respectively. Again, because the 10 ensemble members have different configurations of physical parameterizations in WRF-Solar PHYS, the cloud position and amounts in each ensemble member are highly independent of each other. This diversity can bring a positive impact to cloud detection.
To compare the cloud optical properties of the clouds generated from each experiment, the frequencies of each size range of the COD were compared in four experiments (Figure 12). Looking at the COD range from 0.01 to 1, the SKEBS shows a smaller frequency than the other models. In the COD range from 1 to 10, WRF-Solar PHYS produces the highest frequency, followed by WRF-Solar EPS and SKEBS, and WRF-Solar DET represents the lowest frequency. WRF-Solar DET again shows the lowest frequency in the COD range from 10 to 20, whereas the rest of the models show similar distributions in this range. For optically thick clouds (COD > 20), SKEBS shows a much higher frequency than the other models, and WRF-Solar EPS exhibits much lower frequencies than the other models for clouds with a COD of 30 or more. In summary, WRF-Solar DET produces optically thin (COD < 1) and optically thick clouds (COD > 30) more frequently than the other models, and SKEBS produces optically thick clouds (COD > 20) more frequently than the others. WRF-Solar PHYS and WRF-Solar EPS produce optically medium clouds (1 < COD < 10) more frequently than the other two, and WRF-Solar EPS shows the least amount of optically thick clouds (COD > 20) compared to other models. This could correspond to the previous results, in which WRF-Solar EPS has a more positive bias than the other forecasting systems.

5. Conclusions

WRF-Solar EPS was recently developed by introducing stochastic perturbations into the most relevant physical variables for solar irradiance predictions. In this study, we evaluated the performance of WRF-Solar EPS with day-ahead solar irradiance predictions by comparing it with the WRF-Solar deterministic forecast (WRF-Solar DET), SKEBS, and a WR15ultiphysicsiphysics ensemble (WRF-Solar PHYS) through a forecasting experiment spanning the year of 2018. Satellite-based GHI estimations from the NSRDB were used to quantify the forecasting errors of the four prediction systems. The continuous spatial resolution of the NSRDB across CONUS allowed for analyzing not only diurnal and annual cycles of the forecast errors but also their spatial characteristics in terms of bias, MAE, and correlation. For the three ensemble forecasts, the overall probabilistic attributes of the ensemble, such as reliability and statistical consistency, were analyzed through rank histograms and spread–skill diagrams. We also analyzed the predicted cloud mask from the four forecasting systems using the difference between the clear-sky GHI and the all-sky GHI at each grid point, and we compared the frequencies by size for the predicted cloud optical thickness.
The important features of the evaluation are summarized as follows: (1) The impact of the stochastic perturbations in WRF-Solar EPS is larger in cloudy regions than in clear-sky conditions. (2) WRF-Solar EPS shows slightly higher positive bias than WRF-Solar DET, SKEBS, and WRF-Solar PHYS in the GHI forecast, but the forecast error in terms of MAE decreases by 8.5% compared to WRF-Solar DET with increasing spatial correlation to the observations. In SKEBS and WRF-Solar PHYS, the forecast errors decrease in the central and eastern United States and the southeastern United States, respectively. (3) The three ensemble predictions improve the quality of the GHI forecasts compared to the deterministic predictions, but the ensemble forecast predictions appear to be under-dispersive, unreliable, and overconfident. (4) In cloud mask prediction, the four systems show much lower predictability in some regions in the central United States compared to the NSRDB. Meanwhile SKEBS and WRF-Solar PHYS showed better performance in cloud detection than WRF-Solar EPS. (5) SKEBS produces optically thick clouds (COD > 20) more frequently than the others, whereas WRF-Solar EPS generates clouds with COD greater than 30 with the least frequency.
In SKEBS, the stochastic perturbation is introduced to the stream function and potential temperature, and that impact affects the prediction of the atmospheric momentum over the whole forecast period. On the other hand, in WRF-Solar EPS, stochastic perturbations are added at every model integration time step to 14 variables that directly affect cloud and radiation prediction, but they are subtracted after calculating the tendencies, which limits the perturbation impact on the physics tendencies. Therefore, the uncertainty of cloud position appears to be considered over a wider range in SKEBS than in WRF-Solar EPS. If uncertainty in momentum is accounted for in the future WRF-Solar EPS, it is expected that there will be an improvement in position detection for clouds that do not stand still and move continuously.
Both SKEBS and WRF-Solar EPS used the same physics configuration as WRF-Solar DET. Nevertheless, optically thick clouds were frequently detected in SKEBS, whereas relatively optically thin clouds (1 < COD < 10) were more frequently generated in WRF-Solar EPS. WRF-Solar EPS first changes the amount of cloud by adding stochastic perturbations to the clouds in each ensemble member, and then calculating the average of the 10 ensemble members. This contributed to increasing the overall cloud hit rate compared to WRF-Solar DET, but resulted in a decreased number of optically thick clouds. Remembering that cloud transmittances and reflectances do not have a linear relationship to the cloud size in the radiation process [36], the cloud thickness redistribution of WRF-Solar EPS would have had a significant effect on the positive bias of GHI prediction. Figure 13 shows the results calculated by the FARMS single-column mode, representing that the total amount of GHI changes can lead to a positive bias when the cloud water amount is changed linearly. When the amount of cloud water is increased to 20%, the calculated GHI is decreased to 30.0 W/m2. Conversely, when the cloud amount is decrease to 20%, the GHI is increased to 35.8 W/m2. In the future, this nonlinear relationship between the cloud amount and the GHI should be accounted for when designing the formation of stochastic perturbations.
The high cloud detection rate in WRF-Solar PHYS indicates that there is a possibility to detect more various types of clouds when using various combinations of physical parameterizations. Cloud microphysics is an important one, and the assessment should take into account the representation of cloud optical properties. Therefore, finding several configurations that perform well in GHI prediction and operating them with WRF-Solar EPS would also help to improve surface irradiance predictability.

Author Contributions

Conceptualization, J.-H.K. and P.A.J.; methodology, J.-H.K., P.A.J., J.D., S.A. and J.Y.; investigation, J.-H.K.; data preparation, J.-H.K. and J.Y.; validation, J.-H.K.; draft writing, J.-H.K.; review and revise, P.A.J., J.D., J.Y., M.S. and S.A.; funding acquisition, M.S. and P.A.J.; supervision, P.A.J. All authors have read and agreed to the published version of the manuscript.

Funding

This work was authored by the National Renewable Energy Laboratory, operated by Alliance for Sustainable Energy, LLC, for the U.S. Department of Energy (DOE) under Contract DE-AC36-08GO28308. Funding was provided by the U.S. DOE Office of Energy Efficiency and Renewable Energy Solar Energy Technologies Office (grant number 33503). The views expressed in the article do not necessarily represent the views of the DOE or the U.S. government. The U.S. government retains and the publisher, by accepting the article for publication, acknowledges that the U.S. government retains a non- exclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this work, or allow others to do so, for U.S. government purposes. This material is also based upon work performed at the National Center for Atmospheric Research (NCAR), which is a major facility sponsored by the National Science Foundation under Cooperative Agreement 1852977.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Acknowledgments

We acknowledge high-performance computing support from Cheyenne (https://0-doi-org.brum.beds.ac.uk/10.5065/D6RX99HX) provided by NCAR’s Computational and Information Systems Laboratory, sponsored by the National Science Foundation.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Jimenez, P.A.; Hacker, J.P.; Dudhia, J.; Haupt, S.E.; Ruiz-Arias, J.A.; Gueymard, C.A.; Thompson, G.; Eidhammer, T.; Deng, A. WRF-Solar: Description and clear sky assessment of an augmented nwp model for solar power prediction. Bull. Amer. Met. Soc. 2016, 97, 1249–1264. [Google Scholar] [CrossRef]
  2. Skamarock, W.C.; Klemp, J.B.; Dudhia, J.; Gill, D.O.; Liu, Z.; Berner, J.; Wang, W.; Powers, J.G.; Duda, M.G.; Barker, D. A Description of the Advanced Research WRF Version 4; NCAR Technical Note. NCAR/TN-556+STR; NCAR: Boulder, CO, USA, 2019; 145p. [Google Scholar] [CrossRef]
  3. Jiménez, P.A.; Alessandrini, S.; Haupt, S.E.; Deng, A.; Kosovic, B.; Lee, J.A.; delle Monache, L. The role of unresolved clouds on short-range global horizontal irradiance predictability. Mon. Wea. Rev. 2016, 144, 3099–3107. [Google Scholar] [CrossRef]
  4. Alapaty, K.; Herwehe, J.A.; Otte, T.L.; Nolte, C.G.; Russell Bullock, O.; Mallard, M.S.; Kain, J.S.; Dudhia, J. Introducing subgrid-scale cloud feedbacks to radiation for regional meteorological and climate modeling. Geophys. Res. Lett. 2000, 39, L24809. [Google Scholar] [CrossRef] [Green Version]
  5. Berg, L.K.; Gustafson, W.I., Jr.; Kassianov, E.I.; Deng, L. Evaluation of a modified scheme for shallow convection: Implementation of CuP and case studies. Mon. Wea. Rev. 2013, 141, 134–147. [Google Scholar] [CrossRef]
  6. Herwehe, J.; Alapaty, K.; Spero, T.; Nolte, C.G. Increasing the credibility of regional climate simulations by introducing subgrid-scale cloud-radiation interactions. J. Geophys. Res. Atmos. 2014, 119, 5317–5330. [Google Scholar] [CrossRef]
  7. Deng, A.; Gaudet, B.J.; Dudhia, J.; Alapaty, K. Implementation and evaluation of a new shallow convection scheme in WRF. In Proceedings of the 26th Conference on Weather Analysis and Forecasting/22nd Conference on Numerical Weather Prediction, Atlanta, GA, USA, 2–6 February 2014; American Meteorological Society: Boston, MA, USA, 2014. Available online: https://ams.confex.com/ams/94Annual/webprogram/Paper236925.html (accessed on 18 November 2022).
  8. Pedruzo-Bagazgoitia, X.; Jiménez, P.A.; Dudhia, J.; Vilà-Guerau de Arellano, J. Shallow cumulus representation and its interaction with radiation and surface at the convection grey zone. Mon. Wea. Rev. 2019, 147, 2467–2483. [Google Scholar] [CrossRef]
  9. Bauer, P.; Thorpe, A.; Brunet, G. The quiet revolution of numerical weather prediction. Nature 2015, 525, 47–55. [Google Scholar] [CrossRef]
  10. Sweeney, C.; Bessa, R.J.; Browell, J.; Pinson, P. The future of forecasting for renewable energy. WIREs Energy Environ. 2020, 9, e365. [Google Scholar] [CrossRef]
  11. Li, B.; Zhang, J. A review on the integration of probabilistic solar forecasting in power systems. Sol. Energy 2020, 210, 68–86. [Google Scholar] [CrossRef]
  12. Takamatsu, T.; Ohtake, H.; Oozeki, T.; Nakaegawa, T.; Honda, Y.; Kazumori, M. Regional Solar Irradiance Forecast for Kanto Region by Support Vector Regression Using Forecast of Meso-Ensemble Prediction System. Energies 2021, 14, 3245. [Google Scholar] [CrossRef]
  13. Yang, D.; Wang, W.; Gueymard, C.A.; Hong, T.; Kleissl, J.; Huang, J.; Perez, M.J.; Perez, R.; Bright, J.M.; Xia, X.; et al. A review of solar forecasting, its dependence on atmospheric sciences and implications for grid integration: Towards carbon neutrality. Renew. Sustain. Energy Rev. 2022, 161, 112348. [Google Scholar] [CrossRef]
  14. Molteni, F.; Palmer, T.N. Predictability and finite-time instability of the northern winter circulation. Quart. J. Roy. Meteor. Soc. 1993, 119, 269–298. [Google Scholar] [CrossRef]
  15. Toth, Z.; Kalnay, E. Ensemble forecasting at NMC: The generation of perturbations. Bull. Amer. Meteor. Soc. 1993, 74, 2317–2330. [Google Scholar] [CrossRef]
  16. Buizza, R.; Palmer, T. The singular-vector structure of the atmospheric global circulation. J. Atmos. Sci. 1995, 52, 1434–1456. [Google Scholar] [CrossRef]
  17. Buizza, R.; Houtekamer, P.L.; Toth, Z.; Pellerin, G.; Wei, M.; Zhu, Y. A comparison of the ECMWF, MSC, and NCEP global ensemble prediction systems. Mon. Wea. Rev. 2005, 133, 1076–1097. [Google Scholar] [CrossRef]
  18. Hacker, J.P.; Ha, S.Y.; Snyder, C.; Berner, J.; Eckel, F.A.; Kuchera, E.; Pocernich, M.; Rugg, S.; Schramm, J.; Wang, X. The U.S. Air Force Weather Agency’s mesoscale ensemble: Scientific description and performance results. Tellus 2011, 63A, 625–641. [Google Scholar] [CrossRef] [Green Version]
  19. Berner, J.; Ha, S.-Y.; Hacker, J.P.; Fournier, A.; Snyder, C. Model uncertainty in a mesoscale ensemble prediction system: Stochastic versus multiphysics representations. Mon. Wea. Rev. 2011, 139, 1972–1995. [Google Scholar] [CrossRef] [Green Version]
  20. Berner, J.; Fossell, K.; Ha, S.-Y.; Hacker, J.P.; Snyder, C. Increasing the skill of probabilistic forecasts: Understanding performance improvements from model-error representations. Mon. Wea. Rev. 2015, 143, 1295–1320. [Google Scholar] [CrossRef]
  21. Knutti, R.; Masson, D.; Gettelman, A. Climate model genealogy: Generation CMIP5 and how we got there. Geo Phys. Res. Lett. 2013, 40, 1194–1199. [Google Scholar] [CrossRef]
  22. Eckel, F.A.; Mass, C.F. Aspects of effective mesoscale, short-range ensemble forecasting. Wea. Forecast. 2005, 20, 328–350. [Google Scholar] [CrossRef]
  23. Jankov, I.; Berner, J.; Beck, J.; Jiang, H.; Olson, J.B.; Grell, G.; Smirnova, T.G.; Benjamin, S.G.; Brown, J.M. A performance comparison between multiphysics and stochastic approaches within a North American RAP ensemble. Mon. Wea. Rev. 2017, 145, 1161–1179. [Google Scholar] [CrossRef]
  24. Palmer, T.N. A nonlinear dynamical perspective on model error: A proposal for non-local stochastic-dynamic parameterization in weather and climate prediction. Quart. J. Roy. Meteor. Soc. 2001, 127, 279–304. [Google Scholar] [CrossRef]
  25. Berner, J.; Shutts, G.; Leutbecher, M.; Palmer, T. A spectral stochastic kinetic energy backscatter scheme and its impact on flow-depended predictability in the ECMWF ensemble prediction system. J. Atmos. Sci. 2009, 66, 603–626. [Google Scholar] [CrossRef]
  26. Bowler, N.E.; Arribas, A.; Mylne, K.R.; Robertson, K.B.; Beare, S.E. The MOGREPS short-range ensemble prediction system. Quart. J. Roy. Meteor. Soc. 2008, 134, 703–722. [Google Scholar] [CrossRef]
  27. Sanchez, C.; Williams, K.D.; Collins, M. Improved stochastic physics schemes for global weather and climate models. Quart. J. Roy. Meteor. Soc. 2016, 142, 147–159. [Google Scholar] [CrossRef]
  28. Shutts, G.J. A kinetic energy backscatter algorithm for use in ensemble prediction systems. Quart. J. Roy. Meteor. Soc. 2005, 131, 3079–3102. [Google Scholar] [CrossRef] [Green Version]
  29. Yang, J.; Kim, J.-H.; Jimenez, P.A.; Sengupta, M.; Dudhia, J.; Xie, Y.; Golnas, A.; Giering, R. An efficient method to identify uncertainties of WRF-Solar variables in forecasting solar irradiance using a tangent linear sensitivity analysis. Sol. Energy 2021, 220, 509–522. [Google Scholar] [CrossRef]
  30. Jiménez, P.A.; Yang, J.; Kim, J.-H.; Sengupta, M.; Dudhia, J. Assessing the WRF-Solar model performance using satellite-derived irradiance from the National Solar Radiation Database. J. Appl. Meteor. Climatol. 2022, 61, 129–142. [Google Scholar] [CrossRef]
  31. Kim, J.-H.; Jimenez, P.A.; Sengupta, M.; Yang, J.; Dudhia, J.; Alessandrini, S.; Xie, Y. The WRF-solar ensemble prediction system to provide solar irradiance probabilistic forecasts. IEEE J. Photovolt. 2022, 12, 141–144. [Google Scholar] [CrossRef]
  32. Thompson, G.; Field, P.R.; Rasmussen, R.M.; Hall, W.D. Explicit forecasts of winter precipitation using an improved bulk microphysics scheme. Part II: Implementation of a new snow parameterization. Mon. Wea. Rev. 2008, 136, 5095–5115. [Google Scholar] [CrossRef]
  33. Nakanishi, M.; Niino, H. Development of an improved turbulence closure model for the atmospheric boundary layer. J. Meteor. Soc. Japan. 2009, 87, 895–912. [Google Scholar] [CrossRef] [Green Version]
  34. Chen, F.; Dudhia, J. Coupling an advanced land surface hydrology model with the penn state/ncar mm5 modeling system. part 1: Model description and implementation. Mon. Wea. Rev. 2001, 129, 569–586. [Google Scholar] [CrossRef]
  35. Deng, A.; Seaman, N.L.; Kain, J.S. A shallow-convection parameterization for mesoscale models. Part I: Submodel description and preliminary applications. J. Atmos. Sci. 2003, 60, 34–56. [Google Scholar] [CrossRef]
  36. Xie, Y.; Sengupta, M.; Dudhia, J. A Fast All-sky Radiation Model for Solar applications (FARMS): Algorithm and performance evaluation. Sol. Energy 2016, 135, 435–445. [Google Scholar] [CrossRef] [Green Version]
  37. Mocko, D.M.; Cotton, W.R. Evaluation of fractional cloudiness parameterizations for use in a messoscale model. J. Atmos. Sci. 1995, 52, 2884–2901. [Google Scholar] [CrossRef]
  38. Palmer, T.N.; Buizza, R.; Doblas-Reyes, F.; Jung, T.; Leutbecher, M.; Shutts, G.; Steinheimer, M.; Weisheimer, A. Stochastic parametrization and model uncertainty. ECMWF Tech. Memo. 2009, 598, 44. Available online: https://www.ecmwf.int/sites/default/files/elibrary/2009/11577-stochastic-parametrization-and-model-uncertainty.pdf (accessed on 18 November 2022).
  39. Iacono, M.J.; Delamere, J.S.; Mlawer, E.J.; Shephard, M.W.; Clough, S.A.; Collins, W.D. Radiative forcing by long-lived greenhouse gases: Calculations with the AER radiative transfer models. J. Geophys. Res. 2008, 113, D13. [Google Scholar] [CrossRef]
  40. Thompson, G.; Eidhammer, T. A study of aerosol impacts on clouds and precipitation development in a large winter cyclone. J. Atmos. Sci. 2014, 71, 3636–3658. [Google Scholar] [CrossRef]
  41. Alessandrini, S.; Vandenberghe, F.; Hacker, J. Definition of typical-day dispersion patterns as a consequence of a hazardous release. Int. J. Environ. Pollut. 2017, 62, 305–318. [Google Scholar] [CrossRef]
  42. Grell, G.A.; Freitas, S.R. A scale and aerosol aware stochastic convective parameterization for weather and air quality modeling. Atmos. Chem. Phys. 2014, 14, 5233–5250. [Google Scholar] [CrossRef]
  43. Kain, J.S. The Kain–Fritsch convective parameterization: An update. J. Appl. Meteor. 2004, 43, 170–181. [Google Scholar] [CrossRef]
  44. Tiedtke, M. A comprehensive mass flux scheme for cumulus parameterization in large-scale models. Mon. Wea. Rev. 1989, 117, 1779–1800. [Google Scholar] [CrossRef]
  45. Zhang, C.; Wang, Y.; Hamilton, K. Improved representation of boundary layer clouds over the southeast Pacific in ARW-WRF using a modified Tiedtke cumulus parameterization scheme. Mon. Wea. Rev. 2011, 139, 3489–3513. [Google Scholar] [CrossRef] [Green Version]
  46. Niu, Z.W.; DAN, L.; Yang, Z.Z. The community Noah land surface model with multiparameterization options (Noah-MP): 1. Model description and evaluation with local-scale measurements. J. Geophys. Res. 2011, 116, D12109. [Google Scholar] [CrossRef] [Green Version]
  47. Tao, W.K.; Simpson, J.; Baker, D.; Braun, S.; Chou, M.D.; Ferrier, B.; Johnson, D.; Khain, A.; Lang, S.; Lynn, B. Microphysics, radiation and surface processes in the Goddard Cumulus Ensemble (GCE) model. Meteor. Atmos. Phys. 2003, 82, 97–137. [Google Scholar] [CrossRef]
  48. Lang, S.; Tao, W.-K.; Cifelli, R.; Olson, W.; Halverson, J.; Rutledge, S.; Simpson, J. Improving simulations of convective system from TRMM LBA: Easterly and Westerly regimes. J. Atmos. Sci. 2007, 64, 1141–1164. [Google Scholar] [CrossRef] [Green Version]
  49. Chou, M.-D.; Suarez, M.J. A Solar Radiation Parameterization for Atmospheric Studies; NASA Technical Report NASA/TM-1999-104606; NASA: Washington, DC, USA, 1999; Volume 15, 51p. Available online: https://ntrs.nasa.gov/citations/19990060930 (accessed on 18 November 2022).
  50. Chou, M.-D.; Suarez, M.J. A Thermal Infrared Radiation Parameterization for Atmospheric Studies; NASA Technical Report NASA/TM-2001-104606; NASA: Washington, DC, USA, 2001; Volume 19, 68p. Available online: https://ntrs.nasa.gov/citations/20010072848 (accessed on 18 November 2022).
  51. Tegen, I.; Hollrig, P.; Chin, M.; Fung, I.; Jacob, D.; Penner, J. Contribution of different aerosol species to the global aerosol extinction optical thickness: Estimates from model results. J. Geophys. Res. 1997, 102, 23895–23915. [Google Scholar] [CrossRef]
  52. Ruiz-Arias, J.A.; Dudhia, J.; Gueymard, C.A. A simple parameterization of the short-wave aerosol optical properties for surface direct and diffuse irradiances assessment in a numerical weather model. Geosci. Model Dev. 2014, 7, 1159–1174. [Google Scholar] [CrossRef] [Green Version]
  53. Sengupta, M.; Xie, Y.; Lopez, A.; Habte, A.; Maclaurin, G.; Shelby, J. The national solar radiation data base (NSRDB). Renew. Sustain. Energy Rev. 2018, 89, 51–60. [Google Scholar] [CrossRef]
  54. Yang, J.; Sengupta, M.; Jimenez, P.A.; Kim, J.-H.; Jimenez, P.A.; Xie, Y. Evaluating WRF-Solar EPS cloud mask forecast using the NSRDB. Sol. Energy 2022, 243, 348–360. [Google Scholar] [CrossRef]
  55. Anderson, J.L. A method for producing and evaluating probabilistic forecasts from ensemble model integrations. J. Clim. 1996, 9, 1518–1530. [Google Scholar] [CrossRef]
  56. Hamill, T.M. Interpretation of rank histograms for verifying ensemble forecasts. Mon. Wea. Rev. 2001, 129, 550–560. [Google Scholar] [CrossRef]
  57. Peirce, C.S. The numerical measure of the success of predictions. Science 1884, 4, 453–454. [Google Scholar] [CrossRef] [PubMed]
  58. Mason, I.B. Binary events. In Forecast Verification: A Practitioner’s Guide in Atmospheric Science; Jolliffe, I.T., Stephenson, D.B., Eds.; John and Wiley and Sons: Hoboken, NJ, USA, 2003; pp. 37–76. [Google Scholar]
Atmosphere 13 01932 g001 550
Figure 1. The instantaneous pattern of the stochastic perturbations for the (a) aerosol optical depth (AOD) and (b) turbulent kinetic energy (TKE).
Figure 1. The instantaneous pattern of the stochastic perturbations for the (a) aerosol optical depth (AOD) and (b) turbulent kinetic energy (TKE).
Atmosphere 13 01932 g001
Atmosphere 13 01932 g002 550
Figure 2. Time series of observed (black) and predicted GHI from WRF-Solar DET (red) and WRF-Solar EPS (blue) from 15 April to 17 April 2018 at TBL (Table Mountain, Colorado), BON (Bondville, Illinois), and SXF (Sioux Falls, South Dakota) SURFRAD sites. The thin blue line is the forecast result from each ensemble member, and the thick line indicates the mean of 10 ensembles.
Figure 2. Time series of observed (black) and predicted GHI from WRF-Solar DET (red) and WRF-Solar EPS (blue) from 15 April to 17 April 2018 at TBL (Table Mountain, Colorado), BON (Bondville, Illinois), and SXF (Sioux Falls, South Dakota) SURFRAD sites. The thin blue line is the forecast result from each ensemble member, and the thick line indicates the mean of 10 ensembles.
Atmosphere 13 01932 g002
Atmosphere 13 01932 g003 550
Figure 3. (a) Mean GHI and (b) frequencies of clouds from the NSRDB for 2018.
Figure 3. (a) Mean GHI and (b) frequencies of clouds from the NSRDB for 2018.
Atmosphere 13 01932 g003
Atmosphere 13 01932 g004 550
Figure 4. (a) GHI bias and (b) RMSE (solid lines) as a function of the forecast lead time from WRF-Solar DET (black), WRF-Solar EPS (blue), SKEBS (green), and WRF-Solar PHYS (red) against the NSRDB for 2018 for the CONUS domain, and (b) ensemble spread (dashed lines) of 10 members from WRF-Solar EPS (blue), SKEBS (green), and WRF-Solar PHYS (red).
Figure 4. (a) GHI bias and (b) RMSE (solid lines) as a function of the forecast lead time from WRF-Solar DET (black), WRF-Solar EPS (blue), SKEBS (green), and WRF-Solar PHYS (red) against the NSRDB for 2018 for the CONUS domain, and (b) ensemble spread (dashed lines) of 10 members from WRF-Solar EPS (blue), SKEBS (green), and WRF-Solar PHYS (red).
Atmosphere 13 01932 g004
Atmosphere 13 01932 g005 550
Figure 5. Three-day moving average of the (a) bias, (b) MAE, (c) RMSE, and (d) correlation of the GHI forecasts from WRF-Solar DET (black), WRF-Solar EPS (blue), SKEBS (green), and WRF-Solar PHYS (red) for 2018.
Figure 5. Three-day moving average of the (a) bias, (b) MAE, (c) RMSE, and (d) correlation of the GHI forecasts from WRF-Solar DET (black), WRF-Solar EPS (blue), SKEBS (green), and WRF-Solar PHYS (red) for 2018.
Atmosphere 13 01932 g005
Atmosphere 13 01932 g006 550
Figure 6. Bias (first column), MAE (second column), and correlation (third column) of the GHI forecasts for 2018 from WRF-Solar DET (first row), WRF-Solar EPS (second row), SKEBS (third row), and WRF-Solar PHYS (fourth row).
Figure 6. Bias (first column), MAE (second column), and correlation (third column) of the GHI forecasts for 2018 from WRF-Solar DET (first row), WRF-Solar EPS (second row), SKEBS (third row), and WRF-Solar PHYS (fourth row).
Atmosphere 13 01932 g006
Atmosphere 13 01932 g007 550
Figure 7. Forecast improvements in the MAE with respect to the WRF-Solar DET simulation for the forecast experiments (a) WRF-Solar EPS, (b) SKEBS, and (c) WRF-Solar PHYS.
Figure 7. Forecast improvements in the MAE with respect to the WRF-Solar DET simulation for the forecast experiments (a) WRF-Solar EPS, (b) SKEBS, and (c) WRF-Solar PHYS.
Atmosphere 13 01932 g007
Atmosphere 13 01932 g008 550
Figure 8. Rank histograms for the probabilistic prediction of GHI for (a) WRF-Solar EPS, and (b) SKEBS, and (c) WRF-Solar PHYS. The gray bars show the frequency of occurrence of the observation in each rank. The solid black line, which is 1/(n + 1) when n equals 10, represents a perfect uniform probability for an n-member ensemble.
Figure 8. Rank histograms for the probabilistic prediction of GHI for (a) WRF-Solar EPS, and (b) SKEBS, and (c) WRF-Solar PHYS. The gray bars show the frequency of occurrence of the observation in each rank. The solid black line, which is 1/(n + 1) when n equals 10, represents a perfect uniform probability for an n-member ensemble.
Atmosphere 13 01932 g008
Atmosphere 13 01932 g009 550
Figure 9. Binned spread–skill diagram of GHI [W/m2] for WRF-Solar EPS (blue), SKEBS (green), and WRF-Solar PHYS (red). The ensemble spread is binned into 13 equally populated class intervals.
Figure 9. Binned spread–skill diagram of GHI [W/m2] for WRF-Solar EPS (blue), SKEBS (green), and WRF-Solar PHYS (red). The ensemble spread is binned into 13 equally populated class intervals.
Atmosphere 13 01932 g009
Atmosphere 13 01932 g010 550
Figure 10. Frequency of cloud occurrences in the NSRDB and WRF-Solar DET, WRF-Solar EPS, SKEBS, and WRF-Solar PHYS for 2018.
Figure 10. Frequency of cloud occurrences in the NSRDB and WRF-Solar DET, WRF-Solar EPS, SKEBS, and WRF-Solar PHYS for 2018.
Atmosphere 13 01932 g010
Atmosphere 13 01932 g011 550
Figure 11. Spatial distribution of the Peirce skill score (PSS) of cloud detection for 2018 in WRF-Solar DET, WRF-Solar EPS, SKEBS, and WRF-Solar PHYS.
Figure 11. Spatial distribution of the Peirce skill score (PSS) of cloud detection for 2018 in WRF-Solar DET, WRF-Solar EPS, SKEBS, and WRF-Solar PHYS.
Atmosphere 13 01932 g011
Atmosphere 13 01932 g012 550
Figure 12. The frequency of clouds with respect to various cloud optical depth (COD) in WRF-Solar DET, WRF-Solar EPS, SKEBS, and WRF-Solar PHYS for 2018.
Figure 12. The frequency of clouds with respect to various cloud optical depth (COD) in WRF-Solar DET, WRF-Solar EPS, SKEBS, and WRF-Solar PHYS for 2018.
Atmosphere 13 01932 g012
Atmosphere 13 01932 g013 550
Figure 13. Changes in the cloud mixing ratio for an arbitrary atmospheric condition and corresponding GHI values. The default amount of cloud mixing ratio is 0.367 g kg−1, and the amount of cloud water was subtracted or added linearly by 20%.
Figure 13. Changes in the cloud mixing ratio for an arbitrary atmospheric condition and corresponding GHI values. The default amount of cloud mixing ratio is 0.367 g kg−1, and the amount of cloud water was subtracted or added linearly by 20%.
Atmosphere 13 01932 g013
Table
Table 1. Summary of the characteristics of the 14 stochastic perturbations in WRF-Solar EPS. Each perturbation in the selected modules is characterized by the standard deviation of a Gaussian distribution (σ), the horizontal wavelength (λ), and the decorrelation time (τ), and the variable dimension (ω = 0(1) indicates the variable is 2(3)-dimensional).
Table 1. Summary of the characteristics of the 14 stochastic perturbations in WRF-Solar EPS. Each perturbation in the selected modules is characterized by the standard deviation of a Gaussian distribution (σ), the horizontal wavelength (λ), and the decorrelation time (τ), and the variable dimension (ω = 0(1) indicates the variable is 2(3)-dimensional).
pVariable NameSelected Modules σ λ τ ω
1AlbedoFARMS0.1100,00086,4000
2Aerosol optical depthFARMS0.25100,00036000
3Ångström wavelength exponentFARMS0.1100,00036000
4Asymmetry factorFARMS0.05100,00036000
5Water vapor mixing ratioFARMS, MYNN, Thompson, Noah, Deng, and CLD30.05100,00036001
6Cloud water mixing ratioFARMS, MYNN, Thompson, and Deng0.1100,00036001
7Ice mixing ratioThompson0.1100,00036001
8Snow mixing ratioFARMS and Thompson0.1100,00036001
9Ice number concentrationThompson0.05100,00036001
10Potential temperatureMYNN, Noah, Deng, and CLD30.001100,00036001
11Turbulent kinetic energyMYNN0.0580,0006001
12Soil moisture contentNoah0.180,00021,6001
13Soil temperatureNoah0.00180,00021,6001
14Vertical velocityDeng0.180,00021,6001
Table
Table 2. Summary ultiphysicsiphysics ensemble configuration.
Table 2. Summary ultiphysicsiphysics ensemble configuration.
EnsembleMicrophysicsCumulusShallow CumulusPBLAerosolLSMAlbedoRadiation
1ThompsonnoDengMYNNTegen [51]Unified NoahMonthly albedoRRTMG
2Thompson aerosol awarenessNoDengMYNNThompson and EidhammerUnified NoahMonthly albedoRRTMG
3ThompsonGFMYNN (icloud_bl = 1, ishallow = 0)MYNNTegen [51]Unified NoahMonthly albedoRRTMG
4ThompsonGFGrell (Icloud_bl = 0, ishallow = 1
Edmf = 0)		MYNNTegen [51]Unified NoahMonthly albedoRRTMG
5ThompsonnoDengMYNNTegen [51]Noah MPTableRRTMG
6ThompsonnoDengMYNNRuiz-Arias et al. [52]Unified NoahMonthly albedoGoddard
7GoddardnoDengMYNNTegen [51]Unified NoahMonthly albedoRRTMG
8GoddardnoDengMYNNRuiz-Arias [52]Unified NoahMonthly albedoGoddard
9ThompsonKFicloud_bl = 0,
ishallow = 1
Edmf = 0		MYNNTegen [51]Unified NoahMonthly albedoRRTMG
10ThompsonModified Tiedtkeicloud_bl = 0,
ishallow = 1
Edmf = 0		MYNNTegen [51]Unified NoahMonthly albedoRRTMG
Table
Table 3. Contingency matrix for the NSRDB and WRF-Solar datasets.
Table 3. Contingency matrix for the NSRDB and WRF-Solar datasets.
Forecasting
NSRDBScenarioCloudyCloud-free
CloudyCCCN
Cloud-freeNCNN
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Kim, J.-H.; Jiménez, P.A.; Sengupta, M.; Dudhia, J.; Yang, J.; Alessandrini, S. The Impact of Stochastic Perturbations in Physics Variables for Predicting Surface Solar Irradiance. Atmosphere 2022, 13, 1932. https://0-doi-org.brum.beds.ac.uk/10.3390/atmos13111932

AMA Style

Kim J-H, Jiménez PA, Sengupta M, Dudhia J, Yang J, Alessandrini S. The Impact of Stochastic Perturbations in Physics Variables for Predicting Surface Solar Irradiance. Atmosphere. 2022; 13(11):1932. https://0-doi-org.brum.beds.ac.uk/10.3390/atmos13111932

Chicago/Turabian Style

Kim, Ju-Hye, Pedro A. Jiménez, Manajit Sengupta, Jimy Dudhia, Jaemo Yang, and Stefano Alessandrini. 2022. "The Impact of Stochastic Perturbations in Physics Variables for Predicting Surface Solar Irradiance" Atmosphere 13, no. 11: 1932. https://0-doi-org.brum.beds.ac.uk/10.3390/atmos13111932

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop