3.1. Statistical Performance
We implemented a series of single-value performance metrics to evaluate how well WyoFire simulations performed across a range of landscapes (Figure 3
). Each metric is a unitless index that represent specific facets of simulative model performance and can be formulated to determine rates of Overestimation, Underestimation, and Intersection for each series of the wildfire simulations. Mean performance indices for all simulations are found in Table 3
. Performance outcomes for all simulations varied considerably between the 10 wildfire events. The Area Difference Index was designed as a simple metric to describe wildfire model performance, while the closer a value is to being equal to one suggests a less predictive error in simulation results in contrast to values much greater than one, which suggests more significant amounts of predictive modelling error [6
]. Variables likely driving model over- or under-prediction in different environments are shown in Table 3
Simulation of wildfire events occurring in environments with medium-to-high total fuel loadings dominated by shrubland and grassland vegetation types, such as the Currant, Stallions, and Buffalo fires, produced the highest rates of overestimation–underestimation (ADIoe
) (Table 3
). In contrast, simulations of wildfire events occurring in environments with lower fuel loadings, dominated by mixed-forest, woodland, and shrub-steppe vegetation types, such as the Fishhawk, Saddle Butte, and Corbin fires, yielded the lowest rates of overestimation–underestimation. Wildfires that lie in mixed fuel types, Tannerite, Pole Creek, Pedro Mountain, and Keystone fires, displayed the most balanced performance in terms of overestimation and underestimation rates.
3.2. Principle Components Analysis
PCA was conducted on fuel characteristics and balance of predictive performance indices within each respective burnable environment. The first two principle components account for 78 percent of the total variance in model performance (Figure 4
). The position of each fire relative to one another in the Bi-Plot (Figure 4
) indicates the relative similarity of the models’ performance in predicting actual wildfire perimeters. Examining the burnable environment within each group also helps reveal what vegetation conditions yield over- and under-predictions by WyoFire. PCA yielded three distinct groups of wildfire events (Figure 4
). Group 1 consists of the Corbin, Saddle Butte, Keystone, and Fishhawk fires, which can be observed in Quadrant IV of the Principle Components Analysis Bi-Plot. Group 2 is composed of the Currant, Stallions, and Buffalo fires, which can be seen in Quadrants II and III. Lastly, Group 3 consists of the Pole Creek, Tannerite, and Pedro Mountain fires, which can be found in Quadrant I.
The first Principal Component (PC-1) explains 56.6 percent of the variance in model performance across all simulations. Under-prediction indices were prominent along this axis, showing how the model performed within environments containing forested elements. Fuel loading became an emergent variable along this axis, while each group of individuals is ordered along the continuum by existing vegetation type and fuel-bed characteristics. The Fishhawk and Stallions fire simulations form the end members along the PC-1 axis. PC-1 appears to be primarily described by the existing vegetation type and dominant fuel loading models present within each respective burnable environment. WyoFire tended to under-predict in situations where the fuel load was low to medium with poor fuel-bed continuity. The Fishhawk Fire was characterized by low surface fuel loads but higher canopy fuel loads, as it occurred primarily in Rocky Mountain Subalpine Forest and Woodland vegetation types with a low total fuel loading. The Stallions Fire was primarily in North Western Great Plains Mixed Grass Prairie and Inter-Mountain Basin Big Sage Brush Steppe communities, possessing a low total fuel load and poor fuel-bed continuity.
Performance results for Fishhawk fire simulations displayed a much higher rate of underestimation than overestimation, while results for Stallions simulations show a relatively higher rate of overestimation than underestimation. The Fishhawk fire simulations had a greater rate of under-prediction in congruence with its high ratio of the canopy to surface fuels, further suggesting that this model may struggle to accurately model transitions of a flaming front propagating from the surface to canopy fuel types. Both wildfires burned across landscapes with average terrain complexity, as this variable did not prove to have a significant effect on the outcomes for these fire simulations.
The second Principle Component (PC-2) explains 21.5 percent of the variance in the predictive performance of the wildfire simulations. Certain burnable environments for wildfire simulations that possessed greater terrain complexity, meaning that the landscape has a greater degree of localized elevational variance, displayed overestimation rates similar to those with relatively low levels of terrain complexity, such as Pedro Mountain (6) and Corbin (2) fire simulations. Overestimation indices are pointed toward the Stallions, Currant, and Buffalo fire simulations as a result of simulating wildfire in higher fuel load with increased continuity. The Pedro Mountain and Corbin fires burned through similar environments dominated by Inter Mountain Basin Big Sage Brush Steppes and Artemisia tridentata ssp. vaseyana Shrubland Alliances, yet these two fires appear as opposing end members along the PC-2 axis. The significant difference between these two burnable environments is the influence of grassland vegetation present throughout the Corbin Fire but not the Pedro Mountain Fire.
The Pedro Mountain Fire had a strong influence of limber pine and juniper woodland vegetation types interwoven on the burnable landscape. This disparity may reflect that overall predictive performance results can be a product of the general fuel load and the specific fuel type present within the burnable environment. In this case, the presence-or-absence of canopy fuels may have been a driving factor of model error for these two wildfires. Heterogeneity of fuel loading models and the configuration of existing vegetation types across the landscape were the variables that induced the most significant amount of variance on performance results for each series of wildfire simulations.
Fires in Group 1 are characterized by low total fuel load and low degree of fuel-bed continuity, which resulted in higher rates of under-prediction. Simulation results for these four fires displayed the highest rates of under-prediction out of all model runs. The Corbin and Saddle Butte fires burned through sagebrush steppe and semi-arid shrubland vegetation types, while the Keystone and Fishhawk fires burned predominantly through Subalpine forest and woodland vegetation types such as spruce-fir, lodgepole pine, Douglas-fir, aspen, and other mixed conifers. Existing vegetation type(s) is the only significant difference amongst this grouping of individual wildfire environments, as variations in surface fuel loading appear to be the primary driver of model performance. It can be inferred that the most significant rates of under-prediction are yielded when simulating wildfire events in environments with high ratios of canopy-to-surface fuels. Given the discontinuous nature of sagebrush and semi-arid shrubland vegetation communities across the landscape, the interstitial spacing between clusters of burnable vegetation may result in simulations to under-predicting wildfire activity.
Group 2 is characterized by medium-to-high total fuel loadings and a high level of fuel-bed continuity, which resulted in higher rates of over-prediction. In contrast to Group 1, results for the simulations within Group 2 displayed the highest rates of over-prediction across all model simulations. The Buffalo, Currant, and Stallions fires burned in predominantly of Northwestern Great Plains mixed grassland and sagebrush steppe vegetation types. These landscapes are more homogeneous than landscapes present in Groups 1 and 3. Higher rates of overestimation are achieved when modelling wildfire propagation in herb- and grassland-dominated environments. The Currant and Stallions fire simulations yielded substantially higher rates of overestimation than did model runs for Buffalo Fire, which is likely attributable to the relatively lower level of surface fuel loading within the Buffalo Fire. We infer that higher rates of over-prediction are associated with simulative environments dominated by herb and grassland vegetation types with medium-to-high surface fuel loads. Simulations for wildfires that have burned on landscapes dominated by herb and grassland vegetation types possess a more continuous fuel bed, which results in a more uniform propagation pattern. The grassland vegetation types inherent to these simulation environments create a more continuous fuel bed than shrubland, forest, and woodland vegetation types do.
Group 3 is associated with low-to-medium fuel loads with varying levels of fuel-bed continuity, which resulted in a more accurate prediction with minimal over- or under-prediction. It can be inferred that these landscapes possessed a relatively higher degree of heterogeneity among existing vegetation types and fuel loadings due to the diversification of surface and canopy fuel types within these environments. All fires in this group were burned environments consisting of an increased mixture of canopy and surface fuel types. Tannerite Fire simulations yielded a higher rate of over-prediction than simulations of the Pole Creek and Pedro Mountain fires, as this is likely attributable to the presence of Montane–Foothill–Valley Grassland vegetation types across the burnable landscape for the Tannerite Fire. An increased level of heterogeneity among fuel types in these environments allows the model to simulate more transitionary events of the flaming front propagating from surface to canopy. Simulative results for the Pole Creek and Pedro Mountain fires displayed slightly lower rates of overestimation than the simulations for the Tannerite Fire.