Multi-Criteria Performance Evaluation of Gridded Precipitation and Temperature Products in Data-Sparse Regions

Abstract

Inadequate climate data stations often make hydrological modelling a rather challenging task in data-sparse regions. Gridded climate data can be used as an alternative; however, their accuracy in replicating the climatology of the region of interest with low levels of uncertainty is important to water resource planning. This study utilised several performance metrics and multi-criteria decision making to assess the performance of the widely used gridded precipitation and temperature data against quality-controlled observed station records in the Lake Chad basin. The study’s findings reveal that the products differ in their quality across the selected performance metrics, although they are especially promising with regards to temperature. However, there are some inherent weaknesses in replicating the observed station data. Princeton University Global Meteorological Forcing precipitation showed the worst performance, with Kling–Gupta efficiency of 0.13–0.50, a mean modified index of agreement of 0.68, and a similarity coefficient SU = 0.365, relative to other products with satisfactory performance across all stations. There were varying degrees of mismatch in unidirectional precipitation and temperature trends, although they were satisfactory in replicating the hydro-climatic information with a low level of uncertainty. Assessment based on multi-criteria decision making revealed that the Climate Research Unit, Global Precipitation Climatology Centre, and Climate Prediction Centre precipitation data and the Climate Research Unit and Princeton University Global Meteorological Forcing temperature data exhibit better performance in terms of similarity, and are recommended for application in hydrological impact studies—especially in the quantification of projected climate hazards and vulnerabilities for better water policy decision making in the Lake Chad basin.

1. Introduction

Accurate climate data are critical to the success of modelling processes in order to reduce uncertainty and achieve better prediction in hydrological impact studies. Unfortunately, reliable and long-term observed meteorological datasets are sparse and unavailable in some regions—especially sub-Saharan Africa and the Mediterranean—making hydrological studies a challenging task [1,2].

Alternatively, high-resolution gridded data have been developed to address these shortcomings; however, an understanding of their limitations in terms of observational uncertainties and reliability is important in order to address the twin issues of choice of dataset and suitability to represent basin features.

Some climate data products are more appropriate than others in their applications for climate change impact studies across different regions; therefore, careful and adequate assessment of their strengths and limitations is required in order to provide guidance for future climate and hydrological studies—especially in data-sparse basins. An accurate hydro-climatic impact study requires climate data at high temporal and spatial resolutions. The most accurate measurement devices are rain gauges, and although these are often situated on land and in populated areas for ease of measurement [3], there are a limited number of ground-based rain gauge stations in most parts of the world for effective and efficient hydro-climatic studies with reduced uncertainty in spatial climate prediction. However, weather station records are typically site-specific, while most hydrological studies in environmental sciences research require aerial observations of climate data in order to achieve accurate modelling processes with minimal uncertainty in impact studies [4].

Climate data have been seen to be an important component of hydrologic cycle analysis over time and space. The knowledge and understanding of their spatiotemporal dynamics are essential, and provide useful information for their practical applications in the field of agriculture, aquaculture, water resource and river basin management, and hazard and flood disaster warnings and management [5,6].

Climate and hydrological studies require complete and reliable rainfall and temperature records at good spatial and temporal resolutions [7]. Unfortunately, climate records from various databases contain gaps or missing data points due to systematic errors, which are prevalent in the Mediterranean and sub-Saharan African countries, making hydrological studies difficult [7,8,9]. Several gridded climate data developed by various modelling centres are used as alternatives, owing to their reliability, and are generated from the observed climate station data after quality control, with enhanced reliability analysis and long-term temporal and spatial coverage [10].

In a hydro-climatic studies, the choice of gridded data for the process of bias correction of general circulation models in data-sparse regions indicates an essential procedure for climate change impact assessment studies [11]. However, the choice of reference dataset that is available in either station data, or gridded products derived from observations [12,13], reanalysis data [14,15,16], or remote sensing data [17,18], is critical in the overall uncertainty associated with projected climate change impact studies.

These datasets form the primary input in hydrological modelling studies and climate change impact studies for the accurate assessment of hydrological variables such as streamflow, runoff, soil moisture, evapotranspiration, etc., in order to manage hydropower operations, irrigation scheduling, and early warning systems for landslides and changes in future water availability for climate change and basin hydrological cycle assessment [19].

A study conducted in Africa showed varying degrees of spatial mismatch between observed weather stations and reanalysis data [20]. The techniques and efforts in the analysis vary based on temporal coverage, climate variables involved, and the region of interest [21]. However, gridded and reanalysis data are being updated due to advances in the understanding of climate science over time, and detailed evaluation of their performance at the catchment scale in Africa is rarely found in literature, although this may be attributed to limited availability of reliable long-term climate records, expertise, and ease of access to data [22,23].

Studies have shown that high-resolution gridded data have been developed to provide valuable information on disaster management, initialisation and validation of numerical models, and resolving the diurnal global cycle of precipitation [24,25]. There are many gridded data products available at different timescales (hourly, daily, and monthly) with a finer resolution of 0.5° × 0.5°, which can provide insight that relates (but is not limited) to a model forecast of hydrological cycles, climate change trend analysis, climate downscaling, etc. [13,26,27,28,29,30,31,32,33,34].

The downside in the use of gridded and reanalysis climate data lies in the fact that gridded climate data are a combination of observed station data and quality-controlled statistical interpolation that could result in the attenuation of local climate signals, while reanalysis data are model-based forecasts that require parameterisation of the model, good assimilation technique, and high-quality observation [21,35,36].

Gridded climate products are known to differ in their source, spatial and temporal resolutions, domain size (global coverage), and available timescales, and also exhibit different error bands due to interpolation procedures and considerable differences in general climatology, which are well known and acknowledged [12,37,38,39]. The choice and selection of reference datasets at the catchment scale should be based on observational uncertainty and purpose via critical analysis of the gridded data products. The development and application of gridded climate data is growing rapidly, especially due to the advances in knowledge of their spatiotemporal resolution, latency, and reliability. However, the uncertainty associated with their application across local and regional catchments is still a cause for concern, and has led to some studies related to the ability of the gridded data to replicate or mimic reliable but sparse ground-based data across the globe [40,41,42,43,44,45,46,47].

Furthermore, the performance of the gridded data is predicated on using individual or combined statistical metrics to replicate some particular characteristics of the observed data, and often exhibits contradictory results, making the decision making difficult [6,48,49]. It has been posited that some gridded data have proven to be appropriate compared to others in specific applications and in certain regions around the world [50], and a single statistical metric cannot justify the performance or suitability of a particular gridded dataset. Therefore, it is important to use various metrics to obtain an ideal solution, based on optimal performance across all metrics, especially in data-sparse regions—for prediction efficiency [49,51].

The objective of this study was to employ multi-criteria decision making to assess the performance of five widely used and recently updated gridded precipitation datasets and four temperature datasets in replicating the total and average monthly precipitation and temperature of available gauge-based records in the Lake Chad hydrological basin. This study was necessary to provide guidance on the choice of reference dataset(s) to be adopted for future research in the basin, depending on performance and purpose in hydro-climatic studies, in order to reduce uncertainty in predictions as well as computational time and resource costs.

Furthermore, the choice of the gridded dataset(s) in previous climate studies found in the literature on the Lake Chad basin, for example [52,53,54,55], has been based on their popularity and usage in other basins, without a proper justification of their suitability compared to other available products for improving the reliability of predictions and reducing model biases to provide an accurate representation of basin-scale hydrological features. Additionally, some of the products are only available in monthly time steps, which may not be suitable for downscaling of GCMs with daily time steps as input requirements in some hydrological models and climate change impact studies.

This study employs entropy-based symmetric uncertainty (SU) [56]—a machine learning approach that has been found to be an efficient tool for the assessment of agreement in data that measure the shapes and patterns of data sequences via the concept of mutual information theory, by comparing the similarity between two long time-series climate datasets, and has found its application in various fields [10,57]. The benefit of this method is that it does not depend on the data distribution, unlike the statistical metrics used in other studies [32,40,57]. Four statistical metrics were used in this study—namely, Taylor diagrams, modified index of agreement (md), Kling–Gupta efficiency (KGE), and normalised root-mean-square error (NRMSE)—and then finally trend analysis of the precipitation and temperature data at the annual and seasonal scales of the gridded and observed station records was compared for mean variability and temporal homogeneity across the basin.

2. Study Area and Data

2.1. Study Area

The Lake Chad basin is one of the largest endorheic basins in the world, and occupies an estimated area of ~2,500,000 km²—approximately 8% of Africa [58,59]. The basin cuts across the entirety or part of Algeria, Cameroon, the Central African Republic, Chad, Libya, Niger, Nigeria, and Sudan in Central Africa. The basin is geographically located at latitudes of 5.2°–25.3° N and longitudes of 6.9°–24.5° E, right at the transition zone of the Sahara region and the tropics of the Sudano-Sahelian region of West Africa [60], (Figure 1).

The basin is characterised by a vast and shallow freshwater lake located at its centre, with inflows from Chari and Logone rivers (~90–95%) into the southern pool, the Yobe and Komadugu rivers (~2.5–5%) in the western region, which enters the lake through the northern pool [61,62,63], and other minor rivers such as the Gubio, Ngadda, Yedseram, and El Beid, which supplied only ~1–2% inflow to the lake through the southwestern part of the basin between 1961 and 2013 [64]. The basin serves as the main source of fresh water that supports livelihoods across pastoral land, agricultural land, and fish farming [65], with irrigation agriculture as the major user of the resource that supports a majority (~60%) of the population [66].

The basin is divided into several climatic zones, namely, the Saharan zone located in the north of the basin; the Sahelo-Saharan zone located in the central part of the basin, which covers the north of Diffa, Niger, and Lake Chad; the Sudano-Sahelian zone, which covers N’Djamena in the Chad Republic and the northern part of Cameroon and Nigeria; and the Sudano-Guinean zone located in the south, which covers the south of Chad and the Central African Republic, with annual precipitation of < 100 mm, 100–400 mm, 400–600 mm, and 600–1500 mm, respectively. The average annual temperature in the basin ranges from 35–40 °C in the northern part of the basin to as low as 26.5 °C in the southern part [52], characterised by hot and dry, wet and dry, and cool weather during March–June, June–October, and November–February, respectively [67].

The basin is located in a region that is characterised as having little relief, no surface outlets, and a spatial extent that is quite sensitive to climatic variability; the elevation of the basin ranges from −330 m to 3446 m (Figure 1). However, according to [68,69], the basin is a relatively flat area with an average slope of <1.3%, except for some local hills, plateaus, and mountains in the southern and northern parts of the basin.

2.2. Observational Data and Sources

The observed climate data used in this study were acquired from a number of sources; for example, 12 average monthly observed temperature datasets were obtained from the Global Historical Climatology Network’s monthly temperature dataset, version 4 (

Table 1. List of reliable observed temperature stations’ locations and temporal spans in the Lake Chad basin.

S/No	Station Name	Data Range	Missing Data (%)
1	Bilma	1950–2019	8.2
2	Bossangoa	1954–2016	35.0
3	Bouar	1951–2019	34.7
4	Geneina	1951–2019	11.5
5	Maiduguri	1910–2012	14.0
6	Maina sorda	1951–2019	11.5
7	Moundou	1951–2016	34.3
8	N’Djamena	1951–2019	26.3
9	Ngaoundere	1951–2019	35.2
10	Nguigni	1953–2019	5.8
11	Sahr	1941–2018	39.3
12	Zinder	1923–2019	3.5

Source: [74] http://www.ncdc.noaa.gov/ghcnm/v4.php, accessed on 2 March 2020.

) (http://www.ncdc.noaa.gov/ghcnm/v4.php, accessed on 2 March 2020 [68], while 11 total monthly observed precipitation datasets were obtained from the Lake Chad Basin Commission, and can be found as supplementary datasets online in [52]. We also used 2 station records from the Nigerian Meteorological Agency (NIMET) and the NOAA Global Historical Climatology Network Daily (GHCN-D) version 3.23 http://www.ncdc.noaa.gov/pub/data/ghcn/daily, accessed on 2 March 2020 (

Table 2. List of reliable observed precipitation stations’ locations and temporal spans in the Lake Chad basin.

S/No	Station Name	Data Range	Missing Data (%)
1	Abeche	1985–2015	0.0
2	Banda	1950–2013	0.0
3	Bongor	1950–2013	0.0
4	Bossangoa	1950–2013	0.0
5	Doba	1950–2013	0.0
6	Maiduguri	1979–2010	0.0
7	Moundou	1985–2015	0.0
8	N’Djamena	1985–2013	0.0
9	Nguigni	1968–2020	15.3
10	Potiskum	1980–2010	0.0
11	Sahr	1950–2013	0.0
12	Samry-I	1950–2013	0.0
13	Sategui Deressia	1950–2013	0.0
14	Tsanaga	1950–2013	0.0
15	Zinder	1906–2020	17.1

Source: ((LCBC) [54], NIMET, GHCN-D http://www.ncdc.noaa.gov/pub/data/ghcn/daily, accessed on 2 March 2020.

).

The observed station data considered in this study were carefully selected based on the condition of having fewer missing records and an acceptable temporal span for hydro-climatic analysis in order to achieve effective and reliable predictions. The observed missing climate data records were filled using the multivariate imputation by chained equations (MICE) package, due to its ability to impute continuous two-level data and maintain consistency between imputations while employing passive imputation [70].

The data were checked for 100% completeness after imputation and assessed for comparison using a double-mass curve approach for subjective evaluation of heterogeneity in the datasets [71], and finally subjected to absolute homogeneity tests—namely, the standard normal homogeneity test (SNHT), Pettitt test, and Von Neumann ratio test [72,73].

The double-mass curve showed an almost straight line at all stations without breakpoints (Figure 2a,b), and absolute homogeneity test results were all less than the critical values. The null hypothesis of all of the station data—that the climate data were homogenous at a 95% level of confidence—cannot be rejected; therefore, the quality-controlled data for precipitation and temperature are suitable for performance evaluation of the gridded climate data.

2.3. Gridded Data and Sources

This study analysed five gridded precipitation and four temperature datasets, from the University of East Anglia, Climate Research Unit CRU TS V4.04, German Weather Service, Global Precipitation Climatology Centre (GPCC) v.2018 (precipitation only), US Climate Prediction Centre (CPC), Princeton University Global Meteorological Forcing (PGF) v.2, and University of Delaware (UDEL) V5.01.

Table 3. Summary of gridded datasets considered in this study.

Data Product	Variable(s)	Temporal Resolution	Data Span	Spatial Resolution	Source
CPC	P, Tmax, Tmin	Daily	1979–2020	0.5°	Precipitation https://www.esrl.noaa.gov/psd/data/gridded/data.cpc.globalprecip.html, acessed on 5 June 2020 and temperature https://www.esrl.noaa.gov/psd/data/gridded/data.cpc.globaltemp.html, acessed on 5 June 2020
CRU TS v.4.04	P, Tmax, Tmin	Monthly	1901–2016	0.5°	https://crudata.uea.ac.uk/cru/data/hrg/cru_ts_4.04, acessed on 5 June 2020
PGF v.2	P, Tmax, Tmin	Daily	1901–2012	0.5°	http://hydrology.princeton.edu/data.pgf.php, acessed on 5 June 2020
GPCC v.2018	P	Monthly	1901–2016	0.5°	http://www.esrl.noaa.gov/psd/data/gridded/data.gpcc.html, acessed on 5 June 2020
UDEL V5.01	P Tave	Monthly	1900–2017	0.5°	https://psl.noaa.gov/data/gridded/data.UDEL_AirT_Precip.html, acessed on 5 June 2020

summarises the climatic variables, temporal and spatial resolution, temporal span, and sources of the gridded data.

The gridded datasets were developed from different modelling centres using different interpolation techniques; for example, CPC gridded data were developed via optimal interpolation of station- or gauge-based records of GTS [25]; CRU data were developed via angular distance weighting of monthly observed station data from the World Meteorological Organization, National Oceanic and Atmospheric Administration (NOAA) database, and climate records from national meteorological agencies across the globe [75]; the PGF dataset was developed based on findings from NCEP–NCAR reanalysis and other global data via bilinear interpolation from their native gridded scale [76]; GPCC datasets were developed by the combination of monthly gauged and quality-controlled records from 7000 stations around the world, along with GTS synoptic weather reports, and interpolated to regular grids using an ordinary point Kriging method [13], while UDEL precipitation and temperature data were interpolated using shepherd algorithms for grid-based data from various sources—such as GHCN2, NCAR, GHCN-D, GHCN-monthly version 3, and records from national meteorological agencies [77,78].

3. Research Methodology

The performance of the gauge-based gridded precipitation and temperature data was evaluated by multiple approaches—including machine learning, filter-based symmetric uncertainty, statistical metrics (modified index of agreement, Kling–Gupta efficiency, normalised root-mean-square error, and Taylor diagrams), and time-series analysis of trends exhibited at annual and seasonal timescales—to assess the gridded data in terms of their reliability in mimicking the observed data across all of the stations in the study area for the period 1979–2012. The gridded datasets were sourced from the websites of the providers, as shown in Table 3, extracted using the raster and ncdf4 R packages, and interpolated to the observed data station resolution using the inverse distance weighting method. The detailed methodology of the study is outlined below.

3.1. Symmetric Uncertainty

Symmetric uncertainty is an entropy-based machine learning algorithm (filter method) used in assessing the pairwise agreement between long time-series data. This method utilises information entropy via the concept of mutual information (MI), which measures the commonality between two variables. For example, if $p\left(x\right)$ and $p\left(y\right)$ are considered probability density functions of the observed variable $\left(x\right)$ and the gridded variable $\left(y\right)$, and $p\left(x,y\right)$ represents the mutual probability distribution functions of $x$ and $y$, then mutual information can be evaluated as follows:

$$MI\left(x,y\right)=p\left(x,y\right)\log \frac{p\left(x,y\right)}{p\left(x\right)·p\left(y\right)}$$

(1)

The mutual information shown in Equation (1) can also be evaluated as the difference between the mutual entropy of two time-series variables, and in this case, taking the observed data as $H\left(x\right)$, the gridded data as $H\left(y\right)$, and the mutual entropy of the observed and gridded data time series as $H\left(y,x\right)$, MI can be written as:

$$MI\left(x,y\right)=H\left(y\right)-H\left(x,y\right)$$

(2)

Thus, $H\left(y\right)$ and $H\left(y,x\right)$ indicate the amount of uncertainty inherent in the gridded data, and join the gridded and observed probability density functions of the precipitation and temperature time series data. The two independent variables in Equation (2) can be expressed as:

$$H\left(x,y\right)={{\displaystyle \sum }}_{i=1}^{n}p\left(x,y\right)\log \frac{p\left(x,y\right)}{p\left(x\right)\times p\left(y\right)}$$

(3)

$$H\left(y\right)=-{\displaystyle \int }p\left(y\right)\log \left(p\left(y\right)\right)dx$$

(4)

The entropy estimated in Equation (3) implies the extent of mutual information between the gridded and observed precipitation/temperature data. The mutual information tends to be zero in the absence of common information, and has a value of unity when the model data series can depict the complete information associated with the observed data series. However, biases are inherent when using time series with larger values if there are less similar values between the two variables [79]; this drawback can be addressed through the concept of SU, by dividing the value of mutual information gain and the sum of the entropies of $y$ and $x$, as shown in Equation (5):

$$SU\left(x,y\right)=2\times \frac{MI\left(x,y\right)}{H\left(x\right)+H\left(y\right)}$$

(5)

The value of symmetric uncertainty ranges from 0 to 1, where 0 indicates poor similarity and 1 indicates high similarity between the gridded and observed precipitation/temperature time-series data [80]. This study utilised the FSelector package in R software [81] to assess the similarity between the monthly gridded and observed precipitation/temperature data (

Table 4. Similarity score of gridded precipitation and temperature against observed datasets estimated by SU.

Rank	Precipitation Dataset	SU	Temperature Dataset	SU
1st	CRU	0.395	PGF	0.765
2nd	GPCC	0.387	CRU	0.725
3rd	UDEL	0.369	UDEL	0.722
4th	CPC	0.367	CPC	0.709
5th	PGF	0.365

). Figure 3 shows the distribution of gridded precipitation and temperature data of all available stations within the study area.

3.2. Statistical Metrics

In this study, four statistical metrics were used to evaluate the ability of the selected monthly gridded precipitation and temperature datasets to replicate the observed station time series. Further details of the metrics are outlined below.

3.2.1. Kling–Gupta Efficiency

This is a metric that highlights three components—namely, correlation, bias, and ratio of variances between gridded and observed time series data—as proposed in [82]; the value of KGE varies between 0 and 1, which indicate no agreement and perfect agreement between the gridded ($x$_g) and observed ($x$_obs) data, respectively. The coefficient can be computed as given in Equation (6):

$$KGE=1-\sqrt{{\left(\gamma -1\right)}^2+{\left(1-\frac{{\mu }_g}{{\mu }_{obs}}\right)}^2+{\left(\frac{{\sigma }_g/{\mu }_g}{{\sigma }_{obs}/{\mu }_{obs}}\right)}^2}$$

(6)

3.2.2. Modified Index of Agreement

This is a statistical metric that evaluates the standardised measure of the degree of model prediction error. This metric was modified from its original form proposed in [83], which has shown to be less sensitive to extreme values; it has a value that varies between 0 and 1, indicating no agreement and perfect agreement between the gridded ($x$_g) and observed ($x$_obs) data, respectively. The coefficient can be computed as given in Equation (7):

$$md=1-\frac{{{\displaystyle \sum }}_{i=1}^n{\left(x_{obs,i}-x_{g,i}\right)}^i}{{{\displaystyle \sum }}_{i=1}^n{\left(│x_{g,i}-\stackrel{\rightharpoonup }{x_{obs}}│+│x_{obs,i}-\stackrel{\rightharpoonup }{x_{obs}}│\right)}^i}$$

(7)

3.2.3. Normalised Root-Mean-Square Error

This is a statistical metric that facilitates and summarises the magnitude of errors between model ($x$_g) and observed ($x$_obs) data with different scales, and is defined by the ratio of root-mean-square error and the standard deviation of the data. This metric is considered to be a great measure of precision, and the predictive ability of the model is considered to be accurate with values closer to zero [84,85]. The value of NRMSE can be computed by the following equation:

$$NRMSE=\frac{{\left[\frac{1}{n}{{\displaystyle \sum }}_{i=1}^n{\left(x_{g,i}-x_{obs,i}\right)}^2\right]}^{1/2}}{\frac{1}{n}{{\displaystyle \sum }}_{i=1}^n{x}_{g,i}}$$

(8)

3.2.4. Taylor Diagrams

These are a graphical representation that summarise proximity or similarity between modelled and observed long time-series data. The similarity is quantified based on their correlation, centred root-mean-square difference, and the amplitude of variations (standard deviations). These models are quite useful in gauging the relative performance of models compared to the observed data [86,87].

3.2.5. Trend Analysis

In this study, trend analysis was carried out using the nonparametric Mann–Kendall or modified Mann–Kendall tests [88,89], where significant autocorrelation was observed in the time-series data, trend-free pre-whitening was applied to correct the anomaly [90], and Sen’s slope estimator was used to calculate the statistically significantly increasing or decreasing trends and the magnitude of the trends. The Mann–Kendall statistics are given as:

$$S={{\displaystyle \sum }}_{k=1}^{n-1}{{\displaystyle \sum }}_{j=k+1}^n\mathrm{sgn}\left(x_j-x_k\right)$$

(9)

where $x_j$ and $x_k$ are sequential data values for n time series. The sgn of the series is defined as:

$$\mathrm{sgn}\left(x_j-x_k\right)=\{\begin{array}{c}1 if x_j>x_k\\ 0 if x_j=x_k\\ -1 if x_j<x_k\end{array}$$

(10)

The mean E(S), variance V(S), and the Z statistics can be computed as:

$$E\left(S\right)=0$$

(11)

$$V\left(S\right)=\frac{1}{18}\left\{n\left(n-1\right)\left(2n+5\right)-{{\displaystyle \sum }}_{i=1}^p{t}_i\left(t_i-1\right)\left(2t_i+5\right)\right\}$$

(12)

$$Z=\{\begin{array}{c}\frac{S-1}{\sqrt{V\left(S\right)}} for S>0\\ 0\hspace{1em}for S=0\\ \frac{S+1}{\sqrt{V\left(S\right)}} for S<0 \end{array}$$

(13)

In the equation above, p represents the number of tied groups in the series, and each of the tied groups is indicated by $t_i$. All positive and negative values of the Z statistics represent statistically increasing and decreasing trends in the time-series data.

The magnitude of the detected trends in the time-series data was computed by the nonparametric Sen’s slope estimator, because this method is robust against outliers in time-series analysis, as given below:

$$SS=median\left[\frac{x_j-x_i}{j-i}\right]foralli<j$$

(14)

where $x_i$ represents the value of the data at time step $i$, and $x_j$ for timestep $j$.

4. Results

4.1. Assessment of Gridded Data Using Symmetric Uncertainty

The performance of the gridded monthly total precipitation and average temperature data for the period 1979–2012 was individually assessed at each station location after being downloaded at the station resolutions. The SU score obtained at the Zinder stations located at latitude 13.8° N, longitude 8.9° E is given in Table 4. The results from the table show that CRU and PGF were found to have the highest estimated similarity scores of precipitation and temperature across the station. However, there were inconsistencies in the performance of the gridded data at different stations in replicating the observed data. The CRU, GPCC, and PGF datasets showed a better performance at 53.3%, 33.3%, and 13.3%, respectively, while CRU, PGF, and UDEL also showed a better performance, at 41.7%, 33.3, and 25.0% of the precipitation and temperature stations in the study area, respectively.

The observed precipitation stations located in the Sudano-Guinean zone of the basin—for example, Bossangoa, Samry-I, Sategui, Tsanaga, Doha, and Moundou (SU ≥ 0.478)—were shown to have a better similarity coefficient compared to stations located in the Sahelo-Saharan zone, e.g., Nguigni, Zinder (SU ≤ 0.395), etc. However, the temperature stations located in the Saharan and Sahelo-Saharan zones (i.e., Bilma, Maina-Sorda, Nguigni, and Zinder) performed better in simulating the observed temperature (SU ≥ 0.692), while the worst performance in the study area was in the Sudano-Guinean zone (SU ≤ 0.483). The consistency in the precipitation data in the Sudano-Guinean zone may be attributed to the more numerous and accurate station records in the southern zone of the basin, as well as the reduction and abandonment of ground-based gauged data and errors in taking records resulting in systematic errors in the northern part of the basin, due to migration, political instability, etc.

The mean SU of the gridded precipitation across the stations in the basin for the CPC, CRU, GPCC, PGF, and UDEL datasets was 0.448, 0.504, 0.512, 0.472, and 0.480, respectively, while the SU of the gridded temperature for the CPC, CRU, PGF, and UDEL datasets was 0.482, 0.532, 0.523, and 0.519, respectively. These results show that there is a slight variation in performance across the gridded datasets assessed in this study. Figure 3 shows the distribution of the similarity coefficients of the datasets across the stations considered.

4.2. Statistical Metric Efficiency

The results of the statistical metrics of the gridded precipitation and temperature data considered in this study for the period 1979–2012 show that there is a good agreement between the gridded precipitation datasets and the observed data across the stations within the basin, with the mean KGE and md coefficient in the range of ~0.7 and above (

Table 5. Summary of the performance of statistical metrics of gridded precipitation data against observed data in the Lake Chad basin.

TOTAL MONTHLY PRECIPITATION
STATION	KGE					md					NRMSE
	CPC	CRU	GPCC	PGF	UDEL	CPC	CRU	GPCC	PGF	UDEL	CPC	CRU	GPCC	PGF	UDEL
Abeche	0.64	0.63	0.52	0.24	0.71	0.89	0.87	0.85	0.62	0.91	1.058	1.272	1.237	1.308	1.093
Banda	0.84	0.84	0.80	0.44	0.87	0.93	0.96	0.93	0.73	0.95	0.615	0.455	0.639	0.926	0.508
Bongor	0.71	0.83	0.78	0.32	0.81	0.89	0.95	0.96	0.67	0.95	0.706	0.527	0.468	1.073	0.505
Bossangoa	0.80	0.94	0.87	0.38	0.89	0.94	0.98	0.97	0.69	0.97	0.409	0.244	0.31s2	0.818	0.298
Doba	0.75	0.77	0.76	0.31	0.78	0.94	0.96	0.96	0.70	0.96	0.481	0.405	0.394	0.911	0.401
Maiduguri	0.70	0.83	0.85	0.37	0.44	0.84	0.93	0.93	0.66	0.90	0.949	0.668	0.701	1.188	0.888
Moundou	0.86	0.91	0.94	0.41	0.94	0.95	0.96	0.97	0.71	0.97	0.513	0.443	0.403	0.979	0.392
N’Djamena	0.72	0.66	0.61	0.27	0.73	0.90	0.86	0.86	0.62	0.90	0.827	0.928	0.91	1.273	0.79
Nguigni	0.48	0.45	0.43	0.13	0.47	0.82	0.84	0.82	0.57	0.81	1.097	1.034	1.066	1.421	1.095
Potiskum	0.84	0.67	0.62	0.26	0.69	0.75	0.90	0.88	0.62	0.91	1.045	0.728	0.795	1.209	0.711
Sahr	0.58	0.65	0.53	0.23	0.60	0.83	0.86	0.80	0.65	0.83	0.826	0.711	0.901	0.965	0.802
Samry-I	0.74	0.68	0.78	0.50	0.72	0.89	0.95	0.96	0.81	0.96	0.908	0.706	0.601	0.944	0.617
Sategui Deressia	0.70	0.60	0.64	0.49	0.51	0.91	0.90	0.91	0.86	0.89	0.777	0.893	0.81	0.706	0.975
Tsanaga	0.81	0.87	0.90	0.40	0.67	0.91	0.96	0.99	0.70	0.96	0.704	0.503	0.293	1.069	0.549
Zinder	0.64	0.63	0.52	0.24	0.71	0.89	0.87	0.85	0.62	0.91	0.837	0.899	0.918	1.248	0.772
Mean Value	0.72	0.73	0.70	0.33	0.70	0.89	0.92	0.91	0.68	0.92	0.78	0.69	0.70	1.07	0.69

Bold values indicate stations where gridded datasets have a better performance.

) for the CRU, GPCC, CPC, and UDEL datasets, but not for the PGF data, which performed the worst across the stations, with a mean KGE and md coefficient of 0.33 and 0.68, respectively. The results of NRMSE were consistent with all of the gridded datasets, but had a higher value for the PGF data, with a mean value 1.07. However, the results for the temperature datasets showed a better performance or mean similarity coefficient of all of the gridded data products, with values generally above 0.85 for both KGE and md (

Table 6. Summary of the performance of statistical metrics of gridded temperature data against observed data in the Lake Chad basin.

AVERAGE MONTHLY TEMPERATURE
STATION	KGE				md				NRMSE
	CPC	CRU	PGF	UDEL	CPC	CRU	PGF	UDEL	CPC	CRU	PGF	UDEL
Bilma	0.97	0.97	0.98	0.98	0.99	0.99	0.99	0.99	0.038	0.041	0.042	0.038
Bossangoa	0.81	0.80	0.83	0.86	0.90	0.91	0.92	0.91	0.037	0.035	0.033	0.038
Bouar	0.77	0.73	0.74	0.85	0.83	0.71	0.67	0.89	0.054	0.08	0.091	0.043
Geneina	0.81	0.82	0.84	0.93	0.88	0.93	0.93	0.97	0.076	0.06	0.057	0.043
Maiduguri	0.77	0.76	0.75	0.49	0.92	0.93	0.92	0.68	0.064	0.059	0.063	0.12
Maina Sorda	0.97	0.97	0.96	0.97	0.99	0.99	0.99	0.99	0.033	0.026	0.026	0.026
Moundou	0.83	0.85	0.84	0.78	0.90	0.92	0.90	0.84	0.052	0.046	0.054	0.074
N’Djamena	0.92	0.94	0.94	0.93	0.96	0.97	0.96	0.97	0.047	0.04	0.045	0.042
Ngaoundere	0.70	0.75	0.73	0.73	0.84	0.85	0.81	0.84	0.052	0.051	0.061	0.053
Nguigni	0.98	0.95	0.96	0.98	0.99	0.99	0.99	0.99	0.031	0.024	0.025	0.024
Sahr	0.87	0.90	0.89	0.86	0.91	0.95	0.93	0.92	0.046	0.031	0.039	0.045
Zinder	0.99	0.99	0.99	0.98	1.00	1.00	1.00	0.99	0.019	0.017	0.018	0.022
Mean value	0.87	0.87	0.87	0.86	0.93	0.93	0.92	0.92	0.046	0.043	0.046	0.047

Bold values indicate stations where the gridded dataset has a better performance.

), along with almost similar NRMSE across the stations in the study area.

The results from Table 5 and Table 6 reveal that the CPC, CRU, GPCC, and UDEL gridded precipitation datasets show better performance, by approximately 20%, 20%, 26.7%, and 40%, respectively, in terms of KGE, and 13.3%, 40%, 46.7%, and 46.7%, respectively, in terms of md, in all of the stations. The temperature data showed a better similarity with the observed station data, with CPC, CRU, PGF, and UDEL recording a better performance by 33.3%, 50%, 25%, and 50%, respectively, in terms of KGE, and 33.3%, 75%, 41.7%, and 50%, respectively, in terms of md. The metrics used in this study are presented using boxplots in Figure 4, revealing a consistent variation in their ability to replicate the observed precipitation and temperature data—except for the PGF gridded precipitation data, which may pose a large uncertainty in the prediction of climate variables. This may be due in part to the interpolation technique, source, and quality of the observed data used in its development, covering the entire Lake Chad basin.

The similarity or agreement between the gridded precipitation/temperature datasets and the observed data for the period 1979–2012 at annual, pre-monsoon, and monsoon timescales was evaluated using Taylor diagrams (Figure 5). The results indicate that the polar plots lie in the first quadrant, revealing that all correlation values are positive. The results also reveal a notable variability in normalised standard deviation in the gridded precipitation products in the three timescales. The GPCC and CPC datasets recorded the best performance, with a Pearson’s correlation coefficient greater than 0.5, while PGF had the worst performance due to wider variability and a lower Pearson’s correlation coefficient across the annual and seasonal timescales. However, PGF data showed a better performance in the temperature datasets, with normalised standard deviation (variability) ~1.0 in the annual and monsoon season timescales, as well as a better correlation with the observed data. Although the gridded temperature datasets in general exhibited better performance with low observational uncertainty in replicating the observed data at the annual timescale relatively well compared to the monsoon and pre-monsoon seasons, the differences in their performance were apparent; however, a seemingly similar root-mean-square difference was observed in the precipitation datasets at all timescales.

4.3. Trend Analysis of Gridded Data

The trends of precipitation and temperature were evaluated at the annual and monsoon season timescales for the gridded datasets and observed station records for the period 1979–2012. For the sake of clarity, the performance of the datasets was assessed based on statistically increasing trends, decreasing trends, or no trends—arising due to limited station records—to ease the complexity of analysis. The Z-statistic values (

Table 7. Mann–Kendall Z-statistics values of linear trends in annual gridded and observed precipitation for the Lake Chad basin (1979–2012).

Stations	Precipitation Trend
	OBSERVED	CPC	CRU	GPCC	PGF	UDEL
Abeche	−1.60	−0.85	2.59	1.96	2.23	2.43
Banda	2.02	0.15	1.07	2.05	0.00	1.10
Bongor	−0.71	−1.63	1.17	0.15	1.42	1.63
Bossangoa	0.33	−0.82	0.60	−0.53	0.44	−0.53
Doba	0.16	−1.01	0.21	0.59	0.50	0.33
Maiduguri	−2.19	0.79	2.25	2.16	2.45	−1.90
Moundou	0.69	0.26	1.21	0.18	1.01	0.34
N’Djamena	0.67	2.29	2.02	−0.29	1.42	2.54
Nguigni	−0.20	0.71	2.31	0.92	2.02	2.31
Potiskum	1.82	1.51	2.51	0.19	1.96	1.28
Sahr	0.06	0.98	1.19	2.58	1.81	2.11
Samry-I	1.91	−1.57	1.16	0.15	1.63	1.63
Sategui D.	0.74	0.77	1.10	0.36	1.41	1.30
Tsanaga	1.13	0.27	1.39	1.75	1.69	1.39
Zinder	1.60	1.07	2.13	0.65	2.16	2.16
% Agreement	-	66.67	73.33	60.00	66.67	73.33

Note: Highlighted bold values indicate significant trends.

,

Table 8. Mann–Kendall Z-statistics values of linear trends in monsoon season gridded and observed precipitation for the Lake Chad basin (1979–2012).

Stations	Precipitation Trend
	OBSERVED	CPC	CRU	GPCC	PGF	UDEL
Abeche	−1.24	−0.38	0.24	1.68	2.39	2.07
Banda	1.07	1.17	0.50	1.07	−0.24	−0.27
Bongor	−0.74	−0.95	1.51	−0.09	1.42	1.50
Bossangoa	0.01	−0.47	0.53	−0.92	0.12	−0.83
Doba	0.46	0.83	0.24	−0.06	0.71	0.53
Maiduguri	−1.99	1.18	2.38	2.25	2.94	−1.80
Moundou	0.38	1.44	1.32	−0.14	1.24	0.34
N’Djamena	0.75	2.46	1.88	−0.08	1.51	2.42
Nguigni	−0.47	0.53	2.16	0.62	1.96	2.25
Potiskum	1.40	1.70	2.69	0.54	2.22	1.28
Sahr	0.95	1.84	0.67	1.60	1.60	0.39
Samry-I	2.11	−0.95	1.51	−0.09	1.43	1.50
Sategui D.	0.36	1.33	1.90	0.24	2.16	0.85
Tsanaga	0.98	0.53	1.71	1.33	1.90	1.07
Zinder	1.10	1.39	2.08	0.62	2.11	1.96
% Agreement	-	73.33	73.33	46.67	66.67	66.67

Note: Highlighted bold values indicate significant trends.

,

Table 9. Mann–Kendall Z-statistics values of linear trends in annual gridded and observed temperature for the Lake Chad basin (1979–2012).

Stations	Temperature Trend
	OBSERVED	CPC	CRU	PGF	UDEL
Bilma	3.21	3.18	4.28	3.33	3.39
Bossangoa	1.36	2.53	3.75	3.30	2.25
Bouar	0.50	2.58	3.43	2.65	1.90
Geneina	2.80	−1.79	2.46	3.21	2.00
Maiduguri	0.37	2.55	3.85	3.07	3.03
Maina S.	3.35	3.67	3.61	3.11	1.60
Moundou	−0.37	2.65	3.49	3.30	3.86
N’Djamena	2.37	5.19	3.33	2.93	1.75
Ngaoundere	1.60	−0.02	3.64	3.05	3.83
Nguigni	2.99	2.96	3.42	2.95	2.03
Sahr	0.50	3.25	3.08	3.92	3.41
Zinder	2.76	2.92	3.73	3.14	3.83
% Agreement	-	75.00	91.67	91.67	91.67

Note: Highlighted bold values indicate significant trends.

and

Table 10. Mann–Kendall Z-statistics values of linear trends in monsoon season gridded and observed temperature for the Lake Chad basin (1979–2012).

Stations	Temperature Trend
	OBS	CPC	CRU	PGF	UDEL
Bilma	2.90	2.77	2.71	3.28	2.71
Bossangoa	1.46	1.29	3.33	3.46	1.45
Bouar	0.79	1.13	3.31	2.15	1.11
Geneina	1.41	−2.62	1.69	3.38	1.72
Maiduguri	1.59	0.78	2.57	1.45	2.21
Maina S.	1.60	1.53	1.89	0.92	0.47
Moundou	−1.29	1.07	2.90	3.04	2.98
N’Djamena	−0.45	3.27	2.61	2.02	0.62
Ngaoundere	0.50	−0.82	2.66	1.98	3.33
Nguigni	2.15	1.76	2.15	1.42	0.52
Sahr	−0.82	0.77	3.25	3.31	1.66
Zinder	−0.33	−0.50	1.41	0.62	2.17
% Agreement	-	58.33	66.67	66.67	66.67

Note: Highlighted bold values indicate significant trends.

) showed the unidirectional trends of precipitation and temperature at the annual and monsoon season timescales, respectively. The results revealed that 20% and 26.7% of the observed station data showed a statistically decreasing trend of precipitation at the annual and monsoon season timescales, respectively, while 8.3% and 33.3% of the observed station data showed a statistically decreasing trend of temperature at the annual and monsoon season timescales, respectively. The stations where the analysis recorded a declining precipitation trend—for example, Maiduguri, Nguigni, and Bongor—were situated in the semi-arid and Sudano zones. Declining temperature trends were observed at Sahr, Moundou, N’Djamena, and Zinder in the Sudano-Sahelian and Guinean zones, while other stations indicated statistically increasing unidirectional trends in the study area, consistent with the findings of [91,92,93]. However, the results revealed a variable degree of mismatch between the gridded and observed station records across the basin; for example, the CPC, CRU, GPCC, PGF, and UDEL data showed 66.67%, 73.33%, 60.0%, 66.67% and 73.33% agreement in unidirectional trends at the annual timescale, and 73.33%, 73.33%, 46.67%, 66.67%, and 66.67% agreement in unidirectional trends in the monsoon season, respectively, against the observed data. The evaluated trends indicated that the CRU and CPC datasets have the ability to replicate the observed station records with a higher degree of agreement compared to other datasets. The temperature datasets showed a higher degree of agreement in unidirectional trends against the station records, with fairly similar results—for example, CRU, PGF and UDEL showed a better performance at the annual (91.67%) and monsoon (66.67%) season timescales. However, the CRU gridded datasets showed strong increasing trends, and a possible tendency to overestimate the temperature of the basin. Therefore, the choice of the dataset adopted for hydro-climatic studies is critical for accurate assessment in order to reduce uncertainty in the prediction of hydrological variables in modelling studies and achieve good policy planning in water resource management.

The results of Sen’s slope estimator for the trend analysis of the annual and monsoon season timescales at a 95% level of confidence revealed that there were mismatches or difficulties with the gridded precipitation and temperature datasets in reproducing a similar magnitude of trends relative to the station records (Figure 6, Figure 7, Figure 8 and Figure 9). For example, the gridded datasets overestimated or underestimated the annual and monsoon season precipitation and temperature in most of the stations. However, the mismatch has been acknowledged as one of the sources of uncertainty in modelling processes—although, in this assessment, the median magnitude of the precipitation trends for CPC (1.200) and GPCC (1.000), and of the temperature trends for CRU (0.0185) and CPC/UDEL (0.0075), proved to be better than that of other datasets relative to the observed station data at the annual and monsoon season timescales across the basin.

5. Discussion

The performance of widely used, updated, and recently available high-resolution gridded precipitation and temperature datasets was assessed over the 15 available, quality-controlled, and reliable observed precipitation datasets and 12 temperature station records across the Lake Chad basin. The study’s temporal span was set to be 1979–2012 for consistency across all of the datasets, and met the requirements for a hydrological impact study. The study adopted a 0.5° resolution dataset, and finally extracted and interpolated the gridded dataset to station resolution using the inverse distance weighting method for consistency, making it adequate for regional studies.

In summary, the evaluated gridded climate datasets have proven to differ in their performance across the selected metrics, although the performance exhibited across all of the gridded climate datasets was promising—especially with regards to temperature, because climate models simulate temperature variables better than precipitation variables, as acknowledged in [94] However, these notable differences observed in the performance of the gridded datasets may be misleading in the selection or choice of a dataset based on evaluation using a single metric. Therefore, a good gridded dataset should have the ability to accurately replicate the climate patterns and amplitude of spatial and temporal variability across different performance metrics, which is critical for thorough and accurate hydro-climatic applications. This, however, has been acknowledged by [38], who stated that a single performance metric cannot adequately be relied upon for the selection of gridded datasets for global application. The merit of this methodology is predicated on the fact that the combination of multiple metrics for selection may reduce the chance of underperforming models exhibiting better performance for the wrong reasons [1].

The gridded dataset exhibited some inherent weaknesses across the performance metrics, especially in simulating the trends and magnitude of precipitation and temperature across the annual and monsoon season timescales; for example, the CRU data was shown to have a better performance in replicating the observed station data relative to the other datasets, but overestimated the temperature trends across the annual and monsoon season timescales. However, the performance across the metrics considered in this study

Table 11. Summary of the best two (or three when the metrics have the same performance) gridded precipitation and temperature datasets across the performance metrics considered in this study.

Performance Metric	Precipitation					Temperature
	CPC	CRU	GPCC	PGF	UDEL	CPC	CRU	PGF	UDEL
Symmetric Uncertainty	-	√	√	-	-	-	√	√	-
Kling–Gupta Efficiency	√	√	-	-	-	√	√	√	-
Index of Agreement	-	√	√	-	√	√	√	-	-
NRMSE	-	√	-	-	√	√	√	√	-
Taylor Diagrams	√	-	√	-	-	-	-	√	-
Trend Analysis	√	√	-	-	-	-	√	√	√
Ability across Performance Metrics	50%	83.3%	50%	0	33.3%	50%	83.3%	83.3%	16.7%

, revealed that the CRU data perform better in 5 out of 6 (83.3%) performance metrics considered for both precipitation and temperature, while CPC and GPCC have a relatively worse performance, with 3 out of 6 (50%) for precipitation, and PGF had 5 out of 6 (83%) for temperature in terms of the performance metrics. The superiority of the CRU data may be attributable to their employing a larger number of station data than other gridded datasets. However, the datasets were able to exhibit a satisfactory performance and represent the variability reasonably well, indicating that they possessed the ability to provide reliable hydro-climatic information with a lower level of uncertainty in their predictions.

The findings of this study, using multi-criteria decision making based on the assessment of the datasets by several performance metrics, revealed that the CRU, GPCC, and CPC precipitation datasets and the CRU and PGF temperature datasets showed better performance and, as such, should be recommended for application in hydro- climatic studies in the Lake Chad basin in order to enhance prediction and achieve a low level of uncertainty in terms of similarity to the observed station precipitation and temperature data, respectively. Furthermore, this was the first study to evaluate the performance and uncertainty associated with gridded precipitation and temperature datasets in the Lake Chad hydrological basin, and cannot be compared to previous studies. However, findings from other studies in regions with similar climatology have shown to be consistent with the present findings. For example, a study in the arid region of Pakistan indicated that GPCC was found to have an acceptable agreement with the observed dataset [95]. The CRU dataset was also found to have a better performance in the Niger Delta region of Nigeria [2], while GPCC, CRU, and CPC were consistent with the observed climate data in the mountainous region of South Africa [47].

Furthermore, the findings from our analysis also acknowledge the limitations associated with the development of gridded datasets and their inherent sources of uncertainty—such as spatial aggregation, the temporal period of analysis, and input uncertainty [96]—and offer ways to reduce the level of uncertainty associated with the observed station records—for example, sampling error, bias error, urbanisation effects, and rain gauge and thermometer exposure changes [97]; therefore, caution has been taken in pursuit of a reliable assessment.

6. Conclusions

This paper evaluated the performance of a long-term time series of high-resolution gridded precipitation and temperature datasets and their suitability for hydro-climatic studies in the data-scarce Lake Chad basin. The emphasis in this assessment was to employ multiple performance metrics to evaluate the ability of the selected datasets to replicate the quality-controlled observed meteorological station records available, and to provide methodological guidance based on multi-criteria decision making as to the choice of reference dataset suitable for climate and hydrological impact studies in data-scarce regions.

The results of the analysis revealed that all of the gridded precipitation datasets had the ability to replicate the observed climate record with varying levels of uncertainty—except for the PGF dataset, which exhibited unsatisfactory performance in terms of KGE, md, and NRMSE. However, all of the temperature datasets showed a strong agreement, and were consistent with the observed data. The results from the Taylor diagrams indicate a notable variability in normalised standard deviation across the annual, pre-monsoon, and monsoon season timescales, but have an acceptable Pearson’s correlation coefficient relative to the observed data records.

The trend analysis results show that the temperature datasets exhibit better ability in replicating the trends compared to precipitation datasets. Although there was a varying degree of mismatch in the magnitude of the trends across the stations, the CRU data exhibited a strong increasing temperature trend across all of the stations at the annual and monsoon season timescales, and tended to overestimate the temperature of the basin relative to the observed gauge data. However, the multi-criteria decision-making approach applied based on the performance exhibited across the metrics used in this study revealed that the CRU, GPCC, and CPC datasets are appropriate for precipitation, while the CRU and PGF datasets are appropriate for temperature, providing better replication of the basin’s climatology, with an acceptably low level of uncertainty in the prediction of climatic and hydrological variables for better policy planning and water resource management.

Furthermore, the results of this study highlight that the choice of gridded data is critical for fair representation of historical and projected future climate changes, as acknowledged in [1]. Therefore, due to differences in the temporal resolution of gridded datasets, the daily CPC precipitation and PGF temperature datasets, along with the monthly CRU precipitation/temperature datasets, are recommended for the downscaling and bias correction of global climate models in hydrological modelling processes, depending on the input data requirements of hydrological models in the Lake Chad basin, as well as on the need for improvement in the development of climate models in order to better reproduce satisfactory temporal and spatial variability in climate indices, limit uncertainties, and improve prediction accuracy in the quantification of projected climate hazards and vulnerabilities for better water policy decision making.