Global 15-Meter Mosaic Derived from Simulated True-Color ASTER Imagery

Abstract

This work proposes a new methodology to build an Earth-wide mosaic using high-spatial resolution (15 $\mathrm{m}$) Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) images in pseudo-true color. As ASTER originally misses a blue visible band, we have designed a cloud of artificial neural networks to estimate the ASTER blue reflectance from Level-1 data acquired by the Moderate Resolution Imaging Spectroradiometer (MODIS) on the same satellite Terra platform. Next, the granules are radiometrically harmonized with a novel color-balancing method and seamlessly blended into a mosaic. We demonstrate that the proposed algorithms are robust enough to process several thousands of scenes acquired under very different temporal, spatial, and atmospheric conditions. Furthermore, the created mosaic fully preserves the ASTER fine structures across the various building steps. The proposed methodology and protocol are modular so that they can easily be adapted to similar sensors with enormous image libraries.

1. Introduction

The Terra satellite was launched on 18 December 1999. This satellite platform has five instruments which include the Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) and the Moderate Resolution Imaging Spectroradiometer (MODIS). ASTER was built by the Japanese Ministry of Economy, Trade and Industry (METI) [1], while MODIS was designed by National Aeronautics and Space Administration (NASA), Goddard Space Flight Center (GSFC).

ASTER is a 15 m resolution, 14 bands multispectral instrument. It has been used for change detection, calibration, validation, and land surface studies from individual granules analysis [1]. However, the global monitoring of the Earth and ocean surfaces will be greatly helped by the integration of the satellite granule database from 2003 to 2012 into a unique natural color global mosaic, referred to as CLAMS (Color-Land ASTER MosaicS). The distributed ASTER granules cannot produce natural-color images, since ASTER sensors lack a blue visible band as illustrated by Figure 1. Generally, false color RGB composites are created by assigning red, green and blue to visible near infrared bands 3N, 2, and 1, respectively. However, ice/snow areas appear grey, desert areas yellow, and vegetation red, as illustrated on the left hand side of Figure 2.

In this work, we aim at constructing a global mosaic at 15 m of ground resolution, which blends our true-color visible ASTER images.

Several studies have addressed various ways of generating the simulated true color imagery for optical satellite sensors, which do not have a part of visible bands. Chen and Tsai [2] proposed spectral transformation techniques between Système Probatoire de l’Observation de la Terre (SPOT) false color image and Landsat-5 Thematic Mapper (TM) true color imagery. They used an unsupervised fuzzy c-means classifier and spectral control points. Knudsen [3] proposed a pseudo-natural color methodology for aerial imageries by using a simple least-squares adjusted linear model for the relationship between the blue band and the green, red and near-infrared (NIR) bands. This paper showed the possibilities of cost-efficiency by using the color-infrared (CIR) traditional photogrammetric products recorded by a traditional aerial camera. Patra et al. [4] developed spectral transformation techniques with spectral control points to IKONOS false-color imagery and natural color (simulated true color) imagery, and they evaluated their developed transformation method by using Quickbird, MODIS, Indian remote sensing satellite (IRS-P6) LISS-4, LISS-3, and AWiFS sensors. Huixi and Yunhao [5] applied the atmospheric correction algorithm, ATCOR to Landsat-7 ETM+, SPOT, and Terra ASTER imageries. They used the spectral similarity scale (SSS) method with a spectral library for generating pseudo natural color composites, and could obtain excellent results. Zhu et al. [6] developed a non-linear model based on a spectrum machine learning (SML) method with the spectral library, and they applied it to Landsat-7 ETM+, SPOT, and ASTER imageries. The atmospheric correction algorithm FLAASH from ENVI software was first applied, followed by the SML method to establish an implicit non-linear relationship between the blue band and other bands. The next-generation GOES-R advanced baseline imager (ABI) does not have a green band, and high-resolution atmospheric model simulations have been used to produce the ABI reflective band imagery required for true-color imagery [7,8]. Most researchers suggest that true-color algorithms are affected by sun-target-sensor geometry and atmospheric conditions.

There have been several efforts to construct cloud-free true color base-maps from moderate or high spatial resolution (less than 100 $\mathrm{m}$) satellite data. ASTER has been operated over 19 years since Terra satellite launch in 1999. Thus, the huge available acquisitions database makes it possible to generate cloud-free global mosaics. The large number of ASTER image granules (about 780,000) used in this study were collected over many years, different seasons, and under varying vegetation and illumination conditions. Without the appropriate corrections, the resulting mosaic can appear as a patchwork of individual images. To avoid this, it deems necessary to apply atmospheric corrections and to smoothen seasonal effects. There have been several attempts of true-color mosaic constructions for various satellite optical sensors, mostly using adjustments of radiometric characteristics. Guindon [9] proposed radiometric adjustment for seamlessness mosaic of Landsat-5 MSS for northwestern Ontario area. Liew et al. [10] calculated solar zenith angle corrected radiances, and have used a brightness thresholding method to identify the best cloud-free and non-shadow pixels among the pixels from the multiple images at a given region. They successfully tested their mosaic technique with SPOT images acquired over the South East Asia region. Du et al. [11] applied a radiometric equalization techniques for representative pixel pairs in each overlap area, selected by means of a principal component analysis and calculation of linear correlation coefficients. They proved the methodology by mosaicking 6–7 Landsat-5 TM granules over the Boreal Ecosystem-Atmosphere Study (BOREAS) transect. Bindschadler et al. [12] generated a seamless cloud-free Landsat-7 ETM+ mosaic of Antarctica by radiometric adjustment. Roy et al. [13] produced a mosaic of the conterminous United States (CONUS) using 6521 Landsat-7 ETM+ imageries from December 2007 to November 2008. Choi et al. [14] developed the mosaic algorithm for high resolution images captured by Kompsat-2 sensor. This algorithm can be applied to different images affected by seasonal change, and is applicable to other high resolution optical sensor images. There exist two global mosaics, GeoCover2000 [15] (Landsat-7) and Landsat-8 VNIR maps [16], that have processed 30 $\mathrm{m}$ Landsat images and pansharpened them to 15 $\mathrm{m}$. In summary, most attempts of generating mosaics of high resolution optical imageries were mostly carried out over regional areas, as the construction of a global cloud-free mosaic of high resolution imageries is extremely difficult due to the large amount of spatially and temporarily varying acquisitions.

We propose a protocol to assemble a true-color global mosaic from high-resolution ASTER imagery. The protocol first implies the construction of the missing ASTER blue band making use of the Moderate Resolution Imaging Spectroradiometer (MODIS) Terra instrument as it sits on board of the same satellite platform acquires time and space synchronized images with all visible bands, though at a lower spatial resolution (250–500 m). In 2012, we developed a first algorithm in which, after atmospheric corrections, the ASTER pseudo-blue band was constructed from a single artificial neural network (ANN), making it possible to generate ASTER granules in true color composite. These processed ASTER images are currently distributed by the AIST MADAS system as one of ASTER-VA products [17,18]. This first algorithm was not completely satisfactory and a couple of years ago, we decided to improve the blue color retrieval in particular over the ocean, as described in Section 2. The new retrieval algorithm this time uses a cloud of ANN’s, which preserve the finest details of the atmospheric components, such as dust, smoke and thin clouds.

The second key feature of the constructed CLAMS mosaic is that it is worldwide color-balanced, in the sense that the radiometric differences between the adjacent images introduced by the solar incident angle, atmosphere, and illumination condition are equalized. This is achieved by a novel color-balancing method for ASTER based on a MODIS reflectance reference library (FondsDeSol, FDS) proposed by Gonzalez et al. [19]. By automatically selecting appropriate color reference information from the FDS library according to the geographical scope and acquisition season information of the target images, the proposed approach provides effective solutions for eliminating color error propagations between adjacent granules over the globe. We will illustrate the rendered color quality of the mosaic in Section 3.1 and on the website http://newtec.univ-lille1.fr/LARGEMOSAICSFR/.

The third feature of the CLAMS mosaic is that it preserves the ASTER fine 15 $\mathrm{m}$ structures across the various building steps, as demonstrated in Section 3.2.

2. Methodology for ASTER Blue Reflectance Reconstruction

In the entire document, all presented data correspond to reflectance values from MODIS L1B or ASTER L3A products to which minimum atmospheric corrections were applied to compensate for the Rayleigh scattering contribution, and correct for ozone according to the 6S radiative transfer code [20], with the same equations as in Ref. [21]. These corrected reflectances are denoted hereafter as $\rho$. In all the figures, we present ASTER granules geometrically corrected to Plate Carrée projection (North oriented) within a bounding box of 75 $\mathrm{k}$$\mathrm{m}$ × 75 $\mathrm{k}$$\mathrm{m}$, and we specify both the dates and geolocalization of their central points.

The ASTER granules suffer from two main defects. The first is linked to signal saturations in the $0.56$ $\mathsf{\mu }$$\mathrm{m}$ and $0.66$ $\mathsf{\mu }$$\mathrm{m}$ bands over bright surfaces as explained in Section 2.1, while the second corresponds to the missing $0.4$ $\mathsf{\mu }$$\mathrm{m}$ band which is reconstructed with ANNs using MODIS reflectance data as detailed in Section 2.2. To train the ANN and to reconstruct the saturated ASTER pixels missing values we use as input data the 0.555, 0.645, and $0.858$ $\mathsf{\mu }$$\mathrm{m}$ bands from MODIS and the $0.469$ $\mathsf{\mu }$$\mathrm{m}$ blue band as target output data. To retrieve the ASTER blue band we use as input data the 0.56, 0.66, $0.81$ $\mathsf{\mu }$$\mathrm{m}$ ASTER bands.

2.1. Reconstruction of Saturated ASTER Level1 Pixels

In the process of correcting the bright surface saturation we do not expect to retrieve the real reflectances but simply to create reasonable values giving true color composites close to reality. Saturated values only appear for the 0.56 and $0.66$ $\mathsf{\mu }$$\mathrm{m}$ bands (noted ${\rho }_1$ and ${\rho }_2$). A correction model was empirically determined using MODIS reflectances over the ASTER saturated areas. It turned out that simple first and second-order polynomial functions of the $0.81$ $\mathsf{\mu }$$\mathrm{m}$ (${\rho }_{3N}$) reflectances could be used to assign unsaturated reflectance values ${\rho }_1$ and ${\rho }_2$, as detailed in

Table 1. Polynomial expansions for the saturation correction of bands ${\rho }_1$ ($0.56$ $\mathsf{\mu }$$\mathrm{m}$), ${\rho }_2$ ($0.66$ $\mathsf{\mu }$$\mathrm{m}$) using ${\rho }_{3N}$ ($0.81$ $\mathsf{\mu }$$\mathrm{m}$).

Ocean Colors
${\rho }_{3N}<0.2$	${\rho }_1=-1.627869\times {\rho }_{3N}^2+0.714701\times {\rho }_{3N}+0.370053$
	${\rho }_2=-17.420099\times {\rho }_{3N}^2+6.117370\times {\rho }_{3N}-0.180111$
${\rho }_{3N}>0.2$	${\rho }_1=2.298600\times {\rho }_{3N}^2-1.202424\times {\rho }_{3N}+0.591962$
	${\rho }_2=0.725354\times {\rho }_{3N}+0.202700$
Desert
	${\rho }_1=0.564969\times {\rho }_{3N}-0.002576$
	${\rho }_2=0.808582\times {\rho }_{3N}-0.001048$
Clouds
	${\rho }_1=1.176044\times {\rho }_{3N}-0.081713$
${\rho }_1>0.15$	${\rho }_1=1.096000\times {\rho }_{3N}-0.020926$
	${\rho }_2=1.220622\times {\rho }_{3N}-0.086758$
${\rho }_2>0.15$	${\rho }_2=1.186130\times {\rho }_{3N}-0.090857$

.

Figure 3, Figure 4 and Figure 5 demonstrate that the desaturation algorithm visually corrects the missing values over desert, ocean and cloudy areas.

2.2. ANN to Reconstruct ASTER Blue Reflectances

Fine-tuning the parameters of an ANN (Stuttgart Neural Network Simulator (SNNS) [22]) to faithfully reconstruct ASTER blue reflectances turned out to be a tricky process. In 2012, we designed an ANN that used for the training inputs the 3 MODIS 0.555, 0.645, $0.858$ $\mathsf{\mu }$$\mathrm{m}$ bands (spectrally close to the VNIR ASTER bands, 0.56, 0.66, $0.81$ $\mathsf{\mu }$$\mathrm{m}$), and the blue MODIS band at $0.469$ $\mathsf{\mu }$$\mathrm{m}$ as target values. This ANN is a fully connected feedforward network [23]. The ANN topology we present yields the best blue retrieval out of many tests we carried out. It consists of two hidden layers with 10 nodes each (See Figure 6). The training data set used a selection of about fifty MODIS granules for which we hand-selected representative areas such as water, ocean colors, deserts, rocks, white and bright surfaces, etc. Once trained, the network was tested on various MODIS granules and the training ANN data set was enlarged with those showing inaccurate (poor correlation with respect to MODIS real blue reflectances) reconstructed blue values. However, the blue retrieval was not fully accurate, yielding yellow colors over hazy areas and bright reefs as illustrated by the examples on the left hand side of Figure 7.

Based on our recent experience with ANNs design for aerosol optical depth retrievals [21], we found out that ANNs training and accuracy are improved if applied to specific classes of reflectance values. Thus, our new algorithm uses MODIS values that are classified into three groups according to the values $x=0.2\times {\rho }_{MODIS}\left(0.645\right)+0.8\times {\rho }_{MODIS}\left(0.555\right)$, defining two main areas, the first one corresponding to water, dark areas and vegetation ($x<0.2$), the second one to desert areas ($x>0.2$). The third group encompasses the brightest cloudy pixels selected by according to the following three-fold tests:

$$\begin{array}{c}\hfill \{{\rho }_{MODIS}\left(0.555\right)>0.2\}\&\{{\rho }_{MODIS}\left(0.858\right)>0.2\}\&\left\{\frac{{\rho }_{MODIS}\left(0.555\right)}{{\rho }_{MODIS}\left(0.645\right)}>0.9\right\}.\end{array}$$

(1)

The histograms of the three groups are drawn in Figure 8. Each of the three histograms are split to define subclasses of pixel values defining the inputs of ANNs. The determination of the number of data by classes (bins) was carried out in successive stages by monitoring both the convergence of the networks and the quality of the blue retrieval over about one hundred MODIS granules that compose the training data sets. It soon became clear that the first group requires a more precise division (800 networks) and converges with an average number of 4000 pixels per bin. In group 2, the subdivision is wider with 1900 networks and an average number of 2200 pixels per bin was necessary. The third group requires 1300 networks with an average number of 3200 pixels per bin. Figure 9 demonstrates how the histogram classification is used to split the MODIS data. For each bin, a dedicated ANN with a feedforward topology, with the same architecture as before (3 inputs, 2 hidden layers with 10 nodes each, and 1 output value, see Figure 6), is designed. Altogether an ensemble of 4000 ANNs is trained. The training dataset was enlarged to about one hundred MODIS granules.

For the retrieval, the three ASTER bands (0.56, 0.66, $0.81$ $\mathsf{\mu }$$\mathrm{m}$) are scaled to match the MODIS ones (0.555, 0.645, $0.858$ $\mathsf{\mu }$$\mathrm{m}$) as:

$$\begin{array}{c}\hfill \{{\rho }_1={\rho }_{ASTER}\left(0.56\right)\times 0.9140\};\{{\rho }_2={\rho }_{ASTER}\left(0.66\right)\times 0.8918\};\{{\rho }_3={\rho }_{ASTER}\left(0.81\right)\times 0.8857\},\end{array}$$

(2)

the coefficients being determined from linear regressions.

The central panel of Figure 7 demonstrates that the 2018 blue retrieval algorithm yields enhanced blue reflectances, improving the final image rendering. Furthermore, the absolute quality of the 2018 blue reconstruction is proven by Figure 10 showing the excellent correlations (at least 0.98 with low standard deviations, max of 0.03) between all three bands of ASTER and MODIS. This is further highlighted by Figure 11 where a comparable agreement is found with Landsat-8 OLI L1T products which holds a real blue channel. This Landsat-8 L1T were obtained via the USGS EarthExplorer.

In order to preserve the spectral response of ASTER instrument, the red and green bands of the final visible composite are the ASTER original measurements, while the blue is reconstructed from the cloud of ANNs. As the MODIS and ASTER bands response are slightly shifted (See Figure 1), one can observe in Figure 12 subtle differences for red and green areas in the final color rendering of the natural ASTER composite with respect to MODIS.

To further illustrate the accuracy of the blue reconstruction, we present in Figure 13, examples of heterogeneous natural-color reconstructions. The final results preserve the 15 m ASTER sensitivity, as we notably recognize the signatures of thin clouds on both dark (Figure 13a,b) and bright backgrounds (Figure 13c) with their shadows, as well as the fine patterns of bright sand dunes in deserts (Figure 13d).

3. Building a Global Mosaic

The protocol to construct the CLAMS global mosaic released in 2012 has involved the scanning of the whole distributed ASTER portfolio from 2003 onwards to define a classification algorithm that sorts images with respect to their temporal distance to a seasonal reference date, but also with increasing degree of cloudiness as to minimize cloudy granules. About 780,000 granules where analyzed from where 70% were downloaded and colorized, while 40% contribute to the final mosaic.

3.1. Color-Matching of ASTER Granules to FondsDeSol Reference

In order to construct a seamless global mosaic image, radiometric and phenomenological scene differences have to be minimized across space and time. Helmerand and Ruefenacht [24] used histogram matching of adjacent scenes to build small-size regional mosaics, but this approach leads to increasingly growing bias when applied to large-scale areas. The availability of a world-wide cloud-free surface reflectance database FondsDeSol [19] centered at the end of august is used as reference to equalize the ASTER granules over the whole Earth.

To achieve the color equalization of the ASTER RGB bands and remove as far as possible bidirectional effects and seasonal changes, we have used the iterative distribution transfer (IDT) algorithm proposed by Pitié et al. [25], which allows within few iterations, typically less than 10, an excellent color match to the FondsDeSol reference. It is noteworthy that this colorization process allows to partially dehaze ASTER granules that are blurred by the atmospheric aerosols. An example is displayed in Figure 14, which presents in subset Figure 14 (1) patched natural color ASTER granules tainted by the atmospheric components. Figure 14 (2) shows the same patch colorized by the IDT process, the only visible transitions being that of the granules boundaries. Indeed, seasonal effects are smoothened but may be still be slightly visible when the land surface is significantly altered across various seasons (field structures, forest, rivers, etc.).

3.2. Seamless ASTER Granules Blending

To create seamless mosaics, we relied on an effective methodology using the Laplacian pyramid-based algorithm introduced by Burt and Adelson [26]. Multi-scale Gaussian pyramid representations of the overlapping areas of the patched granules are used to merge all the spatial structural details at the 15 $\mathrm{m}$ resolution, the color matching being already achieved by the FondsDeSol color equalization preceding step. Figure 15 illustrates the reflectance values over transects of two different surface types in the Nusa Tenggara (Indonesia) and Van Diemen Gulf (Australia). The black curves represent the original ASTER natural color granules, the magenta or purple lines refer to the values across IDT colorized granules, while the R,G,B curves correspond to the final results after pyramid blending. Note that the different steps shift the reflectance values but preserve the fine structures of the original data. In Figure 15a, the colorization and blending effects are minor across the granules boundaries (vertical black lines). In Figure 15b we have highlighted with red circles large changes in the reflectance values between the two last steps of the mosaic process (IDT and blending). In this example the blending step (displayed in the (3) inset) brings in thin smoke plumes from the background image (https://earthexplorer.usgs.gov/).

3.3. Filling the Gaps

For limited numbers of very small areas, where we did not find any suitable ASTER granules, we patched the mosaic with less than 100 Landsat-7 ETM+ granules processed by NASA/JPL CalTech, which is derived from GeoCover2000 [15]. These were color-equalized with the IDT algorithm and merged into the mosaic with the pyramid blending method.

3.4. Distribution of the Mosaic

Partial mosaics samples can be seen on http://newtec.univ-lille1.fr/LARGEMOSAICSFR/ with a web interface that allows to visualize and zoom into several subsets of the global mosaic, with the 2012 blue algorithm (France, Saudi Arabia), and the 2018 blue algorithm (Australia, Indonesia, South Africa, Bahamas, Antarctic). The global mosaic with the 2012 blue algorithm will be available on the https://gbank.gsj.jp/madas/?lang=en web-platform in 2019, while the latest higher quality one shall be also available in the near future.

4. Conclusions and Future Work

This work is the first world-wide mosaic of ASTER images in natural colors. Two main problems of the distributed data have been tackled, namely the saturation over bright surfaces, and most importantly the lack of a blue band. The latter was reconstructed using a large cloud of ANNs trained on MODIS reflectance values, improving the blue rendering with respect to the 2012 version. The granules with the 2012 version are readily available on the https://gbank.gsj.jp/madas/?lang=en web-platform, while the latest higher quality ones shall be available in the near future. The availability of true-color ASTER granules is by itself relevant to help comparisons with other huge high-resolution database as Landsat, in particular to monitor surface changes.

The construction of the mosaic implies to harmonize the colors of spatially and temporarily separated granules and to fusion them with seamless blending algorithms. We have demonstrated that the various steps preserve the 15 $\mathrm{m}$ spatial structures of the original ASTER images. The created global mosaic is a real 15 $\mathrm{m}$ resolution mosaic that can serve as a reference to validate the quality of other global maps [15,16]. The presented mosaic protocol with the improved blue retrieval will shortly be applied to create a revised version of the ASTER mosaic. The proposed algorithmic protocol has processed several hundred thousands ASTER images and its robustness and modularity can be adapted to similar sensors with enormous image libraries, such as SPOT.