Advantages of the Boresight Effect in Hyperspectral Data Analysis

Abstract

Dual pushbroom hyperspectral sensors consist of two different instruments (covering different wavelengths) that are usually mounted on the same optical bench. This configuration leads to problems, such as co-registration of pixels and squint of the field of view, known as the boresight effect. Determination of image-orientation parameters is due to the combination of an inertial measurement system (IMU) and global position system (GPS). The different positions of the IMU, the GPS antenna and the imaging sensors cause the orientation and boresight effect. Any small change in the correction of the internal orientation affects the co-registration between images extracted from the two instruments. Correcting the boresight effect is a key and almost automatic task performed by all dual-system users to better analyze the full spectral information of a given pixel. Thus, the boresight effect is considered to be noise in the system and a problem that needs to be corrected prior to any (thematic) data analysis. We propose using the boresight effect, prior to its correction, as a tool to monitor and detect spectral phenomena that can provide additional information not present in the corrected images. The advantage of using this effect was investigated with the AISA-Dual sensor, composed of an EAGLE sensor for the VIS-NIR (VNIR) region (400–970 nm) and HAWK for the SWIR region (980–2,450 nm). During the course of more than six years of operating this sensor, we have found that the boresight effect provides a new capacity to analyze hyperspectral data, reported herein. Accordingly, we generated a protocol to use this effect for three applications: (1) enhancing the shadow effect; (2) generating a 3-D view; and (3) better detecting spectral/spatial anomalies based on sub-pixel edge detection. This paper provides examples of these applications and suggests possible uses from an airborne platform.

1. Introduction

Hyperspectral imaging spectrometers produce data with high spectral resolution domains (in the range of 5 to 15 nm bandwidths) and continuous band configuration. This gives processors the ability to detect subtle spectral features and defined chemical and physical properties of the sensed objects. This powerful capability has been successfully used in many applications, such as: geological mapping [1,2,3]; agricultural monitoring and optimization [4,5]; environmental damage assessment [6,7]; forestry surveys [8,9]; vegetation monitoring [10,11]; water-quality assessment [12,13]; detection of man-made materials [14,15] and more.

The determination of image-orientation parameters of any sensor during data acquisition is made possible by the combined use of an inertial measurement unit (IMU) and global position system (GPS). In this integrated system, the GPS antenna, IMU and imaging sensor are located in different positions on the airborne carrier. The displacement vectors between sensors therefore have to be determined. Similarly, axes of the IMU and imaging sensor are not the same and a misorientation matrix exists between them. System calibration includes both calibration of individual sensors and calibration between sensors. The IMU calibration for drifts and biases and calibration of the imaging sensor for interior orientation parameters are all components of sensor calibration. Alignment between sensors includes the determination of a constant displacement vector between them and a constant misorientation matrix between the IMU body frame and imaging-sensor frame. The boresight effect is a product of different optics and detector matrices between the two sensors. The resulting boresight misalignment is treated by using the IMU information and the sensor bundle block. A small change in the correction of the interior orientation affects the co-registration between the sensors, generating a boresight effect that is proportional to the increase in altitude. When the sensor’s altitude is different from the flight altitude recommended by the manufacturer (for example 6,000 ft, recommended by Specim Ltd.), the boresight misalignment increases significantly. To enable geo-matching between the two sensors, geo-rectification is performed by conventional methods.

AISA-Dual is an airborne imaging spectrometer designed by Specim Limited as a research sensor that is capable of producing medial to high-fidelity hyperspectral remote sensing (HRS) data across the 400 to 2,400 nm wavelength range. The system consists of a sensor head, containing a pair of co‑boresighted grating spectrometers VNIR sensor EAGLE and SWIR sensor HAWK, two electronic racks, and a digital data recorder. It simultaneously acquires images in 198 contiguous spectral bands with spectral resolution in the range of 12 nm in the VNIR region, and 6 nm in SWIR region. Each spectrometer consists of a set of refractive foreoptics that image the scene onto a slit. Light passing through the slit is dispersed perpendicular to the slit by a flat rating and then imaged onto a 2-D focal plane array. One dimension of the array along the slit provides spatial scene information. The second dimension of the array, along which the light from any given point in the slit has been dispersed, provides spectral information. An image is generated by moving the instrument across a scene in pushbroom fashion, perpendicular to the instrument’s slits, and recording frames of spectral and spatial information detected by the VNIR and SWIR sensors. The system is usually operated on an aircraft at an altitude of 10,000 ft which, together with an instant field of view (IFOV) of 1 mrad (for both sensors) provides a spatial resolution of 1.5 m. A standard AISA-Dual dataset is a 3-D data cube in a non-earth coordinate system (raw geometry). It has 286 pixels in the cross-track direction and hundreds of pixels in the along-track direction. Operational performance requires an instrument with fair signal-to-noise ratio (SNR), co-registered spectral bands taken simultaneously by different detectors, accurate location for each pixel, and accurate radiometric calibration. Since the AISA-Dual instrument that was used for this study has no available GPS/INS (global position system/internal navigation system), it is incapable of performing pixel-by-pixel geo-location and rectification of the images generated in the HRS dataset. The idea behind this study is that the boresight effect does not always need to be corrected as it may provide additional, never before considered information to the analysis. To test this notion, three practical applications are suggested and discussed: (1) enhancing the shadow effect; (2) generating a 3-D view; and (3) better detection of borderline anomalies.

2. Boresight Effect in a Multisensor System

Sensor orientation, defined by the transformation between image coordinates specified in the sensor and the geodetic reference frame, requires knowledge of the interior and exterior sensor parameters. The interior parameters, such as focal length and optical distortions, are provided by the manufacturer. The exterior parameters are defined by the position and orientation of the sensor at the time of exposure and are acquired by a recording system during the flight.

The pushbroom system, whose perspective geometry varies with each scan line (along-track), depends on integrated GPS/INS to achieve operationally effective products. In the multisensor pushbroom system, the direct platform orientation is related to the sensor’s reference frame, while in traditional airborne systems, it is related to the INS frame. A GPS/INS calibration is required to resolve the misalignments between the INS frame and the imaging sensors with sufficient accuracy. Since a crucial part of the direct platform orientation is the rigidity of the sensor, no rotation or movement can occur during data acquisition and every rotational component has to be related to the INS frame and transformed to the sensor frame [16]. The boresight transformation is usually resolved by comparing the GPS/INS position/orientation with an independent aero-triangulation solution. However, the boresight transformation can be treated only if no rotation or movement has occurred. The direct geo-referencing of images is described in Equation 1 by a collinearity concept rearranged for object coordinates [17]:

$${\mathit{r}}_{\mathit{G}}^{\mathit{M}}={\mathit{r}}_{\mathit{E}}^{\mathit{M}}\mathbf{\left(}\mathit{t}\mathbf{\right)}+{\mathit{s}}_{\mathit{g}}{\mathit{R}}_{\mathit{c}}^{\mathit{M}}{\mathit{r}}_{\mathit{g}}^{\mathit{c}}\mathbf{\left(}\mathit{t}\mathbf{\right)}$$

(1)

where ${\mathit{r}}_{\mathit{G}}^{\mathit{M}}$ is the vector of coordinates of an arbitrary point in the line frame, ${\mathit{r}}_{\mathit{E}}^{\mathit{M}}\mathbf{\left(}\mathit{t}\mathbf{\right)}$ is the vector of coordinates of the exposure station, ${\mathit{s}}_{\mathit{g}}$ is a scale factor per point per image for each sensor, ${\mathit{R}}_{\mathit{c}}^{\mathit{M}}\mathbf{\left(}\mathit{t}\mathbf{\right)}$ is a matrix to rotate each sensor frame into the line frame ${\mathit{r}}_{\mathit{g}}^{\mathit{c}}$utilizing the three photogrammetric rotation angles, $\mathit{\omega }, \mathit{\phi }$, $\mathit{\kappa }$ [2].

Since the sensors, the GPS antenna, and the INS are not fixed on the same coordinate, Equation (1) can be modified to include orientation offsets:

$${\mathit{r}}_{\mathit{G}}^{\mathit{M}}=\mathbf{\left(}{\mathit{r}}_{\frac{\mathit{G}\mathit{P}\mathit{S}}{\mathit{I}\mathit{N}\mathit{S}}}^{\mathit{M}}\mathbf{\left(}\mathit{t}\mathbf{\right)}+{\mathit{R}}_{\mathit{b}}^{\mathit{M}}\mathbf{\left(}\mathit{t}\mathbf{\right)}{\mathit{a}}_{\mathit{E}}^{\mathit{b}}{\mathit{b}}^{\mathit{b}}\mathbf{\right)}+{\mathit{s}}_{\mathit{g}}\mathbf{\left(}{\mathit{R}}_{\mathit{b}}^{\mathit{M}}\mathbf{\left(}\mathit{t}\mathbf{\right)}{\mathit{R}}_{\mathit{c}}^{\mathit{b}}\mathbf{\right)} {\mathit{r}}_{\mathit{g}}^{\mathit{c}}\mathbf{\left(}\mathit{t}\mathbf{\right)}$$

(2)

where ${\mathit{r}}_{\frac{\mathit{G}\mathit{P}\mathit{S}}{\mathit{I}\mathit{N}\mathit{S}}}^{\mathit{M}}\mathbf{\left(}\mathit{t}\mathbf{\right)}$ is the vector of coordinates of the INS in the line frame, ${\mathit{R}}_{\mathit{b}}^{\mathit{M}}\mathbf{\left(}\mathit{t}\mathbf{\right)}$ is a matrix to rotate each sensor frame into the line frame ${\mathit{R}}_{\mathit{c}}^{\mathit{b}}$, ${\mathit{a}}_{\mathit{E}}^{\mathit{b}}$ is an offset vector between INS and the sensor exposure station and ${\mathit{b}}^{\mathit{b}}$ is an offset vector between the INS and the GPS antenna that is processed during decoding of the GPS/INS data.

A constant boresight is ensured in the static terrestrial mode, where the sensors and the INS are tightly coupled in a metal frame. Thus, the boresight estimation is limited by the quality of the direct platform orientation components. The boresight estimations are performed by comparing aerial triangulation, GPS/INS position and attitude estimation [18]. The resulting boresight standard deviations for the linear displacements and for the rotation angle can be calculated. However, possible variations in the mechanical structure of the sensor or the INS during the flight might explain the rather fair quality of the boresight estimation.

As mentioned earlier, in practice, the AISA-Dual system is operated with no GPS/INS data available. Consequently, it was impossible to geo-locate or rectify the EAGLE and HAWK (VNIR and SWIR image) sensors. Figure 1 shows the spatial/spectral boresight effect of the AISA-Dual sensor used in this study taken from a representative image.

We suggest converting the boresight shift into additional spectral/spatial information by calculating a simple band ratio between the last band of the VNIR sensor and the first band of the SWIR sensor. We calculate a simple ratio and not a different (normalized) index, since the latter is not dynamic enough (being similar to NDVI (Normalized Difference Vegetation Index) and SR (Simple Ratio) of vegetation [19,20,21,22,23]) to be suitable for an estimation of the boresight shift, which is defined on an image-by-image basis. The boresight “band ratio” is calculated based on Equation (3).

$$Boresight band ratio =b_{1010nm}/b_{920nm}$$

(3)

The result is presented in Figure 2. We used the descriptive statistics to summarize the boresight data in a clear and understandable way (Table 1, Figure 3). The exact tests of significance are known when the samples are drawn from a normally distributed dataset [24]. The tests of significance for exponentially distributed data are related to all skew distributions of special cases. In referring to the shape of the frequency or probability distributions, skewness refers to asymmetry of the distribution, which is zero in the normal distributions. Karl Pearson [25] was the first to suggest measuring skewness by standardizing the difference between the mean and the mode. The most commonly used measure of skewness is defined with respect to the third moment about the mean (Equation (4)), which is simply the expected value of the distribution of cubed z scores. The investigation of outliers is prompted by large skewness scores.

$${\mathit{\gamma }}_1=\frac{{\sum }^{\text{}}{\left(\mathit{X}-\mathsf{\mu }\right)}^2}{\mathit{n}{\sigma }^3}$$

(4)

where $\mathsf{\mu }$ is the mean, $\mathsf{\mu }$ is the standard deviation, and $\mathit{n}$ is the number of data points.

Figure 1. Schematic demonstration of the boresight effect. Images I-VNIR and II-SWIR illustrate the boresight: I is the target in the VNIR region and II is the target in the SWIR region, and the spectrum of the target (black dot) indicates that region I is pavement asphalt (spectrum in VNIR matches the asphalt (red) spectrum) and II is concrete roof (spectrum in SWIR matches the concrete (blue) spectrum).

Figure 1. Schematic demonstration of the boresight effect. Images I-VNIR and II-SWIR illustrate the boresight: I is the target in the VNIR region and II is the target in the SWIR region, and the spectrum of the target (black dot) indicates that region I is pavement asphalt (spectrum in VNIR matches the asphalt (red) spectrum) and II is concrete roof (spectrum in SWIR matches the concrete (blue) spectrum).

Figure 2. Additional boresight band (31 October 2009, 10:00 GMT, midlatitude summer model, 28.2° solar zenith, 137.6° azimuth angle): (A) the 948 nm band, (B) the 1,010 nm band, (C) calculated band ratio (920 nm/1,010 nm) interpreted as boresight band.

Figure 2. Additional boresight band (31 October 2009, 10:00 GMT, midlatitude summer model, 28.2° solar zenith, 137.6° azimuth angle): (A) the 948 nm band, (B) the 1,010 nm band, (C) calculated band ratio (920 nm/1,010 nm) interpreted as boresight band.

Table 1. Descriptive statistics.

N	Mean		Std.	Variance	Skewness
Statistic	Statistic	Std. Error	Statistic	Statistic	Statistic	Std. Error
255 255	2,706.325	569.6115	9,095.966	8.3E + 07	4.132	0.153

Table 1. Descriptive statistics.

N	Mean		Std.	Variance	Skewness
Statistic	Statistic	Std. Error	Statistic	Statistic	Statistic	Std. Error
255 255	2,706.325	569.6115	9,095.966	8.3E + 07	4.132	0.153

Figure 3. Distribution of the boresight values.

Figure 3. Distribution of the boresight values.

In the suggested method, the skew is treated as an important characteristic of the variable. We used the SPSS statistical package to estimate the skewness and its standard errors. This test divides the distribution of the variable (boresight ratio) by its standard error to obtain a z test of the ${\mathit{H}}_{\mathbf{0}}$ null hypothesis that the skewness is zero (as would be expected in a normal population). A reliable rule is that with a nominal 95% confidence interval for the mean of a skewed variable, the interval will have at least 94% coverage if the sample size is at least 25 times the absolute value of the (third-moment) skew [26]. For example, in Table 1, if the 255 samples are even enough, the skew can be as large as four. Based on simple data-analysis techniques, we find that boresight rates conform better to non-normal distribution than normal distribution and the patterns of skewness. A suitable measure of the suggested boresight rate variability is based on the extreme observations (more than three standard deviation units from the mean of probability distributions). The estimation for each boresight rate, the descriptive statistics is first computed, and then the mean and variance of normally distributed data are calculated. In contrast to the normally distributed variances, the non-normally distributed variances are consistently larger. This suggests that with the presence of skewness in the distribution, the boresight variance is likely to interpret the outliers' variability.

3. Boresight Applications: Results and Discussion

Three applications were investigated: (1) enhancing the shadow effect; (2) generating a 3-D view; and (3) better detection of border anomalies.

3.1. Enhancing the Shadow Effect

The effects of apparent shadows on the sensor response depend on both the materials present in the image (shape and content) and the wavelength in question. Current implementations of the ATCOR-4 model de-shadowing process use image statistics to gain knowledge on the darkened area, in order to correct for the shadow effect. This routine is not suitable for datasets acquired on cloud shadows as the inter-comparison process lacks the diffuse light conditions. The de-shadowing algorithm consists of a sequence of eight processing steps: atmospheric correction, cloud masking, water-body masking, and five additional statistical manipulations including covariance matrices and matched filters to define a core shadow mask, and the final step of de-shadowing which is applied exclusively to the pixels in the shadow mask [27].

We suggest using the boresight band ratio to produce an internal shadow map for AISA-Dual images. An interpretation of the boresight band identifies core shadow areas with high values of boresight. In Figure 4, a scatter plot computed from the boresight band ratio and VNIR band (941 nm) is displayed. The clusters are spread out and slanted due to the scene-unmixing pixels. However, the isolated cluster is representative of shadowed areas in the selected scene. We simulated geometric landscape and structure by implementing 3-D buildings into the Digital Terrain Model (DTM) using a Kriging interpolation model [28] to create a representative 3-D map of the investigated site. This map provides the ability to simulate illumination sensed at the sensor level for each pixel by calculated hillshade map and shadow model according to the illumination geometry and surface model of the investigated site. The produced shadow map serves as a ground truth map.

Figure 4. Scatterplot (286 × 400) computed from the boresight band ratios (boresight value) and VNIR band (941 nm). The isolated cluster (circled in red) represents the shadow regions in the image (validated by ground truth map).

Figure 4. Scatterplot (286 × 400) computed from the boresight band ratios (boresight value) and VNIR band (941 nm). The isolated cluster (circled in red) represents the shadow regions in the image (validated by ground truth map).

We determined the relationships between the shadow fraction (mapped as ground truth) and the boresight band ratio values. While only a general indistinct relationship emerged ($R^2$ = 0.173) between radiance response of the image and the shadow’s algorithm provided in ATCOR-4 (Figure 5(A)), there were strong negative correlations for selected scenes between shadowed areas and boresight values (Figure 5(B)).

Figure 5. Relationship between shadow fraction and (A) radiance spectra (550 nm) detected by ATCOR-4 algorithm ($y=944x+1411$ ; $R^2$ is 0.173), (B) boresight values of detected shadow areas ($y=-0.74x+1.31$ ; $R^2$ is 0.913).

Figure 5. Relationship between shadow fraction and (A) radiance spectra (550 nm) detected by ATCOR-4 algorithm ($y=944x+1411$ ; $R^2$ is 0.173), (B) boresight values of detected shadow areas ($y=-0.74x+1.31$ ; $R^2$ is 0.913).

On a scene-by-scene basis, we conclude that the boresight band ratio values are much more sensitive to the shadow fraction than the ATCOR-4 shadow detector (Figure 6). In addition, we evaluated ‘darkening’ for each pixel in the produced shadow map using spatial matching between the boresight band ratio values and the shadow fraction in the ground truth map (Table 2). Based on overlapping/spatial matching between these images (ground truth and boresight band ratio), the boresight band ratio values were classified according to the shadow fraction.

To validate our results, we examined classification of the shadow fraction using the same (predefined) classes for different scenes (Figure 7). The results show that the suggested classes are suitable for the shadow fraction classification and scene independently, despite the fact that the boresight values are calculated in image-by-image mode. These results prove that the suggested method is fully automatic and scene independent.

Table 2. Boresight ratio value correspondence to shadow fraction (ground truth).

Class ID	Boresight value (band ratio)	Shadow fraction (ground truth)
#1 (maroon)	4–3.4	80–100%
#2 (magenta)	3.4–2.5	50–80%
#3 (cyan)	2.5–2.1	20–50%
#4 (yellow)	2.1–1.9	10–20%

Table 2. Boresight ratio value correspondence to shadow fraction (ground truth).

Class ID	Boresight value (band ratio)	Shadow fraction (ground truth)
#1 (maroon)	4–3.4	80–100%
#2 (magenta)	3.4–2.5	50–80%
#3 (cyan)	2.5–2.1	20–50%
#4 (yellow)	2.1–1.9	10–20%

Figure 6. Shadow masks as generated from the original AISA-Dual image: (A) RGB natural color composite AISA-Dual image (18 December 2009; 11:30 GMT; 15 km visibility; midlatitude winter model; 60° solar zenith; 205.8° azimuth angle), (B) boresight shadow mask, (C) boresight shadow estimation (#4 (yellow) 10–20%, #3 (cyan) 20–50%, #2 (magenta) 50–80% and #1 (maroon) 80–100%), (D) ATCOR-4 shadow mask.

Figure 6. Shadow masks as generated from the original AISA-Dual image: (A) RGB natural color composite AISA-Dual image (18 December 2009; 11:30 GMT; 15 km visibility; midlatitude winter model; 60° solar zenith; 205.8° azimuth angle), (B) boresight shadow mask, (C) boresight shadow estimation (#4 (yellow) 10–20%, #3 (cyan) 20–50%, #2 (magenta) 50–80% and #1 (maroon) 80–100%), (D) ATCOR-4 shadow mask.

Figure 7. Shadow masks as generated from the original AISA-Dual image: (A) RGB natural color composite AISA-Dual image (09 Nov 2008; 09:00 GMT; 25 km visibility; midlatitude winter model; 30.8° solar zenith; 161° azimuth angle), (B) boresight shadow mask, (C) boresight shadow estimation (#4 (yellow) 10–20%, #3 (cyan) 20–50%, #2 (magenta) 50–80% and #1 (maroon) 80–100%), (D) ATCOR-4 shadow mask.

Figure 7. Shadow masks as generated from the original AISA-Dual image: (A) RGB natural color composite AISA-Dual image (09 Nov 2008; 09:00 GMT; 25 km visibility; midlatitude winter model; 30.8° solar zenith; 161° azimuth angle), (B) boresight shadow mask, (C) boresight shadow estimation (#4 (yellow) 10–20%, #3 (cyan) 20–50%, #2 (magenta) 50–80% and #1 (maroon) 80–100%), (D) ATCOR-4 shadow mask.

We conclude that the suggested technique provides a shadow map for the de-shadowing algorithm as in ATCOR-4, while skipping all of the shadow-identification steps. The proposed fully automatic method was successfully tested on AISA-Dual images. The advantage of the presented method is that it does not depend on human interpretation, it is automatic and it is a fast-processing algorithm that relies exclusively on the calculated boresight band ratio.

3.2. Stereo 3-D Map

Stereo-viewing of images is a very useful technique for mapping and is widely used in photogrammetry and remote sensing. Traditional vision systems generally use laser systems or calibrated cameras with different viewing angles to enable triangulation in order to extract 3-D metrical information on the surface. Another branch of vision research aims to recognize complex 2-D objects on the scene. This is usually based on a single image of the scene and a model database. The matching between the scene and database information is generally solved via probabilistic methods [29,30,31,32,33,34,35] or neural networks, among others. Nonetheless, 2-D object-recognition methods are not used to calculate the geometrical relationships (position, orientation) between 3-D surfaces and hence do not provide a stereoscopic view or 3-D information.

Stereoscopy consists of simultaneous vision (with two eyes/sensors) that produces a visual experience of the third dimension, that is, a vivid perception of the relative distances of objects in space. Two solutions can be obtained: along-track stereoscopy and across-track stereoscopy [36]. The simultaneous across-track stereo-data acquisition has a strong advantage in terms of radiometric variations. Traditional stereoscopic photography consists of creating a 3-D illusion starting from a pair of 2-D images. The easiest way to enhance depth perception in the brain is to provide the eyes with two different images, representing two perspectives of the same object, with a minor deviation exactly equal to the perspectives that both eyes naturally receive in binocular vision. Relative sizes, positions and depths of objects can be perceived from the region of overlap between our eyes, while rotation is derived from both overlapping and non-overlapping regions. The configuration of the eyes/cameras provides a large total field of view (FOV) for the combined system while at the same time allowing the scale to be fixed based on triangulated features in the camera’s region of overlap.

Our method is based on the observation of a visible feature in both images (EAGLE and HAWK) and spectral dimension. The boresight band ratio (which sets together spatial and spectral dimensions) constrains the relative pose of the stereo model to a particular feature, as shown in Figure 8. The same feature, appearing in a different 2-D position in the right and left projections can be projected as a 3-D scene. Therefore, the particular feature has a known position relative to the boresight ratio values (which are pose values). Since the EAGLE and HAWK sensors are part of the same system, and are installed on the same bench, the rotation in the tangent plane might not need to be considered. Thus, the three-view (spatial right and left projections and spectral boresight values) feature leaves two degrees of freedom: one degree for the location and one for the relative pose.

Generation of a Digital Elevation Model (DEM) includes the following processing steps: stereo‑model setup, 3-D stereo intersection captured by image matching, and DEM editing [37]. The main objective of the application suggested herein is to generate and evaluate relative pixel-based DEMs from the boresight VNIR and SWIR images.

In Figure 8, any point T(x, y, z) in a 3-D space that is visible in both the left (VNIR) and right (SWIR) images will be projected on the left and right image planes, producing two image-intensity pixels A(x, y) and B(x', y'). If the point T(x, y, z) is elevated with respect to the ground and is visible in both images, then an assumption of stereo analysis is that the 2-D regions around the projection points A and B will produce a high boresight value. Using this principle, the 3-D stereo intersection is performed using a computed geometrical model to convert the pixel coordinates in both images determined in the image matching of the stereo pair to 3-D data with OrthoEngine^SE v7.0 of the PCI Geomatics Group [38]. The non-earth coordinates are determined for the measured point with the least-squares intersection process based on the geometric model equations and parameters. The result is an irregular grid in the map projection system, which is transformed to a raw regular DEM.

Figure 8. Projection of 3-D scene points on the image (in VNIR and SWIR regions) planes.

Figure 8. Projection of 3-D scene points on the image (in VNIR and SWIR regions) planes.

In our application, the stereo-model setup is computed with an iterative least-squares bundle adjustment. The first step in a traditional stereo-analysis system is to extract a disparity map from the stereo-image pair. Since an error of ±3 pixels along-track and ±1 pixel across-track for the parallax measurements in the automated matching process has been achieved with these different datasets (along-track and across-track), the potential accuracy for the across-track stereo-derived local DEM from AISA-Dual can be on the order of ~3 m (1.5 × 1.5 pixels). The next stage is calculation of relative DEM, in which the lowest region (represented by small boresight values and no stereo information) is defined as 0 and other regions are defined relative to 0. Each pixel represents an interpolated ratio between lowest and highest boresight value and a disparity map is extracted by stereo-analysis. Thus, it is important to note that the suggested method does not produce absolute elevation (above sea level) of the surface, but relative scene-by-scene DEM data.

We used the Kriging [28] model to interpolate the DEM onto a 3-D surface (Figure 9) as described further on. In general, the Kriging model is a combination of a polynomial model with departures of the form suggested as a modeling response [39,40].

Figure 9. Digital Elevation Model (DEM). (A) AISA-Dual image (22 October 2006; 08:55 GMT; midlatitude summer model; 46.4° solar zenith; 164.6° azimuth angle), (B) 3-D stereo model, (C) boresight-calculated band, (D) relative DEM (black border = cloud-masked) modeled with the Kriging algorithm (0 m 30 m), (E) AISA-Dual image in 3-D view (x axis 1,500 m, y axis 500 m, z axis (relative to the scene elevation) 30 m).

Figure 9. Digital Elevation Model (DEM). (A) AISA-Dual image (22 October 2006; 08:55 GMT; midlatitude summer model; 46.4° solar zenith; 164.6° azimuth angle), (B) 3-D stereo model, (C) boresight-calculated band, (D) relative DEM (black border = cloud-masked) modeled with the Kriging algorithm (0 m 30 m), (E) AISA-Dual image in 3-D view (x axis 1,500 m, y axis 500 m, z axis (relative to the scene elevation) 30 m).

Comparing the retrieved DEM (Figure 9(D)) with the topographic map (Figure 10) shows a high similarity in the height differences between the two datasets. Since the presented scene is occupied by Negev dunes, which are geomorphologically highly dynamic, and since the referenced topographic map is from 1979 (with no updates available), we could not complete a quantitative validation of the retrieved DEM.

Figure 10. Topographic map (from 1979) presented on aerial digital photo (national orthophoto 2002) with a resolution of 2 m.

Figure 10. Topographic map (from 1979) presented on aerial digital photo (national orthophoto 2002) with a resolution of 2 m.

Our method uses a novel combination of image correspondences to determine the relative pose of a dual airborne system. The method described here uses feature correspondences in 2-D to boresight band ratio values, which determine the relative pose.

3.3. Unmixing and Anomaly Detection

Most of the pixels collected by HRS airborne sensors contain mixed spectra from the reflected surface radiation of various materials in the sub-pixels. As a result, mixed pixels may exist when the spatial resolution of the sensor is not sufficient to separate the different pure-signature classes. The resulting spectral measurement is a composite of the individual pure spectra weighted by a set of scalar endmember-abundance fractions [41]. Under such conditions, target detection may be carried out at the sub-pixel level. An anomaly indicator enables detecting targets whose signatures are spectrally distinct from their surroundings with no prior knowledge. In general, such anomalous targets are of low dispersion and relatively small size compared to the surrounding pixels. Two approaches are of particular interest: (1) RXD (Reed-Xiaoli Detector), which refers to the RX indicator after Reed and Yu [41,42], and detects spatial/spectral anomalies using the sample covariance matrix. It detects “interesting target” pixels which occur with low probability in the data relative to the Gaussian distribution of the background [41]. The Gaussian Mixture RX algorithm is fitted to the background model by employing a mixture of Gaussians to estimate the multimodal nature of the background [43]. However, the number of required components is not known, so in practice the background model is inaccurately estimated, resulting in poor detection performance (Figure 11(B)). (2) The Support Vector approach, SVDD, is a non-parametric method with several advantages, including a non-Gaussian modeling basis that can model arbitrarily shaped and multimodal distributions, scarcity and high generalization ability [44]. The SVDD approach does not require a covariance matrix inverse computation and is linear with respect to spectral dimensionality [44]. A processed SVDD with kernel statistics is expressed in Equation (7).

$$SVDD \left(y\right)=R^2-K\left(y,y\right)+2{\sum }_i{\alpha }_{i}K\left(y,x_i\right)$$

(7)

where R is the radius of the minimum enclosing hypersphere after optimizing with respect to ${\alpha }_i$ and K is a kernel function:

$$K\left(x,y\right)=<\text{Φ}\left(\text{x}\right),\text{Φ}\left(\text{y}\right)>$$

(8)

As suggested in Banerjee and Burlina [45], we used a Gaussian radial basis function for the kernel function. Thus, the SVDD function is expressed as:

$$SVDD \left(y\right)=C+{\sum }_i{\alpha }_{i}K\left(y,x_i\right)$$

(9)

where C is a constant offset since $K\left(y,y\right)$ = 1. The Gaussian radial basis function includes ${\sigma }^2$ which controls how well the SVDD generalizes the support boundaries of the unseen data. As a result, SVDD is well suited for global background characterization, achieving high detection performance and low false-alarm rates (Figure 11(C)).

We suggest using the boresight band ratio as a spectral/spatial anomaly indicator. Since any anomaly is defined as a small target with a distinct spectrum, an accurate pixel-by-pixel classification of boresight values (band ratios) may accentuate the targets in question (Figure 11(D)).

The spectral investigation of the detected anomalies (RX, SVDD, boresight) is performed by linear unmixing analysis [46], which compares the ground-based reflectance spectra (targets and background) with image spectra. We used an image subset (286 × 192 pixels) taken from a Mediterranean forest by the AISA-Dual airborne image spectrometer (flight and sensor configuration are detailed in the introduction). This scene is part of a designated study region located in a glade containing 24 gray plastic (artificial) bags (35 × 55 cm).

In the linear mixture analysis (Equation (10)), the composite reflectance spectrum $\rho \left(\lambda \right)$ can be written as a linear sum of N endmember spectra $a_n\left(\lambda \right).$

$$\rho \left(\lambda \right)= {\sum }_{n=1}^N{f}_n{a}_n\left(\lambda \right)+\varphi$$

(10)

which is constrained by:

$${\sum }_{n=1}^N{f}_n=1.0 and f_n\gtrsim 0.0$$

(11)

In Equations (10) and (11), $f_n$ represents the fractional abundance for the n^th endmember, and $\varphi$ provides a measure of the error in the unmixing. For this study, the mixed-pixel spectrum is simulated by summing together resampled (for AISA-Dual spectral configuration) ground truth spectra of the soil and plastic bags. Figure 12 shows ROC curves of the probability of detection and false alarm for linear unmixing analysis of global RX, SVDD and boresight band ratio detectors.

Figure 11. Anomaly detection on a bare-soil field: (A) R(630 nm) G(550 nm) B(430 nm) AISA-Dual image (31 October 2009; 10:10 GMT; midlatitude summer model; 30.2° solar zenith, 141.2° azimuth angle, 1.5 m spatial resolution, 198 spectral bands), (B) RXD product, (C) SVDD product, (D) boresight band ratio product

Figure 11. Anomaly detection on a bare-soil field: (A) R(630 nm) G(550 nm) B(430 nm) AISA-Dual image (31 October 2009; 10:10 GMT; midlatitude summer model; 30.2° solar zenith, 141.2° azimuth angle, 1.5 m spatial resolution, 198 spectral bands), (B) RXD product, (C) SVDD product, (D) boresight band ratio product

Figure 12. ROC performance for the global RX, SVDD and boresight band detectors.

Figure 12. ROC performance for the global RX, SVDD and boresight band detectors.

4. Discussion

The purported disadvantage of the boresight effect was transformed into a useful tool for: (1) enhancing the shadow effect, (2) generating a 3-D view, and (3) better detection of border anomalies. We show that the boresight band ratio provides an internal shadow map for AISA-Dual images. In some cases, where dark targets other than shadow areas are included within bright surroundings, the statistical distribution of the boresight (band ratio) values will show two main groups (bright and dark) with no outliers, and thus, will not emphasize the shadowed pixels. Commonly used methods might overestimate the spatial distribution of shadow areas, as in most cases they spectrally identify dark targets as shadows. Dark targets may be corrected by the de-shadowing algorithm, resulting in erroneous spectra. Therefore, it is preferable to underestimate the shadow areas.

In addition, we show that the boresight band ratio provides an internal relative DEM. This application is useful for atmospheric correction and natural 3-D surface presentation without the need for external elevation data and geo-referencing. When GPS/INS data are unavailable, geometric correction by other means might lead to spatial distortions and as a result, spectral disruptions. Thus, using the boresight band ratio as an internal relative DEM is preferable.

Finally, the boresight band ratio constitutes a simple novel method to indicate spectral/spatial anomalies. It provides reliable detection of anomalies and does not require further external algorithms.

5. Conclusions

The boresight effect, considered a defect in the system, is automatically corrected prior to data analysis. Here we show that this effect, rather than being a defect, can actually deliver useful information. Accordingly, three applications were investigated and demonstrated: (1) enhancing the shadow effect; (2) generating a 3-D view; and (3) better detection of sub-pixel anomaly targets. The first application provides an external shadow map for any de-shadowing algorithm. It enables skipping the steps of shadow identification, which are occasionally misidentified with high-contrast dark targets. This application was found to perform better than a routine application suggested by ATCOR-4. The second application deals with the extraction of 3-D information. This application was tested against known topographic information and found to yield promising results. The third application detects spectral/spatial anomalies. Comparison with two well-known anomaly indicators (RXD and SVDD) showed favorable results. We suggest that the boresight effect can play an important role in analyzing HRS data. Thus, we propose the above applications as useful tools to be taken advantage of prior to this effect's automatic correction.