Multi-Classifier Pipeline for Olive Groves Detection

Abstract

Pixel-based classification is a complex but well-known process widely used for satellite imagery classification. This paper presents a supervised multi-classifier pipeline that combined multiple Earth Observation (EO) data and different classification approaches to improve specific land cover type identification. The multi-classifier pipeline was tested and applied within the SCO-Live project that aims to use olive tree phenological evolution as a bio-indicator to monitor climate change. To detect and monitor olive trees, we classify satellite images to precisely locate the various olive groves. For that first step we designed a multi-classifier pipeline by the concatenation of a first classifier which uses a temporal Random-Forest model, providing an overall classification, and a second classifier which uses the result from the first classification. IOTA2 process was used in the first classifier, and we compared Multi-layer Perceptron (MLP) and One-class Support Vector Machine (OCSVM) for the second. The multi-classifier pipelines managed to reduce the false positive (FP) rate by approximately 40% using the combination RF/MLP while the RF/OCSVM combination lowered the FP rate by around 13%. Both approaches slightly raised the true positive rate reaching 83.5% and 87.1% for RF/MLP and RF/OCSVM, respectively. The overall results indicated that the combination of two classifiers pipeline improves the performance on detecting the olive groves compared to pipeline using only one classifier.

1. Introduction

Image classification, in remote sensing, refers to the assignation of thematic classes to image pixels [1] and classification approaches can be mainly divided into unsupervised classification and supervised classification [2]. Unsupervised classification has some benefits such as a minimum user involvement and less time consuming as there is no training process necessary [3]. However, unsupervised classification is usually less accurate compared to the supervised classifiers, especially when the radiometric distance between two classes is minor. On the other hand, while it has the drawback of having to prepare the training data, supervised classification is perceived as more suitable for accurate and complex classification tasks [4] and to identify spatial objects. Some examples of commonly used supervised pixel-based classification include Machine Learning algorithms such as Random Forest (RF), Support Vector Machine (SVM), and Artificial Neural Network (ANN) [5,6].

The algorithms mentioned above propose generally a binary or a multi-class classification approach, meaning that two or more classes are needed to train the model. In some cases, when the ground truth is limited or when users want to target only one land cover type, a one-class classification model could improve the final classification map [7]. For one-class classification, a training data set for the desired land cover type is needed and the model learns only from that positive data set and optimizes the spectral distances between the positive data and other objects [8]. Some examples of one-class classifiers include Support Vector Data Description (SVDD) and One-class Support Vector Machine (OCSVM) [7]. In order to classify satellite imagery, a method based on several classification models which is called Multiple-Classifier System (MCS) is perceived to be able to improve the accuracy of classification performance [9]. According to Brownlee [10], not only does MCS perform better than a single classifier, but it is also more robust especially for the overfitting issue. Du et al. [9] divided MCS into two main categories: concatenation system, and parallel system and some applications of the MCS concept are presented by Bühlmann [11] and Benediktsson [12]. The concatenation system uses the output of the first classifier as an input for the next, while for the parallel system, the results of several independent classifiers are combined based on different possible strategies to produce the final one [9].

In this study, we tested and applied our classification pipeline within SCO-Live, a CNES funded project. SCO-Live is a collaborative project between ACRI-ST, ARGANS and CAPG (Community of agglomeration of the country of Grasse) [13], which aims to use olive trees as a bio-indicator to monitor climate change. If some previous study monitored olive trees health and water status, none present an accurate approach to solely identify olive groves and to correlate their phenological change with climate change. SCO-Live project is the first to propose a complete processing chain and form identification to ecological analysis in the Southeast of France. To follow and analyze olive trees through time, we first need to identify olive groves’ location for our area of interest. A field campaign to localize all olive groves is too expensive and time consuming without the insurance to locate them all. Satellite imagery allow us to visualize a large area (100 km × 100 km for Sentinel-2) and olive groves can be identify and extracted with a classification process. A temporal Random Forest algorithm is used within the IOTA2 processing chain (hereinafter referred to as the baseline pipeline).

Our objective is to improve the performance of olive groves detection from the baseline pipeline using the supervised concatenated multi-classifier pipeline. Furthermore, we also compared the performances between a binary (2-class) and a unary (1-class) approach.

2. Materials and Methods

2.1. Study Area

The area of interest is located in Southeast France (Figure 1) within the Alpes-Maritime over 272.1 km². The experiment is centred in Grasse and its surroundings including Saint-Cezaire-sur-Siagne, Cabris, and Saint-Vallier-de-Thiey. Olive trees can be found around these areas either in the wild, planted in the small-scale private garden, or as a grove in olive farms.

2.2. Satellite Images

Our process is using Sentinel-2 Level 2A data downloaded from the Copernicus open access hub. Multiple images for the tile identified by the reference T32TLP were used to properly identify the olive grove’s reflectance spectrum evolution according to the phenological stages throughout the year. In the Mediterranean area, olive phenological steps are described by Sanz–Cortés [14] and Torres [15] and EO data were selected according to the main stage of development as described in

Table 1. Earth Observation data selection and their corresponding phenological stage.

Phenological Steps	File Name	Acquisition Date	Cloud Cover
Dormancy period after harvesting	S2B_MSIL2A_20190125T103319_N0211_R108_T32TLP_20190125T134253	25 January 2019	6.74%
Flowering period	S2A_MSIL2A_20190430T103031_N0211_R108_T32TLP_20190430T140106	30 April 2019	5.86%
Early-stage of fruit growth	S2B_MSIL2A_20190724T103029_N0213_R108_T32TLP_20190724T135539	24 July 2019	3.67%
Late-stage of fruit growth	S2B_MSIL2A_20190912T103019_N0213_R108_T32TLP_20190912T141218	12 September 2019	0.18%
Harvest season	S2A_MSIL2A_20191106T103231_N0213_R108_T32TLP_20191106T112019	6 November 2019	11.87%

.

The RPG (Registre Parcellaire Graphique) dataset and the SCO-Live project citizen field observations were used to create a reference and to evaluate the resulting classification map. The RPG data is provided by the French National Institute of Geographic and Forest Information (IGN). It is a vector file containing a detailed description of the various vegetation and crop types in France. The document is freely available on the IGN web site and updated every year. Olive groves are identified within the RPG 2019 shapefile by the code “OLI”. The SCO-Live project relies on citizen science to collect field information. Citizen science is a process that involves individuals to perform some experiments requiring a low scientific level [16]. Another aspect of citizen science is to involve individuals in field data collection by providing all the tools required for intuitive data collection. Within the SCO-Live project, an application has been developed and distributed to allow citizens to contribute by identifying and locating olive trees and groves. The resulting data base gathers all observation made and stores the information in a georeferenced point vector file which contains for each location various information on olive trees exploitation such as crops type, stage of growth, possible damage, etc. All olive observations are located within a global 5 m spatial accuracy linked with the individual phone GPS accuracy.

All data set are georeferenced and projected to the coordinate reference system of EPSG:32632 (WGS84/UTM zone 32N).

2.3. Satellite Imagery Classification

Table 2. The list of pipelines used in this study.

Pipeline	Classifier(s)	Description
Baseline pipeline	Random Forest	The pipeline that used one classifier
Multi-classifier pipeline 1	Random Forest + Multi-layer Perceptron	Pipeline using a multi-classifier system random forest and a binary classification approach in its second classifier
Multi-classifier pipeline 2	Random Forest + One-class Support Vector Machine	Pipeline using a multi-classifier system, random forest and a unary classification approach in its second classifier

describes the different pipelines that are used and compared in this study.

2.3.1. Baseline Pipeline

The baseline pipeline using Random-Forest (RF) [17] is implemented within the IOTA2 framework. IOTA2 (infrastructure for land use by automatic processing using Orfeo toolbox application) is an open-source land cover mapping framework developed by the French Center for Spatial Studies of Biosphere (CESBIO) [18]. The framework implements a classification pipeline using several libraries such as Orfeo Toolbox and Scikit-Learn to perform a temporal supervised pixel-based classification. IOTA2 process, as described in Figure 2, produces a land cover map where each pixel is assigned to a class by processing the surface reflectance taken at different times (multi-temporal images). Only the valid pixels (selected based on the various Sentinel-2 masks and the SCL—Scene CLassification—band) are processed and the different features listed in the configuration file are extracted. As part of the default features extracted, various spectral indices, such as NDVI (Normalized Difference Vegetation Index) and NDWI (Normalized Difference Water Index), are calculated to support land cover type identification.

The baseline pipeline classifies the input images into Urban, Forest, Nature, Grassland, and Olive classes. The reference data for the baseline pipeline was created by referencing the RPG dataset for Olive class and by photointerpretation and spectral indices analysis for other classes.

2.3.2. Multi-Classifier Pipeline

The multi-classifier pipeline presented in Figure 3 was built by combining the baseline pipeline with either a binary classification or a unary (one-class) classification. The first classification overclassifies all vegetation types to ensure that all possible vegetation types are captured in one vegetation class, and those pixels are re-classified by the second classifier to separate the vegetation into different classes. The final classification map is obtained by combining the result from the first and second classification processes.

In this study, two possibilities for the second classifier have been tested and are listed as follows: the Multi-layer Perceptron (MLP) [19] and the One-class Support Vector Machine (OCSVM) [20]. The MLP performs a binary classification using the architecture shown in Figure 4, and it consists of one input layer, two hidden layers and one output layer. The input layer has a number of nodes equal to the number of features corresponding to the five dates, ten bands and six spectral indices that were used. Hence, it made an input layer have 80 nodes. For the two hidden layers, the activation function used is Rectified Linear Unit (ReLU). After the activation function, a Batch Normalization layer (BatchNorm) is added to allow faster model training and allow the use of a larger learning rate. Before the output layer, a Softmax function is set to turn the output values into probabilities such that all nodes in the output class will amount to 1. The output layer consists of nodes equal to the number of classes, which is 2 because the MLP will perform binary classification between Olive and Non-olive classes. For the training phase, the Cross Entropy loss function was used, and the weights and biases of the model were adjusted by the Adam optimizer. OCSVM on the other hand, conducts a unary classification that focuses only on the identification of one class. The kernel used for the OCSVM model was the Radial Basis Function (RBF) kernel. The ν parameter, which is the noise tolerance when the model learns to set the boundary for the training data, was set to 0.5, and the γ parameter, which is the width of the Gaussian curve in the RBF kernel, was set to 0.1.

The multi-classifier pipeline firstly classifies the pixels into Forest, Impervious, and Vegetation classes and then further classifies Vegetation pixels into Olive, and Non-olive classes. Impervious class includes the urban areas, bare land and other non-vegetation objects. Forest class comprises forest areas and dense dark vegetation areas. Vegetation class contains olive groves, trees and other vegetation that is not categorized as Forest. The reference data for the multi-classifier pipeline was also created by interpreting the satellite imagery, analyzing spectral indices, and referencing RPG dataset.

2.3.3. Evaluation Method

To evaluate the performance of the multi-classifier pipeline and the accuracy of the resulting maps, various methods are applied. First, we visually compared the three land cover maps produced by each pipeline and to highlight the differences between them, we performed raster subtraction focusing on Olive class. To realize this, the classes that are not olive were regrouped into one Non-olive class and we compared the olive class spatial distribution for the baseline classification and both MLP pipeline and OCSVM pipeline classification maps. The change in area was quantified from the subtraction raster to obtain the surface loss of each class.

To evaluate the performance of the new multi-classifier pipeline, we calculated the True Positive Rate (TPR) and the False Positive Rate (FPR) to check if there is an improvement in the olive trees detection with the multi-classifier pipeline compared to the baseline pipeline. Moreover, the comparison of MLP and OCSVM is also observed. TPR was calculated based on the SCO-Live project citizen field observations using the following equations:

$$TruePositiveRate=\frac{Correctlypredictedpoints}{Totalpoints}\times 100\%$$

(1)

A point is considered correct if there is at least one pixel of predicted Olive class in the point’s location and its eight neighboring pixels as the spatial accuracy of GPS is considered.

On the other hand, the calculation of FPR was based on the non-olive vegetation polygons vector of the RPG dataset that are located inside the area of interest. We defined the incorrect pixels as the Olive class pixels inside those polygons, and computed FPR using following equations:

$$FalsePositiveRate=\frac{Incorrectlypredictedpixels}{Totalpixels}\times 100\%$$

(2)

3. Results

3.1. Classification Pipeline Resulting Maps

From the three experimented pipelines, we have obtained various classification maps that are presented in Figure 5.

The baseline pipeline classified the input images into 5 classes: Urban, Forest, Nature, Grassland and Olive, while the two other ones assigned the pixels into Forest, Impervious, Olive and Non-olive classes. We observed significant visual changes between the map resulting from the one classification process and the ones resulting from the multi-classifier pipeline. We identified a significant decrease in the coverage of the olive class (represented in dark green on all land cover maps) in the multi-classifier pipeline’s map. This can be confirmed particularly in the West, Northwest, and North of the area of interest.

The differences in olive detection can be observed on the subtraction maps presented in Figure 6.

We can confirm that the area of Olive pixels changing to Non-olive pixels, which are represented in orange color, is emphasized in the area where we observed the significant decrease of olive coverage. Furthermore, we also observed that there are more pixels that turned into Non-olive class than into Olive class as orange coverage is broader than the green one. The change of these areas was calculated and results are presented in

Table 3. The total area in km² based on each change category.

Raster Subtraction	Change Category	Area [km2]
Baseline–MLP	Non-olive to Olive	16.8
	Olive to Non-olive	62.5
	No change	192.9
Baseline–OCSVM	Non-olive to Olive	16.3
	Olive to Non-olive	42.9
	No change	212.9

. The quantification of area in orange reached approximately 62.5 km² and 42.9 km² for the baseline-MLP and the baseline-OCSVM respectively. On the other hand, the area that changed from Non-olive to Olive in both subtractions is about 16 km².

3.2. Performance Evaluation

The evaluation of performance intends to check if there is an improvement in the olive trees detection by the multi-classifier pipeline compared to the baseline pipeline.

Table 4. True positive rate of the three pipelines.

Pipeline	True Positive Rate
IOTA2	81.7%
IOTA2 + MLP	83.5%
IOTA2 + OCSVM	87.1%

shows the TPR of the baseline pipeline and the multi-classifier pipelines. The multi-classifier pipeline using MLP raised the TPR to 83.5% while the pipeline using OCSVM reached 87.1%. The improvement is also observed in FPR, which is described in

Table 5. False positive rate of the baseline pipeline and the multi-classifier pipelines.

Pipeline	False Positive Rate
IOTA2	42.5%
IOTA2 + MLP	1.4%
IOTA2 + OCSVM	28.9%

. The multi-classifier pipeline decreased the FPR by approximately 40% and 13% using MLP and OCSVM respectively, compared to the FPR calculated in the baseline pipeline.

4. Discussion

The results have demonstrated that the significant difference between the baseline pipeline and the multi-classifier pipeline was highlighted in the change of pixels from Olive class to Non-olive class. As we simultaneously observed a decrease of FPR with both MLP and OCSVM, meaning that we reduced the number of wrongly classified pixels in Olive class, we can assume that most of pixels firstly classified as olive were misclassified. For confirmation, we conducted a field survey for some areas where the classification maps showed different predictions between baseline and multi-classifier pipeline. Most of our field observations validated that the pixels firstly classified as olive by the baseline pipeline but not by multi-classifier pipeline were indeed not olive trees in the field. In some areas, we could find green oak and white oak trees in the misclassified regions of the baseline pipeline. We hypothesize that these oak trees may possibly contribute to the confusion of the classification by the model in the baseline pipeline.

In addition to the decrease in misclassification of olive trees, we could also observe from the TPR calculation of the SCO-Live project field observation data that the correct detection of olive trees rose slightly despite the reduction of Olive class coverage in the multi-classifier pipeline. This suggests that the multi-classifier pipeline improves the overall performance of olive detection, which aligned with the theory of using more than one classifier in a system to target a better accuracy at the price of complexity [21]. Furthermore, the improvement of detection can also be linked to the division of classification tasks in the multi-classifier pipeline. By assigning the first classifier to predict all vegetations first and letting the second classifier to further classify the vegetations into olive trees or not, the pipeline performs hierarchical classification [22], where a specified land type is identified after a more a general classification.

With SVM as a one-class approach and MLP as a two-class approach, the results proved that both have raised the performance of olive detection. In our use-case, because we need to minimize the noise around of the classified olive trees pixels, we perceived that the two-class MLP is more suitable for our project as it lowered the FPR significantly while maintaining a good TPR. However, we found that using one-class SVM in our data can achieve even better results in detecting the olive trees (positive data). Therefore, the use of one-class approach can be an interesting and promising solution for other use cases where we only have data for one category of land cover.

5. Conclusions

Pixel-based classification is widely used for satellite imagery analysis. In this paper, we presented the use of multi-classifiers to compare its results and performances with the scenario using only one classifier. Within the frame of the SCO-Live project that aims to identify olive groves to monitor climate change impact, we evaluated our results based on the SCO-Live database and field data. The results indicated that multi-classifier pipelines (MLP and OCSVM) showed better performances in both detecting olive trees and lowering the misclassification compared to single classifier pipeline. In addition, we observed that using both one-class and two-class approaches as the second classifier in multi-classifier pipeline improves the quality of detection. However, from our field campaign, we found that a confusion between oak trees and olive trees often occurs during the classification process. A further and more complete spectral analysis of both species might highlight some spectral differences at specific season allowing a better discrimination of both types. Furthermore, with the newest version of the SCO-Live database, we can access more detailed information on the various farming conditions (irrigation, health) that affect olive trees’ spectral response. This new information needs to be considered to further improve the resulting classification. Moreover, quantifying the computational cost and comparing other pairs of classifiers in the pipeline will be an interesting experiment to give more insight on the multi-classifier pipeline.