Artificial Intelligence and Novel Sensing Technologies for Assessing Downy Mildew in Grapevine

Abstract

Plant diseases and pests cause a large loss of world agricultural production. Downy mildew is a major disease in grapevine. Conventional techniques for plant diseases evaluations are time-consuming and require expert personnel. This work investigates novel sensing technologies and artificial intelligence applications for assessing downy mildew in grapevine under laboratory conditions. In our methodology, machine vision is applied to assess downy mildew sporulation, while hyperspectral imaging is used to explore its potential capability towards early detection of this disease. Image analysis applied to RGB leaf disc images is used to estimate downy mildew (Plamopara viticola) severity in grapevine (Vitis vinifera L. cv Tempranillo). A determination coefficient (R²) of 0.76 ** and a root mean square error (RMSE) of 20.53% are observed in the correlation between downy mildew severity by computer vision and expert’s visual assessment. Furthermore, an accuracy of 81% is achieved to detect downy mildew early using hyperspectral images. These results indicate that non-invasive sensing technologies and computer vision can be applied for assessing and quantify sporulation of downy mildew in grapevine leaves. The severity of this key disease is evaluated in grapevine under laboratory conditions. In conclusion, computer vision, hyperspectral imaging and machine learning could be applied for important disease detection in grapevine.

1. Introduction

Plant diseases, pests and weeds cause large losses of food production in agriculture. Traditionally, plant diseases are identified by visual observations by the growers in the field of biological techniques in the laboratory [1]. Nevertheless, these techniques are time-consuming, susceptible to human error and/or require qualified personnel [1,2]. Non-invasive sensing technologies as RGB (Red, Green and Blue) images, thermography, multispectral and hyperspectral imaging have been postulated as potential non-invasive tools for detecting plant diseases in agriculture with several advantages versus conventional methods [1,2,3], even for detecting diseases when the symptoms are not visible [4]. Computer vision, machine learning and artificial intelligence technologies can be applied to identify, classify and quantify crop diseases in data-driven agriculture [5,6,7,8].

In the European project NoPest (Novel Pesticides for a Sustainable Agriculture), non-destructive proximal sensing technologies and artificial intelligence are being developed for assessing fungal diseases in key crops as grapevine and potato. Downy mildew is a key grapevine disease in world viticulture. Nowadays, the evaluation of this disease has been based mostly on visual assessment of leaves in the vineyards or histological analyses at the laboratory [9]. Computer vision and artificial intelligence could be very useful to recognise and quantify some diseases in grapevine [10,11,12].

This work examines non-invasive imaging technologies and artificial intelligence applications for assessing downy mildew in grapevine under laboratory conditions. Computer vision was applied to evaluate downy mildew severity, while hyperspectral imaging was used to detect this disease early in grapevine.

2. Materials and Methods

This approach could be separated into three main sections (Figure 1): (i) RGB and hyperspectral image acquisition under laboratory conditions; (ii) image processing using computer vision techniques and hyperspectral preprocessing to improve image features and assess downy mildew sporulation; and (iii) machine learning modelling, used to detect the disease early using hyperspectral imaging.

2.1. Image Acquisition under Laboratory Conditions

Under laboratory conditions, leaf discs from grapevine (Vitis vinifera L., cv. Tempranillo) plants were placed in Petri dishes with the abaxial side up. Two groups of the study were defined: One group was infected with the downy mildew agent (Plasmopara viticola), and the other group was used as control. Images were taken in the laboratory every day until nine days after inoculation using a digital RGB camera (Canon EOS 5D, Japan). Additionally, a push broom hyperspectral visible (VIS) camera (spectral range from 400 to 1000 nm) was used to detect downy mildew early under lab conditions. For validation, the percentage of leaf area showing downy mildew sporulation in the leaf discs was visually evaluated by a panel of eight experts, providing a disease severity value for each disc. The average evaluation of each disc was used as a reference.

2.2. Processing of RGB and Hyperspectral Images

The RGB images were processed to detect and quantify downy mildew sporulation in grapevine leaves (Figure 2). The colour space was transformed to HLS (Hue, Lightness, Saturation) to bring the perception of the colour reflected in the digital image closer to the human eye by obtaining colour, brightness and saturation values from red, green and blue values of the images. Saturation values seemed to show different values in areas of the leaves with sporulation and the rest of the leaf, unlike brightness and colour values, thus these values were used to locate sporulation in the leaf. Classical preprocessing techniques were applied to enhance image features that help to analyse downy mildew sporulation, such as median filter, used to smooth the image, and Contrast Limited Adaptive Histogram Equalisation (CLAHE) [13], used to improve the contrast. Hough Transform [14] was used to localise the leaf discs in the image, differentiating them from the rest of the image. Each leaf disc image was segmented with the Otsu method [15], separating the pixels representing downy mildew sporulation and the rest of the leaf. The segmentation of leaf disc images was used to estimate downy mildew severity as the percentage of downy mildew sporulation that appears in the grapevine leaf discs (Figure 3).

On the other hand, hyperspectral images were processed. The values (I) were transformed to reflectance (R) to correct the images using a dark current (DC) and a white reference (WR) values using the following equation:

$$R=\frac{I-DC}{WR-DC}$$

(1)

Savitzky–Golay filter [16] was also applied to smooth the spectra of each image (Figure 4), with a grade 2 polynomial and a size 15 window. Then, a standardisation was applied.

Leaf disc location on hyperspectral images was obtained by applying the watershed segmentation algorithm [17], once the leaf spectra were separated from the background, as indicated in Figure 1. This spectra separation is detailed in the next section.

2.3. Machine Learning Modelling

For hyperspectral images, a two-stage machine learning analysis was designed for (i) the detection of spectra belonging to leaves (segmentation) and (ii) modelling and prediction using the leaf spectra as input. The segmentation of the leaves was carried out by manually selecting spectra belonging to the positive class (leaf spectra) and negative class (spectra from the background elements). From these data, a binary classifier was trained using Support Vector Machines (SVMs) and applied for the automated segmentation of all the images. The modelling for the pathogen detection was done after leaf spectra extraction in the previous step (by binary classifying spectra according to the disc they belong to, between spectra of infected and non-infected discs) and the training using different models trained with Convolutional Neural Networks (CNNs), k-Nearest Neighbour (KNN), MultiLayer Perceptrons (MLPs) and Partial Least Square-Discriminant Analysis (PLS-DA). The CNN architecture was composed of two convolutional blocks, with a one-dimensional convolutional layer and a max-pooling layer on each one, to obtain features from spectra; and three fully connected blocks, to classify the obtained features in the convolutional blocks.

2.4. Implementation

All the experiments were developed with the Python 3.7.4 programming language. The RGB and hyperspectral images were processed using the OpenCV 4.2.0.32 and scikit-learn 0.22.2 libraries on an Intel Core i7 4770 CPU (16 GB RAM). On the other hand, RGB images taken under field conditions were processed with an NVIDIA GeForce RTX 2080 Ti GPU (11 GB memory), optimising the execution of the CNN developed with the Keras 2.3.1 framework and the Tensorflow 2.1.0 backend.

3. Results and Discussion

Results of two approaches used for assessing downy mildew in grapevine are summarised in

Table 1. Methods and results of severity estimation and early detection of downy mildew in grapevine. Results of the early detection method were selected from the 9th day post inoculation (DPI).

Aim	Image Type	Techniques	Method Information
			Training Time	Testing Time	Results		Detection Model
Severity estimation	RGB	Computer vision	None	Middle-Low	R2	RMSE	-
					0.76 **	20.53%	-
Early detection	Hyperspectral	Hyperspectral preprocessing, computer vision and machine learning	Middle	Low	Accuracy (%)	F1-score	Model
			High	Low	82	0.81	CNN
			Low	Low	66	0.62	KNN
			Middle	Low	81	0.80	MLP
			Low	Low	53	0.35	PLS-DA

. Computer vision was applied to assess downy mildew severity, while hyperspectral imaging was employed for early detection. A strong and significant relationship (determination coefficient of 0.76** and a root mean square error of 20.53%) was observed between downy mildew severity measured using computer vision and visual assessment by the experts in grapevine leaf discs. As can be seen in Figure 2 and Figure 3, this new method provided a sporulation location for each leaf disc, which adds interpretability to the results, giving information on the estimated severity of downy mildew.

Disease severity estimation of downy mildew using computer vision techniques obtained similar results to expert evaluation. This severity estimation could consider the expert subjectivity, achieving a greater relationship between automatic and manual assessment [11]. The possibility to adopt new sensing technologies for detecting grapevine downy mildew and for evaluating disease severity gives new opportunities for disease assessment in other key commercial crops in agriculture.

Regarding early detection of downy mildew using hyperspectral imaging, classification accuracy between control and downy mildew inoculated discs after several days post inoculation (DPI) using different machine learning models is showed in Figure 5 and Table 1. Artificial neural networks differed from the other models, with an accuracy close to 81% and an f1-score close to 0.80 (Table 1). This implies that the spectral features that differentiate the spectra of infected discs from those from uninfected discs were difficult to find, and the convolution blocks do not provide a better performance of the model, since a shallower neural network, such as an MLP, was capable of achieving results similar to a CNN.

Early detection of downy mildew using machine learning techniques achieved high accuracies in hyperspectral images. This method could be combined with that used to estimate the severity of the disease to obtain a localisation and quantification of the sporulation or by applying machine learning for image segmentation [7,18]. These results indicate that computer vision can be applied for assessing and quantify sporulation of downy mildew in grapevine leaves. Moreover, hyperspectral imaging could be applied to detect this disease early in grapevine.

Both approaches used for assessing downy mildew in grapevine were developed in cv. Tempranillo, however, can be applied to other grapevine varieties with similar results [19]. Better results would be achieved by re-training the models with more grapevine varieties, allowing greater generalisation and achieving a model that is better adapted to different grapevine varieties.

This work tested using two techniques, such as computer vision and hyperspectral imaging, to assess downy mildew in grapevine. The method employed for downy mildew severity estimation, using computer vision techniques, required no training and no modelling, so its computational cost was low. The higher spectral dimension of hyperspectral images allows for the application of more powerful machine learning techniques for better classification. The method for classifying hyperspectral images, despite requiring the training of machine learning models, most of them could be executed with a CPU, training each one of them in less than one hour. On the other hand, for the classification of hyperspectral images using a CNN, a GPU was used to train the CNN, which took several days, due to the computational complexity of this training. Still, but once the model is trained, its prediction can be very fast, classifying more than 100 images in less than a minute. Moreover, RGB cameras, in general, are much cheaper, with higher availability and easiness of use than hyperspectral imaging.

Machine learning and computer vision algorithms can be very useful for assessing downy mildew in grapevine. Our results indicate that downy mildew can be evaluated in grapevine using novel technologies and artificial intelligence techniques in data-driven agriculture.

4. Conclusions

The results exposed in this work indicate that downy mildew in grapevine can be automatically evaluated using emerging technologies. New sensing technologies and machine learning have shown promising results for assessing downy mildew in Tempranillo, but it could be applied to other grapevine varieties. Computer vision, hyperspectral imaging and artificial intelligence can be applied for monitoring downy mildew in grapevine. Machine vision was applied to assess downy mildew severity, and hyperspectral imaging was used to detect this disease early.

Following the example of downy mildew in grapevine, it can be suggested that computer vision and hyperspectral imaging could be used for assessing major diseases in other key crops in precision agriculture, with several advantages versus conventional methods. Artificial intelligence can also be applied in plant pathology for early disease detection and quantify the symptoms.