Comparative Evaluation of Color Correction as Image Preprocessing for Olive Identification under Natural Light Using Cell Phones

Abstract

Integrating deep learning for crop monitoring presents opportunities and challenges, particularly in object detection under varying environmental conditions. This study investigates the efficacy of image preprocessing methods for olive identification using mobile cameras under natural light. The research is grounded in the broader context of enhancing object detection accuracy in variable lighting, which is crucial for practical applications in precision agriculture. The study primarily employs the YOLOv7 object detection model and compares various color correction techniques, including histogram equalization (HE), adaptive histogram equalization (AHE), and color correction using the ColorChecker. Additionally, the research examines the role of data augmentation methods, such as image and bounding box rotation, in conjunction with these preprocessing techniques. The findings reveal that while all preprocessing methods improve detection performance compared to non-processed images, AHE is particularly effective in dealing with natural lighting variability. The study also demonstrates that image rotation augmentation consistently enhances model accuracy across different preprocessing methods. These results contribute significantly to agricultural technology, highlighting the importance of tailored image preprocessing in object detection models. The conclusions drawn from this research offer valuable insights for optimizing deep learning applications in agriculture, particularly in scenarios with inconsistent environmental conditions.

1. Introduction

Olive oil production is a significant economic activity in many countries, with a global market value estimated at over €14 billion in 2022 [1]. To ensure high-quality olive oil production, it is essential to identify and classify olives accurately according to their ripeness. Establishing the “right time” to harvest is of the utmost significance [2].

In agriculture, accurate detection and evaluation of olive ripeness is crucial for determining the quality and yield of extracted oil [3]. Traditional methods of assessing olive ripeness rely on experts’ visual inspection, which can be subjective and time-consuming [4].

The literature includes similar initiatives for a range of crops. In fact, the literature has computer vision proposals for in-the-field fruit recognition for a variety of cases, including orchard crops like apples [5], mangoes [6], sweet peppers [7], almonds [8], and tomatoes [9], and in vineyards [10,11].

Regarding olive fruit, there have also been earlier studies [12,13] that addressed determining the size and mass of the fruits, as well as those [14,15,16] that addressed categorizing the fruits according to their surface condition. While Ponce et al. [17] achieved this distinction by examining the fruits’ external features, [4,18] used endocarp image analysis to identify different varieties of olive fruit. A neural-network-based image analysis algorithm for identifying olive fruits in tree brunches was demonstrated by Gatica et al. [19]. Also, post-harvest research is available [20,21,22], in which maturity is determined in-line, right before milling [23]. The work of Aljaafreh et al. [24], who used CNN in natural light, is one noteworthy exception.

In our study, we explored image preprocessing techniques like histogram equalization and adaptive histogram equalization, widely recognized in precision agriculture [25]. Notably, we observed their efficacy in controlled environments, such as watershed preprocessing and HSV in olive identification [12], and histogram equalization for plant disease detection [26]. Additionally, the usage of Gaussian filters, HSV color space [12], OTSU [17], LAB color space [18], thin-plate spline interpolation [20], and Kuwahara filters [22] were noted for assessing maturity indexes. Under varying lighting conditions, adaptive histogram equalization (AHE) has shown promise [27,28].

Computer vision and image processing techniques have emerged as promising solutions for automating olive detection and ripeness assessment. Object detection in agricultural settings faces unique challenges. Variations in lighting conditions, camera optics, and image quality can significantly hinder the accuracy of detection systems. Traditional image processing methods and earlier iterations of object detection algorithms, including the YOLO (You Only Look Once) series, have shown limitations in effectively addressing these challenges [29]. The accuracy of fruit detection based on color is affected by variations in fruit color due to its maturity level, fruit variety, uncertain and varying background features, and variable lighting conditions [30].

To overcome these limitations, this study aims to enhance YOLOv7’s performance in detecting olive fruits through advanced image preprocessing techniques. A key focus is color correction using ColorChecker—considering each image includes use of a color chart—enabling precise calibration and correction [31]. This method is compared with images without any color correction and other color correction methods, like adaptive histogram processing and histogram equalization, to assess their efficacy in addressing color imbalances and imaging condition variations. Additionally, data augmentation methods, including image rotation and bounding box rotation, are utilized to improve the model’s adaptability to diverse olive fruit orientations and backgrounds.

The primary objective is to assess the impact of these image-preprocessing techniques on the accuracy of YOLOv7 for olive fruit detection. The hypothesis is that implementing these preprocessing and data augmentation techniques will significantly enhance the mean average precision (mAP) of YOLOv7-based object detection systems in diverse imaging conditions [32].

2. Materials and Methods

2.1. Experimental Set-Up

The study utilized the built-in cameras of various cell phone models, namely the iPhone 6 ™ (8-megapixel) (manufactured by Apple, Zhengzhou, China), iPhone X ™ (12-megapixel) (manufactured by Apple, Zhengzhou, China), iPhone SE ™ (12-megapixel) (manufactured by Apple, Zhengzhou, China), and Motorola e4 ™ (8-megapixel) (manufactured by Motorola, Tianjin, China), along with the Xrite ColorChecker PASSPORT ™color chart (Figure 1). The focus of the investigation was the Arbequina variety of olive, captured under natural light between 10:00 a.m. and 4:00 p.m. in four distinct locations across Brazil. The locations were as follows: Farm São Sepé Prosperado, São Sepé, RS, coordinates 30º21′56.68″ S, 53º30′41.98″ W, notable for producing the acclaimed Prosperato olive oil; Azeite Don Jose, Caçapava do Sul, RS, coordinates 30º37′25.00″ S, 53º20′44.19″ W, recognized for the production of Don Jose olive oil; Olivas do Sul—Pomar, Cachoeira do Sul, RS, coordinates 30º00′33.23″ S, 52º51′59.51″ W, known for producing Olivas do Sul olive oil, and Fazenda Oliq, São Bento do Sapucaí, SP, coordinates 22º37′19.70″ S, 45º41′25.72″ W, home to the producer of Oliq olive oil.

The image collection occurred in February 2023 on different days, resulting in a comprehensive dataset comprising approximately 2400 images for training and 180 for validation (90 for cross-validation 1 and 90 for cross-validation 2). It is important to note that all images were meticulously acquired with a color-checker integrated into the frame, ensuring consistency and accuracy in the dataset.

2.2. Image Color Correction

2.2.1. Color Correction Based on ColorChecker

Color correction based on ColorChecker aligns with best practices by considering ColorChecker reference information, including D65 illuminant details. Each image undergoes processing, incorporating ColorChecker detection techniques as a key step in color decoding [33,34]. This ensures precise calibration and adherence to industry standards for consistent and accurate color representation. Crucially, the CAT02 chromatic adaptation transform was used to convert between different illuminants, an advanced feature of color management systems and critical in maintaining color consistency across different lighting conditions [35]. This is performed during the conversion from RGB to xyY color spaces, ensuring accurate color representation [36]. In the following step, Finlayson’s 2015 [37] method is utilized for color correction, a respected technique in color constancy [37]. After correcting the colors, we converted the images back to sRGB.

2.2.2. Adaptive Histogram Equalization (AHE)

The thinking behind AHE comes from our eyes, which adapts to the local context of the image to evaluate the contents. The procedure to accomplish it is the image divided into a grid of rectangular contextual regions in which the optimal contrast must be calculated. The optimal number of contextual regions depends on the type of input image.

The histogram of the contained pixels is calculated for each of these contextual regions. Calculating the corresponding cumulative histogram results in an assignment table that optimizes the contrast in each contextual region [38]. AHE is an algorithm that aims to enhance the contrast in an image by compensating for the differences between the various regions in the image. Unlike other methods, it operates locally. It considers the pixel’s location to perform the enhancement. This method allows adapting to the various features of an image to preserve details better and enhance local contrast. One disadvantage is that it can over-enhance noise in the image’s almost uniform regions [33,39].

2.2.3. Histogram Equalization

HE is a computer image processing technique that redistributes the intensity values of an image to enhance contrast [40]. This method operates by transforming the image’s intensity distribution. The image histogram is relatively flat by enhancing the peak portion’s contrast and reducing the valley portions’ contrast on both sides. The fundamental concept is to ensure that each grey level appears with the same frequency so that the probability of each grey level is evenly distributed, leading to a flat histogram, and the image also becomes clear [41]. In essence, histogram equalization aims to achieve a balanced representation of intensity levels, thereby improving the overall visual quality of the image (Figure 2).

2.3. Data Augmentation

Data augmentation is one of the crucial approaches in object detection, which encompasses adjusting the picture that contains objects to increase the size of the dataset. This helps to increase the generality of a model and application of the model for practical problem-solving. This study employed two types of geometric [42] data augmentation: bounding box and image rotations [43]. Bounding box rotation involves rotating the bounding boxes of the olive fruits by 90 degrees in three directions: vertical flipping, horizontal flipping, and vertical and horizontal flipping. This allows the neural network to learn in different orientations [44]. This geometric rotation was applied to the image rotation level, vertical flipping, horizontal flipping, and vertical and horizontal flipping. This technology introduces rotations on the olive tree fruit’s perspective, changing the background for their object learning [45]. The dataset is augmented with an added bounding box rotation and a rotated image to ensure that the olive fruits are captured under different scenes.

2.4. Model

Deep learning methods have significantly advanced the object detection field, with the YOLO algorithm emerging as a notable development [32]. The YOLO, an acronym for “You Only Look Once”, algorithm works by dividing the input image into a grid and predicting the object classes and bounding box coordinates for each grid cell. This approach enables YOLO to detect multiple objects in a single pass, making it more efficient than other object detection algorithms [46]. YOLOv7 represents a continuation and refinement of the YOLO series YOLOv2 [47], YOLOv3 [48], YOLOv4 [49], and YOLOv5, renowned for its innovative approach to object detection.

This approach, involving a single-stage process for the simultaneous prediction of bounding boxes and object classification, marks a significant departure from conventional methods that adapted classifiers for object detection. YOLOv7 is celebrated for its state-of-the-art (SOTA) performance [50]; YOLOv7 achieved the highest mAP when compared to YOLOv3, YOLOv5, and Faster RCNN in the detection of Camellia oleifera fruit in field environments. Also, for marine creature detection, YOLOv7 outperformed previous YOLO versions [51]. This advancement has been well-documented in several key publications [52,53,54]. Wang et al. [32] and Liu et al. [55] describe the YOLOv7 as comprising an input, backbone, and head network, and a prediction network, as explained below (Figure 3):

Input module: To ensure that the input color images are uniformly scaled to a 640 × 640 size and meet the requirements for the input size of the backbone network, the preprocessing stage of the YOLOv7 model uses mosaic and hybrid data enhancement techniques. It also uses the adaptive anchor frame calculation method established by YOLOv5.

Backbone network: The three primary parts of the YOLOv7 network are MP1, E-ELAN, and CBS. The CBS module comprises SiLU activation functions, batch normalization, and convolution. The E-ELAN module preserves the original gradient path and helps the network learn more features by directing various feature group computational blocks to learn more varied features. This improves the network’s capacity for learning. MP1 is split into upper and lower branches and comprises CBS and MaxPool. The upper branch reduces the image’s length and width by half using MaxPool and reduces the image’s channels by half using CBS with 128 output channels. The lower branch utilizes a concatenation (Cat) operation to combine the features extracted from both branches, halving the image length and breadth. It also uses the picture channels using a CBS with a 1 × 1 kernel and stride. The network’s capacity to extract features is enhanced by MaxPool, which extracts the maximum value information from tiny local areas, and CBS, which extracts all the value information from small local areas.

Head network: The head network of YOLOv7 is organized using the PANet-based feature pyramid network (FPN) architecture. This network includes the extended efficient layer aggregation network (E-ELAN), MaxPool-2 (MP2), several convolutional, batch normalization, and SiLU activation (CBS) blocks, as well as the introduction of a spatial pyramid pooling and convolutional spatial pyramid pooling (Sppcspc) structure. Adding a convolutional spatial pyramid (CSP) structure within the spatial pyramid pooling (SPP) structure and a large residual edge to facilitate optimization and feature extraction, the Sppcspc structure enhances the perceptual field of the network. The ELAN-H layer, a combination of multiple E-ELAN-based feature layers, further improves feature extraction. With a minor change to the number of output channels, the MP2 block’s structure is similar to that of the MP1 block.

Prediction network: YOLOv7’s prediction network uses a Rep structure to modify how many image channels the head network’s features output should have. Then, it applies a 1 × 1 convolution to predict the confidence, category, and anchor frame. RepVGG [56] serves as the model for the Rep structure, which adds a unique residual design to facilitate training. In practical predictions, this special residual structure reduces to simple convolution, which reduces network complexity without compromising predictive performance.

2.5. YOLOv7 Training

The training of YOLOv7 was conducted on the Google Colab platform with NVIDIA A100 GPU with 40 GB of memory (manufactured by NVIDIA, Hsinchu, Taiwan). Unlike the original broader configuration, this setup was specifically tailored to identify two classes: “olive fruit” and “not olive fruit”. The training was set to run for 100 epochs since multiple epochs can be found in the literature, like 100, 50, and 45 [57,58,59]. The input image resolution was set to 1280 × 1280 pixels, adapting the YOLOv7-E6 model through transfer learning. This decision was based on optimizing the model’s performance for the specific task at hand without any adjustments to the default configuration of hyp.scratch.p5.yaml [32].

2.6. Metrics

2.6.1. Precision

Precision can be considered as the model’s ability to identify all the detected olives correctly. It measures the accuracy of the system’s detections for identifying olives in images. It is calculated by dividing the true positives $\left(TP\right)$ by the sum of $TP$ and the false positives $\left(FP\right)$, as expressed by Equation (1).

$$Precision=\frac{TP}{TP+FP}$$

(1)

where for true positives $\left(TP\right)$, olives are present in the image, and the model predicts their presence correctly; for false positives $\left(FP\right)$, olives are not in the image but are falsely detected.

2.6.2. Recall

Recall reflects the system’s ability to detect all actual olives in the images. It measures how many of the actual olives in an image were correctly identified by the system. It is calculated as the ratio of the true positives $\left(TP\right)$ to the sum of $TP$ and the false negatives $\left(FN\right)$ in Equation (2):

$$Recall=\frac{TP}{TP+FN}$$

(2)

where for true positive $\left(TP\right)$, olives are present in the image, and for false negative (FN), an olive is present in the image but is not detected by the model.

2.6.3. mAP

The mean average precision (mAP) is a metric in object detection tasks that measures the system’s accuracy in detecting objects, such as olives, across images. It is derived from the precision and recall values for olives at a certain threshold or over a range of thresholds. The mAP is typically computed as the area under the precision–recall curve for olives, as expressed by Equation (3):

$$mAP=\frac{1}{C}\sum _{i=1}^{C}A{P}_i$$

(3)

where, $A{P}_i$ is the average precision for olives, and C represents the number of image sets or categories where olives are to be detected. In the case of a single category, such as olives, the mAP would be equal to the average precision $\left(AP\right)$ for that category [60].

2.6.4. Paired t-Test

A paired t-test was used to compare the mAP of two related treatments. A p-value below 0.05 indicates a statistically significant difference between the control group (no treatment) and the corresponding treatment group [61]. The paired t-test is appropriate because the two mAP groups are related [62], as expressed by Equation (4).

$$t=\frac{\overline{d}}{s_d/\sqrt{n}}$$

(4)

where t is the t-value, $\overline{d}$ is the mean of the differences between the paired observations, $s_d$ is the standard deviation of the differences, and n is the number of pairs.

2.6.5. ANOVA

In the case of augmentation and preprocessing, we used ANOVA (analysis of variance), a statistical method used to determine whether there are statistically significant differences between the means of two or more groups [63]. The total sum of squares $\left(TSS\right)$ is the sum of all the squared differences between the mean of a sample and the individual values in that sample:

$$TSS=\sum _{i=1}^k\sum _{j=1}^{n_i}{(Y_{ij}-\overline{Y})}^2,$$

(5)

where $Y_{ij}$ is the observation, $\overline{Y}$ is the overall mean, k is the number of groups, and $n_i$ is the number of observations in the ith group.

Sum of squares between $\left(SSB\right)$: For each subject, compute the difference between its group mean and the grand mean.

$$SSB=\sum _{i=1}^k{n}_i{(\overline{Y_i}-\overline{Y})}^2,$$

(6)

where $\overline{Y_i}$ is the mean of the ith group, $\overline{Y}$ is the overall mean, k is the number of groups, and $n_i$ is the number of observations in the ith group. Sum of squares within $\left(SSW\right)$ is the sum of the squared differences between a value and its sample mean for all values.

$$SSW=\sum _{i=1}^k\sum _{j=1}^{n_i}{(Y_{ij}-\overline{Y_i})}^2,$$

(7)

where $\overline{Y_i}$ is the mean of observations within the ith group, $\overline{Y}$ is the overall mean, k is the number of groups, and $n_i$ is the number of observations in the ith group.

$MSB$ is the mean sum of squares between the groups.

$$MSB=\frac{SSB}{k-1},$$

(8)

where $k-1$ is the degrees of freedom for groups, and $SSB$ is the sum of squares between. $MSW$, the mean sum of squares, is calculated by dividing the sum of squares within the groups by the error degrees of freedom.

$$MSW=\frac{SSW}{N-k},$$

(9)

where N is the total number of observations and k is the number of groups, and $SSW$ is the sum of squares within.

To obtain the F-statistic, we want to compare the “average” variability between the groups to the “average” variability within the groups. We take the ratio of the between mean sum of squares to the error mean sum of squares.

$$F=\frac{MSB}{MSW},$$

(10)

where $MSB$ and $MSW$ are the mean squares between and within groups.

2.6.6. Tukey

The Tukey test, also known as Tukey’s honest significant difference (HSD), is a post hoc test used after an ANOVA has been conducted and found to be significant. The Tukey test is a multiple comparison procedure which controls the family-wise error rate (FWER) [64], as expressed by Equation (11):

$$Q=q_{\alpha }(k,df)\times \sqrt{\frac{MSE}{n}}$$

(11)

where $q_{\alpha }$ is the studentized range statistic q for a given alpha level $\alpha$, number of groups k, and degrees of freedom $df$. $MSE$, is the mean square error from the ANOVA. n, is the number of observations in each group.

3. Results

This study evaluated four treatments in the building dataset, namely: H1—no preprocessing and no augmentation; H1.1—no preprocessing and image rotate augmentation; H1.2—no preprocessing and bound box augmentation; H2—color correction based on ColorChecker preprocessing and no augmentation; H2.1—color correction based on ColorChecker preprocessing and image rotate augmentation; H2.2—color correction based on ColorChecker preprocessing and bound box augmentation; H3—AHE preprocessing and no augmentation; H3.1—AHE preprocessing and image rotate augmentation; H3.2—AHE preprocessing and bound box augmentation; H4—HE preprocessing and no augmentation; H4.1—HE preprocessing and image rotate augmentation; H4.2—HE preprocessing and bound box augmentation (

Table 1. Cross-validation (CV) evaluation of each treatment.

Treatment	CV1	CV1	CV1	CV2	CV2	CV2
No preprocessing	Precision	Recall	mAP	Precision	Recall	mAP
H1	0.593	0.672	0.641	0.572	0.756	0.686
H1.1	0.762	0.786	0.84	0.738	0.794	0.848
H1.2	0.651	0.872	0.821	0.669	0.828	0.814
ColorChecker preprocessing	Precision	Recall	mAP	Precision	Recall	mAP
H2	0.6651	0.872	0.821	0.663	0.759	0.726
H2.1	0.718	0.842	0.847	0.78	0.78	0.86
H2.2	0.69	0.757	0.785	0.662	0.835	0.774
AHE	Precision	Recall	mAP	Precision	Recall	mAP
H3	0.74	0.81	0.78	0.772	0.79	0.84
H3.1	0.633	0.729	0.713	0.66	0.736	0.744
H3.2	0.701	0.669	0.725	0.675	0.738	0.745
HE	Precision	Recall	mAP	Precision	Recall	mAP
H4	0.676	0.793	0.775	0.664	0.814	0.785
H4.1	0.72	0.829	0.832	0.723	0.817	0.839
H4.2	0.647	0.785	0.752	0.661	0.809	0.784

).

A paired t-test was used to compare the mAP of two related treatments. A p-value below 0.05 indicates a statistically significant difference between the control group (no treatment) and the corresponding treatment group [61]. The paired t-test is appropriate because the two mAP groups are related [62]. For this, it could be used to compare the mAP metric from H1 versus H2, H3, and H4 to test which color adjustment improves the mAP metrics (

Table 2. H1 X H2, H3 and H3.

H1	H2	H3	H4
p-value	0.3608	0.0326	0.0949

). In this comparison, they have the same number of images. The study comprehensively evaluates color correction as a preprocessing step for olive identification using cell phones under natural light. The results align with the findings, highlighting the challenges and importance of color correction in agricultural image analysis [65,66].

The paired t-test results revealed that AHE (hypothesis H3) preprocessing is statistically significantly different compared to the other methods, providing empirical evidence for its efficacy in this specific application. AHE separates an image into several sub-blocks, and each sub-block is processed by histogram equalization. It creates a local equalization, making it more interesting for natural light images, wherein the same image in many regions might be darker or lighter. Moreover, the HE performs this for the entire image without considering each region [67]. Color correction, even when using a sophisticated matrix-based method to adjust the colors in an image, often applies corrections uniformly across the entire image [68]. In the case of augmentation and preprocessing, we used ANOVA or variance analysis. The ANOVA p-value was $1.3318434825276204\times {10}^{-5}$, showing statistically significant differences in the mean mAP scores among the different color correction methods and augmentations. To investigate this, we proceeded with the Tukey test (

Table 3. Tukey HSD Test.

Group1	Group2	Meandiff	p-adj	Lower	Upper	Reject
H1	H1.1	0.1805	0	0.1155	0.2455	True
H1	H1.2	0.154	0.0001	0.089	0.219	True
H1	H2.1	0.19	0	0.125	0.255	True
H1	H2.2	0.116	0.0011	0.051	0.181	True
H1	H3.1	0.065	0.0501	0	0.13	False
H1	H3.2	0.0715	0.0294	0.0065	0.1365	True
H1	H4.1	0.172	0	0.107	0.237	True
H1	H4.2	0.1045	0.0024	0.0395	0.1695	True
H1.1	H1.2	0.0265	0.7794	0.0915	0.0385	False
H1.1	H2.1	0.0095	0.9993	0.0555	0.0745	False
H1.1	H2.2	0.0645	0.0522	0.1295	0.0005	False
H1.1	H3.1	0.1155	0.0012	0.1805	0.0505	True
H1.1	H3.2	0.109	0.0018	0.174	0.044	True
H1.1	H4.1	0.0085	0.9997	0.0735	0.0565	False
H1.1	H4.2	0.076	0.0205	0.141	0.011	True
H1.2	H2.1	0.036	0.4779	0.029	0.101	False
H1.2	H2.2	0.038	0.4197	0.103	0.027	False
H1.2	H3.1	0.089	0.0074	0.154	0.024	True
H1.2	H3.2	0.0825	0.0123	0.1475	0.0175	True
H1.2	H4.1	0.018	0.9609	0.047	0.083	False
H1.2	H4.2	0.0495	0.1785	0.1145	0.0155	False
H2.1	H2.2	0.074	0.024	0.139	0.009	True
H2.1	H3.1	0.125	0.0006	0.19	0.06	True
H2.1	H3.2	0.1185	0.001	0.1835	0.0535	True
H2.1	H4.1	0.018	0.9609	0.083	0.047	False
H2.1	H4.2	0.0855	0.0097	0.1505	0.0205	True
H2.2	H3.1	0.051	0.1583	0.116	0.014	False
H2.2	H3.2	0.0445	0.2634	0.1095	0.0205	False
H2.2	H4.1	0.056	0.1053	0.009	0.121	False
H2.2	H4.2	0.0115	0.9974	0.0765	0.0535	False
H3.1	H3.2	0.0065	1	0.0585	0.0715	False
H3.1	H4.1	0.107	0.002	0.042	0.172	True
H3.1	H4.2	0.0395	0.379	0.0255	0.1045	False
H3.2	H4.1	0.1005	0.0032	0.0355	0.1655	True
H3.2	H4.2	0.033	0.5719	0.032	0.098	False
H4.1	H4.2	0.0675	0.0408	0.1325	0.0025	True

).

The results of the Tukey HSD test provide a detailed comparison of the mean differences between the various groups using different augmentation techniques. Here is a summary of the key findings:

H1 vs. Others:

• Significant differences were found between H1 and several other groups (H1.1, H1.2, H2.1, H2.2, H3.2, H4.1, H4.2), indicating that the use of any augmentation or preprocessing method generally improves performance compared to no augmentation/preprocessing.

H1.1 vs. Others:

• H1.1 (no preprocessing with image rotation augmentation) significantly differs from H3.1, H3.2, and H4.2, indicating differences in performance when comparing image rotation augmentation with various preprocessing methods.

H1.2 vs. H3.1 and H1.2 vs. H3.2:

• These comparisons show significant differences, suggesting that the type of preprocessing used with bbox rotation augmentation can impact performance.

H2.1 vs. H2.2:

• A significant difference exists, indicating that the choice between image rotation and bbox rotation in color correction preprocessing can impact the results.

H3.1 vs. H4.1:

• This comparison reveals a significant difference, suggesting that histogram equalization preprocessing combined with image rotation performs better than histogram adaptive preprocessing with the same augmentation.

H4.1 vs. H4.2:

• A significant difference is noted here, highlighting the impact of the type of augmentation (image rotation vs. bbox rotation) in histogram equalization preprocessing.

Based on the Tukey HSD test results, the best treatment in terms of [email protected] performance among the ones tested can be identified by looking for the groups that consistently showed superior performance. The key is to find the group that was significantly better than most others and was not significantly outperformed by any other group. Here is a summary based on our results:

• H1.2 (No preprocessing with image rotation): While it showed significant improvements over H1, it was outperformed by H3.1, H3.2, and H4.2.
• H2.1 (Color correction with image rotation): This group did not show significant differences when compared to H1.1, H3.1, and H4.1, and it was only significantly better than H2.2, H3.1, H3.2, and H4.2.
• H4.1 (Histogram equalization with image rotation): H4.1 stands out as no other group significantly outperformed it in our test, and it showed significant improvements over several groups, including H3.1 and H3.2.

4. Discussion

Aljaafreh et al. [24] conducted a similar test using an RGB camera to collect data from 10 olive farms in Jordan under natural light. Methodologically, they employed YOLOv5 and tested four hyperparameters to determine the best olive fruit detection performance. Hyperparameter D (with four anchors, a learning rate of 0.01, and a weight decay of 0.05) used with YOLOv5x, which accepts images of 640 × 640 pixels, achieved the highest mAP of 0.7708. All the mAP metrics are available in

Table 4. Aljaafreh et al., 2023 [24], YOLOv5 olive fruit object detection under natural light mAP.

Name	Hyperparameter	mAP
YOLOv5x	D	0.7708
YOLOv5s	D	0.7265
YOLOv5x	C	0.7116
YOLOv5s	C	0.6827
YOLOv5x	B	0.733
YOLOv5s	B	0.7384
YOLOv5x	A	0.7559
YOLOv5s	A	0.7413

.

In this study, the mAP (mean average precision) metrics are compared with those reported by Aljaafreh et al. [24]. Our results demonstrate a significant improvement: an increase of 7.94% in CV1 (H4.1: 0.832 compared to 0.7708 mAP (Equation (12)) and an increase of 8.8% in CV2 (H4.1: 0.839 compared to 0.7708 mAP (Equation (13)).

$$\left(\frac{0.832-0.7708}{0.7708}\right)\times 100\approx 7.94\%$$

(12)

$$\left(\frac{0.839-0.7708}{0.7708}\right)\times 100\approx 8.85\%$$

(13)

We observed a significant improvement when comparing our H2.1 results with the best mAP achieved by Aljaafreh et al. [24]. Specifically, in CV1, H2.1 surpassed their results by 9.9% (Equation (14)), and in CV2, the improvement was even more pronounced at 11.57% (Equation (15)).

$$\left(\frac{0.847-0.7708}{0.7708}\right)\times 100\approx 9.90\%$$

(14)

$$\left(\frac{0.86-0.7708}{0.7708}\right)\times 100\approx 11.57\%$$

(15)

This finding would allow farmers to use mobile devices to identify the olives for counting and later research for maturity index and disease identification. Such an approach does not require experience or skills with a camera for image acquisition, therefore becoming an ally for farmers. Since the light condition is a challenge, artificial light can be used to improve the results. In [69], which used Yolov5 with 500 epochs for peach recognition, it was concluded that artificial light resulted in better results when compared to natural light, with an F1 score of 0.81, highlighting that natural light presents issues like excessive and shade sun fleck conditions. Aquino et al. [18], using artificial light to identify olives at night and convolution neural network (CNN)-based models, reached F1 scores of 0.83 and 0.84 for precision using Inception-ResNetV2 with 64 epochs.

In the study which focused on using deep learning for olive fruit detection, particularly employing the YOLOv7 model, notable limitations emerge that warrant attention for future research. Firstly, the study’s findings are inherently tied to specific conditions, notably the use of cell phones for image capture under natural lighting, which may not be representative of internal areas and controlled light environments. Additionally, while exploring a range of preprocessing techniques, including histogram equalization and image rotation, the research may not have encompassed the entire spectrum of available or potentially more effective methods. This limited scope may have resulted in overlooking other innovative techniques that might enhance detection accuracy. Furthermore, the reliance on the YOLOv7 model as the sole CNN tool raises questions about the generalizability of the findings across different object detection models, each potentially responding differently to the same preprocessing and augmentation techniques. These limitations highlight the need for more extensive research to validate and extend these findings under varying conditions, with different olive varieties, with diverse resolutions, and across multiple deep learning models.

The study comprehensively evaluated color correction as a preprocessing technique for olive identification using cell phones in natural light conditions. Various methods were compared, including histogram equalization (HE), adaptive histogram equalization (AHE), and color correction based on use of ColorChecker. The findings indicated that any preprocessing or augmentation method generally improved performance compared to scenarios without such treatments.

5. Conclusions

Based on the results of the Tukey test, it can be concluded that using histogram equalization preprocessing with image rotation augmentation (H4.1) performs the best in terms of [email protected] performance compared to the other tested preprocessing methods and augmentation techniques. The results in terms of image augmentation agree with [70], with the method also performing better on the image level as opposed to the box level.

Notably, the AHE preprocessing method (H3) demonstrated statistically significant differences over the other methods, highlighting its suitability for image capture under natural light with diverse lighting conditions. The study revealed that different preprocessing and augmentation techniques impact the performance of the YOLOv7 model in olive detection.

Among the treatments, the combination of histogram equalization preprocessing and image rotation (H4.1) emerged as the most effective. While sophisticated color correction methods were effective, they were less impactful than AHE preprocessing. This study highlights the importance of carefully selecting image preprocessing and augmentation techniques, with AHE preprocessing and image rotation augmentation identified as key contributors to enhancing olive detection accuracy using YOLOv7 in natural lighting conditions.

In addition, our comprehensive analysis and comparative study with Aljaafreh et al. (2023) [24] have led to notable advancements in olive fruit detection using YOLOv5. We achieved significant improvements in the mean average precision (mAP) when using YOLOv5 [24], with increases of 7.94% and 8.8% in CV1 and CV2 for H4.1, and even more substantial gains of 9.9% and 11.57% in CV1 and CV2 for H2.1, respectively. These results demonstrate the efficacy of our methodological enhancements and provide valuable insights for future research in precision agriculture using deep learning techniques.