Evaluation of the Storage Performance of ‘Valencia’ Oranges and Generation of Shelf-Life Prediction Models

Round 1

Reviewer 1 Report

The assessed work entitled “Evaluation of the storage performance of 'Valencia' oranges and 2 generation of shelf-life prediction models” conducted a large-scale, high-throughput phenotyping analysis of the effects of vari-16 ous pre-harvest and postharvest features on the quality of 'Valencia' oranges. The topic is interesting, although it is not new, it may be of interest to readers. However, work before it is suitable for publication requires corrections and additions. My comments on the individual sections of the manuscript are outlined below:

Abstract:

In the abstract, the author mainly describes some experimental design statements, and the result analysis part is less described.

line 22, “The examined features significantly affected (p < 0.001) fruit weight loss, firmness, 22 decay, color, peel damage, internal dryness, TSS, acidity, vitamin C, ethanol levels, and flavor 23 and acceptance scores”. What is the effect of examined features on fruit quality, for better or for worse.

Materials and Methods

Why are the rootstocks used in harvest 2 fruits inconsistent with others, and will this affect the quality of the fruits?

Table 1 should use a three-line table.

How the fruit was handled after harvest and what fruits were the fruit quality measured on?. E.g. has all fruit been frozen immediately after harvest? Possibly how many of them? On what fruits and how exactly was the colour measured? This information should be provided to better understand the whole experimental procedure.

All experimental methods are not described in sufficient detail. Add the references to all used methods in the material and methods part.

Results:

A ruler should be added to Figure 2.

Author Response

In the abstract, the author mainly describes some experimental design statements, and the result analysis part is less described – we revised the abstract in order to add more data of the observed results, thus within the journals limitation of just 200 words (p. 1, lines 22-27).

line 22, “The examined features significantly affected (p < 0.001) fruit weight loss, firmness, decay, color, peel damage, internal dryness, TSS, acidity, vitamin C, ethanol levels, and flavor and acceptance scores”. What is the effect of examined features on fruit quality, for better or for worse – This data has been removed from the abstract. However, the effects of the various features on fruit quality parameters are described in more details in the results chapter (p. 6-7, lines 243-269).

Materials and Methods - Why are the rootstocks used in harvest 2 fruits inconsistent with others, and will this affect the quality of the fruits? – in harvest 2 we purposely chose an orchard with a different rootstock (namely Volka mariana instead of sour orange used in the other orchards) in order to evaluate the possible influence of rootstock selection on the postharvest storage performance of 'Valencia' oranges.

Table 1 should use a three-line table - we revised Table 1 according to the reviewer's suggestion.

How the fruit was handled after harvest and what fruits were the fruit quality measured on?. E.g. has all fruit been frozen immediately after harvest? Possibly how many of them? On what fruits and how exactly was the colour measured? This information should be provided to better understand the whole experimental procedure – after harvest the fruit were treated and sorted in a commercial packinghouse, and afterwards transferred to the Volcani Institute and placed in cold storage rooms as described (M&M, sections 2.1 and 2.2). A sample of fruit were taken for quality evaluations at time zero, and additional quality evaluations were conducted at 2-week intervals along a 20-week storage period (M&M, section 2.3). We tested only fresh fruit, and not frozen fruit. Quality evaluations were conducted on samples of fruit after removal from cold storage. Peel color was measured as described (M&M, section 2.3.3).

All experimental methods are not described in sufficient detail. Add the references to all used methods in the material and methods part – we added more information to the M&M section.

Results: A ruler should be added to Figure 2 - Unfortunately, the pictures were already taken, and thus we cannot add a ruler. Nonetheless, we assume the readers are familiar with the approximate size of an average orange.

Reviewer 2 Report

In this experiment, 'Valencia' citrus fruits of different harvest period were stored in different temperature and humidity environment, and 14 kinds of quality parameters were tracked. Through statistical analysis, the factors that affect the quality of fruits after harvest were found, so as to develop quality prediction model. The sample size of this study is large, and the results are reliable, which can supplement the postharvest study well. The reviewers recommend reconsideration of your manuscript following major revision.

Line 107: i.e.,

Line 120：Please add the formula of weight loss rate

Line 128：Please add the formula.

Line 128：Please add the formula of TA.

Line 138：Please add the formula.

Line 145：37 ℃，Precede each unit with a space,please correct all mistakes

Line 152：The number of sensory judges is only 3, which is too small and subjective.

Line 155：The number of sensory judges is only 3, which is too small and subjective.

Line 231：The format of the table should be uniform.Table 1 can be changed into a three-line Table

Line 262：It is recommended not to calculate the decay rate of each stage separately, but the total decay rate of each time node should be calculated up to this time. With the extension of time, the decay rate should always rise.

Line 263：where are the figures relevant to RH level (70%, 90% or 95%) in Figure 1? In addition, Why is there no standard deviation in Figure 1? Please check all the figures.

Line 279：where is the “8 ℃”  in Figure 3? And where is “RH (70%, 90% or 95%)” in Figure 3?

Author Response

Line 107: i.e., - we added the missing comma.

Line 120：Please add the formula of weight loss rate – we added the formula of calculating weight loss (p. 3, line 126)

Line 128：Please add the formula - we added the formula of calculating the percentage of peel damage, decay and internal dryness (p. 4, line 139)

Line 128：Please add the formula of TA – acidity levels were measured using an automatic titrator which automatically provides the TA percentages, and thus we did use a specific formula (M&M, section 2.3.5)

Line 138：Please add the formula - we added the formula of calculating vitamin C levels (p. 4, line 151)

Line 145：37 ℃，Precede each unit with a space, please correct all mistakes – we added spaces before °C along the entire manuscript.

Line 152：The number of sensory judges is only 3, which is too small and subjective – we agree with the reviewer that it would have been better to conduct wide-scale consumer acceptance tests with dozens of panelists instead of just 3 trained judges. However, that is impossible in case it is necessary to evaluate 120 samples as done in the current study. 

Line 231：The format of the table should be uniform. Table 1 can be changed into a three-line Table - we corrected the format of Table 1 according to the reviewer's suggestion.

Line 262：It is recommended not to calculate the decay rate of each stage separately, but the total decay rate of each time node should be calculated up to this time. With the extension of time, the decay rate should always rise – the decay percentages were evaluated every two weeks, but each time using different cartons of fruit. Therefore, in some cases we observed biological variations and fluctuations in the measured decay rates, but overall there is an increasing trend of decay rates along storage.

Line 263：where are the figures relevant to RH level (70%, 90% or 95%) in Figure 1? In addition, Why is there no standard deviation in Figure 1? Please check all the figures – the different RH levels were evaluated only in fruit from harvest 4.  In Figs. 1, 3 and 4, the yellow line represents low RH and the gray line high RH.

Line 279：where is the “8 ℃”  in Figure 3? And where is “RH (70%, 90% or 95%)” in Figure 3? – we apologize for this mistake.  We tested only two temperatures of 2 °C and 5 °C, and thus deleted the mistaken indications of 8 °C (see legends of Figs. 3-4).

Reviewer 3 Report

In the submitted manuscript by Ron Porat and colleagues, entitled “Evaluation of the storage performance of 'Valencia' oranges and generation of shelf-life prediction models”, the authors conduct a large-scale, high-throughput phenotyping analysis of quality traits of 'Valencia' oranges to develop a prediction model. Non-linear Support Vector Regression (SVR) was the most effective approach to establish this model. I should admit that I am not an expert in machine learning and artificial intelligence technologies to evaluate in-depth the current work. Overall, the manuscript is well-written, and generally, the author's data and statical analysis are clear. The aim and scope of the Horticulturae journal are in line with the current manuscript. However, there are some issues that should be carefully addressed.

1. Does it enough the dataset to develop this prediction model or you should incorporate more orange farmers?

2. Do you test the proposed model with a small batch of oranges?

3. What is the optimum RMSE to propose a model? Please, give some examples and background in the discussion section.

4. In other similar studies (postharvest quality and prediction models), which prediction model fits better the fruit dataset (incorporate in the discussion section).

Author Response

1. Does it enough the dataset to develop this prediction model or you should incorporate more orange farmers? - The dataset size by itself is sufficient for the development of the model. The quality of all models was tested using a K-fold cross-validation method with 5 folds and 6 repetitions producing 30 samples.  The statistical significance of the models all have a p-value of p<0.01.

 This said, there is indeed a valid point regarding future data with potentially different statistical characteristics. For example, it is possible that produce from other farmers have different physical characteristics that would eventually generate a larger prediction error. If that is indeed the case, it is possible to retrain the model using a larger dataset that includes also data from the specific new farmers. Moreover, it is also possible, that produce from different harvest years would have different statistical characteristics which are currently not controlled by the model. This non-stationary behavior of the data, will probably increase the prediction error of the proposed model. By increasing the training dataset, it will be possible to mitigate this as well. The current study have demonstrated the feasibility of using prediction models for the First Expired, First Out (FEFO) logistics strategy.

Following the reviewer's comment, we included the above paragraph in the Discussion section (p. 15, lines 491-499).

2. Do you test the proposed model with a small batch of oranges? - Indeed the dataset in this study is relatively small when considering machine learning applications. However, we employed a very careful approach using 5-fold cross validation with 6 repetitions. According to this approach, each orange data point was left out of the training of the model and used for evaluation of the model 6 times:

 The model was tested using a 5-fold cross-validation method which means that on each fold, 80% of the data was used for training, and 20% (24 data points) of the data was hidden during training and used only for evaluation. Then, a different fold is chosen for testing, and the other remaining 4 folds are used to train a completely new model in order to evaluate on the new test-fold. Therefore, in 5-fold cross-validations, each of the 5 folds is used once for testing, and 4 times for training. Moreover, we repeated the above 5-fold scheme using 6 different random allocations into folds, which produced overall all 30 test-sets each consisting of 24 data points. This enabled us to measure the statistical significance of the predictions for all the models which yielded p-values of p<0.01.

3. What is the optimum RMSE to propose a model? Please, give some examples and background in the discussion section - The model was aimed to predict the fruit acceptance score which was assigned using a 5-grade scale (1 = very bad, 2 = poor, 3 = fair, 4 = good, and 5 = excellent).

Generally, the desired error is determined the by application for which the model is designed. Ideally, we aim for the lowest Root Mean Squared Error (RMSE) possible, that is that the model’s prediction is as close as possible to the expert’s judgment. In this study, we reached an RMSE of 0.195, which means that the average error of the model is ~0.2 points above or below the expert’s judgment. For FEFO management use, this is a reasonable error.

4. In other similar studies (postharvest quality and prediction models), which prediction model fits better the fruit dataset (incorporate in the discussion section). - The choice of machine learning model depends upon the statistical properties of the problem at hand. According to Occam’s razor theory, we should seek the most simple model that explains the observed data. In our case, we observed that a linear model such as linear regression or a linear support vector machine had difficulties explaining the data which translated into higher prediction errors. This indicates the need for a non-linear model in order to describe the data. We found that the non-linear support vector machine based on radial basis functions were adequate to the problem at hand.

To address this issue, the following chapter has been added to the discussion (p. 14, lines 465-476):

"A large arsenal of potential machine learning models can be applied, and the choice of model should depend upon the statistical properties of the problem at hand. According to Occam’s razor theory, we should seek the simplest model that explains the observed data. In this study, we examined five regression models: Linear Regression, Linear SVR, Non-linear SVR, RF, and XGBoost. We noticed that the linear models had difficulties ex-plaining the data which translated into higher prediction errors. This indicated the need for a non-linear model in order to describe the data. We found that the Non-linear SVR model based on radial basis kernel was adequate (Table 3)".

Round 2

Reviewer 1 Report

Most of my suggestions were taken into account by the Authors during the revision. I believe the manuscript can be accepted in its current form.