3D22^MX: Performance Subjective Evaluation of 3D/Stereoscopic Image Processing and Analysis

Round 1

Reviewer 1 Report

I think the structure of the presentation could be improved; currently, it is too fragmented into lists containing few words and short explanations.

Also, for the audience of this journal, I would expect more theoretical considerations of the proposed methods, rather than a very applicative and specific article like the one proposed.

Therefore, I request a restructuring of the article with a clear discussion of the theoretical aspects of the proposed methods.

Moreover, the experiments consist of the comparison of the performances of several methods but an ablation study is missing.

Author Response

Please see the attachment. We prepare our response by adding a letter at the beginning of the modified article.

Author Response File: Author Response.pdf

Reviewer 2 Report

In this paper authors explain a methodology for building a 3D/ stereoscopic database, then they implemented a software tool for degradation of this images, and finally they do a psychophysical experiment for a specific type of noise. The database use 15 indoor images and 5 outdoor ones spatial-calibrated and lighted for different types of scenarios, with different criteria calibration. They use Matlab for their computations, by using 10 different types of noises for degradation, each with 5 levels of degradation. Finally, they apply a criterion to carry out psychophysical experiments with 3D/stereoscopic images.

The paper is well-written, the method is clear, and the results are remarkable. The way of explaining all the concepts is very didactic and helps a reader little introduced to the subject to understand everything in a good way.

The introduction explains the state of the art of the problem in detail and puts the problem to be solved in context with the appropriate references.

The method used is well explained and could easily be reproduced following the paper.

The conclusions are well written and summarize the article very well and propose a research line that could be continued in the future.

However, I would ask the authors to make these changes:

-       In the images that appear in the document, the place where they were acquired is not mentioned. Which ones are made by the authors, and which ones are not? This information should appear at least, in the caption of each of them, indicating the responsible for their rights.

-       In order to explain the noise-types (section 3.2), it would be better if each of them was explained without the need to create a new subsection. A different paragraph would be enough.

-       It would be good to represent the data that appears in table 1 in a linear graph where it can be seen that the MOS increases when the level of degradation increases.

-       In figure 17, authors should explain in the caption the meaning of each of the lines and points that appear in the figure.

-       Authors in Ref 2 are written in capital letters, change it.

As a conclusion, I have enjoyed reading this paper and I would recommend publishing it after making these minor changes.

Author Response

Please see the attachment. We prepare our response by adding a letter at the beginning of the modified article.

Author Response File: Author Response.pdf

Reviewer 3 Report

While the science seems sound, the communication/presentation of ideas requires considerable English language improvements; As a result of this communication problem and the consequent difficulties I faced comprehending the different aspects of the manuscript, I do not recommend publication in the current form. The following points should be addressed in addition to this language issue.

-- The abstract also needs to clearly communicate the novelty of the work. It was difficult to ascertain this.

-- Thereby, it is essential to develop databases containing not only different images with added noises but also Most Opinion Scores (MOS) from viewers in order to subjectively assess the quality of each 3D/Stereoscopic-image.
If the database is entirely based on subjective assessments of individuals, how can we guarantee the wide-ranging applicability of the developed algorithms? The authors should comment on this.

-- The authors should comment on why the sizes of the datasets are relatively low (10, 20 etc.) compared to what will be expected (100’s – 1000’s) to give a robust performance of the developed algorithms.

-- Please include axes titles in Fig. 10.

-- The methodology section is too detailed; it leaves the reader waiting too long to see the results. For example, the description of the user interface developed can be shortened, as this has lower scientific value.

-- While the development of the database is commendable, the reviewers should attempt to compare their results with standard algorithms in MATLAB or other image processing software to demonstrate potential improvements to what already exists.

-- Figure 17, shows results for only J2K noise, more results can be included to capture other noise types; thus strengthening the paper.

-- Given the level of subjective assessments in this work, it would have been great to report the uncertainty associated with measurements by conducting randomised repeats. The authors should comment on this.

Author Response

Please see the attachment. We prepare our response by adding a letter at the beginning of the modified article.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

I thank the authors for the efforts done in the paper revision.

Reviewer 3 Report

The authors have satisfactorily addressed my technical comments. I now recommend the publication of this paper. 

However, I encourage the authors to further conduct a thorough and final proofread to ensure adequate elimination of ALL grammar-related errors.