Evaluation of the Accuracy of Four Digital Methods by Linear and Volumetric Analysis of Dental Impressions

Round  1

Reviewer 1 Report

The article is interesting, however the volumetric analysis technique is already known. In dentistry the use of Geomagic for this controls is used and the results are proved. 

This study compared 4 comparison procedures and the conclusion is that MATLAB could be used. It is another type of analysis. However, it is essential to know the way of the alignment between the CMM and digitalized tooth. Each software has its own alignment algorithms and therefore it should be shown the alignment deviation in each case, before the making the comparisons.

Other comments: 

In the Abstract "precision" is used instead of "accuracy".

Author Response

The authors wish to thank the reviewers for his/her important contribution.

In particular, the authors appreciate the reviewer's effort aimed at improving the quality of the

paper and at increasing the clarity of the presentation.

The paper has been amended following all reviewer's suggestions, as better elucidated at the

following lists.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

This is an interesting study about the new methodology for the accuracy evaluation by volumetric analysis in the dental field. However, it is still not clear to me following matters as far as I read the manuscript.

Title

In this study, the phrase 'dental material impressions' is not appropriate because the single tooth is scanned with various scanners and the accuracy of the acquired data is compared. And the authors did not evaluate the data quality among different tooth material. Please, replace it with "dental impressions."

Abstract

Line 20: The purpose of this study is to confirm the validity of the volumetric error distribution by MATLAB algorithms, compared to the existing methods of evaluating the accuracy and precision of optical scanners used in the dental field. First of all, this study compares the data obtained from various optical scanners to evaluate the performance of them and seems to be distant from the scope of the 'materials' journal.

Line 22: It is not clear what the author wants to say in this sentence. Please, make it clear which message you want to deliver, the limitation of this study, or the fact that it is also possible in the full arch model. As far as I know, it is not possible to acquire the complete-arch scan including the interproximal embrasure with CMM equipment. Also, the authors don't need to mention the limitations of this study in the abstract.

Line 36: No conclusions in the abstract. It is recommended that the authors clearly state what message they want to deliver in this study.

Line 37: The keywords do not seem to cover all of the representative contents of this study. It is recommended to include more specific words such as 'volumetric error distribution'.

Introduction

Overall, the contents of the text are not organized well. It is recommended that the sentences be reconnected throughout the manuscript and grouped into separate paragraphs. The language is poor, and the text is unclear to understand. I could find lots of non-English expressions. I recommend that the authors use easy explanations to promote the 'materials' readers to catch the intention of the authors. Also, revision by one who uses English as a native language is needed.

Line 109: The authors reported that the linear analysis conducted with commercial software show inconsistency in terms of predictability and repeatability. However, there is no reference to this, and there is no evidence to support the authors' opinions. Since this is the primary goal of this study, the authors should state the logical basis of this opinion more objectively and scientifically.

Line 115: The author explains that the purpose of this study is to evaluate the precision of various scanning systems, and it is also important to conduct it under in vivo situations. However, in line 128, the authors mentioned that they used the gypsum model of one of the authors as the subject of this study, which was previously acquired for other purposes. In this way, they didn't get the IRB approval, but on the contrary, this study is not a clinical study anymore. You need to explain more.

Materials and Methods

Line 130: It does not seem to be a clinical study and the experiment was conducted on a single tooth. Therefore it does not seem necessary to mention the exclusion criteria. It is considered that the dentoform tooth may be enough.

Line 134: This study evaluates the accuracy performance of scanners and it does not include the evaluation of the various properties of materials used in dentistry. Therefore, the contents of lines 134 through 141 do not seem to be important, and it may be shortened.

Line 135: Please, explain about the 'double impression technique'.

Line 142: 'sixth lower molar' is not a common term in dentistry. Please check if the English translation is correct.

Line 169: Previous studies provide more detailed information on the specifications of the scanners used in this study. It is recommended that a more detailed description be included in table 2.

Line 189: Describe what 'de facto' means in English.

Line 195: Please, let me know more about the algorithms used in this study. This is a key part of the study, but it is not easy to understand. To my understanding, the 'barycenter' of the reference data is calculated as a point, and it seems that the vector is calculated between the 'barycenter' point and that of the test group dataset. But it is not easy to understand how the 'volumetric error' is calculated. Identify whether this method was newly developed by the authors, and how it was introduced. Please, add references to this methodology.

Line 201: Please state only objective facts in the 'Materials and Methods' section. Remove the phrase 'in our opinions'.

Line 203: Identify the level of significance used in this statistical method.

Line 209 to 212: Move Table 3 to the 'Results' section. The content of the text is not the explanation of the statistical analysis either. Describe the statistical methodology used in this study.

Results

Line 224: Indicate the unit in the measurement presented. Figure 2: (d) is different from (a), (b), (c). Please, replace (d) with one which was captured at the same angle as the other images. Also, the scale bar shown on the right side of the color map is not the same among them, so I can not compare four pictures on the same basis. Adjust the scale to have the same maximum and minimum value.

Line 228: The teeth shown in Figures 2 and 3 look different. To my understanding, the authors got scan data of the first molar from one person. Explain why the shape of these teeth is different. If you scanned the multiple teeth, please give me the information about the sample size.

Line 233: The two sentences described by the author do not seem to be helpful in the description of the text. Modify it to match with the context.

Line 239: Figure 4 is the picture that can also be captured in Geomagic or 3-matic software. Please let me know why the authors presented this picture just for MATLAB group.

Line 244: Likewise, it is advisable to adjust the maximum and minimum values of the color scale bar to be the same. The result of (c) seems to be poor, but it seems to be reduced.

Discussion

At the former part of this section, the sentences are arranged as very short paragraphs, while the extremely long paragraph continues in the middle. It is recommended that the authors describe the text more logically and coherently. There is also an insufficient reference to existing studies in this research area. I also recommend that the authors consider the following studies;

- Renne, W.; Ludlow, M.; Fryml, J.; Schurch, Z.; Mennito, A.; Kessler, R.; Lauer, A. Evaluation of the accuracy of 7 digital scanners: An in vitro analysis based on 3-dimensional comparisons. J Prosthet Dent 2017,118:36-42.

- Kim, RJ.; Park, JM.; Shim, J.S. Accuracy of 9 intraoral scanners for complete-arch image acquisition: A qualitative and quantitative evaluation. J Prosthet Dent 2018;120:895-903.

- Nedelcu, R.; Olsson, P.; Nyström, I.; Thor, A. Finish line distinctness and accuracy in 7 intraoral scanners versus conventional impression: an in vitro descriptive comparison. BMC Oral Health 2018;18:27.

Line 280: The mean dimensional error is 50% higher and the standard deviation is 150% higher than the other group in the intraoral scanner group. Please, describe how this calculation came out and modify the table for the understanding of 'materials' readers. It is an important part of saying that the proposed analysis method is different from the existing methods. It is not easy to agree with the author's opinion because I could not understand the calculations the authors conducted.

Line 288: The authors described the variations in the results were due to the different methodologies. I cannot understand clearly why and how MATLAB algorithms were used for minimizing the point-to-surface distance, and why this procedure minimizes the volume and what is the effect and meaning of this procedure. Please describe it in easy-to-understand terms.

Line 323: Using MATLAB, the authors stated that the accuracy of the intraoral scanner was not good in the existing method, but, actually, it was not that bad. It is difficult for me to confirm this fact from the table shown in the result. Presenting raw data in the Appendix was an unfamiliar way for me. It is recommended that the authors make a more convincing statement.

Conclusions

Line 331: There was not enough explanation about why MATLAB was a reliable method and why the existing method was inconsistent. Please state more about the basis for this judgment clearly in the 'discussion' section. Otherwise, I can not agree with this conclusion.

Line 334: This is a study of how accurately an optical scanner reads the shape of a tooth. About the authors' mention of the future study regarding the kind of material, I doubt the effectiveness of this suggestion. And this sentence should be moved to the end of the 'discussion' section.

Author Response

The authors wish to thank the reviewers for his/her important contribution.

In particular, the authors appreciate the reviewer's effort aimed at improving the quality of the

paper and at increasing the clarity of the presentation.

The paper has been amended following all reviewer's suggestions, as better elucidated at the

following lists.

 Author Response File: Author Response.pdf

Round  2

Reviewer 2 Report

This is an interesting study about the new methodology for the accuracy evaluation by volumetric analysis in the dental field. 

However, the answers to the reviewer's questions were not accurately presented and are not reflected in the text. 

Some points of the study design, interpretation of results are not clear and will need improvement or clarification.

The language is poor, and the text is unclear to understand

This manuscript is not acceptable for publication in its present form.

Author Response

The authors wish to thank the reviewers for his/her important contribution.

In particular, the authors appreciate the reviewer's effort aimed at improving the quality of the

paper and at increasing the clarity of the presentation.

The paper has been amended following all reviewer's suggestions, as better elucidated at the

following lists.

Reviewers' comments:

 Reviewer #1: 

The article is interesting, however the volumetric analysis technique is already known. In dentistry the use of Geomagic for this controls is used and the results are proved. 

As indicated in the text Geomagic results are indicated in mm. This is an attempt to use volumetric analysys in dental field.

Several previous studies have used commercial programs to determine the trueness of intraoral scanners. In this, however, an attempt was made to apply a volumetric error analysis to the dental field. (line 256)

This study compared 4 comparison procedures and the conclusion is that MATLAB could be used. It is another type of analysis. However, it is essential to know the way of the alignment between the CMM and digitalized tooth.

The authors added in the text and in reference section a previous paper of the ssame group in wich validation and explanation of this technique was gave.

 S. Catalucci, R. Marsili, M. Moretti, and G. Rossi, "Comparison between point cloud processing techniques," Measurement: Journal of the International Measurement Confederation, vol. 127, pp. 221-226, 2018.

Each software has its own alignment algorithms and therefore it should be shown the alignment deviation in each case, before the making the comparisons.

The commercial software, as well as the one proposed in Matlab, uses iterative registration algorithms.

It is not possible to know in detail how to register.

Other comments: 

In the Abstract "precision" is used instead of "accuracy".

The term was replaced

This is an interesting study about the new methodology for the accuracy evaluation by volumetric analysis in the dental field. However, it is still not clear to me following matters as far as I read the manuscript.

Title

In this study, the phrase 'dental material impressions' is not appropriate because the single tooth is scanned with various scanners and the accuracy of the acquired data is compared. And the authors did not evaluate the data quality among different tooth material. Please, replace it with "dental impressions."

The statement was modified in the text

Abstract

Line 20: The purpose of this study is to confirm the validity of the volumetric error distribution by MATLAB algorithms, compared to the existing methods of evaluating the accuracy and precision of optical scanners used in the dental field. First of all, this study compares the data obtained from various optical scanners to evaluate the performance of them and seems to be distant from the scope of the 'materials' journal.

We agree with reviewer’s consideration. Starting from this point, as indicated in the text

“The comparison of different intraoral scanning systems is very important for the clinician for choosing the most suitable materials for prosthetic reconstruction (ceramic, metal, zirconia, etc.)” lines 110

This aspect is very important for a material choice, in our opinion

  Line 22: It is not clear what the author wants to say in this sentence. Please, make it clear which message you want to deliver, the limitation of this study, or the fact that it is also possible in the full arch model. As far as I know, it is not possible to acquire the complete-arch scan including the interproximal embrasure with CMM equipment. Also, the authors don't need to mention the limitations of this study in the abstract.

The authors removed the sentence in the abstract section, moving this consideration to limitation of the study as indicated below

“The limitation of this study is that some investigations must be done about full arch in term of volumetric analisys .312

 Line 36: No conclusions in the abstract. It is recommended that the authors clearly state what message they want to deliver in this study.

The following sentences were removed from the abstract

We have removed the claim “The most accurate method to compare trueness and precision of new digital impressions resulted to be the volumetric technique. In particular, D1 and I scanners had a higher accuracy respect to D2 and D3 scanners” ed aggiunto “

For the results of our study the volumetric technique can represent an excellent analysis system

 Line 37: The keywords do not seem to cover all of the representative contents of this study. It is recommended to include more specific words such as 'volumetric error distribution'.

volumetric error distribution (removed old word)

Introduction

Overall, the contents of the text are not organized well. It is recommended that the sentences be reconnected throughout the manuscript and grouped into separate paragraphs. The language is poor, and the text is unclear to understand. I could find lots of non-English expressions. I recommend that the authors use easy explanations to promote the 'materials' readers to catch the intention of the authors. Also, revision by one who uses English as a native language is needed.

The article, modified as requested from the reviewer was checked by a native English teacher

Line 109: The authors reported that the linear analysis conducted with commercial software show inconsistency in terms of predictability and repeatability. However, there is no reference to this, and there is no evidence to support the authors' opinions. Since this is the primary goal of this study, the authors should state the logical basis of this opinion more objectively and scientifically.

The characterization of the metrological performances of scanning systems pass through linear distance measurements. In the present study, linear distorting methotology is used for performance comparison, and a technique is used in parallel with these.

 Line 115: The author explains that the purpose of this study is to evaluate the precision of various scanning systems, and it is also important to conduct it under in vivo situations. However, in line 128, the authors mentioned that they used the gypsum model of one of the authors as the subject of this study, which was previously acquired for other purposes. In this way, they didn't get the IRB approval, but on the contrary, this study is not a clinical study anymore. You need to explain more.

We absolutely agree with the reviewer. The authors removed this part of materials and methods section.

Materials and Methods

Line 130: It does not seem to be a clinical study and the experiment was conducted on a single tooth. Therefore it does not seem necessary to mention the exclusion criteria. It is considered that the dentoform tooth may be enough.

The exclusion criteria were removed

Line 134: This study evaluates the accuracy performance of scanners and it does not include the evaluation of the various properties of materials used in dentistry. Therefore, the contents of lines 134 through 141 do not seem to be important, and it may be shortened.

The part were modified and shortened as requested and indicated below

Starting from a cast model an alginate imprint was obtained and a referring zirconium model produced. One of the authors with a complete dentition was recruited for a full arch imprinting, by using silicon material (Elite HD, Zhermack,Italy). The impressions were disinfected for 10 minutes (Impresept, 3M ESPE) and poured in type IV dental stone (Moldastone CN, Heraeus Kulzer).

 Line 135: Please, explain about the 'double impression technique'.

We changed this part, speaking about silicon material imprint

 Line 142: 'sixth lower molar' is not a common term in dentistry. Please check if the English translation is correct.

 Statement modified with A first inferior molar

Line 169: Previous studies provide more detailed information on the specifications of the scanners used in this study. It is recommended that a more detailed description be included in table 2.

The table 2 was modified as request by the reviewer

Line 189: Describe what 'de facto' means in English.

Modified the statement in is widely used line 175

 Line 195: Please, let me know more about the algorithms used in this study. This is a key part of the study, but it is not easy to understand. To my understanding, the 'barycenter' of the reference data is calculated as a point, and it seems that the vector is calculated between the 'barycenter' point and that of the test group dataset. But it is not easy to understand how the 'volumetric error' is calculated. Identify whether this method was newly developed by the authors, and how it was introduced. Please, add references to this methodology.

Citation insered about previous study and in the text by authors, line 478:

61. Catalucci, S., Marsili, R., Moretti, M., Rossi, G., Comparison between point cloud processing techniques, (2018) Measurement: Journal of the International Measurement Confederation, 127, pp. 221-226, DOI: 10.1016/j.measurement.2018.05.111    

 Line 201: Please state only objective facts in the 'Materials and Methods' section. Remove the phrase 'in our opinions'.

Statement removed

Line 203: Identify the level of significance used in this statistical method.

Level of significance added   line 194 e 195

Line 209 to 212: Move Table 3 to the 'Results' section. The content of the text is not the explanation of the statistical analysis either. Describe the statistical methodology used in this study.

The table 3 was replaced in result section

Results

Line 224: Indicate the unit in the measurement presented. Figure 2: (d) is different from (a), (b), (c). Please, replace (d) with one which was captured at the same angle as the other images. Also, the scale bar shown on the right side of the color map is not the same among them, so I can not compare four pictures on the same basis. Adjust the scale to have the same maximum and minimum value.

Figure 3 has been adjusted and an image has been added

 Line 228: The teeth shown in Figures 2 and 3 look different. To my understanding, the authors got scan data of the first molar from one person. Explain why the shape of these teeth is different. If you scanned the multiple teeth, please give me the information about the sample size.

To highlight the distribution of values there is an occlusal view of the same tooth

The images of figures 2 and 3 have all been modified.

 Line 233: The two sentences described by the author do not seem to be helpful in the description of the text. Modify it to match with the context.

Sentences modified

Line 239: Figure 4 is the picture that can also be captured in Geomagic or 3-matic software. Please let me know why the authors presented this picture just for MATLAB group.

Figure added for 3-matic

Line 244: Likewise, it is advisable to adjust the maximum and minimum values of the color scale bar to be the same. The result of (c) seems to be poor, but it seems to be reduced.

Figure improved

Discussion

At the former part of this section, the sentences are arranged as very short paragraphs, while the extremely long paragraph continues in the middle. It is recommended that the authors describe the text more logically and coherently. There is also an insufficient reference to existing studies in this research area. I also recommend that the authors consider the following studies;

- Renne, W.; Ludlow, M.; Fryml, J.; Schurch, Z.; Mennito, A.; Kessler, R.; Lauer, A. Evaluation of the accuracy of 7 digital scanners: An in vitro analysis based on 3-dimensional comparisons. J Prosthet Dent 2017,118:36-42.

- Kim, RJ.; Park, JM.; Shim, J.S. Accuracy of 9 intraoral scanners for complete-arch image acquisition: A qualitative and quantitative evaluation. J Prosthet Dent 2018;120:895-903.

- Nedelcu, R.; Olsson, P.; Nyström, I.; Thor, A. Finish line distinctness and accuracy in 7 intraoral scanners versus conventional impression: an in vitro descriptive comparison. BMC Oral Health 2018;18:27.

References added n. 58, 59 and 60

Line 280: The mean dimensional error is 50% higher and the standard deviation is 150% higher than the other group in the intraoral scanner group. Please, describe how this calculation came out and modify the table for the understanding of 'materials' readers. It is an important part of saying that the proposed analysis method is different from the existing methods. It is not easy to agree with the author's opinion because I could not understand the calculations the authors conducted.

Statement modified without misure values. Line 263 and 264

Line 288: The authors described the variations in the results were due to the different methodologies. I cannot understand clearly why and how MATLAB algorithms were used for minimizing the point-to-surface distance, and why this procedure minimizes the volume and what is the effect and meaning of this procedure. Please describe it in easy-to-understand terms.

The wording has been modified and clarified in the text. See line 304 - 309

  Line 323: Using MATLAB, the authors stated that the accuracy of the intraoral scanner was not good in the existing method, but, actually, it was not that bad. It is difficult for me to confirm this fact from the table shown in the result. Presenting raw data in the Appendix was an unfamiliar way for me. It is recommended that the authors make a more convincing statement.

Appendixes were modified

Conclusions

Line 331: There was not enough explanation about why MATLAB was a reliable method and why the existing method was inconsistent. Please state more about the basis for this judgment clearly in the 'discussion' section. Otherwise, I can not agree with this conclusion.

The conclusion section was completely rewritten . The important aspect in authors opinion is that commercial softwartes are predictable and suitable methods as referred in literature. This is a study about another volumetric technique that can not replace the other ones but that can help clinician in material and scanners selection.  Line 315 - 317

The following conclusions were drawn: all of the digital impression systems were able to measure the specific tooth structure; each tested system showed different levels of trueness and precision values; Commercial software are reliable methods to analyze accuracy and precision; A different approach based on volumetric error calculation has been proposed in addition to classical linear error calculation; results seems to demonstrate the good reliability of the procedure.

 Line 334: This is a study of how accurately an optical scanner reads the shape of a tooth. About the authors' mention of the future study regarding the kind of material, I doubt the effectiveness of this suggestion. And this sentence should be moved to the end of the 'discussion' section.

As requested the sentence was removed

Author Response File: Author Response.pdf