A Whole-Slide Image Managing Library Based on Fastai for Deep Learning in the Context of Histopathology: Two Use-Cases Explained

Round 1

Reviewer 1 Report

A whole slide image managing library based on fastai for deep learning in the context of histopathology: Two use-cases explained

In the proposed paper, the authors develop an image processing library and present its application on two prediction tasks for distinguishing types of neuropathologies based on histological image data using deep learning models. The presented work builds on previous research conducted by the authors, who aim to facilitate and standardize similar research in the future by publishing the developed code which was built on a well-known platform. It consists of many practical tools created to be used by researchers not experienced in technical details of deep learning or large image data handling. The main contribution of the article is the application of mostly previously developed pipelines on two novel use cases.

The article is generally sound and well-structured. However, it abounds with grammatical errors and typos. Style should be improved in accordance with the academic way of writing. Most paragraphs are difficult to follow and understand and should be rewritten. For instance, the abstract, first paragraph in the Introduction section, paragraph through lines 268-278, and others. Style should be improved, reducing the colloquial jargon and sentences like “So only if you zoom…” (line 196), “checkout the code” (line 208), starting the sentence with a number (line 325), avoiding choppiness (line 270, 271), and other. Also, refrain from citing Wikipedia, as it is often an aggregation of existing facts, and the matter is probably expressed in more detail elsewhere, potentially with added peer review assurance.

As the focus of the article is application of the pipeline for novel use cases, please provide relevant references of previous attempts for the similar classification or state that there were none to be found and provide a clear outline of what is new.

The evaluation of the models should be improved. What was the inter-rater variability of your baseline? Please consider adding Brier score as an evaluation metric, as it adds more information on you model performance. Also, consider using model calibration in prediction of the class probabilities to better match the expected distribution observed in the data. Both techniques could potentially strengthen your article.

Please provide some comments on misclassified tiles. Do they share some common feature? Was the tissue percentage relevant for the prediction?

What was the most useful predictive information in tile classification? Is it possible to implement some of the model explainability techniques, such as SHAP values? Please provide a comment on this and consider adding it to the paper to increase the interpretability of your results and potentially adding novel insights in pathological tissue classification.

Other:

Line 49 – please provide a few relevant references.

Line 614 – probably want to remove “please add”

Please consider reducing the paragraph on Otsu thresholding for brevity, as it is a well-known, basic image processing technique

Author Response

"Please see the attachment." 

Author Response File: Author Response.docx

Reviewer 2 Report

This article presents two case studies on histopathology based on deep learning.

In both cases, the authors used off-the-shelf ConvNets to perform diseases classification from hematoxylin and eosin (H&E) stained slides.

The work is interesting, well-motivated, and the results are outstanding. 
Moreover, the authors addressed two entirely novel issues at state of the art.

However, the manuscript presents some weaknesses that need to be addressed before publication.

1. The introduction should provide a profound overview of the study.
I think that it is unclear what unique challenges are associated with this task.
In addition, the introduction should also contain more details about the open research problems and the research challenges associated with these tasks.

2. The related work section does not exist, and, after a careful search, it is undoubtedly due to the novelty of the tasks faced. 
However, even though they are a few, I suggest the authors integrate into the introductory section, or in an appropriate section, to provide a general state of the art of the works that face similar issues, at least in the context of histopathology.
Some non-exhaustive examples are the following:
- https://0-www-ncbi-nlm-nih-gov.brum.beds.ac.uk/pmc/articles/PMC6257102/
- https://0-link-springer-com.brum.beds.ac.uk/chapter/10.1007/978-3-319-55524-9_14
- https://0-link-springer-com.brum.beds.ac.uk/chapter/10.1007/978-3-319-68560-1_31
- https://0-www-nature-com.brum.beds.ac.uk/articles/nature21056?spm=5176.100239.blogcont100708.20.u9mVh9
- https://jamanetwork.com/journals/jama/fullarticle/2588763/
- https://0-www-nature-com.brum.beds.ac.uk/articles/s41598-019-50587-1

3. In consequence of point #2, the authors should stress the research gap between this work and the limitations of other existing work, considering they are using well-known off-the-shelf networks.

4. I cannot find any motivation for the reason why the authors exploited ResNet50 and ResNeXt-101. Is there any specific reason? Would you mind offering fundamental explanations for these choices?

5. In Sec. 2.8, the authors decided to employ Data Augmentations to increment the number of samples and avoid overfitting. However, I think they should motivate if it can condition
the network to make correct predictions because, typically, this kind of
image has particular characteristics. Would you please give some motivation?

6. In the limitations subsection, the authors could also motivate the robustness based on the obtained results.
For example: can their proposal be applied to clinical practice?

7. A conclusion section is missing, but I think it is necessary to make a point of the work and give insights about the results. Here, the authors should emphasize the real advantages of their experimental results over existing ones and depict the future directions. I know that a specific subsection of Sec. 4 makes something similar, but the conclusion section is needed to conclude the work and make the pros and cons of the proposals.

Minor issues:
- I appreciate that the authors shared their code to make the experiments reproducible. However, there are too many links around the paper, even referred to the color spaces, which reduce the readability. Would you please insert them as footnotes?
- Is the dataset planned to be public realized? If not, it could be essential to add some images representing the use cases to depict the scenario and the difficulties faced by the authors.
- There are some English-language typos. I suggest a grammar check on the entire manuscript.

Author Response

"Please see the attachment." 

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

In the second review I can still see some minor errors in the text, such as missing commas. It would really improve the readability of the paper if the english could be polished a bit more.

A paragraph of interpretation of calibration plots could also add to your paper. When is the model overconfident, and when is it underconfident? What does that mean for your purpose. If you could reflect on this I believe it would add to your paper.

Author Response

Dear Reviewer,

thank you for your suggestions. We have reread the text and tried to address the language issues in a way that the manuscript has hopefully improved.

Also we addressed the issue of adding a paragraph of interpretation of the calibration plots in the results section at the end of the explanation of the results for the two use cases.

Reviewer 2 Report

Dear Authors,

you addressed the majority of my concerns.

However, I think that a related work section is needed for such a topic.

Author Response

Dear reviewer,

we are thankful for your suggestion. we have added a related work section to the paper.