Optimizing System Reliability in Additive Manufacturing Using Physics-Informed Machine Learning

Round 1

Reviewer 1 Report

This paper proposes a machine learning (ML) method for predicting the system response and optimizing printing parameters in fused filament fabrication (FFF). This is an innovative use of ML to aid FFF prototyping and printing. The authors also claimed their approach as physics-informed machine learning (PIML), which is a trending research topic in ML and engineering. The paper has notable research value. But there are several major issues to be addressed. Case study results were not well presented, which also compromised the validity of this work.  

Major comments:

1. In introduction, the authors abruptly brought up PIML and indicated that PIML was a promising methodology for AM. But PIML was not defined nor explained. This may be confusing to readers who did not know PIML. Please define PIML concepts and introduce PIML models and applications briefly. There are some existing literature that may help:

• Shenghan Guo, Mohit Agarwal, Clayton Cooper, Qi Tian, Robert X. Gao, Weihong Guo, Y.B. Guo, Machine learning for metal additive manufacturing: Towards a physics-informed data-driven paradigm, Journal of Manufacturing Systems, Volume 62, 2022, Pages 145-163, ISSN 0278-6125, https://0-doi-org.brum.beds.ac.uk/10.1016/j.jmsy.2021.11.003.
• McGowan, E.; Gawade, V.; Guo, W. A Physics-Informed Convolutional Neural Network with Custom Loss Functions for Porosity Prediction in Laser Metal Deposition. Sensors2022, 22, 494. https://0-doi-org.brum.beds.ac.uk/10.3390/s22020494

 

2. Section 2 is essentially a method overview. The authors did well in reviewing the types of ML models they used for method development. However, it is suggested that the authors add a brief explanation (or overview) at the beginning of Section 2 to enlighten the readers why these models were adopted and how they were suitable for developing the solution in this study.

 

3. Domain knowledge (DK) was brought up in Section 1. There is no definition or explanation of DK. It is unclear what information would be considered as DK in this study.

 

4. The case study results are not provided. Section 4.1 only shows narrative content of the case study procedure, which is not enough. Please add the results from implementing the proposed method and then interpret them and discuss the effectiveness and validity of the method.

 

5. The proposed method is certainly based on ML models. However, how is it “physics-informed” ML? The models are more of conventional neural networks. Please clarify.

 

Minor comments:

1. In abstract, line 15, “… time and cost efficiently”, this sentence is grammatically unsmooth.

 

2. Page 2, line 43-45 seems like two sentences merged. Putting them separately will be better.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

The authors focussed to highlight the method to improve the quality of Fused Filament Fabrication (FFF) parts by applying ANNs. The following issues need to be addressed for further considerations.

 

1. Authors need to do critical review, with a focus on all individual parameters (print speed, bed temperature, printing time, print material, overhang angle etc.) affecting the quality of FFF process and parts. Critical review of individual parameters needs to be highlightened.
2. Highlight the cost-effective experimental methods for developing models, empirical relationships that could provide sufficient input-output data requirements for training neural networks.
3. Authors need to highlight the reverse modelling and their significance in practical cases for better quality parts.
4. Critical review on past studies of ANNs and artificial intelligence in optimizing the parts with process parameters need to be highlightened.
5. Significance of Design of experiments, their main and interaction effects in defining the process insights need to be highlightened.
6. ANNOVA, MANOVA and parameter contribution estimation with DOE need to be further elaborated.
7. Significance of hybrid ANN-artificial intelligence algorithm model concepts need to be addressed as well.
8. Critical review on ANN parameters (hidden neurons, hidden layers, feedback neurons, bias, activation functions, learning rate, alpha etc.) and requirements of tuning parameters and their influence on prediction accuracy need to be highlightened.
9. It would be better to propose the framework in graphical illustration for experimentation, developing neural models, and optimization.
10. Furthermore, scope for future work need to be highlightened.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

The article is a relevant subject but the results obtained have a low level.
So I recommend a major revision of the article.

The title of the article completely corresponds to it.
Comments to the article:

• The presented model is an important step, but its effectiveness has not been confirmed. The main comment is the lack of verification of the adequacy of the method of machine learning through experiments. The constructed figures 6 according to other authors do not give anything to readers. This is due to the fact that the authors do not check the results.
• Why is it a bad idea to check the results of other authors? Because the authors got their results on different 3D printers. And this technique is very sensitive to changes in design.
• Kinematics of the printer, type of extruder, type of blowing, method of measuring defects ... all this also does not take into account the model.
• Another parameter that will have a critical impact is the software (slicer) used to create the G-code. Even if the printer and all the settings are the same, but the slicer will be different you will get different results. Therefore, the authors must conduct the experiments themselves. And check the adequacy of the developed method of machine learning.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

The response and revisions from the authors are insufficient. The response is overly brief. The revisions did not address key issues related to method and case study.

For my comment 2, the revision is basically a list of what the author did but did not offer any useful information about why these things are necessary. 

Case study (my comment 4) remains a major concern about this work. The initial manuscript did not have case study result. The authors responded that the case study was not ready. I don't think this should be the status of a ready-to-publish paper. More work needs to be accomplished to demonstrate the developed method. Case study should be done before considering publication.

My comment 5 was not addressed either. It does not offer useful information by saying that the method combines physics and machine learning. How physics and machine learning were integrated is important. There is clear definition of "physics-informed machine learning". The authors should explain how the proposed method fits in the concept and definition of PIML.

Author Response

 

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

Congratualtions

Author Response

Much obliged!

Reviewer 3 Report

The author practically did not bring the article to the correct condition. The comments were only the fact that this is just the beginning of the implementation. However, since there are no normal research results, I think that the article should be rejected.

The authors responded to almost all comments in the following tone:
"We are at a very early stage of our method and the main focus is just on the print bed adhesion."
or
"We have taken your point into consideration, but at this point the method is still at its early stages and the purpose of our paper is to determine whether the method is promosing enough for further research."

Author Response

We regret that we have not been able to address your comments with our responses.

The reason why we were not able to address your comments, is because our work is primarily in basic research where the focus of university research lies and your comments 3 and 4 are aimed at application. The value of the paper lies in proposing a method for parameter selection that is superior to the popular approach. The popular approach is to change parameters as defects occur based on the known overall effects. Although our method has not been fully implemented yet, we are able to show by comparison, that it is superior to the popular approach (chapter 4.3). Based on this, a case study comparing the methods would not yield different results. We would like to publish the method under our name at this time, as first attempts already show that the method is superior to others. However, a case study can show the performance of the presented method and therefore is still being caried out and will be published as well.

Round 3

Reviewer 1 Report

I am okay with the authors' response to my points 2 and 5. But I am not convinced by the response of point 4. This paper is not rigorous and valid due to the missing case study.

This paper states that a physics-informed machine learning method was developed to solve the parameter selection problem in Additive Manufacturing (AM). However, the proposed method was not implemented or validated with real, or even simulated, AM data. This is unusual to machine learning (ML) related studies.

In AM, there have been plenty of ML-based methods developed for defect prediction and process control. Case study is crucial to demonstrate the validity and usefulness of a new ML (or PIML) model. Otherwise, why would people choose this model over the existing ones? How do people know that this method will work on real AM data?

The authors stated in the response that their Section 4.3 showed the superiority of their PIML model. Yet, again, Section 4.3 is pure narrative. Several strong statements were made, e.g., the proposed method can "setup time". Without quantitative results, such statements are not supported. Rigorous case study is needed to actually show the merits and superiority of the proposed PIML model before making these statements.

A final point is that many citations seem missing in Section 4.3. When the authors mention literature or existing approaches, citations should be provided to show the origins. 

Author Response

We are happy to have addressed your points 2 and 5.

Based on your remarks we have added a case study to support our hypotheses in our manuscript. We have also made it clear for the readers, that at this point of our research some of our statements in 4.3 cannot be fully validated.

We have not added more citations to 4.3 as the popular approach is mostly present in the non-scientific world, we had already cited a source that had a certain added value. (see [25]). Critical statements to the statistical approach are covered by the case study and the statement.” Deviations in print process could make all observations obsolete.” is something we think everybody could agree on.

Reviewer 3 Report

Unfortunately, the authors cannot convince that their paper can be accepted. Therefore, I recommend rejecting the manuscript.
1 - "The value of the paper lies in proposing a method for parameter selection that is superior to the popular approach." How did the authors prove it? The authors did not conduct research to show the results of their method.
2 - "Although our method has not been fully implemented yet, we are able to show by comparison, that it is superior to the popular approach" If the method is not fully implemented then what is the point of publishing it? Authors must complete the method and present the results of the comparison, then it can be published. The results of the comparison should include graphical, numerical confirmations.

Author Response

We are sorry we could not address your comments in our previous revision.

Based on your remarks we have added a case study to our manuscript to prove our hypotheses. We hope to show that our method is working by implementing it and comparing it to another ML approach. A graphical illustration has been added to our manuscript to show this comparison.

Round 4

Reviewer 1 Report

The added case study makes the work more solid and rigorous. There are some questions associated with the it:

1. What is the statistical approach used? Need to specify the name and the associated papers, as well as a brief summary of the method.

2. The relevance of this selected statistical approach with respect to the proposed PIML method needs to be summarized.

3. The data description of the case study is vague. A summary of experiment and data collection will make it stronger. More detailed description of data characteristics (e.g., sample size, data form, any cleaning procedure) and method training is needed.

Author Response

Thank you for your very helpful and pertinent comments.

We have added additional information to our text to adress your comments. We have explained the statistical approach used and its relevance for our proposed method. A more detailed summary of the experiments has been added and other detailed information.

Reviewer 3 Report

The authors reviewed the version of the article by adding subsection 4.4. However, the results obtained are practically not described. The choice of test sample and error parameters raises many questions. Therefore, I recommend a major revision.

Comment:
- What is the root mean square error of the authors determined (what indicator or size or defect)? What tools were used to determine the error? Did the authors calibrate the device after changing the filament coil, or was the filament in a special box?
- Why was the test cube chosen for the experiment? The test cube cannot check most of the parameters that changed during printing. Overhanging elements, openings, supports, bridges, most settings have an impact on these elements. Therefore, the use of a test cube as an experiment does not make sense.

Author Response

Thank you for your pertinent and helpfull comments. Based on your comments we have added additional information and explanations to our manuscript. Nevertheless, here are detailed answers.

During the print measurements the filament coil was not changed. The printers are in a special, temperature and humidity controlled room.

Our focus lies on the print bed adhesion and therefore the parameters directly influencing Overhangs, bridges, supports etc were not part of the case study (such as overhang-threshold, support-material-speed, …). In the linked data repository at figshare you can see in the file “parameter.info” the parameters that were used. We tried to stick to parameters which actually might change the print.