Research on Contactless Bio-Signal Measurement Technology for Improving Social Awareness of Individuals with Communication Challenges

Round 1

Reviewer 1 Report

From the title, I expect a method is proposed for improving social awareness of individuals with communication challenges in this paper. From Abstract, bio-signals of patients with autism spectrum will be discussed. However, the authors only mention the acquisition of four types of bio-signals, and the experimental results include only heart rate and respiration rate. Four types of bio-signal acquisition are not shown to be superior to those of previous methods. I recommend that the paper be rejected because the validity of the whole system is not confirmed well, and the performance of the individual methods has not been sufficiently evaluated.

If the goal is to create a system like Fig. 20, this figure should be shown earlier, and the usefulness of this system should be fully verified. I am also concerned that the various estimation results in Fig. 20 doesn't look good enough. The heart rate and respiratory rate should not fluctuate as much as shown in the figure. It is not clear what kind of people participated in the experiment. I would like you to clarify what level of people the authors mean by "8 people with weak communication." It is also unclear what Fig 21 shows.

Although the individual estimation methods look to be revised from fully naive processes, it is not shown whether they are really superior to the state-of-the-art methods. If this could be shown enough, a paper could be structured only by the discussion. I cannot understand what kind of data and what kind of processing are shown in some of the figures, such as Figs. 1 and 2. The aspect ratio of some figures should be refined.

The explanation at the beginning is too broad for the proposed method. It should be a specific problem that can be solved by the end of the paper.

Author Response

Please see the attachment

Author Response File: Author Response.docx

Reviewer 2 Report

Although I believe the paper can be recommended for publication in its current form, I am convinced it could be improved by further describing the test group used in section 4.

Author Response

Please see the attachment

Author Response File: Author Response.docx

Reviewer 3 Report

A brief summary

The study describes a fascinating solution in order to collect and analyze the biometric data, including optical blood flow (heart rate), respiration, and interaction techniques such as facial expression, gaze, facial movement, and hand movement, with a contactless webcam level camera. It may help measure the bio states of individuals with communication weakness. But no literature reviews or evidence, the four kinds of biodata applied in the ASD or HFA research, is described in the manuscript.

Specific comments

• The need for a literature review of how the biometric data used for measuring the state data of the communication weak people should be raised in the introduction section or related works section.
• The need for a literature review of image-based measurement technology with non-contact or contact devices, including the optical blood flow signal acquisition, Respiration signal acquisition, Face feature point detection and facial expression recognition implementation, Gaze and facial movement tracking interaction, Hand movement tracking interaction, should be raised in related works section.
• The description of the statistical information about the error of measurement between non-contact and contact collecting from the experiment is a bit poor. What is the difference between 8 people with weak communication and 15 people in the control group? 
• Two Figure 1. 
• Most of the Figures were not mentioned in the context. Only Figure1 and Figure 12 were denoted in the context.
• Check out the text format of the Figures.  
• Reference 1 and Reference 2 were not mentioned in the context.
• The limitation section is totally missing.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 4 Report

Introduction:

The paper considers the use of bio-signal technology as a means of improving communication challenges in individuals on the autistic spectrum. It describes the use of image recognition tracking to acquire optical blood flow (for HRV), respiration, facial expression and gesture/interaction (based on gaze and hand movement). In general, I found the technical aspect of this work to be its greatest strength whilst the theory as it connects to the practical application to be its most limiting factor. The process of acquiring and processing raw bio-signal data is detailed, but the precise affective and cognitive states being inferred and the mechanism by which these interpretations are to be applied are unclear.

Originality/Novelty:

The use of biosensing technology in this particular way is certainly innovative and I am not presently aware of similar work, either published or in progress. Utilising non-invasive techniques via a single method of acquisition has limitations but the rationale for doing so is well-established and the leveraging of multiple signals within smartphone technology adds to the innovative value of the work.

Significance:

This is difficult to fully appraise due to the lack of detail provided on the application of the research. I feel that the authors could either reduce the references to ASD applications of this technology and explicitly emphasise the broader analytical significance of the underlying technical methods by placing them in the wider context of existing research and clearly identifying enhancements afforded by their approach. Otherwise, the alternative would be to add quite a bit of extra content to fully explain how this technology could interpret psychological state and how this could be leveraged to support autistic individuals. Personally, I feel that the first of these two options would produce the strongest paper.

Scientific soundness:

There are a few instances where important points are presented as common knowledge and not supported (e.g., increasing cases of ASD, L23-24). I would prefer such points to be evidenced, despite being well-known, as they are so foundational in justifying the need for the project. The introduction itself is limited to only a single source and, whilst I don’t expressly disagree with the points raised, I would argue that this section would be stronger with more consistent usage of citations to evidence these positions. This is particularly prevalent regarding the justification for the project’s broad approach, as there is no review of existing analysis, diagnostic or support methods against which the proposed work can be compared.

As noted in the introduction, there is comprehensive detail provided regarding the acquisition and processing (particularly noise filtering and frequency analysis), but the subsection for each signal ends at the point of physiological state determination (e.g. reaching a heart-rate variability measure from optical blood flow) but does not proceed to the psychological (affective/cognitive) stage. Facial recognition is the exception here and the paper does explain the process of interpreting discrete emotional states, but not how knowledge of these states will be applied.

Presentation quality:

As non-native speakers, the language of the paper is generally good but there are quite a few grammatical errors throughout and a review of these would be helpful. I don’t feel that there are any language issues that genuinely obscure the meaning of the text but addressing these errors would add some further polish and add a little more clarity also. The images used are helpful and most are well-captioned but the first two figures and figure 4 would benefit from being a higher resolution. Lastly, section 4 is a little too short and details of the methodology feel absent, making interpretation of the scientific robustness difficult.

Interest to readers:

I would argue that the work fits within the remit of the journal though its direct contribution may be more foundational/technical than applied at this stage.

Summary and notes:

Overall, the work is impressive with regards to its technical component but is somewhat undermined by its lacking in theoretical and applied grounding. Minor points aside, I would argue that the paper is publishable on condition that the issues raised in the ‘significance’ section of this review are addressed. I have identified this as a ‘minor revision’ as I feel it should be possible to add this content relatively quickly without conducting any extensive research, but I do emphasise that whilst the revision is minor, it is still critical.

• L8-9: “Communication” awkwardly appears as a sub-category of communication skills
• L62 and 89: Both figures are labelled “Figure 1”

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

Unfortunately, the impression of this manuscript is the same as the one before the revision. I recommend to reject this manuscript.

There are large gaps between the goals set, the proposed methods, and the experimental results shown. The individual techniques seem to be improvements on existing ones, but there is no verification that they have been improved. Among the four bio-signals to be extracted, (3) facial expression has not been confirmed to captured images with a webcam. There is no result shown for (4). Although subjects with ASD and ID seem to have participated, there is no indication that the system "can train and improve the social awareness". Before discussing the quality of individual techniques or the validity of the system as a whole, the story of the entire manuscript should be reconsidered.

The aspect ratio of many figures is not appropriate, and the figures degrade the appearance of the paper.

Author Response

Please see the attachment

Author Response File: Author Response.docx

Reviewer 3 Report

Thank you for rewriting long parts of the paper, which has improved the content to the point that it is now reasonably interesting to the reader.

Author Response

Thank you very much. The time given was very short.

Round 3

Reviewer 1 Report

Since the other three reviewers consider it worthy, I think it may be worthwhile. I will leave the decision of acceptance or rejection to the Associate Editor.

My opinion is the same as in the previous review. The title says "improving social awareness of individuals with communication challenges," however, I cannot confirm that this has been achieved. Therefore, I think the paper should not be accepted. I wrote "the story of the entire manuscript should be reconsidered" as a reason for rejection, not as a condition for acceptance. Limited time cannot be an excuse for not revising a manuscript.

At least the following points about appearance should be revised:

• Figure 1 has low resolution.
• The text in Figure 6 is stretched horizontally.
• The top of the frame in Figure 13 is unnaturally missing.
• The circles representing the samples in Figures 14 and 15 are stretched vertically.
• The text sizes in Figures 14 and 15 are unnaturally different.
• Figure 17 has low resolution.
• The captured image in Figure 21 is unnaturally stretched vertically. The same image can be seen in Figure 19, but it is not stretched vertically.
• The captured image on the left in Figure 25 is unnaturally stretched vertically.

Author Response

Please see the attachment

Author Response File: Author Response.docx