Smart Phone-Based Motion Capture and Analysis: Importance of Operating Envelope Definition and Application to Clinical Use

Round 1

Reviewer 1 Report

Comments on the form

• Inconsistent units (spaces between numbers and units)
• All images show poor quality. Some images look like screenshots
• The tables should be converted from images into LaTeX tables
• Instead of using feet (nowadays only common in the US), the decimal metric system should be used 
• Not necessary to underline text: Use paragraphs instead (e.g. in lines 247 and 257)
• Line 162: The defined accuracy metric should be in its own equation numbered with (1). Also, ALL variables need to be explained/referenced in the surrounding text  
• Section 2.2 is formally missing
• Section 3.3. is formally missing 
• Line 399: Speed should be provided as distance over time, 96 bpm is a frequency

Comments on the presented content

• This study basically deals with object/marker tracking on 2D images, which is a very evolved discipline in computer vision. However, a review of the state-of-the-art, or at least related work, is completely missing. The only related reference is from 2014, since then a lot of work has been published in the field of marker-based and marker-less motion capture 
• The here presented system operates on 2D images only, where current related work in the field is able to derive 3D skeleton information, even on 2D images only. This should at least be mentioned in related work
• It would have been good to actually present real photographs of the system setup, in contrast to only showing synthetic images, especially of the markers
• The core functionality of the tracking algorithm is only explained in a few sentences with concrete references to Matlab functions. It would have been great to see actual footage of the video stream to get an impression of the difficulty level of the marker tracking, especially for the different lighting conditions
• The initial study and method, as presented in reference [16], is dealing with gait analysis on a treadmill and presents the system in use. This "follow-up" study uses health as motivation but is not actually presenting a medical use case. Instead, it is only evaluating a rather trivial image processing algorithm. Therefore, table 2 is not useful for the here presented study
• For tracking markers on 2D images using this image processing algorithm, the camera’s resolution is crucial. In the present study, the iPhone 6 model was used with a resolution of 1334 x 750 px. However, even the “affordable” iPhone X model from 2017 has an almost 3-times higher camera resolution of 2436×1125 px. An increased accuracy, even with a 4-year-old smartphone, can be expected but remains unknown
• The evaluation of the here presented methods seems to be valid. However, in order to accept this paper, all of the above-mentioned issues should be addressed. Also, a specific medical use case (e.g. gait analysis on a walkway, where evaluating the tracking distance would make sense) should be demonstrated to prove the system's usefulness

Author Response

Please see the attachment

Author Response File: Author Response.docx

Reviewer 2 Report

Defining the Operating Envelope of a Smart Phone-Based Mo-1 tion Capture and Analysis System

In a previous paper (reference 16), some of the authors present a system MO2CA for capturing motion.  Specifically, they affix a colored marker to human limbs (the heel) and show that they can track its motion using a smart phone camera (iPhone 7) if the motion is in a 2D plane, such as watching the subject run on a treadmill from the side.  They take the images and run some MATLAB functions o them to extract the motion of the marker as a sequence of x,y values.  They show that this approach works as well as much more expensive systems for this limited domain.  This is a testament to the quality of modern smart phone cameras.

The current paper determines the envelope over which this will work, distance, marker size, light intensity, through a series of experiments.

But this is information for what is now a fairly old iPhone.  At least they could have related the values to the specifications of the camera so that we will know the values in the future.  Or is Applied Sciences going to accept a new paper with the same set of experiments every time a new iPhone comes out?

The previous result was not surprising but perhaps useful.  For example, patients have been consulting with doctors over Zoom.  With the MO2CA "system", the doctor might be able to conduct a useful measurement of range of motion using the patient's cell phone.  The patient would need a tripod, however.

So how much does the current result help?  If the room is too dim for MO2CA, then it won't work, and the doctor will ask the patient to move to a brighter room or get some brighter lights.

What we need is a paper that shows that MO2CA is useful for telemedicine or some other setting where more expensive equipment is unavailable.  Is measuring motion in 2D medically useful?  That is the question.  They have already shown it can measure motion in 2D.

It seems well proof-read except perhaps right at the end:

large join
-->
large joint

Advance of this system offers the utility of fixed motion analysis systems in a personal, mobile, adaptable user-friendly format.
???
Advance --> Advantages?

Author Response

Please see attachement

Author Response File: Author Response.docx

Reviewer 3 Report

The paper is easy to read and well written. 

Good luck

Author Response

Please see attachment

Author Response File: Author Response.docx

Reviewer 4 Report

Thank you very much for sending the article titled: thank you very much for sending the article titled: Defining the Operating Envelope of a Smart Phone-Based Motion Capture and Analysis System. Generally, the paper is quite interesting to my mind, however, the authors should refer to the following statements:

• In the Introduction section authors describe analyzed human motion by using optical systems. Sometimes it is difficult provide experiment with this kind of system and it this case Inertial Measurement Units could be useful. I suggest describe wireless IMUs as a toll for this kind of experiment. These tool are useful because in simply way we can obtain kinematic parameters (position with Euler angles, angular velocity and acceleration). Moreover, with optical system it can be a great tool for compare these parameters. For example in article titled: A Kinematic Model of a Humanoid Lower Limb Exoskeleton with Hydraulic Actuators, Sensors 2020, authors obtained hip, knee and ankle angles, which were useful for calculation exoskeleton limb gait parameters. To improve article quality I suggest writing 3-4 sentences in Introduction or Discussion section (as a future research) with including this reference. 
• please add an experiment Flow chart
• line 286 - why authors use p-value as letter P?
• I suggest move subsections 4.1-4.4 to Introduction. Authors described a goal of the experiment.
• Limitation of the study and Next steps can be in discussion section (without subsections title)

Author Response

Please see attachment

Author Response File: Author Response.docx

Reviewer 5 Report

Defining the Operating Envelope of a Smart Phone-Based Motion Capture and Analysis System

The authors developed a system -- called Mobile Motion Capture (MO2CA) - employing a smart-phone and colored markers and define its operating envelope to enable human movements.

The paper's main contribution is the measurement of the quantitative characteristics of this system improving also perviously published results of some of the authors [16]. The measurements are concerned with basic envelop features such as the max distance calculations for establishing the overall operating range for different use cases and motions.

While neither the approach nor the algorithm provides any scientific novelty, the experimental results seem to be sound and the paper is overall well written.

I have one editorial suggestion prior to publication required for better readability:

While the paper discusses the operating envelope of the MO2CA system, the reader is not provided with a clear description of this sytem, i.e. of MO2CA itself, in particular the algorithms that are used. In the paper [16] which is referenced, the authors provide a description, however, it requires the reader to consult this paper and - as the information is spread over several sections in this paper - it is cumbersome to gather this information and to put everything together. Furthermore, as the use case(s) in [16] and the use cases in this paper are not exactly the same, the reader has to make some spurious assumptions to relate this information. Therefore,I strongly suggest to include a short, self-contained description of MO2CA in this paper describing not only the hardware but also the software algorithms in detail.

Author Response

Please see attachment

Author Response File: Author Response.docx

Round 2

Reviewer 2 Report

I complained that the paper did not establish medical relevance.  The authors added material on measuring range of motion.  That is acceptable.

I complained that the paper was limited to a single iPhone model, and it would not be appropriate to be allowed to submit a new paper every time a new iPhone came out.  The authors have reframed the paper as a PROTOCOL for measuring the effectiveness of any iPhone.  That is acceptable.

I complained that the single iPhone model they used was very old:  an iPhone 6.  They switched to an iPhone 8 but report the SAME numbers.  This is not credible.  The specs they mention on line 151 are for an iPhone 6 even though they update the text to say iPhone 8.  (On line 211, they forgot to update iPhone 6 to iPhone 8.)  Incidentally, the screen dimensions given on line 151 and 152 are not relevant, are they?  That has nothing to do with the camera.

So I question the ethics here.  They seem to be reporting iPhone 6 results but just changed the 6 to an 8 to answer my objection.

Since you are providing a "general approach for defining the 51 operational bounds of future video capture technologies that arise for potential clinical use", please apply that approach to at least two different models of iPhone.  According to line 140, you have iPhones 8, 10, and 13 available to you.  If you choose to reuse your original iPhone 6 data as one of the data sets, please identify it that way.

Author Response

See attached

Author Response File: Author Response.docx