Energy Management of Hybrid UAV Based on Reinforcement Learning

Round 1

Reviewer 1 Report

Hybrid UAVs are highly topical. The solution proposed for Energy management is interesting and has potential. Unfortunately, the learning algorithm is described very briefly. I intuited, for example, what Q1 and Q2 mean. I also noticed inconsistency in the use of regular and italic style in mathematical formulas and pseudocode. 
A revision of the text would be needed to eliminate these shortcomings.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

Dear Authors,
Thank you for your attempt to present your work in this manuscript. As far as I understood, in this work, an energy management algorithm is used to optimize the consumption of energy in a hybrid UAV. The concept has a promising application; however, the paper is purely written and has many issues to convey the concept. Some of these grammatical and conceptual issues are listed below. These issues made it extremely hard to understand the merits of the work; therefore, the current format of the paper is not acceptable, and I decided to vote to reconsider this work after major revision. This, however, does not mean that the next attempt would be necessarily the same, and in the case of observing the same issues, I would reject the paper.

1. (Line 9, Page 1) Please, define what UAV stands for.
2. (Line 14, Page 1) Please, define what is RL
3. (Lines 25-28, Page 1) Please, rewrite the sentence (, When the number of training … in the actual flight control.)
4. (Line 33, Page 1) change “Due to” to “Owing to” (Due to is usually used for negative aspects)
5. (Line 37, Page 1) what does “power source is more relys too much on power …” mean?
6. (Lines 44-47, Page 1) The sentence “Due to the addition of battery, … the volume and weight of the whole machine.” does not make any sense. Please, rewrite this sentence.
7. (Lines 51-52, Page 2) change “rule-based and optimization algorithm” to “rule-based and optimization algorithms”.
8. (Line 57, Page 2) remove “a kind of”
9. (Line 66, Page 2) explain what is “disaster of dimensionality”.
10. (Line 67, Page 2) rewrite “which is also the online control strategy, it only …”.
11. (Lines 74-77, Page 2) rewrite “Because the application scope ….it is difficult to achieve satisfactory results.”
12. (Lines 87-88, Page 2) rewrite “It aims to prove that …. Intelligence in the future.”
13. (Lines 114-117, Page 2) rewrite “Similarly, the specific application mode of … and control effect.”
14. (Line 151, Page 3) what ECU stands for?
15. (Lines 166-168, Page 4) use “is” instead of “represents”.
16. (Figure 5, Page 6) change “Current/A”, “Voltage/V”, and “time/s” to “Current (A)”, “Voltage (V)”, and “Time (s)”, respectively.
17. (Figure 5, Page 6) increase the font size; it is not readable.
18. (EQ 4, Page 6) include U_t in the left-hand vector, and U_OCV(S_OC)-IR_0-U_e-U_d in the right-hand vector.
19. (Line 208, Page 6) change “HPPC(Hybrid Pulse Power Characteristic)” to “Hybrid Pulse Power Characteristic (HPPC)”
20. (Line 215, Page 7) find a better wording for “ground bench test”
21. (Line 227, Page 7) what does “one of reinforcement learning algorithm” mean? Modify the sentence.
22. (Line 229, Page 7) “current state s in” ?
23. (Line 231, Page 7) “rewarde”?
24. (Line 231, Page 7) “, The purpose of …”?!
25. (Page 7) Draw a diagram for what is explained in subsection 3.1.
26. (EQ 5, Page 8) change ”A” to “a” and define this variable.
27. (Lines 241-245, Pages 7-8) Rewrite the sentence (too long and hard to understand).
28. (Line 247, Page 8) “unknowable” or “unknown”?
29. (Line 254, Page 8) what does “MDP” stand for and what is its meaning?
30. (Lines 262 and 273, Page 8) “research object” or “research objective”?!
31. (Line 266, Page 8) “for comparison and research” or “for comparison and further studies”?
32. (Line 276, Page 8) what does it mean by “fixedly”, find a better word for it.
33. (EQ 9, Page 9) define “g” and “s” and explain why “g” is divided by “s”.
34. (Line 286, Page 8) add another subsection and explain the control strategy used in this work. t is very unclear what is done in this work.
35. (Line 303, Page 9) explain TPF: SxAxS -> [0,1]
36. (Lines 305-307, Page 9) rewrite the sentence “Put it another way, it learning …”
37. (Line 307, Page 9) use “S” instead of “s” in “Q(s,a)”.
38. (EQ 10, Page 9) use “a_t” instead of “A_t”
39. (Figure 9, Page 10) it is very unclear.
40. (Table 2, Page 10) the Pseudocode is not clear; following my previous comment, carefully explain your approach and modify Table 2 accordingly.
41. (Table 2, Page 10) Fully elaborate on the long-term and short-term scenarios and the conditions mentioned in this table.
42. (Line 344, Page 11) change “the curves is” to “the curves are”
43. (Line 367, Page 12) correct the sentence, “return area is are equal …”
44. (Figure 11, Page 12) It Is not a schematic diagram.
45. (Line 376, Pare 12) correct the wording “tens of times of …”
46. (Figure 12, Page 13) Increase the font size.
47. (Figure 13, Page 13) Change “Torque/Nm” and “Speed/rpm” to “Torque (Nm)” and “Speed (rpm)”.
48. (Line 404, Page 13) “remaining oil” or “remaining fuel”?
49. (Figure 14, Page 14) There are two Figure 14s, remove one.
50. (Figure 14, Page 14) Correct the units and increase the font size.
51. (Line 436, Page 14) “missio”?!
52. (Figure15) Similar to other figures, the units should not be written after the slash. Please write them in brackets and increase the font size.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

Dear Authors

Only one very good specialist in UAV can understand your work. In my opinion is to readapt manuscript to broader audience to get high impact

Now - explain why Markov and not other IA approaching? need to explain your selection in this study

Explain direct parameters in eq 5 - R for example, g in table 3 ( note under table) and more similar mathematics in text

Fig 2& 3; 4&5  in one and self explanatory or describe them in text under figures; also 6&7

Section 3 revise to be selfexplanatory reanalyze fig 8&9 table with pseudocode. Take in account targeted audience and impact

Conclusions- clarify this aspect

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 3 Report

Minor English, Revise wordiness 

Check the last paragraph - Conclusions or discussions?!

Take advise from editor

Author Response

Please see the attachment.