Evaluation of the Diagnostic Potential of Recombinant Coxiella burnetii Com1 in an ELISA for the Diagnosis of Q Fever in Sheep, Goats and Cattle
, Heinrich Neubauer 1 and Katja Mertens-Scholz 1,*Round 1
Reviewer 1 Report
This is overall a systematically executed and presented evaluation of the performance of a cloned immuno-dominant antigen for use as a diagnostic tool for detecting Coxiella infection in ruminants. A few comments/questions that may strengthen the report and/or broaden its interest:
(1) In the Introduction (lines 39-40) the authors note that symptomless and serologically negative animals shedding bacteria have been reported, and that missing these is a short-coming of existing diagnostics. Were samples from any such animals included in the set used in assay evaluation in this study, or are any such available to the authors? Demonstration that the described assay could detect these animals would broaden the interest in the potential applications of the new assay.
(2) The Materials and Methods are very thoroughly described. However, from the assay validation perspective, it would be of interest to know whether a set of samples was assayed repeatedly on 2 or ideally 3 different days. I.e., does the assay as described reproducibly give the same results for the same samples in independent assay runs?
(3) As a another means of defining assay sensitivity, and interpreting the selected cutoff values for defining positive results, was a standard curve performed using defined anti-Com1 antibodies spiked into normal sera at known concentrations, to estimate the level of circulating antibodies that produce a positive assay signal? If an appropriate anti-Com1 antibody is available, addition of this experiment would strengthen the benchmarking of the assay.
(4) The Figure numbers are inappropriately referenced in the text (Fig 1 is referred to as Fig 2 in the text, Fig 2 is referred to as Fig 3).
(5) The criteria by which serum samples were excluded from the study are rather vague and confusing (lines 263-267, 325-329, and 376-380). How were outliers defined as appropriate for exclusion? It seems odd that exactly four samples were excluded for precisely the same reasons for each of the three sample sets (sheep, goats, cattle).
(6) Line 291 references location c, but the preceding list of location identifiers (line 286) only includes numbers.
Author Response
Dear Reviewer,
we thank you for revising our manuscript and the very helpful comments and nice advice. We tried to answer all your questions and changed the text accordingly to your suggestions.
(1) In the Introduction (lines 39-40) the authors note that symptomless and serologically negative animals shedding bacteria have been reported, and that missing these is a short-coming of existing diagnostics. Were samples from any such animals included in the set used in assay evaluation in this study, or are any such available to the authors? Demonstration that the described assay could detect these animals would broaden the interest in the potential applications of the new assay.
This is indeed a very critical and interesting point. We unfortunately did not have the required amount of serum samples fulfilling the criteria of serological negative shedders. We could only include two serum samples from sheep, which are negative by commercial ELISA and positive by qPCR:
16QC1583 positive for antibodies for phase I and phase II, negative by IDEXX ELISA and qPCR positive, Com-1 negative
16QC1587 positive for antibodies for phase II only, negative by IDEXX ELISA, qPCR positive, Com-1 positive
It would be a very interesting project to collect this type of samples and compare commercial ELISA, phase-specific ELISA and reactivity of recombinant proteins. I think this will be a very promising approach for improvement of Q fever serology. Thank you for this helpful suggestion.
(2) The Materials and Methods are very thoroughly described. However, from the assay validation perspective, it would be of interest to know whether a set of samples was assayed repeatedly on 2 or unless otherwise stated ideally 3 different days. I.e., does the assay as described reproducibly give the same results for the same samples in independent assay runs?
We tested the majority of the sheep and goat sera repeatedly in a pilot trial over several days. The values we obtained were similar, but a higher variability was observed for the control sera on individual plates. We tried to avoid normalization and performed all species-specific analyses one day under the same conditions. If we pursue this ELISA further, also in combination with other epitopes we should include a normalization step and use e.g. a pool of positive and negative sera as controls.
(3) As a another means of defining assay sensitivity, and interpreting the selected cutoff values for defining positive results, was a standard curve performed using defined anti-Com1 antibodies spiked into normal sera at known concentrations, to estimate the level of circulating antibodies that produce a positive assay signal? If an appropriate anti-Com1 antibody is available, addition of this experiment would strengthen the benchmarking of the assay.
Yes indeed this would be a very good control, but unfortunately we do not have an anti-Com1 antibody and to my knowledge there is no commercial Com1-antibody available.
(4) The Figure numbers are inappropriately referenced in the text (Fig 1 is referred to as Fig 2 in the text, Fig 2 is referred to as Fig 3).
The numbering was changed.
(5) The criteria by which serum samples were excluded from the study are rather vague and confusing (lines 263-267, 325-329, and 376-380). How were outliers defined as appropriate for exclusion? It seems odd that exactly four samples were excluded for precisely the same reasons for each of the three sample sets (sheep, goats, cattle).
Yes, it seems a little odd, but two sera of each animal species gave us questionable results in the commercial ELISA. Because of this uncertainty, we excluded them from our sample set. We also eliminated serum samples with the highest and lowest OD-values per species as outliers.
(6) Line 291 references location c, but the preceding list of location identifiers (line 286) only includes numbers.
Changed to: Another false positive sample showed a positive result for phase II antibodies but was negative for phase I antibodies (location (6)) PhI-/PhII+, confirming…
Reviewer 2 Report
This study recombinantly expressed Com1 in E. coli, an outer membrane protein of Coxiella burnetii, to be used in indirect ELISA for the improved detection of Q fever in sheep, goats and cattle. Q fever is an important reproductive disease particularly of domestic ruminants, but is also zoonotic, causing often acute, sometimes chronic, illness in people that come in contact with infected animals. Commercially available indirect ELISAs are available for veterinary diagnosis, however, both the sensitivity and specificity of these tests vary considerably due to the use of different target proteins. The aim of the study was to evaluate recombinant Com1 as an antigen in ELISA to diagnose Q fever in domestic ruminants. Com1 has been used successfully to diagnose Q fever in human sera.
Overall, this manuscript presents a body of work that uses a protein target known to be useful in human diagnosis of Q fever and applied this to animals. The conclusions of the manuscript are consistent with the evidence and arguments presented. Namely, the authors observed “outstanding”, “excellent” and “acceptable” discrimination of goat, sheep and cattle sera using the Com1 ELISA and ROC and AUC analysis. The authors clearly indicate that poorer discrimination is likely due to the smaller numbers of animals used, and/or the variation in Com1 positive animals within each population, and that Com1, used together with other immunogenic proteins in a diagnostic test, is a promising candidate for the improved detection of Q fever in ruminants.
Specific comments:
Line 27: Move ‘were’ to after ‘sheep.
Line 150-151: You have repeated the Coomassie blue stain components.
Line 156: ‘Gently’ not ‘gentle’.
Line 168 & 171: On line 168 you use sheep, goats and cattle, but on line 171 you use ovine, caprine and bovine. Whichever you choose to use, please make it consistent throughout the paper.
Line 175: ‘The’ not ‘this’, and ‘result’ not ‘results’.
Line 181-192: You mention 3 different commercial ELISAs, but in your results you refer to the ‘commercial ELISA’ as your reference test. Which of these tests was used as the reference test?
Line 202: Were field sera samples tested in duplicate or triplicate?
Line 212: ‘Rectified’, do you mean corrected?
Line 214-215: Define p. Are you assessing diagnostic or analytical sensitivity and specificity? Your equation for sensitivity doesn’t appear to take false negatives into account, and although you have an equation to estimate false positives, that alone cannot be used to calculate specificity. As your methods need to repeatable, please update your sensitivity and specificity equations with the detail necessary for them to be repeated.
Line 252-255: Were the negative sera samples true negatives, i.e. ELISA confirmed by PCR or isolation? Were they sampled from true negative populations?
Line 271-273: You state that you only used one very high OD serum and one very low OD serum as your positive and negative controls. Why not use a pool of very high sera’s and very low sera’s as your controls? Wouldn’t this be a better representation of the population that you are testing to minimise single samples not falling within the range of the controls?
Line 301: Do you mean flocks, not herds?
Line 428-435: Why did you not assess specificity using Q fever true negative sera?
Line 501-503: You state “The results of this study show that Com1 can be used in conjunction with other C. burnetti-specific proteins…”. Please explain how you have demonstrated this when you did not compare these other proteins in your study.
Comments on Figures:
Figure 1: Do you require both shape and colour to differentiate between sheep, goat and cows? I would suggest to select one or the other to simplify the figure.
Comments on Tables:
Table 2: What is FLI? Are these results based on your testing, or from the NRL? Was a PCR or isolation used to confirm results (reference test)? What does ‘excluded because of lowest/highest density’ mean? Please include all relevant information and descriptions in the table heading so that it can be read without the need to refer to the body of the paper.
Author Response
Dear Reviewer,
we thank you for revising our manuscript and the very helpful comments and nice advice. We tried to answer all your questions and changed the text accordingly to your suggestions.
Line 27: Move ‘were’ to after ‘sheep.
Changed to: …specificities for sheep were 85 % …
Line 150-151: You have repeated the Coomassie blue stain components.
Changed to: … Coomassie blue-staining (20 % ethanol, 1.6 % phosphoric acid, 8 % ammoniumsulfate, 1 % Coomassie Brilliant Blue G250 (CBB G250; SERVA) in deionized water) overnight…
Line 156: ‘Gently’ not ‘gentle’.
Changed to: …temperature and gently shaken.
Line 168 & 171: On line 168 you use sheep, goats and cattle, but on line 171 you use ovine, caprine and bovine. Whichever you choose to use, please make it consistent throughout the paper.
Thank you for this advice. We changed ovine, caprine and bovine accordingly to sheep, goat and cattle throughout the manuscript.
Line 175: ‘The’ not ‘this’, and ‘result’ not ‘results’.
Changed to: …of C. burnetii in the herd as well as with a serological positive result in a commercial…
Line 181-192: You mention 3 different commercial ELISAs, but in your results you refer to the ‘commercial ELISA’ as your reference test. Which of these tests was used as the reference test?
Sorry for the confusion. We tested all sera with the IDEXX Q Fever Ab Test except 34 cattle sera obtained from the NRLs of tuberculosis (n=22) and chlamydiosis (n=12). These sera were tested using the ID Screen® Q Fever Indirect Multi-species ELISA and PrioCHECK Ruminant Q Fever Ab Plate Kit with the same results for each sample. We assumed that the results will be the same using the IDEXX ELISA and did not perform this assay.
Line 202: Were field sera samples tested in duplicate or triplicate?
Sorry, we forgot to mention this. We tested all sera in duplicate and changed the text to: … field sera from sheep, goats or cattle were added in duplicate per well in a dilution of…
Line 212: ‘Rectified’, do you mean corrected?
Changed to: …from Com1-ELISA were corrected by deletion of…
Line 214-215: Define p. Are you assessing diagnostic or analytical sensitivity and specificity? Your equation for sensitivity doesn’t appear to take false negatives into account, and although you have an equation to estimate false positives, that alone cannot be used to calculate specificity. As your methods need to repeatable, please update your sensitivity and specificity equations with the detail necessary for them to be repeated.
We calculated the diagnostic specificity and sensitivity and p refers to probability. We tested the majority of the sheep and goat sera repeatedly in a pilot trial over several days. The values we obtained were similar, but a higher variability was observed for the control sera on individual plates. We tried to avoid normalization and performed all species-specific analyses one day under the same conditions. If we pursue this ELISA further, also in combination with other epitopes we should include a normalization step and use e.g. a pool of positive and negative sera as controls.
Line 252-255: Were the negative sera samples true negatives, i.e. ELISA confirmed by PCR or isolation? Were they sampled from true negative populations?
This is an important topic and true negative sera are hard to find. We defined true negative sera as from apparently healthy animals, which are serological negative and PCR-negative. However, because of intermitted shedding patterns, the possibility of serological negative animals and the worldwide endemic status of Q fever, it is very difficult to identify truly negative animals. In our study we could include one sheep flock 17QC0185-0195 and one goat flock 17QC0739-0748, which were serological, and qPCR negative.
Line 271-273: You state that you only used one very high OD serum and one very low OD serum as your positive and negative controls. Why not use a pool of very high sera’s and very low sera’s as your controls? Wouldn’t this be a better representation of the population that you are testing to minimize single samples not falling within the range of the controls?
A pool of different sera as positive and negative controls would be a good alternative. Thank you.
Line 301: Do you mean flocks, not herds?
Changed to: The 10 sera with false negative results in the recombinant Com1-ELISA were from four geographically different located flocks (false negative sera/total samples per location: (1) 1/29; (5) 2/10; (6) 5/16; (8) 2/10).
Line 428-435: Why did you not assess specificity using Q fever true negative sera?
Unfortunately, we only have true negative serum samples from one flock of sheep and one flock of goats. This set of samples is too small.
Line 501-503: You state “The results of this study show that Com1 can be used in conjunction with other C. burnetti-specific proteins…”. Please explain how you have demonstrated this when you did not compare these other proteins in your study.
We did not test other proteins and rephrased the sentence accordingly. Sorry for the confusion.
Comments on Figures:
Figure 1: Do you require both shape and colour to differentiate between sheep, goat and cows? I would suggest to select one or the other to simplify the figure.
We included both colors and shapes to have a choice between color printing and black and white printing.
Comments on Tables:
Table 2: What is FLI? Are these results based on your testing, or from the NRL? Was a PCR or isolation used to confirm results (reference test)? What does ‘excluded because of lowest/highest density’ mean? Please include all relevant information and descriptions in the table heading so that it can be read without the need to refer to the body of the paper.
All in table A1 listed ELISA results designated as “result commercial ELISA”, “result phase I antibodies” and “result phase Ii antibodies” were performed by the NRL for Q fever and working group of rickettsiology. For confirmation of positive samples we only had results/available material from 76 samples, whereof 45 were positive and 31 were negative by qPCR. We added these results in the supplemental table. We eliminated serum samples with the highest and lowest OD-values per species as outliers.
Reviewer 3 Report
The authors here present a study that aims to evaluate recombinant Com1 as an antigen in ELISA to diagnose Q fever in domestic ruminants. This protein which is an outer membrane protein of Coxiella burnetii was recombinantly expressed in E. coli, to be used in indirect ELISA for the improved detection of Q fever in sheep, goats and cattle. In this study, the authors also compared the newly designed serological assay to the commercially available ELISA kits that are available for veterinary diagnosis. The authors provide substantial evidence that suggests this new diagnostic tool is an excellent way to not only screen for Q fever in animals but also in humans.
Minor comments:
Line 27: The sensitivities and specificities for sheep were.
Line 30: recombinant Com1 can be provide; delete be
Line 149-151: Repetition of the components.
Author Response
Dear Reviewer,
we thank you for revising our manuscript and the very helpful comments and nice advice. We tried to answer all your questions and changed the text accordingly to your suggestions.
Line 27: The sensitivities and specificities for sheep were.
Changed to: The sensitivities and specificities for sheep were 85 % and 68 %
Line 30: recombinant Com1 can be provide; delete be
Changed to: In summary, recombinant Com1 can provide a basis for more sensitive and specific diagnostic tools in veterinary medicine.
Line 149-151: Repetition of the components.
Changed to: … Coomassie blue-staining (20 % ethanol, 1.6 % phosphoric acid, 8 % ammoniumsulfate, 1 % Coomassie Brilliant Blue G250 (CBB G250; SERVA) in deionized water) overnight…
Reviewer 4 Report
The method presented by the authors is an alternative to an ELISA test, with the aim of achieving more accurate infection results.
The work presented has a great design and the methodology has been detailed. Although the design should be confirmed with the possible genetic variation of the protein against which the test was designed, for this reason it would be convenient if the analysis was always accompanied by the methodology that has been carried out so far.
I end by congratulating the authors on their great work.
Author Response
Dear Reviewer,
we thank you for revising our manuscript and the very helpful comments and nice advice.
No comments were made.
