Next Article in Journal
Effects of Clockwise and Counterclockwise Job Shift Work Rotation on Sleep and Work-Life Balance on Hospital Nurses
Next Article in Special Issue
The Health Consequences of Social Mobility in Contemporary China
Previous Article in Journal
Relationship of Time-Activity-Adjusted Particle Number Concentration with Blood Pressure
Previous Article in Special Issue
Determinants of Health Care-Seeking Delay among Tuberculosis Patients in Rural Area of Central China
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

The Quality of Tuberculosis Care in Urban Migrant Clinics in China

1
School of Economics and Management, Northwest University, Xi’an 710069, China
2
Department of Health, Sport, and Exercise Sciences, School of Education, University of Kansas, Lawrence, KS 66045, USA
3
Center for Experimental Economics in Education, Shaanxi Normal University, Xi’an 710127, China
4
Faculty of Liberal Arts, Northwest University, Xi’an 710069, China
5
Rural Education Action Program, Freeman Spogli Institute for International Studies, Stanford University, Stanford, CA 94305, USA
6
School of Economics and Management, Wuhan University, Wuhan 430072, China
7
School of Economics, Northwest University of Political Science and Law, Xi’an 710122, China
8
School of Economics and Finance, Xi’an Jiaotong University, Xi’an 710061, China
9
Institute of Social Medicine and Health Administration, School of Public Health, Shandong University, Jinan 250012, China
10
Department of Health Policy and Management, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
*
Authors to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2018, 15(9), 2037; https://0-doi-org.brum.beds.ac.uk/10.3390/ijerph15092037
Submission received: 9 August 2018 / Revised: 11 September 2018 / Accepted: 13 September 2018 / Published: 18 September 2018
(This article belongs to the Special Issue Poverty, Inequality and Public Health in China)

Abstract

:
Large and increasing numbers of rural-to-urban migrants provided new challenges for tuberculosis control in large cities in China and increased the need for high quality tuberculosis care delivered by clinics in urban migrant communities. Based on a household survey in migrant communities, we selected and separated clinics into those that mainly serve migrants and those that mainly serve local residents. Using standardized patients, this study provided an objective comparison of the quality of tuberculosis care delivered by both types of clinics and examined factors related to quality care. Only 27% (95% confidence interval (CI) 14–46) of cases were correctly managed in migrant clinics, which is significantly worse than it in local clinics (50%, 95% CI 28–72). Clinicians with a base salary were 41 percentage points more likely to demonstrate better case management. Furthermore, clinicians with upper secondary or higher education level charged 20 RMB lower out of pocket fees than less-educated clinicians. In conclusion, the quality of tuberculosis care accessed by migrants was very poor and policies to improve the quality should be prioritized in current health reforms. Providing a base salary was a possible way to improve quality of care and increasing the education attainment of urban community clinicians might reduce the heavy barrier of medical expenses for migrants

1. Introduction

Major improvements have been made in tuberculosis (TB) management in China during recent decades. Prevalence of smear-positive tuberculosis (TB) and TB-related mortality in China have been on the decline, decreasing by 63% and 80%, respectively, from 1990 to 2010 [1]. This is in the wake of several government health reforms, including the scale up of tuberculosis control programs and the introduction of three major health insurance programs that have led to more than 95% of Chinese citizens possessing basic coverage [1,2]. However, despite recent progress in improving health care delivery, China still has the third highest number of TB incident cases in the world [3]. Therefore, TB management in China remains a priority for international and Chinese public health teams. China still has a long way to go to achieve the goal outlined in the new post-2015 End TB Strategy by 2025 [4].
About 247 million migrants have moved from rural to urban areas for higher-wage jobs, and this population of migrants comprises a significant and increasing portion of the total national population in China (18%), which presents new challenges for TB control in large cities [5,6,7]. The rapid influx of migrants, combined with inadequate investment in fundamental public health infrastructure in these communities, has created high incidence rates and low detection and treatment rates for infectious diseases including TB [8,9]. Management of TB in migrant communities presents a unique challenge for several reasons. First, migrants usually share crowded living conditions with poor sanitation [8]. Second, due to limitations of the household registration system (Hukou), many migrants lack medical insurance and pay out of pocket for care, leading migrants facing budget constraints to choose less expensive private and community-level clinics over other sources of care [10]. Migrants also lack knowledge about the disease and have a greater delay between the onset of symptoms and clinical diagnosis [11,12,13,14]. Limited leisure time and absence of sick days aggravate poor healthcare utilization by migrants [15]. These factors contribute to the prevalence of TB in migrant communities increase the need for high quality care for this population. It is essential that clinics in urban migrant communities adequately diagnose and treat tuberculosis to reduce inequities in healthcare utilization [16].
Studies using patient satisfaction surveys have shown migrants to be less satisfied with primary care in comparison to local residents [17]. Other studies using patient perception of health service quality found that migrants gave higher scores for tertiary and secondary hospitals than for primary care clinics located in their communities [18,19]. Although patient satisfaction and perception of quality are important, they do not provide objective measures of quality because satisfaction and perception are subjective and patient recall may be biased. In addition, varying interpretations of quality of care due to patients’ differing medical knowledge, different disease cases and other characteristics make it very hard to compare the quality of care between clinics. However, previous research has rarely taken a relatively objective approach to examining quality of care in urban clinics [20,21,22]. Even less research has focused on health care in urban migrant communities [6,7,23].
This study uses standardized patients (SPs) to avoid the influence of differences in patient disease, patient characteristics and recall bias when comparing quality between different clinicians. SPs were extensively trained to present a specific, consistent case of a certain illness to multiple unsuspecting clinicians in order to measure how clinicians act in organic interactions. SPs have been used to empirically evaluate clinical practice in developed countries since the 1950s [24]. In the past decade, SP methodology has been applied to medical practice in the developing world [25,26,27,28,29,30] and has been previously validated in studies of quality care in China [22,23,31]. SP methodology offers unique advantages compared to other methods of evaluating clinical practice [28,32,33,34]. The first advantage is that observing clinicians who are not aware of being observed produces a more accurate evaluation of their practice [22,34,35]. Another advantage is the ability to compare results across clinicians since the disease is presented in the same way at every visit.
The main goal of this paper is to provide objective evidence on the quality of TB care in urban migrant community clinics. We approach this goal in three ways. First, we describe the characteristics of clinics in migrant communities. We compare differences between clinics that mainly serve migrant workers (here on referred to as “migrant clinics”) and clinics that mainly serve local urban residents (referred to as “local clinics”). Second, we evaluate the quality of TB care provided by both migrant and local clinics using SP visits. To interpret the results of SP visits, we compare the observed practice of clinicians to the International Standards for Tuberculosis Care (ISTC) and Chinese national standards for the diagnosis and treatment of tuberculosis [36,37,38]. Finally, we look at factors that influence clinicians’ practice. In addition to type of clinic, we look at how clinician characteristics such as education, salary, and facility characteristics can lead to high or low quality of health care.

2. Materials and Methods

Ethical approval: Approval was granted by the institutional review boards of Stanford University, USA (protocol number: 25904) and Sichuan University, China (protocol number: K2015025). All clinicians who participated in the study gave informed verbal consent during the initial facility survey conducted approximately ten days before SP visits. Clinicians were asked to consent to visits by SPs “at some point in the next month”. Every individual in the role of SP was given instructions on how to avoid invasive tests or procedures.

2.1. Selection of Facilities, Data Collection, and Study Size

The results presented here were from a cross-sectional survey on quality of care in urban migrant communities of Shaanxi province in northwestern China. The sample for this study was drawn from one prefecture (the administrative level below the province and above the county) in Shaanxi and the sample prefecture has an overall prevalence of TB (urban and rural combined) on par with the national average [39].
The 48 clinics included in this study were selected from two of nine total urban districts in the selected prefecture. Twenty-four migrant communities were chosen out of a list of migrant communities suggested by locals in two selected districts. Since both migrants and locals live together in these communities, we wanted our sample to include both clinics that mainly serve migrant populations (migrant clinics) and clinics that mainly serve local populations (local clinics). To do so, we used a random sampling procedure to identify sample households in each migrant community. After ascertaining the location of the community committee, we used a website to randomly generate another latitude and longitude point within one kilometer of the community committee, and went to the household located nearest to this point to conduct a short household survey. After finishing this survey, our enumerators were instructed to exit the house, turn left, and count three households down the road. This household then became our next sample household. This procedure continued until we had five migrant and five local residents in each community. For example, if the first five households were all migrants, any migrant households after that would be skipped and the procedure would continue until five local resident households were found. To generate a list of candidate clinics, we asked the respondents to name the top three clinics or hospitals they usually visited when they were sick. Based on the clinic list, we contacted clinics with the highest frequency of responses until we found two clinics in each community (48 clinics in 24 communities) that agreed to be visited by SPs at some point in the next month. The communities represented by this sample serve a population of 475,000 migrants [40].
We conducted three separate waves of data collection (Figure 1). The first wave of data collection consisted of an initial short facility survey that was conducted in early July 2015. Clinician consent for SP visits was obtained at this time. Actual SP visits started ten days after the initial survey in late July 2015. SPs simulating symptoms of TB visited clinicians in all sampled clinics. SPs were randomly assigned to clinics. Within each clinic, SPs visited the clinician following the normal procedures for any walk-in patient. Given a choice of which clinician to visit, SPs randomly chose a clinician following a pre-determined randomization protocol. Our study was therefore designed to represent the care a walk-in patient would receive at each of the sampled clinics. Finally, in October 2015, we conducted a follow-up survey with those clinicians visited by SPs. In this survey, we asked clinicians whether they had detected any SPs and administered a detail facility and clinician survey. This survey collected information including number of patients, number of clinicians, total value of equipment in facility, and clinician’s age, gender, education, medical certification, and salary.

2.2. Standardized Patients

The SP case used in this study was adapted to our previous study [22]. In this presumptive TB case, the SP presents with a “cough that is not getting better, and a fever”. Upon appropriate questioning by the clinician, the SP reveals symptoms consistent with a classic case of presumed tuberculosis including a cough duration of 2–3 weeks, fever with night sweats and loss of appetite and weight. The SPs, although recruited specifically to fit the TB case in terms of health and physical characteristics, differed in age, gender, height and weight. Our recruitment standard for TB SPs was that they be around 35 years old, of average weight and height, and healthy with no obvious signs of illness or other conditions that could prejudice diagnoses. The average age of recruited SPs was 37; the youngest was 28 and the oldest 43. Twenty-four SPs (thirteen male, eleven female) were selected after intensively training over a two-week period to consistently present the case to clinicians in the sample.
Following the interaction, SPs purchased all medications prescribed and paid clinicians their usual fee. After each visit, SPs were debriefed using a structured questionnaire and SP responses were confirmed against a recording of the interaction taken using a concealed recording device. Our previous study provides further details and the SP case script (in Mandarin with an English translation) [22].

2.3. Outcomes

Whether the case was correctly managed and evaluated was assessed in comparison to national standards for TB Care in China and the International Standards for TB Care, 3rd Edition [36,37,38]. The principal outcome of interest was correct case management, which we followed previous researchers Das et al. and Sylvia et al. in defining as the meeting of one of three criteria: referral (either verbal or written) to an upper level health system provider or the Chinese Center for Disease Control and Prevention; recommendation for a chest radiograph; or recommendation for further sputum testing for TB (i.e. smear microscopy, PCR, or culture) [22,26]. The reason that we chose a lenient definition of correct management is that unnecessary and/or harmful medicines are not penalized—a stricter definition would exclude from correct management those cases where additional (unnecessary or even harmful) medicines were given, reducing the proportion of correct management by a significant margin. However, we report the use of (unnecessary) antibiotics, and fluoroquinolones. Antimicrobial resistance is a severe public health concern in China while fluoroquinolones tend to mask underlying TB [41,42,43]. We also collected information on the charged fees of each visit. Since all SPs did not have any medical insurance during visits, all fees charged by clinicians were out of pocket.
In addition to whether SPs were correctly managed, we also assess the clinicians’ adherence to a pre-specified checklist of recommended questions and examinations. We also indicate a short list of “essential” items according to experts from the Center of Tuberculosis Control, Chinese Center for Disease Control and Prevention.

2.4. Statistical Analysis

Since we want to compare quality of TB care between migrant clinics and local clinics, an appropriate criterion to distinguish from clinic types is very important. We used data from the household survey in migrant communities. If one clinic was only mentioned by either migrants or local residents in their list of top three frequently visited clinics in the community, the clinic was defined a migrant clinic or a local clinic, respectively. However, some clinics were mentioned by both types of households. For those clinics, we counted the number of times each clinic was mentioned as a households’ first choice. A clinic was defined as a migrant clinic if it was mentioned more times by migrant workers than local residents, and vice versa. If there was a tie for first choice clinics, we counted the second choice, and then the third choice. Using this criteria, all 48 clinics were divided into 30 migrant clinics and 18 local clinics.
We calculated the proportion or mean and 95% CI by facility type (migrant or local) for correct case management and its individual components (referral, chest radiograph or sputum test) as well as for the use of antibiotics and fluoroquinolones, and the number and percent of checklist items completed.
To assess the difference in case management between clinic types and SP interactions, we used logistic regressions for binary dependent variables and Ordinary Least Squares (OLS) regressions for continuous dependent variables. We also used logistic and OLS regressions to assess correlations between clinician and facility characteristics and case management outcomes. Two models were used in correlations analysis. The first model included the type of clinic and clinician characteristics: age, gender, education, physician certification and whether the clinician received a base salary. The second model additional included facility characteristics: the number of physicians in the facility, the total value of equipment and whether the building was owned by clinicians. We report these results as margins effects in logistic regressions and coefficients with standard errors in OLS regressions. All analyses were done using Stata 14 (Stata Corporation, College Station, TX, USA).

3. Results

Completion rates for each phase of data collection are shown in Figure 1. The short consent survey was completed in all 48 clinics included in the sample. Out of 48 attempted SP interactions, 42 were successfully completed (16 in local clinics and 26 in migrant clinics). No clinicians were present at the time of SP visits six times in sampled clinics. Of the 42 completed SP interactions, matching facility survey was all completed, and clinician survey was completed for 40.

3.1. Primary Description of Facilities

The facility survey suggested better facility infrastructure at local clinics in comparison of migrant clinics (Table 1). For instance, although on overage, migrant clinics had 0.34 more full-time staff, there were 0.5 more full-time clinicians in local clinics. Similarly, the total value of equipment was higher in local clinics, at 14,800 RMB, compared to 10,100 RMB in migrant clinics. In addition, 56% clinicians in local clinicians received tuberculosis-specific training in 2014 compared to 33% in migrant communities.
In contrast, migrant clinics received many more patients last year than local clinics: 9153 versus 5858. Salary paid to clinicians in migrant clinics was 7150 RMB per month, compared to only 3060 RMB in local clinics. Furthermore, 88% clinicians in migrant clinics had an upper secondary or higher education attainment, compared to 69% in migrant clinics. There was no large difference in clinicians’ age, physician certification or received base salary.

3.2. Case Management of SPs

Table 2 shows the outcomes of interactions between SPs and clinicians in migrant communities. Overall, 15 out of 42 (36%, 95% CI 23–51) cases were correctly managed at clinics in migrant communities. Patients were considered correctly managed if the clinician suggested a chest radiograph (29%, 95% CI 17–44) or a sputum smear test (2%, 95% CI 0–12), or made a referral (29%, 95% CI 17–44). Though not considered incorrect in our lenient definition of correct management, frequent prescriptions of antibiotics unrelated to the treatment of TB were common, occurring in 57% (95% CI 42–71) of interactions, as were 5% (95% CI 1–16) prescriptions for fluoroquinolone. Patients were charged an average of 15.31 (95% CI 8.25–22.1) RMB (2.36 USD). Clinicians spent 9.44 (95% CI 7.23–11.65) minutes on average for the interactions with SPs, and only completed 17% (95% CI 14–20) of checklist items based on ISTC standard and 21% (95% CI 18–25) for China national standard.
As shown in Table 2 and Figure 2, quality of TB care provided by local clinics for SPs was significantly better than in migrant clinics. SPs were correctly managed in eight out of 16 interactions (50%, 95% CI 28–74), compared to seven out of 26 interactions in migrant clinics (27%, 95% CI 14–46). This higher correct management rate in local clinics compared to migrant clinics mainly came from suggesting a chest radiograph (38% versus 23%) and referring SPs to higher level hospitals (38% versus 23%). Although unnecessary prescription, because there was no anti-TB medicine available in clinic, were highly frequent in both types of clinics, clinicians in migrant clinics significantly gave more unnecessary prescriptions than those in local clinics: 73% (95% CI 54–86) versus 50% (95% CI 28–72). In addition, the combined out of pocket fees of consultation and medicines was significantly higher in migrant clinics at 18.62 (95% CI 8.82–28.41) RMB (2.86 USD), compared to 9.94 (1.21–18.67) RMB (1.53 USD) in local clinics. Furthermore, clinicians in local clinics asked slightly more questions and performed more examinations during the consultation process according to both ISTC and China national standards than clinicians in migrant clinics.
Completion of each checklist item is shown in Figure 3 and Table A1. More than half of clinicians in both types of clinics performed auscultations, measured temperature, and inquired about cough duration and medications taken. The most common checklist item completed was cough duration, one of four items considered essential questions, asked by 81% of clinicians in both types of clinics. However, two checklist items indicated as essential questions—past tuberculosis and family tuberculosis—were not asked by any clinicians. Fifty percent of clinicians in migrant clinics asked about the production of sputum, another essential question, versus 62% of clinicians at local clinics. Furthermore, questions about fever duration were asked by 38% of clinicians at migrant clinics, which is much less compared to 50% of clinicians in local clinics.

3.3. Correlations Between the Quality of Tb Care and Clinic Characteristics

Table 3 shows the factors correlated with quality of TB care. After controlling for clinic types, clinicians with upper secondary or higher education level charged 20 RMB out of pocket fees lower than less-educated clinicians, which is even larger than the difference caused by clinics types, at 13.30 RMB. In addition, clinicians with a base salary were 41% more likely to demonstrate better case management which was mainly contributed by being 48% more likely to refer patients to higher level hospitals, and significantly charged 22 RMB lower out of pocket fees, compared to those clinicians with salary all composed from medical consultation and medicine sales. In addition, in clinics with more clinicians, SPs were more likely to be referred to higher-level hospitals, while SPs were less likely to be referred if the total value of the equipment in the clinic was higher.

4. Discussion

Our study used SPs to evaluate the management of TB among healthcare clinicians in rural-to-urban migrant communities in China. The rate of correct case management detected by our study was 36%, indicative of poor quality TB care. Clinicians indicated a preference for radiograph over microbiological testing in contrast to international and national recommendations for TB diagnosis, as radiograph often yield significant false positives [36,37,38]. The poor quality of TB management we found is consistent with results of a previous study evaluating the same disease case in rural China [22]. Poor-quality TB case management contributes significantly to the problem of delayed detection of TB in rural and urban China [11,12,14].
Deficits in TB care were significantly different between migrant and local clinics, consistent with previous literature studying patient perception of care for migrants [18,19]. Comparing to previous literature studying the quality of TB care in rural China, although the rates of correct case management was higher in local clinics than in township health centers which make up the middle tier of China’s rural health system, the rates of correct management in migrant clinics was even lower than rates in rural village clinics, meaning migrants did not share the benefits of higher quality of care in the urban health system, even when they migrated from rural to urban areas [10,22]. The lower quality of TB care delivered by migrant clinics exacerbated gaps in health care equality between migrants and local residents [6,14]. Our results indicated that improving the quality of migrant TB care should be prioritized in current health reforms.
Aside from clinic types, we found clinicians with higher educational attainment significantly charged lower out of pocket fees, mitigating the budget barrier for migrants [14]. This is consistent with previous studies that reiterate ongoing reforms increasing the educational attainment of primary care providers in China [2,35]. We also found that clinicians with a fixed base salary were more likely to refer patients to an upper-level clinician and charged lower fees, indicating they made less profit from patient visits, compared to clinicians without a base salary. In China, clinicians highly depend on profits from medicine sales to supplement their nominal salary [21,44]. Research has shown that physician-induced demand is in part responsible for the high rates of unnecessary medication (e.g., antibiotics) prescriptions in China [42,45]. This suggests a base salary could be a potential way to discourage over-prescription and encourage appropriate referral, a direction in which China seems to be moving [2,20,21]

Limitations

There are several limitations to our study. First, we evaluated the management of a single SP case depicting a classic presentation of presumptive TB. As a result, we cannot ascertain how clinicians may deal with more complicated TB presentations (e.g., patients with recurrent TB symptoms and suggestive of drug-resistant TB), and whether clinicians can correctly treat known cases of TB. Das et al. (2015) [26], for instance, evaluated clinician management of four different cases including a confirmed case of TB and suspected MDR-TB.
Second, we evaluated one-time new patient interactions. Due to increased risk of detection, SPs did not complete follow-up visits with the 17% of clinicians who asked patients to return (see Table 2). Moreover, SPs were completely unknown to clinicians; clinician treatment of known, regular patients may differ. Differences of stability in patient–clinician relationships were more likely in local clinics than in migrant clinics considering the high mobility of migrant workers [10].
Third, we selected clinics and defined their types from a simple survey of households in migrant communities, however we did not collect more information on patient sorting behavior and health status of both migrants and local residents. A survey of both migrants and local residents in the future would lead to deeper understanding of their choices for clinics.
Finally, although we selected these migrant communities to meet our research design in consideration of the geographical variation of migrants, our study sample was not randomly representative of urban migrant communities.

5. Conclusions

Clinics in urban migrant communities provided poor quality care in the diagnosis and management of a classic case of presumptive TB. There was a large gap in correct case management between migrant clinics and local clinics, reinforcing that policies to improve the quality of migrant TB care should be prioritized in current health reforms. Given the significant correlations with correct case management and out of pocket fees, providing a base salary is a possible way to improve quality of TB care and increasing the education attainment of urban community clinicians might help reduce the heavy barrier of medical expenses for migrants.

Author Contributions

Conceptualization, H.X., J.N., C.Z., Y.S. and S.S.; Data curation, H.X., Q.A., K.L. and J.Z.; Formal analysis, H.X., Q.A., K.L. and J.Z.; Methodology, H.X., J.H., J.N., A.W. and S.S.; Resources, J.Y., H.L., J.N., C.Z. and Y.S.; Visualization, Q.A. and K.L.; Writing—original draft, H.X., J.H., B.Y. and A.W.; and Writing—review and editing, H.X., J.H., J.Z., E.A. and S.S.

Funding

The authors are supported by the 111 Project (Grant No. B16031), the Laboratory of Modern Teaching Technology of the Ministry of Education, Shaanxi Normal University, the National Natural Science Foundation of China (Grant No. 71703083), the National Social Science Fund Youth Project (Grant No. 15CJL005), the National Natural Science Foundation of China (Grant No. 71703084), the National Natural Science Foundation of China (Grant No. 71473152), the China Medical Board (Grant No. CMB-16-257) and the Knowledge for Change program at The World Bank (Grant No. 7172469). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Acknowledgments

We thank the SPs for their hard work and dedication; Alexis Medina for assistance managing the project; and Ming Mu, Andrew Hartnett and students in CEEE health groups for excellent research assistance.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, and in the decision to publish the results.

Appendix A

Table A1. Completion of Checklist Items.
Table A1. Completion of Checklist Items.
TotalLocal ClinicsMigrant Clinics
Questions
Cough duration 1,2,*0.81 (0.40)0.81 (0.40)0.81 (0.40)
(Produced) Sputum 1,2,*0.55 (0.50)0.62 (0.50)0.50 (0.51)
Past tuberculosis 2,*0.00 (0.00)0.00 (0.00)0.00 (0.00)
Family tuberculosis 2,*0.00 (0.00)0.00 (0.00)0.00 (0.00)
Blood in sputum 1,20.02 (0.15)0.00 (0.00)0.04 (0.20)
Fever (duration) 1,20.43 (0.50)0.50 (0.52)0.38 (0.50)
Fever type 1,20.19 (0.40)0.19 (0.40)0.19 (0.40)
Chest pain 20.19 (0.40)0.25 (0.45)0.15 (0.37)
(Loss of) Appetite 1,20.05 (0.22)0.12 (0.34)0.00 (0.00)
Lost weight 1,20.05 (0.22)0.06 (0.25)0.04 (0.20)
Breathing difficulty 1,20.05 (0.22)0.00 (0.00)0.08 (0.27)
Wheezing 1,20.07 (0.26)0.06 (0.25)0.08 (0.27)
(Take) Any medicines 20.60 (0.50)0.56 (0.51)0.62 (0.50)
Smoking 20.10 (0.30)0.00 (0.00)0.15 (0.37)
Diabetes 20.00 (0.00)0.00 (0.00)0.00 (0.00)
High blood pressure or hypertension 20.00 (0.00)0.00 (0.00)0.00 (0.00)
HIV/AIDS 20.00 (0.00)0.00 (0.00)0.00 (0.00)
Alcohol 20.10 (0.30)0.12 (0.34)0.08 (0.27)
Age 20.24 (0.43)0.19 (0.40)0.27 (0.45)
Family symptoms 20.00 (0.00)0.00 (0.00)0.00 (0.00)
Have cough throughout the day 20.26 (0.45)0.31 (0.48)0.23 (0.43)
Weakness 10.02 (0.15)0.00 (0.00)0.04 (0.20)
Are there any night sweats present? 10.02 (0.15)0.06 (0.25)0.00 (0.00)
Exams
Weight 1,20.00 (0.00)0.00 (0.00)0.00 (0.00)
Pulse 20.10 (0.30)0.06 (0.25)0.12 (0.33)
Blood pressure 20.05 (0.22)0.06 (0.25)0.04 (0.20)
Temperature 1,20.62 (0.49)0.62 (0.50)0.62 (0.50)
Auscultation 1,20.62 (0.49)0.56 (0.51)0.65 (0.49)
Chest radiograph 1,20.29 (0.46)0.38 (0.50)0.23 (0.43)
Sputum smear test (Sputum AFB) 1,20.02 (0.15)0.00 (0.00)0.04 (0.20)
HIV test 20.00 (0.00)0.00 (0.00)0.00 (0.00)
Diabetes test 20.00 (0.00)0.00 (0.00)0.00 (0.00)
Mantoux Tuberculin Skin Test (TST) 1,20.00 (0.00)0.00 (0.00)0.00 (0.00)
Sputum culture test 10.00 (0.00)0.00 (0.00)0.00 (0.00)
Notes: Data are proportion (standard deviation). 1 Chinese Standard, 2 International (ISTC) Standard, * essential items.

References

  1. Wang, L.; Zhang, H.; Ruan, Y.; Chin, D.P.; Xia, Y.; Cheng, S.; Chen, M.; Zhao, Y.; Jiang, S.; Du, X.; et al. Tuberculosis prevalence in China, 1990–2010: A longitudinal analysis of national survey data. Lancet 2014, 383, 2057–2064. [Google Scholar] [CrossRef]
  2. Liu, G.G.; Vortherms, S.A.; Hong, X.Z. China’s Health Reform Update. Annu. Rev. Public Health 2017, 38, 431–448. [Google Scholar] [CrossRef] [PubMed]
  3. WHO. Global Tuberculosis Report 2017. Available online: http://www.who.int/tb/publications/global_report/en/ (accessed on 14 September 2018).
  4. WHO. Global Strategy and Targets for Tuberculosis Prevention, Care and Control After 2015. Available online: http://www.who.int/tb/post2015_strategy/en/ (accessed on 14 September 2018).
  5. The National Health and Family Planning Commission of the Floating Population Family Planning Service Management: China Floating Population Development Report 2016. Available online: http://www.nhfpc.gov.cn/ldrks/s7847/201610/d17304b7b9024be38facb5524da48e78.shtml (accessed on 14 September 2018).
  6. Yang, C.; Lu, L.; Warren, J.L.; Wu, J.; Jiang, J.; Zuo, T.; Gan, M.; Liu, M.; Liu, Q.; DeRiemer, K.; et al. Internal migration and transmission dynamics of tuberculosis in Shanghai, China: An epidemiological, spatial, genomic analysis. Lancet Infect. Dis. 2018, 18, 788–795. [Google Scholar] [CrossRef]
  7. Li, X.; Yang, Q.; Feng, B.; Xin, H.; Zhang, M.; Deng, Q.; Deng, G.; Shan, W.; Yue, J.; Zhang, H.; et al. Tuberculosis infection in rural labor migrants in Shenzhen, China: Emerging challenge to tuberculosis control during urbanization. Sci. Rep. 2017, 7, 4457. [Google Scholar] [CrossRef] [PubMed]
  8. Hu, X.; Cook, S.; Salazar, M.A. Internal migration and health in China. Lancet 2008, 372, 1717–1719. [Google Scholar] [CrossRef]
  9. Jia, Z.W.; Jia, X.W.; Liu, Y.X.; Dye, D.; Chen, F.; Chen, C.S.; Zhang, W.Y.; Li, X.W.; Cao, W.C.; Liu, H.L. Spatial analysis of tuberculosis cases in migrants and permanent residents, Beijing, 2000–2006. Emerg. Infect. Dis. 2008, 14, 1413–1419. [Google Scholar] [CrossRef] [PubMed]
  10. Peng, X. China’s demographic history and future challenges. Science 2011, 333, 581–587. [Google Scholar] [CrossRef] [PubMed]
  11. Wang, W.; Jiang, Q.; Abdullah, A.S.; Xu, B. Barriers in accessing to tuberculosis care among non-residents in Shanghai: A descriptive study of delays in diagnosis. Eur. J. Public Health 2007, 17, 419–423. [Google Scholar] [CrossRef] [PubMed]
  12. Zhou, C.; Tobe, R.G.; Chu, J.; Gen, H.; Wang, X.; Xu, L. Detection delay of pulmonary tuberculosis patients among migrants in China: A cross-sectional study. Int. J. Tuberc. Lung Dis. 2012, 16, 1630–1636. [Google Scholar] [CrossRef] [PubMed]
  13. Tang, Y.; Zhao, M.; Wang, Y.; Gong, Y.; Yin, X.; Zhao, A.; Zheng, J.; Liu, Z.; Jian, X.; Wang, W.; et al. Non-adherence to anti-tuberculosis treatment among internal migrants with pulmonary tuberculosis in Shenzhen, China: A cross-sectional study. BMC Public Health 2015, 15, 474. [Google Scholar] [CrossRef] [PubMed]
  14. Li, Y.; Ehiri, J.; Tang, S.; Li, D.; Bian, Y.; Lin, H.; Marshall, C.; Cao, J. Factors associated with patient, and diagnostic delays in Chinese TB patients: A systematic review and meta-analysis. BMC Med. 2013, 11, 156. [Google Scholar] [CrossRef] [PubMed]
  15. Mou, J.; Griffiths, S.M.; Fong, H.; Dawes, M.G. Health of China’s rural-urban migrants and their families: A review of literature from 2000 to 2012. Br. Med. Bull. 2013, 106, 19–43. [Google Scholar] [CrossRef] [PubMed]
  16. WHO. The World Health Report 2008—Primary Health Care (Now More Than Ever). Available online: http://www.who.int/whr/2008/en/ (accessed on 14 September 2018).
  17. Hu, R.; Liao, Y.; Du, Z.; Hao, Y.; Liang, H.; Shi, L. Types of health care facilities and the quality of primary care: A study of characteristics and experiences of Chinese patients in Guangdong Province, China. BMC Health Serv. Res. 2016, 16, 335. [Google Scholar] [CrossRef] [PubMed]
  18. Li, H.; Chung, R.Y.N.; Wei, X.; Mou, J.; Wong, S.Y.S.; Wong, M.C.S.; Zhang, D.; Zhang, Y.; Griffiths, S. Comparison of perceived quality amongst migrant and local patients using primary health care delivered by community health centres in Shenzhen, China. BMC Fam. Pract. 2014, 15, 76. [Google Scholar] [CrossRef] [PubMed]
  19. Zeng, J.; Shi, L.; Zou, X.; Chen, W.; Ling, L. Rural-urban Migrants’ Experiences with Primary Care under Different Types of Medical Institutions in Guangzhou, China. PLoS ONE 2015, 10, e0140922. [Google Scholar] [CrossRef] [PubMed]
  20. Li, X.; Lu, J.; Hu, S.; Cheng, K.K.; De Maeseneer, J.; Meng, Q.; Mossialos, E.; Xu, D.R.; Yip, W.; Zhang, H.; et al. The primary health-care system in China. Lancet 2017, 390, 2584–2594. [Google Scholar] [CrossRef]
  21. Yip, W.C.-M.; Hsiao, W.C.; Chen, W.; Hu, S.; Ma, J.; Maynard, A. Early appraisal of China’s huge and complex health-care reforms. Lancet 2012, 379, 833–842. [Google Scholar] [CrossRef]
  22. Sylvia, S.; Xue, H.; Zhou, C.; Shi, Y.; Yi, H.; Zhou, H.; Rozelle, S.; Pai, M.; Das, J. Tuberculosis detection and the challenges of integrated care in rural China: A cross-sectional standardized patient study. PLoS Med. 2017, 14, e1002405. [Google Scholar] [CrossRef] [PubMed]
  23. Xu, D.R.; Hu, M.Y.; He, W.J.; Liao, J.; Cai, Y.Y.; Sylvia, S.Y.; Hanson, K.; Pan, Y.L.; Zhou, Z.L.; Zhang, N.; et al. Assessing Quality of Primary Health Care in 7 Chinese Provinces with Unannounced Standardized Patients: Protocol of a Cross-Sectional Survey. BMJ Open 2018. accepted. [Google Scholar] [CrossRef]
  24. Barrows, H.S. An overview of the uses of standardized patients for teaching and evaluating clinical skills. Acad. Med. 1993, 68, 443. [Google Scholar] [CrossRef] [PubMed]
  25. Das, J.; Holla, A.; Das, V.; Mohanan, M.; Tabak, D.; Chan, B. In Urban and Rural India, A Standardized Patient Study Showed Low Levels of Provider Training and Huge Quality Gaps. Health Aff. 2012, 31, 2774–2784. [Google Scholar] [CrossRef] [PubMed]
  26. Das, J.; Kwan, A.; Daniels, B.; Satyanarayana, S.; Satyanarayana, S.; Subbaraman, R.; Bergkvist, S.; Das, R.K.; Das, V.; Pai, M. Use of standardised patients to assess quality of tuberculosis care: A pilot, cross-sectional study. Lancet Infect. Dis. 2015, 15, 1305–1313. [Google Scholar] [CrossRef]
  27. Daniels, B.; Dolinger, A.; Bedoya, G.; Rogo, K.; Goicoechea, A.; Coarasa, J.; Wafula, F.; Mwaura, N.; Kimeu, R.; Das, J. Use of standardised patients to assess quality of healthcare in Nairobi, Kenya: A pilot, cross-sectional study with international comparisons. BMJ Glob. Health 2017, 2, e000333. [Google Scholar] [CrossRef] [PubMed]
  28. Das, J.; Holla, A.; Mohpal, A.; Muralidharan, K. Quality and Accountability in Health Care Delivery: Audit-Study Evidence from Primary Care in India. Am. Econ. Rev. 2016, 106, 3765–3799. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  29. Das, J.; Chowdhury, A.; Hussam, R.; Banerjee, A.V. The impact of training informal health care providers in India: A randomized controlled trial. Science 2016, 354, aaf7384. [Google Scholar] [CrossRef] [PubMed]
  30. Das, J.; Woskie, L.; Rajbhandari, R.; Abbasi, K.; Jha, A. Rethinking assumptions about delivery of healthcare: Implications for universal health coverage. BMJ 2018, 361, k1716. [Google Scholar] [CrossRef] [PubMed]
  31. Sylvia, S.; Shi, Y.; Xue, H.; Tian, X.; Wang, H.; Liu, Q.; Medina, A.; Rozelle, S. Survey using incognito standardized patients shows poor quality care in China’s rural clinics. Health Policy Plan. 2015, 30, 322–333. [Google Scholar] [CrossRef] [PubMed]
  32. Peabody, J.W.; Luck, J.; Glassman, P.; Dresselhaus, T.R.; Lee, M. Comparison of Vignettes, Standardized Patients, and Chart Abstraction: A Prospective Validation Study of 3 Methods for Measuring Quality. JAMA 2000, 283, 1715–1722. [Google Scholar] [CrossRef] [PubMed]
  33. Glassman, P.A.; Luck, J.; O’Gara, E.M.; Peabody, J.W. Using Standardized Patients to Measure Quality: Evidence from the Literature and a Prospective Study. Joint Comm. J. Qual. Improv. 2000, 26, 644–653. [Google Scholar] [CrossRef]
  34. Leonard, K.L.; Masatu, M.C. Using the Hawthorne effect to examine the gap between a doctor’s best possible practice and actual performance. J. Dev. Econ. 2010, 93, 226–234. [Google Scholar] [CrossRef]
  35. Woodward, C.A.; McConvey, G.A.; Neufeld, V.; Norman, G.R.; Walsh, A. Measurement of physician performance by standardized patients: refining techniques for undetected entry in physicians’ offices. Med. Care 1985, 23, 1019–1027. [Google Scholar] [CrossRef] [PubMed]
  36. TB CARE I. International Standards for Tuberculosis Care, Edition 3. 2014. Available online: http://www.istcweb.org (accessed on 14 September 2018).
  37. National Health and Family Planning Commission of the People’s Republic of China. Standards of Out-Patient Diagnosis and Treatment of Pulmonary Tuberculosis. 2012. Available online: http://www.moh.gov.cn/mohyzs/s3586/201202/54119.shtml (accessed on 14 September 2018).
  38. National Center for Tuberculosis Control and Prevention, China CDC. Diagnostic Criteria for Pulmonary Tuberculosis (WS 288-2008). 2008. Available online: http://tb.chinacdc.cn/zcfg/dfzcfg/201208/t20120814_66809.htm (accessed on 14 September 2018).
  39. National Health and Family Planning Commission of the People’s Republic of China. China Health Statistics Yearbook 2015. Available online: http://www.stats.gov.cn/tjsj/ndsj/2015/indexeh.htm (accessed on 14 September 2018).
  40. Shaanxi Bureau of Statistics. Shaanxi Statistical Yearbook; Shaanxi Statistics Press: Xian, China, 2016; Available online: http://www.shaanxitj.gov.cn/upload/2016/tongjinianj/2015/indexch.htm (accessed on 14 September 2018).
  41. Wang, J.; Wang, P.; Wang, X.; Zheng, Y.; Xiao, Y. Use and Prescription of Antibiotics in Primary Health Care Settings in China. JAMA Intern. Med. 2014, 174, 1914–1920. [Google Scholar] [CrossRef] [PubMed]
  42. Xue, H.; Shi, Y.J.; Huang, L.; Yi, H.M.; Zhou, C.C.; Zhou, H.; Kotb, S.; Tucker, J.D.; Sylvia, S.Y. Drivers of Inappropriate Antibiotic Prescriptions: A Quasi-Experimental Study of Primary Care Providers in Rural China. J. Antimicrob. Chemother. 2018. Forthcoming. [Google Scholar] [CrossRef]
  43. Hogan, C.A.; Puri, L.; Gore, G.; Pai, M. Impact of fluoroquinolone treatment on delay of tuberculosis diagnosis: A systematic review and meta-analysis. J Clin. Tuberc. Mycobact. Dis. 2017, 6, 1–7. [Google Scholar] [CrossRef]
  44. Yip, W.C.-M.; Hsiao, W.; Meng, Q.; Chen, W.; Sun, X. Realignment of incentives for health-care providers in China. Lancet 2010, 375, 1120–1130. [Google Scholar] [CrossRef]
  45. Currie, J.; Lin, W.; Meng, J. Addressing antibiotic abuse in China: An experimental audit study. J. Dev. Econ. 2014, 110, 39–51. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Figure 1. Flow of participants though study.
Figure 1. Flow of participants though study.
Ijerph 15 02037 g001
Figure 2. Differences of SP main outcomes between local and migrant clinics.
Figure 2. Differences of SP main outcomes between local and migrant clinics.
Ijerph 15 02037 g002
Figure 3. Differences of the Completion of Checklist Items between local and migrant clinics. Notes: 1 Chinese Standard; 2 International (ISTC) Standard; * essential items.
Figure 3. Differences of the Completion of Checklist Items between local and migrant clinics. Notes: 1 Chinese Standard; 2 International (ISTC) Standard; * essential items.
Ijerph 15 02037 g003
Table 1. Facility and clinician characteristics.
Table 1. Facility and clinician characteristics.
Full SampleLocal ClinicsMigrant ClinicsLocal vs. Migrant
p-Value
(N = 42)(N = 16)(N = 26)
Panel A: Facility Characteristics
Number of patients received in 20147866.73 (2666.13–13,067.34)5857.5 (3578.6–8136.4)9152.64 (527.52–17,777.76)0.539
Average distance to patients who visit clinic1.59 (1.18–2.01)1.43 (0.89–1.96)1.7 (1.09–2.31)0.517
Number of staffs working full time at the facility3.27 (2.47–4.07)3.06 (2.01–4.12)3.4 (2.21–4.59)0.684
Number of clinicians working full time at the facility2.07 (1.59–2.56)2.38 (1.44–3.31)1.88 (1.3–2.46)0.321
Total value of equipment (10,000 RMB)1.2 (0.79–1.6)1.48 (0.65–2.31)1.01 (0.57–1.46)0.266
Physicians who own their own facilities5 (12%, 5–26%)3 (19%, 7–43%)2 (8%, 2–25%)0.318
Panel B: Clinician Characteristics
Age (years)28 (70%, 55–82%)10 (63%, 39–82%)18 (75%, 55–88%)0.792
Male clinicians32 (80%, 65–90%)11 (69%, 44–86%)21 (88%, 69–96%)0.401
Clinicians with upper secondary or higher education21 (53%, 37–67%)8 (50%, 28–72%)13 (54%, 35–72%)0.158
Clinicians with Practicing Physician Certificate 5 (13%, 5–26%)2 (13%, 3–36%)3 (13%, 4–31%)0.796
Clinicians with Assistant Practicing Physician Certificate11 (28%, 16–43%)6 (38%, 18–61%)5 (21%, 9–40%)––
Clinicians with Rural Physician Certificate 28 (70%, 55–82%)11 (69%, 44–86%)17 (71%, 51–85%)0.253
Clinicians receiving base salary28 (70%, 55–82%)10 (63%, 39–82%)18 (75%, 55–88%)0.888
Monthly salary (1000 RMB)5.51 (1.25–9.76)3.06 (1.55–4.56)7.15 (–0.02–14.31)0.347
Clinicians who received Tuberculosis-specific training in 201417 (43%, 29–58%)9 (56%, 33–77%)8 (33%, 18–53%)0.155
Notes: Data are mean (95% CI) for continuous variables and number (mean, 95% CI) for dummy variables.
Table 2. Main outcomes of interactions with standardized patients.
Table 2. Main outcomes of interactions with standardized patients.
Full SampleLocal ClinicsMigrant ClinicsLocal vs. Migrant
p-Value
Patient-clinician interactions421626
Case management
Correctly managed the case §3650270.086
(23–51)(28–72)(14–46)
Ordered a chest radiograph 2938230.290
(17–44)(18–61)(11–42)
Ordered a sputum smear test2040.497
(0–12)(0–19)(1–19)
Referred to higher level hospitals2938230.179
(17–44)(18–61)(11–42)
Referred to CDC or DOTs, if referral2040.497
(0–12)(0–19)(1–19)
Medicine Prescribed6450730.024
(49–77)(28–72)(54–86)
Asked patient to return176230.141
(8–31)(1–28)(11–42)
Gave any antibiotic 5750620.282
(42–71)(28–72)(43–78)
Gave any fluoroquinolone5640.907
(1–16)(1–28)(1–19)
Process
Time with clinician (min)9.449.169.610.449
(7.23–11.65)(5.47–12.85)(6.66–12.57)
Number of questions and examinations (ISTC)5.385.55.310.828
(4.48–6.28)(3.75–7.25)(4.22–6.4)
% of questions and examinations (ISTC)1718170.828
(14–20)(12–23)(14–21)
Number of questions and examinations (China)3.8143.690.663
(3.18–4.44)(2.77–5.23)(2.95–4.44)
% of questions and examinations (China)2122210.663
(18–25)(15–29)(16–25)
% of essential history checklist asked by clinician (Both Standards)3436330.739
(29–39)(26–46)(26–40)
Out of pocket fees of consultation and medicines combined (RMB) 15.319.9418.620.024
(8.52–22.1)(1.21–18.67)(8.82–28.41)
Out of pocket fees of consultation and medicines combined (US dollars) * 2.361.532.860.024
(1.31–3.4)(0.19–2.87)(1.36–4.37)
Diagnosis
Mentioned tuberculosis5640.182
(1–16)(1–28)(1–19)
Notes: Data are mean or percent (95% CI). * 1 USD = 6.5 RMB. § Correctly managed for Tuberculosis is defined as a chest radiograph, sputum test, or referral. “ISTC” is ISTC standard and “China” is China national standard.
Table 3. Correlates of correct case management, chest radiograph, referral, medicine prescription and out of pocket fees of interactions with standardized patients among local and migrant clinics.
Table 3. Correlates of correct case management, chest radiograph, referral, medicine prescription and out of pocket fees of interactions with standardized patients among local and migrant clinics.
Correct Case ManagementChest RadiographReferralMedicine PrescriptionOut of Pocket Fees
(1)(2)(3)(4)(5)(6)(7)(8)(9)(10)
Service for migrant workers−0.29 **−0.29 *−0.28 *−0.36 **−0.24 *−0.30 *0.29 **0.32 **15.52 *13.30 *
(0.11)(0.12)(0.13)(0.14)(0.11)(0.12)(0.10)(0.12)(6.14)(6.20)
Clinician age (years)−0.01−0.01−0.010.00−0.000.000.010.010.340.38
(0.01)(0.01)(0.01)(0.01)(0.01)(0.01)(0.01)(0.01)(0.28)(0.27)
Male clinician0.060.040.030.060.190.17−0.06−0.07−4.27−8.42
(0.16)(0.15)(0.17)(0.17)(0.16)(0.14)(0.17)(0.16)(7.19)(7.29)
Clinician education, upper secondary or higher0.120.040.080.290.190.20−0.23−0.24−23.48 *−20.22 *
(0.22)(0.24)(0.27)(0.34)(0.22)(0.23)(0.23)(0.26)(8.62)(8.59)
Practicing physician certificate−0.24 *−0.22−0.35 **−0.41 **−0.21−0.210.130.154.7310.81
(0.12)(0.12)(0.12)(0.15)(0.13)(0.12)(0.14)(0.14)(6.17)(6.33)
Clinician has base salary0.280.41 *----0.210.48 **−0.01−0.18−9.13−22.00 *
(0.15)(0.21)----(0.15)(0.18)(0.16)(0.22)(6.95)(8.39)
Number of clinicians in a clinic 0.06 −0.14 0.12 * −0.07 −6.52 *
(0.06) (0.09) (0.05) (0.07) (2.63)
Total value of equipment (10,000 RMB) −0.05 0.06 −0.12 * 0.07 0.00
(0.06) (0.06) (0.05) (0.06) (2.38)
Building of clinic owned by clinicians 0.15 −0.34 0.04 −0.05 12.99
(0.23) (0.25) (0.18) (0.23) (10.19)
Number of Observations40402828404040404040
Mean of Outcome0.360.290.290.6415.31
Notes: Results are reported as marginal effects of logistic regressions in Columns (1)–(8) and OLS regressions in Columns (9) and (10). Standardized errors are reported in parentheses under coefficients. Correct case management is defined as a chest radiograph or sputum test or referral. * p < 0.05, ** p < 0.01, *** p < 0.001.

Share and Cite

MDPI and ACS Style

Xue, H.; Hager, J.; An, Q.; Liu, K.; Zhang, J.; Auden, E.; Yang, B.; Yang, J.; Liu, H.; Nie, J.; et al. The Quality of Tuberculosis Care in Urban Migrant Clinics in China. Int. J. Environ. Res. Public Health 2018, 15, 2037. https://0-doi-org.brum.beds.ac.uk/10.3390/ijerph15092037

AMA Style

Xue H, Hager J, An Q, Liu K, Zhang J, Auden E, Yang B, Yang J, Liu H, Nie J, et al. The Quality of Tuberculosis Care in Urban Migrant Clinics in China. International Journal of Environmental Research and Public Health. 2018; 15(9):2037. https://0-doi-org.brum.beds.ac.uk/10.3390/ijerph15092037

Chicago/Turabian Style

Xue, Hao, Jennifer Hager, Qi An, Kai Liu, Jing Zhang, Emma Auden, Bingyan Yang, Jie Yang, Hongyan Liu, Jingchun Nie, and et al. 2018. "The Quality of Tuberculosis Care in Urban Migrant Clinics in China" International Journal of Environmental Research and Public Health 15, no. 9: 2037. https://0-doi-org.brum.beds.ac.uk/10.3390/ijerph15092037

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop