Next Article in Journal
Effect of a Shock Micro-Cycle on Biochemical Markers in University Soccer Players
Next Article in Special Issue
COVID-19 Severity and Neonatal BCG Vaccination among Young Population in Taiwan
Previous Article in Journal
Normative Data of Objectively Measured Physical Activity and Sedentary Time in Community—Dwelling Older Japanese
Previous Article in Special Issue
Vitamin D Levels Are Reduced at the Time of Hospital Admission in Sicilian SARS-CoV-2-Positive Patients
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Fine-Scale Space-Time Cluster Detection of COVID-19 in Mainland China Using Retrospective Analysis

1
State Key Laboratory of Remote Sensing Science, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100101, China
2
School of Geography, Geomatics and Planning, Jiangsu Normal University, Xuzhou 221116, China
3
China Electronic Technology Group Corporation, Institute of Electronic Science, Beijing 100041, China
4
Jiangxi Academy of Sciences, Nanchang 330098, China
5
Beijing Research Center of Urban Systems Engineering, Xizhimen Nan Da Jie 16, Xicheng District, Beijing 100035, China
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2021, 18(7), 3583; https://0-doi-org.brum.beds.ac.uk/10.3390/ijerph18073583
Submission received: 23 February 2021 / Revised: 22 March 2021 / Accepted: 23 March 2021 / Published: 30 March 2021

Abstract

:
Exploring spatio-temporal patterns of disease incidence can help to identify areas of significantly elevated or decreased risk, providing potential etiologic clues. The study uses the retrospective analysis of space-time scan statistic to detect the clusters of COVID-19 in mainland China with a different maximum clustering radius at the family-level based on case dates of onset. The results show that the detected clusters vary with the clustering radius. Forty-three space-time clusters were detected with a maximum clustering radius of 100 km and 88 clusters with a maximum clustering radius of 10 km from 2 December 2019 to 20 June 2020. Using a smaller clustering radius may identify finer clusters. Hubei has the most clusters regardless of scale. In addition, most of the clusters were generated in February. That indicates China’s COVID-19 epidemic prevention and control strategy is effective, and they have successfully prevented the virus from spreading from Hubei to other provinces over time. Well-developed provinces or cities, which have larger populations and developed transportation networks, are more likely to generate space-time clusters. The analysis based on the data of cases from onset may detect the start times of clusters seven days earlier than similar research based on diagnosis dates. Our analysis of space-time clustering based on the data of cases on the family-level can be reproduced in other countries that are still seriously affected by the epidemic such as the USA, India, and Brazil, thus providing them with more precise signals of clustering.

1. Introduction

A novel coronavirus was first reported in China in December 2019 and was named “COVID-19” by the World Health Organization (WHO) [1,2].The COVID-19 was discovered in the capital of Hubei province, a Central China city named Wuhan, where the traffic system is well developed, and spread rapidly to nearly every part of the world causing global pandemic [3]. The coronavirus is a large virus family, which is known to cause Middle East respiratory syndrome (MERS), severe acute respiratory syndrome (SARS), and other serious diseases [4]. The COVID-19 virus is a new strain of coronavirus that has never been found in a human body before. The virus is mainly transmitted by droplets of coughing or sneezing. In view of this mode of transmission, most cases are related to direct contact, mainly during close contact [5,6,7].
Geography Information System (GIS) is an effective means to visualize and analyze spatial characteristics based on epidemic data [8,9,10]. Combined with spatial statistics, GIS can be used to help mitigate the epidemic through scientific information, find spatial correlations with other variables, and identify transmission dynamics and clustering [11,12,13,14].
Chinese regions differ from one another quite to a great degree, especially in terms of population density, meteorology, transmission net, industry, and economy [15,16,17,18,19,20]. Exploring spatio-temporal patterns of disease incidence can help to identify areas of significantly elevated or decreased risk, providing potential etiologic clues [21,22]. Only through mastering the temporal and spatial distribution of the epidemic can we achieve the target of accurate prevention and control. Quantitative research on the spatial and temporal characteristics of the clusters and the internal diffusion of the virus can not only comprehensively and profoundly help to understand the spatio-temporal law and internal mechanism of epidemic transmission, but also reflects the impact of emergency prevention and control measures for the spread of epidemic and serve as scientific evidence for policy adjustment.
However, most of the studies focus on the single analysis of spatial or temporal clustering. Single analysis of spatial clustering cannot reflect the dynamic change of epidemic situation, and single analysis of temporal aggregation can provide only some fuzzy information, namely whether there exist clusters in a certain time. However, the analysis of space-time clustering can not only indicate whether there exist clusters, but also can detect the spatial location and time of duration. Therefore, the detection results of space-time clustering are more conducive to the disease control departments taking timely response measures. If an outbreak causes the incidence of the whole area to increase at the same time, the temporal clustering method can easily detect the existence of clustering. However, for an outbreak that started in a local area and then gradually spread to the whole area, the incidence curve of the whole area will show an upward trend due to the increase in number of cases in the local area, so the clusters will be detected by single temporal clustering with a lag. The warning results of space-time clustering are more accurate and timelier due to the full use of the temporal and spatial information in the data. It is of great importance for early warning and prevention of future outbreaks [23,24,25].
Currently, epidemiologists pay more attention to the identifying of space-time clusters on a small scale, such as communities, counties, or provinces, and there are very few research works on large-scale analysis of space-time clustering, especially at the national scale [26] Even where there is, due to the availability of data, many studies are conducted with counties or provinces as the smallest unit [27,28]. In fact, it is very important to analyze the clustering of an epidemic on a national scale at the unit of family-level because it is of great significance for the development of accurate prevention, control, and work resumption policy at the national level.
Although we can roughly observe the clustering areas of COVID-19according to the existing epidemic map, the results of clusters number, clustering sizes, and time of duration obtained by different clustering methods at different spatial and temporal scales are very different. Currently, most of the national scale epidemic clustering detection is based on the administrative boundaries of counties, cities, or provinces. However, in the process of infectious disease transmission, the separation of administrative divisions cannot become the barrier of disease transmission. If the detection is carried out in isolation with provinces, cities, and counties as the units, some important clusters may not be detected timely due to the lack of information in the surrounding areas. Therefore, it is necessary to take the family as the smallest unit of clusters detection.

2. Materials and Methods

2.1. Materials

The COVID-19 cases are collected from the database of diagnosis and suspected cases of COVID-19 in mainland China established by a special group of big data analysis, which is subordinate to the Joint Prevention and Control Mechanism of the State Council. The database is generated based on China’s National Infectious Disease Information System (IDIS), which requires each COVID-19 case to be reported electronically by the responsible doctor as soon as a case has been diagnosed. It includes cases that are reported as asymptomatic, and data are updated in real time. The dataset includes the records of all confirmed COVID-19 cases from the onset of the outbreak on 2 December 2019 to 20 June 2020 in Mainland China (excluding Hong Kong, Macao, and Taiwan regions). Each record contains the information of the patient’s name, gender, ID, date of onset, date of diagnosis, administrative code, home address, and so on. The administrative boundary map of China was acquired from National Geomatics Center of China (http://ngcc.sbsm.gov.cn/, accessed date: 3 March 2018), which is in the format of shapefile.
The daily number of new cases of COVID-19 in mainland China between 2 December 2019 and 20 June 2020 was the data used for Figure 1. The total number of COVID-19 cases is 83,377 as of 20 June 2020. There are two peaks in the variation curve. One is on 24 January 2020 with a count of 3756 confirmed cases. The other one is on 1 February 2020 with a count of 5089 cases. That is because the prevention and control plan of COVID-19 (5th edition) issued by General Office for National Health Commission of China has added “clinical diagnosis” in the case diagnosis classification of Hubei province so that the patients can be diagnosed as early as possible according to the epidemic characteristics of Hubei on 13 February 2020, which led to a surge in new cases near 1 February 2020 as the date of onset.

2.2. Methodology

2.2.1. Spatialization of the Case Date

The traditional method of spatialization for epidemic data is based on the administrative code of the patient’s home on county, city, or province scale. However, this method cannot catch the exact location of the case. In this paper, we employ the software of XGeocoding v2 to translate the patient’s home address into coordinate information. In this way, all the recorded confirmed cases were inputted into Microsoft Excel 2010 (Microsoft, Redwoods, WA, USA), geo-coded according to their residential addresses and determined its longitude and latitude coordinates by using ArcGIS10.2 (ESRI Inc., Redlands, CA, USA), which was assumed to represent the location of case outbreak.

2.2.2. Space-Time Cluster Detection

The method employed for cluster detection is space-time scan statistic. Scan statistic is a method widely used in epidemic clustering analysis. It can effectively detect the increase of local time and/or spatial incidence of cases, and test whether the increase is caused by random variation. It can not only detect whether there is clustering in a certain area, but also accurately locate the clustering.
Space-time scan statistics is an extension of spatial scan statistics put forward by Kulldorf, professor of Harvard Medical School in 1997 [29,30,31]. It adds the time dimension to the original spatial scanning statistics so that the scanning statistics can detect the clustering in time and space at the same time. Therefore, compared with the circular window of the spatial scanning statistics, its scanning window is also correspondingly changed into a cylinder, where the bottom of the cylinder corresponds to the spatial range, while the height corresponds to a certain length of time segment. Because the size and position of the cylinder scanning window are constantly changing, the space-time scan statistics can be used for the time and place of epidemic onset. The size of the point and its scale are analyzed in depth, so as to realize the early identification of epidemic outbreak.
The specific detection process of Space-time Scan Statistic can be divided into four steps. Firstly, select a random spatial point in the study area as the center of the bottom surface of the cylinder scanning window. Then, they gradually increase the radius and height of the bottom surface of the cylinder scanning window. The continuous change of the bottom area of the cylinder corresponds to the change of the geographical area covered by the scanning window, and the continuous change of the height of the cylinder corresponds to the change of the bottom area until reaching the maximum space and time limit of the scan window. All positions of the cylinder scan window in the study area repeat the same scanning process. For each scan window, the expected incidence can be calculated according to the actual number of cases and population. Thirdly, the expected incidence can be calculated according to the number of cases in the scan window and outside the scan window. The log likelihood of the test statistics can be constructed from the actual and expected incidence Ratio; LLR (Log Likelihood Ratio) is used to evaluate the abnormal degree of the number of cases in the log likelihood ratio scanning window. It is necessary to select the window with the largest log likelihood ratio as the window with the highest abnormal degree of the number of cases will produce a large number of scanning windows. Finally, use the method of Monte Carlo simulation to evaluate the statistical significance of the window.
The retrospective analyses of space-time permutation is employed as the probability model of clustering detection. Its principle is as follows.
First of all, assume that the number of infections in an area z during d days is Czd, which corresponds to a scanning cylinder. Then, the total number of infections C in the whole study area during all time segments can be expressed as the following function.
C = z d C z d
Thus, the number of infections per day in each region μ z d can be described as:
μ z d = ( z C z d ) ( d C z d ) C
Therefore, we can calculate the expected number of infections μ A per scanning window A according to the number of infections of each unit μ z d .
μ A = ( z , d ) A μ z d
If the observed number of infections in cylinder A is CA, then CA obeys the hypergeometric distribution of mean μ A . The probability function of CA can be calculated as:
P ( C A ) = z A C z d C A C z A C z d d A C z d C A C d A C z d
When z A C z d and d A C z d are very small relative to the total number of infections C, CA approximately obeys the Poisson distribution of mean μ A . Based on this approximation, the generalized likelihood ratio (GLR) is used to measure whether the number of infections in cylinder A is abnormal.
In this paper, we show how information about the location and spatial extent of such events can be estimated from the spatial and temporal array of all calls by using the space-time permutation scan statistic.
G L R = C A μ A C A C C A C μ A C C A
The probability that a cluster will form by chance is assigned using Monte Carlo hypothesis testing by employing the likelihood of the statistic in question. For this, the time stamps of data points are shuffled and the statistic is calculated again. The process is then repeated 999 times.
The conventional techniques of space-time scan statistic commonly used the administrative boundary such as province, city, and county to be the minimum spatial unit of detection and use the regional center coordinate as all the cases location, which outbreak in this region [27,28].However, the separation of administrative divisions may become the barrier of disease transmission, because some important clusters may not be detected timely due to the lack of information in the surrounding areas if use the provinces, cities, and counties as the minimum unit of the detection. Our method employed the locations of the patients’ community or family to be the basic statistic unit by translating the patients’ home addresses to coordinate in order to detect the space-time clusters in finer scale.

3. Results

All 83,377 of the COVID-19 new cases in mainland China between 2 December 2019 and 20 June 2020 are geocoded and spatialized. The spatial distribution of COVID-19 cases are shown in Figure 2. The COVID-19 cases are distributed throughout all the provinces of China. No province is immune. Most of the cases are located in Hubei province, where Wuhan is its provincial capital. Tibet has only one case, which is the least of any province in China.
All the geo-coded cases were inputted into the model retrospective analyses of space-time permutation using the software of SatScan V9.6 (Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, MA, USA). The space-time permutation scan statistic employed in the study utilizes millions of overlapping cylinders to define the scanning window, each being a possible candidate for an outbreak. It can use different parameter settings of the maximum scan cylinders radius and height to consider the spatio-temporal independence of the clusters. When the outbreak location distance of two cases exceeds the maximum scanning radius or the interval time of them exceeds the scanning height, they are considered to be independent, and will not be detected in the same cluster. In this study, we set the maximum cluster size of the spatial window to a circle with a 100 km radius. The maximum temporal cluster size is seven days. The time precision is one day. Table 1 summarizes the characteristics of the statistically significant space-time clusters of COVID-19 at the family-level with a maximum spatial scanning window size of 100km in mainland China. Figure 3 illustrates the distribution of all the corresponding space-time clusters. According to the statistics, 43 clusters were detected with a p-value less than 0.05, in which 10 clusters were identified in Hubei province and 4 clusters were in Hebei province. Inner Mongolia and Shandong province had three clusters in each. Heilongjiang, Gansu, Sichuan, Liaoning, Guangdong, Fujian, and Zhejiang province had two clusters in each. Tianjin, Jilin, Jiangxi, Shaanxi, Guizhou, Anhui, Chungking, Shanghai, and Shanxi province had one cluster in each. All the clusters were detected in the central or eastern provinces. No cluster was detected in the western provinces.
The earliest space-time cluster was identified in Zhejiang province from 15–21 January 2020 with a radius of 88.1 km. The earliest cluster in Hubei province was detected from 19–25 January with a radius of 97.1 km. The third province, autonomous region, or municipality directly under the central government that shows a cluster appear is in Inner Mongolia from 2–4 February 2020 with a radius of 7.9 km. Jiangxi and Anhui are the adjacent provinces of Hubei and detected clusters separately from 2–3 February 2020. There were nine clusters detected in Zhejiang, Chongking, Hebei, Hubei, Shandong with a radius of 0 km. This means the cases of the cluster are located in the same communities, and it is likely to be a cluster of the same family. The last cluster was identified in Hebei province from 10–16 June 2020 with a radius of 68.2 km. The observed number of cases is 171, while the expected one is 1.1. In fact, the last cluster is caused by Beijing’s new confirmed COVID-19 cases related to Xinfadi market. Cases in Hebei province account for only a small proportion. With the maximum spatial scanning window size set as 100 km, this cluster contains both areas in Beijing and Hebei, and the cluster center is located in Hebei.
In order to discover finer-scale clusters, we set the maximum cluster size of the spatial window to a circle with a 10 km radius. The maximum temporal cluster size and time precision are still set as seven days and one day. Table 2 summarizes the characteristics of the statistically significant space-time clusters of COVID-19 at the family-level with a maximum spatial scanning window size of 10 km in mainland China. Figure 4 illustrates the distribution of all the corresponding space-time clusters. As statistics, 88 clusters were detected with a p-value less than 0.05. The clusters are distributed in 19 provinces, autonomous regions, or municipalities directly under the central government, as shown in Table 3. The earliest clusters are identified in Hubei from 20–25 January, and the last one is in Beijing from 14–20 June. There are 25 clusters located in Hubei province and 8 clusters were in Shanghai city. Each of Beijing, Guangdong, and Inner Mongolia has seven clusters. These areas including Shanghai, Beijing, and Guangdong are China’s top-three well-developed provinces or cities. They have developed transportation network with Hubei province especially Wuhan city. The large population of trade and migrant flow lead to the virus transmitting from Hubei to these areas more quickly than other provinces.
In order to discover more detailed characteristics of the clusters, we enlarged the clustering maps of both Wuhan and Beijing city, which are reported as the original epidemic areas, respectively, in the first and second waves of COVID-19 in China as shown in Figure 5 and Figure 6.There are 16 clusters identified in Wuhan city. All the clusters are detected in February 2020. These clusters are more uniformly distributed in all areas of Wuhan, regardless of whether in urban or suburban areas. In order to help to understand the detail spatio-temporal structure of the clusters, we colored the points of cases by the cluster according to the time from the first case reported in China. The early clusters formed in the east areas of the Yangtze River during 6–10 February 2020, which is about 10 weeks after the first case was reported, although the early cases were reported in Hankou district, which is in the west of the Yangtze River, as shown in Figure 5.
Although there are seven clusters identified in Beijing city, the clustering times range from February 2020 to June 2020. The clusters in Beijing are mainly distributed in the urban area. Similarly, we colored the points of cases by the cluster, and found the clusters can be divided into two stages. The first stage includes four clusters from February to March that are located in the northern urban area of Beijing, which is 12 to 17 weeks from the first reported case. These clusters belong to the first wave of COVID-19 in China, which originated in Wuhan. The second stage includes three clusters of outbreak in June 2020 and is located in the southern urban area of Beijing, which is 27 to 29 weeks from the first reported case. These clusters belong to the second wave of COVID-19 in China, which may have originated in Xinfadi market, which is a large wholesale market that sells fruits, vegetables, and meat located in Beijing’s Fengtai District and has been caught in the spotlight after new COVID-19 clusters were linked to it in June 2020, as shown in Figure 6.

4. Discussion

The study employed the retrospective analyses of space-time permutation to detect the space-time clusters of COVID-19 in mainland China on a fine scale. Based on the data obtained from China’s National Infectious Disease Information System (IDIS), we geo-coded each case and translate the patient’s home address into coordinate information in order to catch the exact location of the case. Former studies on space-time clusters detecting of COVID-19 in a county are mainly at the county-level [27,28]. It is the first time for a study to identify the clusters at the family-level in a large country.
Epidemic diagnosis time can scientifically evaluate and comprehensively reflect the emergency level and physical therapy capacity of a national or local health department. Most studies use COVID-19 case data based on diagnosis time from Johns Hopkins University’s Center for Systems Science and Engineering GIS dashboard to do the retrospective analysis. Although these data are updated daily, the statistics of daily cases are based on the diagnosis date. In fact, the average epidemic diagnosis time for COVID-19 outbreak from early onset to diagnosis is 7.35 days in mainland China [32]. It means the detected clusters start and end times will delay 7.35 days using the same space-time scan model based on the diagnosis dates rather than dates of onset. Therefore, it is more scientific and effective to use the case dates of onset when detecting the space-time clusters of an epidemic.
In order to account for the characteristics of the disease in a small region and to improve the probability of detecting smaller clusters, we set the maximum cluster size of the space-time clusters to two fine scales: One is 100 km and another one is 10 km. On the maximum cluster size of 100 km, we have detected 43 clusters during the study period, 10 of which were located in Hubei province. However, to our surprise, the earliest space-time cluster was identified in Zhejiang province from 15–21 January 2020 with a radius of 88.1 km. This may be explained by there being a large flow of population between Hubei and Zhejiang, especially in late January, with many infected students and migrant workers returning to Zhejiang from Wuhan before the closure of the city on 23 January. It also verified one table result in the former study that identifies a significant number of people who entered Wenzhou from Hubei Province, which explains why this city was the first outside the epicenter where confinement was adopted. While the maximum cluster size of 10 km, 88 clusters were detected by space-time scan statistic. The detected clusters are finer than that of 100 km. We have compared the characteristic of the clusters in the city of Wuhan with Beijing. The clusters identified in Wuhan are all detected in February and they are uniformly distributed in all areas of Wuhan, regardless of whether in urban or suburban areas. However, the seven detected clusters in Beijing are mainly distributed in the urban area. Four clusters ranging from February to March are located in the northern urban area. The other three clusters from the outbreak in June are located in southern urban area of Beijing, which may have originated in Xinfadi market and belong to the second wave of COVID-19 in China.
Regardless of whether it isa maximum clustering size of 100 km or 10 km scale, the province where the most clusters are located is Hubei, and the month with the most clusters is February, which indicates that China’s COVID-19 epidemic prevention and control strategy is effective and has successfully prevented the virus from spreading from Hubei to other provinces and lasting too long. When the first wave of the pandemic hit, the virus-testing capabilities were not strong, as it was unbeknownst to us at the time. However, China did a satisfactory job in data tracking, patient tracking, community quarantine, and the early warning from front-line fever clinics, which ensured there were no loopholes left. China was under great pressure when Wuhan city was closed on 23 January 2019. That strategy relieved the situation that the epidemic might transmit into the other areas of China through the Spring Festival holiday and formed more clusters.
There are also some limitations in the study. Firstly, because we use the home coordinate as the unit of space-time scan statistic, no population data at the home address-level can be collected in China. Therefore, we can only select the space-time permutation rather than the Poisson model in the discrete scan statistics. Secondly, the clusters detected are circular. In fact, changes in geography and cultural practices will in many cases invalidate this. Non-circular clusters may help to improve the detection. Thirdly, it is difficult to present all the fine space-time clusters in such a large country as China, especially when there are 88 clusters with a maximum cluster size of 10 km to be exhibited on the map. Finally, because of the change of statistical standards, previously only the number of people who had been diagnosed by accounting instruments were counted as confirmed cases. However, from 12 February 2020, clinical diagnosis cases recognized by doctors were also included in the statistics of confirmed cases, thus causing a sharp increase of cases and clusters in early February for a week from early onset to diagnosis. Lastly, using the conventional space-time scan statistic to detect the fine-scale space-time clusters at the family-level in whole China takes a lot of time. Combined with some of the optimization algorithms such as the particle swarm optimization [33], the probabilistic cellular automata model [34] and the coupled spring forced bat algorithm [35] may help improve the efficiency of the method.

5. Conclusions

In this study, we use the retrospective analysis of space-time scan statistic to detect clusters of COVID-19 in mainland China on two fine scales: With maximum clustering radii of 100 km and 10 km. Different from the other study, our analysis is based on case dates of onset, which are collected from the database of diagnosis and suspected cases of COVID-19 in mainland China established by the special group of big data analysis, which is subordinate to the Joint Prevention and Control Mechanism of the State Council. In addition, it is the first time to identify the space-time clusters of COVID-19 in a large country at the family-level. The results show that the detected clusters vary with the maximum clustering radius. Forty-three space-time clusters were detected with a maximum clustering radius of 100 km and 88 clusters with a maximum clustering radius of 10 km from 2 December 2019 to 20 June 2020. Using a small clustering radius may identify finer clusters. Hubei has the most clusters regardless of scale. Most of the clusters were generated in February. That indicates China’s COVID-19 epidemic prevention and control strategy is effective and has successfully prevented the virus from spreading from Hubei to other provinces and lasting too long. Well-developed provinces or cities that have large populations and developed transportation network are more likely to generate space-time clusters. The analysis based on the data of cases of onset may detect the start times of clusters seven days earlier than the same research that is based on the diagnosis dates. Our analysis of space-time clustering based on the data of cases of onset on the family-level can be reproduced in other countries that are still seriously affected by the epidemic, such as the USA, India, and Brazil, providing them with more precise signals of clustering.

Author Contributions

M.X.: Conceptualization, validation, writing—original draft. X.Z.; data curation, methodology, visualization, software. C.C.; conceptualization, data curation, writing—review and editing. H.L.; software, validation. Z.Y.; visualization, investigation. S.Z.; data curation, visualization. Z.H.; data curation, visualization. R.S.D.; Writing—review and editing. All authors have read and agreed to the published version of the manuscript.

Funding

This study was funded by National Key Research and Development Program of China (No. 2016YFB0501505), the Key Research and Development Program of Jiangxi, China (20181BBG70036) and the National Natural Science Foundation of China (No. 41971394).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

No additional data available.

Acknowledgments

We thank all the persons involved in response to this outbreak.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. WHO. Statement on the Second Meeting of the International Health Regulations (2005) Emergency Committee regarding the Outbreak of Novel Coronavirus (2019-nCoV). 30 January 2020. Available online: https://www.who.int/newsroom/detail/30-01-2020-statement-on-thesecond-meeting-of-the-international-healthregulations-(2005)-emergency-committeeregarding-the-outbreak-of-novelcoronavirus-(2019-ncov) (accessed on 1 February 2020).
  2. Wang, R.; Zhang, X.; Irwin, D.; Shen, Y. Emergence of SARS-like coronavirus poses new challenge in China. J. Infect. 2020, 80, 350–371. [Google Scholar] [CrossRef] [PubMed]
  3. Rothe, C.; Schunk, M.; Sothmann, P.; Bretzel, G.; Hoelscher, M. Transmission of 2019-nCoV Infection from an Asymptomatic Contact in Germany. N. Engl. J. Med. 2020, 382, 970–971. [Google Scholar] [CrossRef] [Green Version]
  4. Dong, E.; Du, H.; Gardner, L. An interactive web-based dashboard to track COVID-19 in real time. Lancet Infect. Dis. 2020, 20, 533–534. [Google Scholar] [CrossRef]
  5. Riou, J.; Althaus, C.L. Pattern of early human-to-human transmission of Wuhan 2019-nCoV. Eurosurveillance 2020, 25, 2000058. [Google Scholar]
  6. Li, Q.; Guan, X.; Wu, P.; Wang, X.; Zhou, L.; Tong, Y.; Ren, R.; Leung, K.; Lau, E.; Wong, J.; et al. Early Transmission Dynamics in Wuhan, China, of Novel Coronavirus–Infected Pneumonia. N. Engl. J. Med. 2020, 382, 1199–1207. [Google Scholar] [CrossRef]
  7. Chan, J.W.; Yuan, S.F.; Kok, K.H.; To, K.K.W.; Chu, H.; Yang, J.; Xing, F.; Liu, J.; Yip, C.C.; Poon, R.W.; et al. A familial cluster of pneumonia associated with the 2019 novel coronavirus indicating person-to-person transmission: A study of a family cluster. Lancet 2020, 395, 514–523. [Google Scholar] [CrossRef] [Green Version]
  8. Franch-Pardoa, I.; Napoletanob, B.; Rosete-Vergesa, F.; Billa, L. Spatial analysis and GIS in the study of COVID-19. A review. Sci. Total Environ. 2020, 739, 140033. [Google Scholar] [CrossRef] [PubMed]
  9. Mollalo, A.; Vahedib, B.; Rivera, K. GIS-based spatial modeling of COVID-19 incidence rate in the continental United States. Sci. Total Environ. 2020, 728, 138884. [Google Scholar] [CrossRef]
  10. Sarwar, S.; Waheedab, R.; Sarwar, S.; Khand, A. COVID-19 challenges to Pakistan: Is GIS analysis useful to draw solutions? Sci. Total Environ. 2020, 730, 139089. [Google Scholar] [CrossRef]
  11. Dong, N.; Yang, X.; Ye, L.; Chen, K.; Chan, E.; Yang, M.; Chen, C. Genomic and protein structure modelling analysis depicts the origin and infectivity of 2019-nCoV, a new coronavirus which caused a pneumonia outbreak in Wuhan, China. bioRxiv 2020. [Google Scholar] [CrossRef]
  12. Arab-Mazar, Z.; Sah, R.; Rabaan, A.A.; Dhama, K.; Rodriguez-Morales, A.J. Mapping the incidence of the COVID-19 hotspot in Iran—Implications for travellers. Travel Med. Infect. Dis. 2020, 38, 1477–8939. [Google Scholar] [CrossRef] [PubMed]
  13. Rezaei, M.; Nouri, A.A.; Park, G.S.; Kim, D.H. Application of geographic information system in monitoring and detecting the COVID-19 outbreak. Iran. J. Public Health 2020, 49, 114–116. [Google Scholar]
  14. Giuliani, D.; Dickson, M.M.; Espa, G.; Santi, F. Modelling and Predicting the Spatio-Temporal Spread of Coronavirus Disease 2019 (COVID-19) in Italy (3/20/2020). SSRN 2020. [Google Scholar] [CrossRef] [Green Version]
  15. Yao, Y.; Pan, J.; Wang, W.; Liu, Z.; Kan, H.; Qiu, Y.; Meng, X.; Wang, W. Association of particulate matter pollution and case fatality rate of COVID-19 in 49 Chinese cities. Sci. Total Environ. 2020, 741, 140396. [Google Scholar] [CrossRef]
  16. Qi, H.; Xiao, S.; Shi, R.; Ward, M.; Chen, Y.; Tu, W.; Su, Q.; Wang, W.; Wang, W.; Zhang, Z. COVID-19 transmission in Mainland China is associated with temperature and humidity: A time-series analysis. Sci. Total Environ. 2020, 728, 138778. [Google Scholar] [CrossRef]
  17. Zhang, Z.; Xue, T.; Jin, X. Effects of meteorological conditions and air pollution on COVID-19 transmission: Evidence from 219 Chinese cities. Sci. Total Environ. 2020, 741, 140244. [Google Scholar] [CrossRef] [PubMed]
  18. Liu, J.; Zhou, J.; Yao, J.; Zhang, X.; Li, L.; Xu, X.; He, X.; Wang, B.; Fu, S.; Niu, T.; et al. Impact of meteorological factors on the COVID-19 transmission: A multicity study in China. Sci. Total Environ. 2020, 726, 138513. [Google Scholar] [CrossRef]
  19. Ma, Y.; Zhao, Y.; Liu, J.; He, X.; Wang, B.; Fu, S.; Yan, J.; Niu, J.; Zhou, J.; Luo, B. Effects of temperature variation and humidity on the death of COVID-19 in Wuhan, China. Sci. Total Environ. 2020, 724, 138226. [Google Scholar] [CrossRef]
  20. Tosepu, R.; Gunawan, J.; Effendy, D.S.; Lestari, H.; Bahar, H.; Asfian, P. Correlation between weather and Covid-19 pandemic in Jakarta, Indonesia. Sci. Total Environ. 2020, 725, 138436. [Google Scholar] [CrossRef]
  21. Kang, D.; Choi, H.; Kim, J.; Choi, J. Spatial epidemic dynamics of the COVID-19 outbreak in China. Int. J. Infect. Dis. 2020, 94, 96–102. [Google Scholar] [CrossRef]
  22. Zhou, C.; Su, F.; Pei, T.; Zhang, A.; Du, Y.; Luo, B.; Cao, Z.; Wang, J.; Yuan, W.; Zhu, Y.; et al. COVID-19: Challenges to GIS with big data. Geo Sustain. 2020, 1, 77–87. [Google Scholar] [CrossRef]
  23. Ngui, A.; Appariciob, P.; Fleuryd, M.; Lesagec, A.; Grégoiree, J.; Moisane, J.; Vanasse, A. Spatio-temporal clustering of the incidence of schizophrenia in Quebec, Canada from 2004 to 2007. Spat. Spatiotemporal Epidemiol. 2013, 6, 37–47. [Google Scholar] [CrossRef] [PubMed]
  24. Zare, M.; Rezaianzadehm, A.; Tabatabaee, H.; Aliakbarpoor, M.; Faramarzi, H.; Ebrahimi, M. Spatiotemporal clustering of cutaneous leishmaniasis in Fars province, Iran. Asian Pac. J. Trop. Biomed. 2017, 7, 862–869. [Google Scholar] [CrossRef]
  25. Milne, G.; Graham, J.; Allen, A.; Lahuerta-Marin, A.; McCormick, C.; Presho, E.; Skuce, R.; Byrne, A.W. Spatiotemporal analysis of prolonged and recurrent bovine tuberculosis breakdowns in Northern Irish cattle herds reveals a new infection hotspot. Spat. Spatiotemporal Epidemiol. 2009, 28, 33–42. [Google Scholar] [CrossRef]
  26. Cordes, J.; Castro, M. Spatial analysis of COVID-19 clusters and contextual factors in New York City. Spat. Spatiotemporal Epidemiol. 2020, 4, 100355. [Google Scholar] [CrossRef] [PubMed]
  27. Desjardinsa, M.; Hohlb, A.; Delmelle, E. Rapid surveillance of COVID-19 in the United States using a prospective space-time scan statistic: Detecting and evaluating emerging clusters. Appl. Geogr. 2020, 18, 102202. [Google Scholar] [CrossRef] [PubMed]
  28. Hohl, A.; Delmelle, E.; Desjardins, M.; Lan, Y. Daily surveillance of COVID-19 using the prospective space-time scan statistic in the United States. Spat. Spatiotemporal Epidemiol. 2020, 34, 100354. [Google Scholar] [CrossRef]
  29. Kulldorff, M. Spatial Scan Statistic. Commun. Stat. Theory Methods 1997, 26, 1481–1496. [Google Scholar] [CrossRef]
  30. Kulldorff, M.; Athas, W.F.; Feuer, E.J.; Miller, B.A.; Key, C.R. Evaluating Cluster Alarms: A Space-Time Scan Statistic and Brain Cancer in Los Alams, New Mexico. Public Health Briefs 1998, 88, 1377–1380. [Google Scholar] [CrossRef] [Green Version]
  31. Kulldorff, M.; Heffernan, R.; Hartman, J.; Assunção, R.; Mostashari, F. A Space-time Permutation Scan Statistic for Disease Outbreak Detection. PLoS Med. 2005, 2, 216–224. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  32. Zhang, X.; Lin, H.; Zhu, C.; Wang, J.; Li, M.; Xu, C.; Hu, M.; Meng, B.; Cao, C. Spatiotemporal Characteristics and Dynamic Process Analysis of Epidemic Diagnosis Time of COVID-19 at Early Stage in Chinese Mainland. Geo Spat. Inf. Sci. 2020, 45, 791–797. [Google Scholar]
  33. Zhang, H.; Hui, Q. A Coupled Spring Forced Bat Searching Algorithm: Design, Analysis and Evaluation. In Proceedings of the 2020 American Control Conference (ACC), Denver, CO, USA, 1–3 July 2020; pp. 5016–5021. [Google Scholar]
  34. He, S.; Peng, Y.; Sun, K. SEIR modeling of the COVID-19 and its dynamics. Nonlinear Dyn. 2020, 101, 1667–1680. [Google Scholar] [CrossRef] [PubMed]
  35. Ghosh, S.; Saumik, B. A data-driven understanding of COVID-19 dynamics using sequential genetic algorithm based probabilistic cellular automata. Appl. Soft Comput. 2020, 96, 106692. [Google Scholar] [CrossRef] [PubMed]
Figure 1. Daily number of COVID-19 cases in mainland China between 2 December 2019 and 20 June 2020 (dates of onset are used for the statistics).
Figure 1. Daily number of COVID-19 cases in mainland China between 2 December 2019 and 20 June 2020 (dates of onset are used for the statistics).
Ijerph 18 03583 g001
Figure 2. The spatial distribution of COVID-19 in mainland China between 2 December 2019 and 20 June 2020 (Taiwan, Hang Kong and Macau not included).
Figure 2. The spatial distribution of COVID-19 in mainland China between 2 December 2019 and 20 June 2020 (Taiwan, Hang Kong and Macau not included).
Ijerph 18 03583 g002
Figure 3. Space-time clusters of COVID-19 at the family-level with a maximum spatial scanning window size of 100 km in mainland China from 2 December 2019 to 20 June 2020.
Figure 3. Space-time clusters of COVID-19 at the family-level with a maximum spatial scanning window size of 100 km in mainland China from 2 December 2019 to 20 June 2020.
Ijerph 18 03583 g003
Figure 4. Space-time clusters of COVID-19 at the family-level with a maximum spatial scanning window size of 10 km in mainland China from 2 December 2019 to 20 June 2020.
Figure 4. Space-time clusters of COVID-19 at the family-level with a maximum spatial scanning window size of 10 km in mainland China from 2 December 2019 to 20 June 2020.
Ijerph 18 03583 g004
Figure 5. Zoomed space-time clusters of COVID-19 at the family-level with a maximum spatial scanning window size of 10 km in Wuhancity from 2 December 2019 to 20 June 2020.
Figure 5. Zoomed space-time clusters of COVID-19 at the family-level with a maximum spatial scanning window size of 10 km in Wuhancity from 2 December 2019 to 20 June 2020.
Ijerph 18 03583 g005
Figure 6. Zoomed space-time clusters of COVID-19 at the family-level with a maximum spatial scanning window size of 10 km in Beijing city from 2 December 2019 to 20 June 2020.
Figure 6. Zoomed space-time clusters of COVID-19 at the family-level with a maximum spatial scanning window size of 10 km in Beijing city from 2 December 2019 to 20 June 2020.
Ijerph 18 03583 g006
Table 1. Characteristics of the statistically significant space-time clusters of COVID-19 at the family-level with a maximum spatial scanning window size of 100 km in mainland China from 2 December 2019 to 20 June 2020 (clusters where all cases happen in the same geolocation are reported as having 0 km radii).
Table 1. Characteristics of the statistically significant space-time clusters of COVID-19 at the family-level with a maximum spatial scanning window size of 100 km in mainland China from 2 December 2019 to 20 June 2020 (clusters where all cases happen in the same geolocation are reported as having 0 km radii).
SNCluster CenterRadius (km)Start DateEnd DateTemporal Size (d)p-ValueObservedExpected
1Zhejiang88.115 January 202021 January 20207<0.00113060.35
2Hubei97.119 January 202025 January 20207<0.001933674.71
3Hubei100.022 January 202028 January 20207<0.001487341.09
4Hubei83.223 January 202027 January 20205<0.001353237.13
5Hubei99.728 January 202031 January 20204<0.001670378.61
6Inner Mongolia7.92 February 20204 February 20203<0.001304.33
7Jiangxi43.72 February 20205 February 202040.005338.62
8Anhui39.43 February 20207 February 20205<0.0016422.88
9Hubei26.83 February 20208 February 202060.0026425.17
10Guizhou85.26 February 20207 February 202020.001202.94
11Hebei24.88 February 202014 February 202070.001244.19
12Zhejiang0.09 February 202011 February 20203<0.001201.94
13Chungking0.011 February 202011 February 20201<0.00170.10
14Hebei0.011 February 202011 February 202010.04070.21
15Hubei86.514 February 202020 February 20207<0.001712369.38
16Sichuan22.515 February 202019 February 20205<0.001252.94
17Hubei0.017 February 202018 February 20202<0.00131111.86
18Hubei0.019 February 202021 February 20203<0.001851.18
19Shandong0.019 February 202019 February 20201<0.0011221.25
20Hubei0.024 February 202028 February 20205<0.001250.29
21Hubei0.025 February 202025 February 20201<0.001230.13
22Hebei17.625 February 202027 February 202030.00950.04
23Gansu6.34 March 202010 March 20207<0.001350.14
24Guangdong65.512 March 202018 March 20207<0.001141.04
25Gansu73.713 March 202014 March 20202<0.00150.01
26Fujian57.618 March 202024 March 20207<0.001240.46
27Shanghai53.619 March 202025 March 20207<0.0011032.64
28Fujian30.919 March 202025 March 20207<0.001120.28
29Guangdong29.221 March 202027 March 20207<0.001512.05
30Shandong76.422 March 202024 March 202030.00770.16
31Tianjin41.523 March 202029 March 20207<0.001270.43
32Heilongjiang7.325 March 202031 March 20207<0.001330.24
33Liaoning8.827 March 202031 March 20205<0.00170.03
34Shanxi92.21 April 20207 April 20207<0.001440.43
35Heilongjiang20.05 April 202011 April 20207<0.001590.35
36Inner Mongolia82.87 April 202013 April 20207<0.0012382.11
37Inner Mongolia72.214 April 202020 April 20207<0.001320.32
38Shaanxi3.621 April 202027 April 20207<0.001330.05
39Liaoning5.48 May 202012 May 202050.0033<0.01
40Jilin60.09 May 202015 May 20207<0.001230.03
41Sichuan0.828 May 20201 June 20205<0.001110.01
42Shandong0.029 May 202029 May 20201<0.0013<0.01
43Hebei68.210 June 202016 June 20207<0.0011711.10
Table 2. Characteristics of the statistically significant space-time clusters of COVID-19 at the family-level with a maximum spatial scanning window size of 10 km in mainland China from 2 December 2019 to 20 June 2020 (clusters where all cases happen in the same geo-location are reported as having 0 km radii).
Table 2. Characteristics of the statistically significant space-time clusters of COVID-19 at the family-level with a maximum spatial scanning window size of 10 km in mainland China from 2 December 2019 to 20 June 2020 (clusters where all cases happen in the same geo-location are reported as having 0 km radii).
SNCluster CenterLatitudeLongitudeRadiusStart TimeEnd TimeTemporal Size (d)p-ValueObservedExpected
1Hubei30.9662113.99028.320 January 202025 January 202060.00516395.3
2Hubei31.6174113.82949.425 January 202031 January 20207013264.8
3Hubei32.0496112.13325.226 January 202031 January 202060188101.4
4Hubei31.6887113.44468.929 January 202031 January 20203015354.1
5Hubei30.7732112.63437.73 February 20207 February 202050368.3
6Hubei30.4947113.54322.83 February 20204 February 202020233.4
7Hubei30.6153114.4855106 February 202010 February 202050564378.2
8Hubei30.674114.65677.47 February 202012 February 202060.0054514.8
9Sichuan31.0197101.15551.28 February 20208 February 202010120.5
10Zhejiang29.0978119.065209 February 202011 February 202030201.9
11Hubei30.6753114.30198.611 February 202016 February 2020601255806.7
12Hubei30.807114.78139.911 February 202017 February 2020707830.1
13Chungking29.6338106.3409011 February 202011 February 20201070.1
14Hebei41.7145114.7829011 February 202011 February 202010.02770.2
15Hubei30.3781114.42758.512 February 202017 February 2020606919.9
16Hubei31.0169114.2449712 February 202017 February 202060316.5
17Guangdong22.1659113.28338.113 February 202013 February 202010.00280.2
18Inner Mongolia44.9239127.177014 February 202015 February 202020.01170.2
19Hubei30.3177114.08551014 February 202019 February 2020609833.4
20Hubei30.9102114.59589.614 February 202020 February 2020705012.3
21Hubei30.4973114.899.214 February 202014 February 2020104510.7
22Hubei30.504113.97269.816 February 202021 February 2020705215.8
23Hubei30.5878114.2588017 February 202018 February 20202031111.9
24Hubei30.6244114.11628.518 February 202024 February 20207017264.4
25Hubei31.2838114.28694.618 February 202018 February 202010190.3
26Hubei30.8288114.3178.918 February 202024 February 2020706615.4
27Hubei31.1484114.34559.818 February 202021 February 202040212
28Hubei30.4977114.3479019 February 202021 February 202030851.2
29Shandong35.5204116.5637019 February 202019 February 2020101221.2
30Inner Mongolia46.7797131.81947.92 February 20204 February 202030304.3
31Anhui32.9803117.32497.92 February 20207 February 202060.0225622.1
32Beijing39.9807116.3938022 February 202026 February 202050100.1
33Hubei30.515114.08290.822 February 202027 February 202060240.4
34Hubei30.554114.3122024 February 202028 February 202050250.3
35Hubei30.4946114.2993025 February 202025 February 202010230.1
36Hebei41.4276114.95662.225 February 202025 February 202010.01840
37Gansu36.0742103.76486.34 March 202010 March 202070350.1
38Guangdong22.5322113.99516.98 March 202012 March 20205080.1
39Guangdong22.9355113.42419.58 March 202014 March 20207070.1
40Shanghai31.1188121.65589.910 March 202013 March 202040100.1
41Gansu35.6057103.2067013 March 202014 March 20202050
42Beijing40.0674116.57928.114 March 202020 March 202070610.4
43Beijing40.1854116.39730.415 March 202021 March 202070170.1
44Fujian24.5038118.04417.115 March 202021 March 20207090.1
45Fujian24.724118.70837.816 March 202021 March 20206090
46Shanghai31.3344121.60089.918 March 202024 March 202070.01170.2
47Tianjin39.0995117.23729.719 March 202025 March 202070160.3
48Shanghai31.2015121.47789.419 March 202025 March 202070500.9
49Fujian26.0801119.37668.419 March 202025 March 202070.02260.1
50Shanghai31.0122121.41340.120 March 202025 March 202060.00150
51Sichuan30.5619103.92253.121 March 202026 March 20206050
52Guangdong23.1856113.33229.321 March 202027 March 202070320.8
53Heilongjiang40.9065111.9804022 March 202022 March 20201060
54Zhejiang28.0965120.33178.722 March 202027 March 202060.01350
55Tianjin39.0041117.7655023 March 202028 March 20206080
56Heilongjiang40.8446111.73037.325 March 202031 March 202070330.2
57Hebei39.3359117.83912.827 March 202029 March 20203040
58Liaoning38.9591121.61838.827 March 202031 March 20205070
59Beijing40.092116.37513.428 March 202029 March 20202050
60Hebei38.0112114.5161028 March 20203 April 20207060
61Guangdong23.4414113.31138.429 March 20204 April 202070.02250
62Shanxi37.7926112.53598.131 March 20206 April 202070290.2
63Shanghai30.8993121.16501 April 20204 April 202040.00340
64Shanghai31.0648121.743703 April 20209 April 202070310.3
65Shanghai31.2074121.718703 April 20209 April 202070160.1
66Heilongjiang49.48117.688404 April 202010 April 202070300.1
67Heilongjiang49.6056117.48982.95 April 202011 April 202070310.2
68Inner Mongolia44.6009129.61442.26 April 202012 April 202070250.2
69Shanxi37.9272112.566637 April 20207 April 202010120
70Inner Mongolia44.4032131.17045.17 April 202013 April 2020702081.6
71Inner Mongolia44.9222130.54284.913 April 202017 April 20205090
72Inner Mongolia45.7625126.65786.214 April 202020 April 202070280.2
73Tianjin39.1706117.38751.316 April 202016 April 202010.00330
74Shaanxi34.2054108.98673.621 April 202027 April 202070330.1
75Inner Mongolia45.2905130.28653.325 April 202030 April 20206040
76Jilin44.4123126.97141.65 May 202011 May 202070110
77Liaoning41.7046123.41625.48 May 202012 May 202050.00230
78Jilin43.8046126.532810 May 202015 May 202060170
79Shanghai31.339121.4488013 May 202013 May 20201040
80Sichuan30.6483104.05820.828 May 20201 June 202050110
81Shandong35.1304119.2983029 May 202029 May 20201030
82Guangdong23.0409113.25946.531 May 20206 June 20207050
83Sichuan30.5971104.15955.93 June 20208 June 20206050
84Guangdong23.1688113.45281.58 June 202013 June 202060100
85Beijing39.7775116.17316.49 June 202012 June 202040.02440
86Hebei38.8718115.97679.310 June 202015 June 20206070
87Beijing39.7966116.34989.911 June 202017 June 2020701440.7
88Beijing39.9131116.17728.414 June 202020 June 202070130.1
Table 3. Number of clusters in each province at the family-level with a maximum spatial scanning window size of 10 km.
Table 3. Number of clusters in each province at the family-level with a maximum spatial scanning window size of 10 km.
Province NameNumber of Clusters
Anhui1
Beijing7
ChongKing1
Fujian3
Gansu1
Guangdong7
Hebei5
Heilongjiang4
Hubei25
Inner Mongolia7
Jilin2
Liaoning2
Shaanxi1
Shandong2
Shanghai8
Shanxi2
Sichuan4
Tianjin3
Zhejiang2
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Xu, M.; Cao, C.; Zhang, X.; Lin, H.; Yao, Z.; Zhong, S.; Huang, Z.; Shea Duerler, R. Fine-Scale Space-Time Cluster Detection of COVID-19 in Mainland China Using Retrospective Analysis. Int. J. Environ. Res. Public Health 2021, 18, 3583. https://0-doi-org.brum.beds.ac.uk/10.3390/ijerph18073583

AMA Style

Xu M, Cao C, Zhang X, Lin H, Yao Z, Zhong S, Huang Z, Shea Duerler R. Fine-Scale Space-Time Cluster Detection of COVID-19 in Mainland China Using Retrospective Analysis. International Journal of Environmental Research and Public Health. 2021; 18(7):3583. https://0-doi-org.brum.beds.ac.uk/10.3390/ijerph18073583

Chicago/Turabian Style

Xu, Min, Chunxiang Cao, Xin Zhang, Hui Lin, Zhong Yao, Shaobo Zhong, Zhibin Huang, and Robert Shea Duerler. 2021. "Fine-Scale Space-Time Cluster Detection of COVID-19 in Mainland China Using Retrospective Analysis" International Journal of Environmental Research and Public Health 18, no. 7: 3583. https://0-doi-org.brum.beds.ac.uk/10.3390/ijerph18073583

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop