Next Article in Journal
Spatio-Temporal Research Data Infrastructure in the Context of Autonomous Driving
Next Article in Special Issue
A Blockchain Solution for Securing Real Property Transactions: A Case Study for Serbia
Previous Article in Journal
Investigating the Relationship between the Built Environment and Relative Risk of COVID-19 in Hong Kong
Previous Article in Special Issue
An Illumination Insensitive Descriptor Combining the CSLBP Features for Street View Images in Augmented Reality: Experimental Studies
Article

An Efficient Row Key Encoding Method with ASCII Code for Storing Geospatial Big Data in HBase

by 1,2, 1,3,4, 1, 5, 1, 1, 1,3,4, 1,3,4 and 1,3,4,*
1
College of Land Science and Technology, China Agricultural University, Beijing 100083, China
2
Center for Spatial Information Science and Systems, George Mason University, 4400 University Dr, Fairfax, VA 22030, USA
3
Key Laboratory of Remote Sensing for Agri-Hazards, Ministry of Agriculture, Beijing 100083, China
4
Key Laboratory of Agricultural Land Quality and Monitoring, Ministry of Natural Resources, Beijing 100083, China
5
State Key Laboratory of Earth Surface Processes and Resource Ecology, Beijing Normal University, Beijing 100875, China
*
Author to whom correspondence should be addressed.
ISPRS Int. J. Geo-Inf. 2020, 9(11), 625; https://0-doi-org.brum.beds.ac.uk/10.3390/ijgi9110625
Received: 1 September 2020 / Revised: 13 October 2020 / Accepted: 22 October 2020 / Published: 25 October 2020
(This article belongs to the Special Issue GIS Software and Engineering for Big Data)
Recently, increasing amounts of multi-source geospatial data (raster data of satellites and textual data of meteorological stations) have been generated, which can play a cooperative and important role in many research works. Efficiently storing, organizing and managing these data is essential for their subsequent application. HBase, as a distributed storage database, is increasingly popular for the storage of unstructured data. The design of the row key of HBase is crucial to improving its efficiency, but large numbers of researchers in the geospatial area do not conduct much research on this topic. According the HBase Official Reference Guide, row keys should be kept as short as is reasonable while remaining useful for the required data access. In this paper, we propose a new row key encoding method instead of conventional stereotypes. We adopted an existing hierarchical spatio-temporal grid framework as the row key of the HBase to manage these geospatial data, with the difference that we utilized the obscure but short American Standard Code for Information Interchange (ASCII) to achieve the structure of the grid rather than the original grid code, which can be easily understood by humans but is very long. In order to demonstrate the advantage of the proposed method, we stored the daily meteorological data of 831 meteorological stations in China from 1985 to 2019 in HBase; the experimental result showed that the proposed method can not only maintain an equivalent query speed but can shorten the row key and save storage resources by 20.69% compared with the original grid codes. Meanwhile, we also utilized GF-1 imagery to test whether these improved row keys could support the storage and querying of raster data. We downloaded and stored a part of the GF-1 imagery in Henan province, China from 2017 to 2018; the total data volume reached about 500 GB. Then, we succeeded in calculating the daily normalized difference vegetation index (NDVI) value in Henan province from 2017 to 2018 within 54 min. Therefore, the experiment demonstrated that the improved row keys can also be applied to store raster data when using HBase. View Full-Text
Keywords: geospatial big data; HBase; row keys; large scale; storage; GF-1 imagery geospatial big data; HBase; row keys; large scale; storage; GF-1 imagery
Show Figures

Figure 1

MDPI and ACS Style

Xiong, Q.; Zhang, X.; Liu, W.; Ye, S.; Du, Z.; Liu, D.; Zhu, D.; Liu, Z.; Yao, X. An Efficient Row Key Encoding Method with ASCII Code for Storing Geospatial Big Data in HBase. ISPRS Int. J. Geo-Inf. 2020, 9, 625. https://0-doi-org.brum.beds.ac.uk/10.3390/ijgi9110625

AMA Style

Xiong Q, Zhang X, Liu W, Ye S, Du Z, Liu D, Zhu D, Liu Z, Yao X. An Efficient Row Key Encoding Method with ASCII Code for Storing Geospatial Big Data in HBase. ISPRS International Journal of Geo-Information. 2020; 9(11):625. https://0-doi-org.brum.beds.ac.uk/10.3390/ijgi9110625

Chicago/Turabian Style

Xiong, Quan, Xiaodong Zhang, Wei Liu, Sijing Ye, Zhenbo Du, Diyou Liu, Dehai Zhu, Zhe Liu, and Xiaochuang Yao. 2020. "An Efficient Row Key Encoding Method with ASCII Code for Storing Geospatial Big Data in HBase" ISPRS International Journal of Geo-Information 9, no. 11: 625. https://0-doi-org.brum.beds.ac.uk/10.3390/ijgi9110625

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop