Utilizing Building Offset and Shadow to Retrieve Urban Building Heights with ICESat-2 Photons

Abstract

Building height serves as an essential feature of urban morphology that provides valuable insights into human socio-cultural behaviors and their impact on the environment in an urban milieu. However, openly accessible building height information at the individual building level is still lacking and remains sorely limited. Previous studies have shown that the ICESat-2′s ATL03/08 products are of good accuracy for urban building heights retrieval, however, these studies are limited to areas with available data coverage. To this end, we propose a method for extracting urban building height by using ICESat-2 ATL03 photons and high-resolution remote sensing images. We first extracted the information of building roof to footprint offsets and building shadows from high resolution imagery using multitasking CNN frameworks. Using the building height samples calculated from ICESat-2 ATL03 photons, we developed a building height estimation method that combines building offset and shadow length information. We assessed the efficacy of the proposed method in the Wujiaochang area of Shanghai city, China. The results indicated that the proposed method is able to extract building height with a MAE of 4.7 m, and outperforms the traditional shadow-based and offset-based method. We believe that the proposed method is a good candidate for accurately retrieving building heights on a city-wide scale.

1. Introduction

Information on urban building morphology is urgently needed for urban sustainable development, and offers a comprehensive view of land use planning [1,2,3,4], urban infrastructure [5], economic development [6], ecological processes [7,8,9], and well-being [10,11]. Building height plays a crucial role in urban building morphology as it serves as a fundamental attribute that reflects human activities and the interactions between humans and their environment in an urban context. Many urban applications, such as sustainable urban planning [12,13], urban climate modeling [14,15,16], population estimation [17,18], and three-dimensional (3D) building reconstruction [19,20,21], all have a close relationship with the height of urban buildings. Additionally, the determination of building heights also plays a crucial role in assessing the associated risks of natural hazards. Natural hazards can have detrimental effects, encompassing the loss of human lives, destruction of property, and disruption of economic activities [22]. Building structures and heights are instrumental in estimating both the economic and population losses. This significance becomes particularly pronounced in high mountain regions, where conventional approaches for monitoring building structures are inadequate, thereby impeding the monitoring of alterations linked to natural disasters. Therefore, calculating building height efficiently and accurately is significant to facilitate a broad range of urban applications.

Advancements in remote sensing technology have facilitated the automatic estimation of building heights, thanks to the increasing availability of remote sensing data. The primary data used to extract urban building heights include photogrammetry, high-resolution images, and airborne LiDAR data [23,24,25,26,27,28,29]. There is currently a growing interest in utilizing space-borne lasers to retrieve building heights [30]. These lasers offer a synoptic view and have the capability to capture data on a global scale, making them a valuable tool for obtaining accurate information about building heights. The Ice, Cloud, and Land Elevation Satellite-2 (ICESat-2) mission has provided a solution for investigating urban structures using a space-borne laser. ICESat-2 is an Earth observation satellite with coverage up to 88°N–88°S latitude [31]. It is equipped with a photon-counting technology supported by the Advanced Topographic Laser Altimeter System (ATLAS) instrument [32,33]. At an altitude of about 500 km, the ATLAS sensor produces three pairs of laser beams which are 3.3 km apart from each other across the beam track [34]. Each pair consists of a strong signal beam and a weak signal beam (energy ratio of 4:1). The two beams of different intensities form observation tracks on the ground at a distance of approximately 90 m from each other.

Though ICESat-2 was designed to supervise changes in sea ice [35,36,37,38], it has been used in urban built environment fields like urban terrain and building heights estimation [39,40,41,42]. Zhao et al. [43] evaluated the performance of ICESat-2 ATL03 and ATL08 data products in urban areas, and they found ICESat-2 products have been shown to have a good accuracy with a height deviation smaller than 1.5 m in dense urban areas. For estimating building height, Dandabathula et al. [40] demonstrated that it is feasible to obtain building heights from ICESat-2 geolocated photons and the accuracy is up to 17 cm. Lao et al. [44] developed a method which is based on random sample consensus (RANSAC) linear fitting and mathematical statistics to extract the building height. Their result shows that the ATL03 estimated building height is compatible with that derived by terrestrial laser scanning. All these studies indicated that the ICESat-2 data allow for detailed and accurate measurements of the ground objects, especially the profile along the track of each beam. However, ICESat-2 altimetry data are limited by the spatial resolution, especially the inter-beam observation interval, resulting in the primary data processing still being track-based [33,45,46].

In order to continuously extract building heights over a large area using ICESat-2 data, additional remote sensing data, particularly high-resolution images that offer a cost-effective means of acquiring building information with extensive spatial coverage, need to be incorporated. For example, Zhao et al. [46] developed a method that combines Google Earth Satellite images with ICESat-2 photon data to extract building heights. Based on the building height samples obtained by ICESat-2 data, they developed an improved shadow-based model for building height estimation. For building height extraction using high-resolution images, numerous deep learning models, including convolutional neural networks [47,48], residual convolutional–deconvolutional networks [49], deep convolutional neural fields [50], generative adversarial networks [51], encoder-decoder networks [52], deep ordinal regression networks [53], and multi-scale refinement networks [54], have been successfully employed for estimating building heights using a single aerial image in recent years. Furthermore, multitask learning has also demonstrated its efficacy, particularly when combined with image segmentation and extraction of multi-spectral features [55,56,57,58]. The integration of deep learning techniques allows for the automatic extraction of multi-level features from remote sensing images, utilizing deeper models with skip connections between layers, and has proven to be highly effective for building height estimation. However, deep learning methods heavily depend on having a substantial quantity of training data and height annotations. Given the wide range of architectural styles and building heights observed across different regions, the transferability of deep learning methods is often limited. Most current studies used building shadows [25,59,60] and “roof to footprint” offset vectors [23,58,61,62] derived from deep learning approaches to estimate building heights. However, using only one of these pieces of information for building height estimation can lead to deviations.

Therefore, the objective of this study is to address how to utilize gapped ICESat-2 ATL03 photons for estimating building heights continuously in urban areas. We have made two contributions in this study. Firstly, to address the challenge of obtaining building heights in urban areas continuously using ICESat-2 ATL03 photons, we proposed a method that integrates ICESat-2 ATL03 photons with high-resolution remote sensing imagery. This approach allows us to estimate the heights of individual buildings within urban areas. Secondly, during the building height estimation, we innovatively integrated two height estimation features, the length of building shadows and building offset vectors. These additions greatly enhance the accuracy of building height estimation through optimization techniques. In the rest of this paper, we first present the study areas and the data used in this study in Section 2. We then describe the proposed method in Section 3. In Section 4 and Section 5, we conduct an analysis and discussion of the results. Finally, we draw our conclusions in Section 6.

2. Study Area and Data

Our research focuses on Wujiaochang, an urban sub-center situated in the Yangpu District of Shanghai, China. Located in the northeast of the city center, Wujiaochang is one of the most energetic and vibrant urban sub-centers in Shanghai. Figure 1 displays the 5 × 5 km² Wujiaochang area. Wujiaochang is renowned for its extensive array of commercial buildings, including prominent establishments such as Wanda Plaza and Bailian Youyicheng Shopping Mall. These commercial entities contribute to Wujiaochang’s appeal as a highly sought-after shopping destination. In addition to the commercial sector, the area is also home to various hotels, residential buildings, office spaces, and several esteemed universities, such as Fudan University and Tongji University. The unique characteristic of Wujiaochang lies in its diverse building distribution patterns and densities, making it an ideal location for validating the effectiveness of the method proposed in our study.

In this study, we utilized a total of three types of data, including ICESat-2 altimetry data, Google Earth satellite images, and reference data for building height.

ICESat-2 altimetry data. As of now, multiple versions of the ICESat-2 ATL03 product have been publicly released. For the purposes of this study, we utilized the most recent release, version 5, which can be accessed from https://search.earthdata.nasa.gov/ (accessed on 10 December 2021). The data within this version of the product were collected during the time period spanning from October 2018 to November 2021. The collected photon data are displayed in Figure 1. In total, we collected 348,148 ATL03 photons.

Google Earth satellite images. The Google Earth satellite image was downloaded from Google Earth. Figure 1 displays the high-resolution satellite image of the study area captured on 7 February 2021. The image has a spatial resolution of 0.3 m.

Reference data for building height. To assess the accuracy of estimated building heights, the building heights, which were derived from the DSM data provided by the Shanghai Surveying and Mapping Institute, were utilized as reference building heights. More detailed information can be found in references [46,63].

3. Methods

The workflow of our proposed method is depicted in Figure 2. The method includes five steps: roof-to-footprint offset extraction, shadow length extraction, height samples calculation from ICESat-2 data, building height retrieval, and accuracy assessment. The specific details are as follows.

3.1. Roof-to-Footprint Offset Extraction

For building height retrieval from remote sensing imagery, the offset vectors from building roof to building footprint (Figure 3c) are considered good indicators. Generally, the boundary of building footprint is often occluded by its façade in the images (see Figure 3a). Therefore, the building footprints can be estimated by translating the roof boundaries based on the roof-to-footprint offset vectors (see Figure 3d). To simultaneously extract building roofs, offset vectors, and building footprints, we employed the learning offset vector (LOFT) model proposed by Wang et al. [58] to finish the multitasking problem. The LOFT model predicts the building roofs and their associated offset vectors to the footprints by adopting the Mask R-CNN model. During the training stage, the LOFT network takes the remote sensing images as input to the backbone ResNet with feature pyramid network and produce feature map. Then, the region proposal network generates region proposals from the feature map to generate building bounding boxes, roof masks, and offset vectors. During the inference stage, the predicted offset vectors are used to move the roof masks to the footprint masks. For details of the LOFT model, please refer to Wang et al. [58].

3.2. Shadow Length Extraction

As previously mentioned, the shadow length of building is an important indicator in estimating building heights. In our prior research [46], we constructed a U-Net model for shadow extraction based on our own labelled building shadow training data. Our results demonstrated that the well-trained U-Net model was able to effectively extract shadows in GES imagery. Based on the building footprint data extracted in Section 3.1, we utilized the sun’s azimuth angle to segment the shadows and identified the shadows corresponding to individual buildings based on their proximity to the building footprint. We then employed the parallel line method as used in previous studies [25,64] to determine the length of the shadows corresponding to each building. A set of equidistant parallel lines with a spacing of 0.2 m were generated using the sun’s azimuth angle to quantify the length of shadows. Subsequently, to mitigate the influence of irregularities such as spots and gaps within the shadows, the parallel lines were subjected to the Pauta criterion for filtering. The resulting set of parallel lines that meet the criterion were averaged to obtain the representative length of each shadow. For specific details regarding the building shadow length calculation, please refer to our previous study [46].

3.3. Height Samples Calculation from ICESat-2 Data

Data obtained directly from ICESat-2 need to be cleaned, and some preliminary processing is needed to facilitate subsequent use. Here, we use a simple but effective confidence filter method for data cleaning. The ICESat-2 ATL03 product has its own noise recognition algorithm, every photon has attached a confidence attribute (signal_conf_ph) before release. By creating histograms of the number of photons versus the height and computing the signal-to-noise ratio of each histogram bin, the ATL03 product’s noise reduction algorithm identifies each photon as either a likely signal photon or a background photon [65]. In our study, we therefore selected the ATL03 photons with medium and high confidence levels (signal_conf_ph ≥ 3).

To obtain the heights of buildings which locate along the ICESat-2 tracks, it is necessary to clearly identify the intersection situation between the building footprints and ICESat-2 photons. Furthermore, a building footprint usually intersects with several ICESat-2 photons; it is thus important to define the ground photons and the roof photons. The difference between the ground photons and the roof photons can be leveraged to determine the building height.

Here, we employed a buffer zone approach for building height calculation. As shown in Figure 4, we first marked all the photons that fell in the building footprint as candidate roof photons (red points) and labeled the photons located within 5 m buffered area [46] of the footprint as candidate ground photons (black points). Among all candidate roof photons, the ones with larger z values are more likely to be real roof photons. Conversely, for all candidate ground photons, the ones with smaller z values are more likely to be real ground photons. Therefore, we used a percentile approach to determine the elevation of the building roof. That means that the elevation of a building roof was calculated as a specific percentile of all elevations of the candidate roof photons on the corresponding roof part. In our experiments, we selected ten buildings as examples and used seven numbers—50, 60, 70, 80, 85, 90, and 95—as percentiles to calculate the roof elevation. Their results were compared with reference roof elevation. Our results indicated that the 90 percentile be selected for roof elevation calculation because of its relatively better performance. For ground elevation calculation, the 10 percentile was selected due to the smaller errors. Finally, the building height was determined by the difference between the roof elevation and the ground elevation.

3.4. Building Height Retrieval

For building height calculation, both the roof-to-footprint offset ($L_{offset}$) and building shadow length ($L_{shadow}$) have a linear relationship with building height ($H$): $H=k_1{L}_{offset}$ or $H=k_2{L}_{shadow}$, where $k_1$ and $k_2$ are the proportionality coefficients between building height and offset length and shadow length, respectively. As these two methods have different performances in calculating the building heights, we therefore propose to combine these two height calculations for better calculating building height, as shown in Equation (1):

$$H={\omega }_1{L}_{offset}+{\omega }_2{L}_{shadow}+\epsilon$$

(1)

where ${\omega }_1$ and ${\omega }_2$ represent the weights of the two height calculation methods and $\epsilon$ is the error term. Using a number of building height samples, the optimization of the above equation is performed using the least squares method to obtain the optimal parameters (${\omega }_1$, ${\omega }_2$, and $\epsilon$). The least squares method is a mathematical technique used to find the best-fitting curve or line that minimizes the sum of the squared differences between the real building heights ($H_i^{real}$) and the estimated building heights ($H_i^{est}$) from the model. Using the height samples, we may define the error ($E\left({\omega }_1,{\omega }_2\right)$) associated to Equation (1) by:

$$E\left({\omega }_1,{\omega }_2\right)={{\displaystyle \sum }}_{i=1}^n{\left(H_i^{real}-H_i^{est}\right)}^2$$

(2)

The goal of the least squares method is to find values of ${\omega }_1$ and ${\omega }_2$ that minimize the error. In multivariable calculus, the minimum of $E\left({\omega }_1,{\omega }_2\right)$ allows us to find the values of (${\omega }_1$, ${\omega }_2$), such that:

$$\frac{\partial E}{\partial {\omega }_1}=0, \frac{\partial E}{\partial {\omega }_2}=0$$

(3)

Using the building height samples obtained in Section 3.3, we solved the parameters in Equation (1) and used the equation to finish the estimation of the height of all buildings in the study area.

3.5. Accuracy Evaluation

The accuracy of building height estimation was validated with the reference building height data. Three error metrics are used to measure the accuracy of the building height estimation, including Mean Absolute Error (MAE), RMSE, and R Squared (R² score). MAE is the average difference between the estimated height ($H_i$) and the reference height (${\check{H}}_i$). The smaller the value, the better the result:

$$MAE=\frac{1}{n}{{\displaystyle \sum }}_{i=1}^n\left|H_i-{\check{H}}_i\right|$$

(4)

RMSE is the square root of the ratio of the square of the deviation of the estimated height from the reference height to the number of buildings. The RMSE is used to measure the deviation between the estimated height and the reference height.

$$RMSE=\sqrt{\frac{1}{n}{{\displaystyle \sum }}_{i=1}^n{\left(H_i-{\check{H}}_i\right)}^2}$$

(5)

R² score was used to determine how well the estimated height fits the reference height:

$$R^2=1-\raisebox{1ex}{${{\displaystyle \sum }}_{i=1}^n{\left(H_i-{\check{H}}_i\right)}^2$}\!\left/ \!\raisebox{-1ex}{${{\displaystyle \sum }}_{i=1}^n{\left({\overline{H}}_i-{\check{H}}_i\right)}^2$}\right.$$

(6)

RMSE, MAE, and R² are commonly used evaluation metrics for assessing the effectiveness of building height estimation models. RMSE is sensitive to outliers and penalizes larger errors more heavily. Lower RMSE values indicate better model performance, with a value of 0 indicating a perfect fit. RMSE is widely used because it provides a measure of the average magnitude of errors in the predicted values. MAE is less sensitive to outliers compared to RMSE and treats all errors equally. Like RMSE, lower MAE values indicate better model performance, with a value of 0 indicating a perfect fit. MAE is often used when outliers are expected to have a significant impact on the model’s performance. R² ranges from 0 to 1, where 0 indicates that the model does not explain any of the variance, and 1 indicates a perfect fit. In summary, RMSE and MAE provide measures of the average error between predicted and actual values, while R² indicates the proportion of variance explained by the model. These metrics were considered together to comprehensively evaluate the effectiveness of our building estimation model.

4. Results

The LOFT model was trained on the BONAI dataset [58], which consists of 3300 images with a size of 1024 × 1024 pixels. We divided the dataset into three subsets: a training set, a validation set, and a test set, using a ratio of 7:2:1. We also followed the parameter settings recommended in reference [58], which involved training for 24 epochs. The learning rate started at 0.02 and decreased by a factor of 0.1 at the 16th and 22nd epoch. Stochastic gradient descent, with a weight decay of 0.0001 and a momentum of 0.9, was used. The LOFT model was implemented in PyTorch. The U-Net model used for building shadow extraction was trained on a dataset of 1057 GES images used in reference [46]. These images have a size of 1024 × 1024 pixels. The dataset was randomly divided into training, testing, and validation sets in the same ratio of 7:2:1. The U-Net model was trained for 300 epochs using an initial learning rate of 0.001 and a batch size of 16. The U-Net model was implemented using TensorFlow and Keras. Both the LOFT model and U-Net model were implemented on a consumer-level PC equipped with an Intel Core i7-8700 CPU running at 3.20 GHz and an Nvidia GeForce GTX 1080 8 G graphics card.

Following the method described in Section 3, we first obtained the building roofs, shadows, and roof-to-footprint offsets for the study areas, as shown in Figure 5. Visually, the building footprints, shadows, and roof-to-footprint offsets were effectively extracted and they matched well with the GES image. In total, 8216 building roofs and their corresponding shadows, roofs, and offsets were initially extracted. We employed various metrics, including overall accuracy (OA), precision, recall, F-score, and the intersection over union (IOU), to assess the accuracy of extracting building roofs and shadows, with reference to the manually labeled 50 building roofs and building shadows. The results are summarized in

Table 1. The extraction accuracies of building roofs and building shadows.

	Building Roof	Building Shadow
TP	368,922	398,656
TN	1,304,989	1,256,822
FP	36,656	47,200
FN	35,811	43,700
OA	0.96	0.95
Precision	0.91	0.89
Recall	0.91	0.90
F-Score	0.91	0.89
IOU	0.84	0.81

. From Table 1, it can be observed that, in the study area, the average OA for building roofs and shadows extraction reached 96% and 95%, respectively. The IOU of building roofs was 84%, while the IOU of building shadows was 81%. We also found that the false positive error of building shadows was mainly concentrated in areas with dense vegetation, where it was challenging to distinguish building shadows from overlapping vegetation. The false positive error of building footprints was mainly concentrated in areas with high building density, where mutual occlusion of buildings increased the difficulty of detecting building roofs.

In total, we extracted 74,807 ATL03 photons with medium and high confidence levels in the study area, of which 8712 were intersected with the extracted building footprints. Using the method introduced in Section 3.3, we obtained 468 valid building height samples. Using the extracted building shadow length and building offset, the resulted equation for extracting building height is: $H=1.06L_{offset}+0.63L_{shadow}+3.86$. The algorithm reached an accuracy of MAE = 4.90 m, RMSE = 6.91 m, and R² = 0.72 on the building height samples. The calculation of parameters ${\omega }_1$ and ${\omega }_2$ (Equation (1)) in our proposed method was obtained using the least squares method, which ensures the optimal selection of parameters and the best accuracy. From the equation, we found that the weight of offset vector was higher than that of the shadow length in the study area, suggesting that the offset length is more important than shadow length for building height calculation.

Subsequently, we calculated the heights of all the individual buildings in the study area (see Figure 6a). Using the estimated building heights, the 2D building footprints were extruded vertically to generate LOD-1 3D building models, as shown in Figure 6c. We further calculated the maximum, minimum, and average values of building heights, as well as the number of buildings within specified height ranges. The statistical results indicate that most of the buildings have heights ranging from 6 to 115 m, and the average building height in study area is 20.24 m. Additionally, there are 230 buildings with heights greater than 50 m and 218 buildings with heights less than 10 m in the study area.

Based on the building height reference data, we then computed the accuracy metrics (RMSE, MAE, and R²) for all buildings, excluding the 468 buildings used for constructing the building height estimation model. Figure 6b shows the statistical results of the three height estimation metrics (MAE, RMSE, and R²) in the study area. The result shows that the estimated building height matched well with the reference height data, with an R² value of 0.72 in the study area. The RMSE value for the study area is 6.75 m and the MAE in the study area is 4.70 m, indicating that the proposed method has good accuracy. Furthermore, we mapped the height deviations in Figure 7. It can be observed from the distribution of height deviations that 46% of the height deviations are concentrated within a range of −3 m to 3 m. Overlaying the building height deviations onto the high-resolution GES image, we found that buildings with larger MAE values were mainly concentrated in densely populated areas. Building shadows and offset vectors in these areas were affected by occlusions between buildings, resulting in relatively large deviations.

Figure 8 illustrates the MAEs at different ranges of building heights. The result reveals that there was no substantial correlation between the MAE of height estimation and the building height. Nevertheless, a general trend emerged, indicating that lower buildings exhibit smaller MAEs, whereas taller buildings tend to have larger MAEs. Specifically, buildings with heights ranging from 10 to 25 m exhibit the minimum height estimation error, while structures exceeding 30 m in height often display the largest estimation error.

5. Discussion

We compared our proposed method with the shadow-length-based and offset-vector-based methods. We first compared the performance of different height estimation models on height samples. We calculated the MAEs and RMSEs of the shadow-based, offset-based, and our proposed method, as shown in

Table 2. The performance of shadow-based and offset-based methods on building height samples.

	Shadow-Based	Offset-Based
Equation	$\mathrm{BH}=1.10 \times L_{shadow}$ + 6.64	$\mathrm{BH}=1.56 \times L_{offset}$ + 7.96
R2	0.54	0.60
MAE	6.11	5.79
RMSE	8.87	8.24

. Table 2 indicates that, among 468 building height samples, the MAEs of the shadow-based and offset-based were 6.11 m and 5.79 m, respectively. Therefore, our proposed method has the best performance by comprehensively considering both the building shadow length and offset length.

Figure 9 provides a summary of the accuracy of building height estimation achieved by these two methods. As shown in Figure 8, the shadow-based method shows the lowest precision with an R² value of 0.57 and a MAE of 5.7 m. The offset-vector-based method performs slightly better than the shadow-length-based method with an R² value of 0.59 and a MAE of 5.6 m. Therefore, our proposed method has the highest accuracy in building height estimation with an R² value of 0.72 and a MAE of 4.7 m.

We also compared our method with the improved shadow-based method proposed in the literature [46]. It is worth mentioning that the improved shadow-based method proposed in the literature [46] divided the buildings into different categories based on building azimuths and further constructed local shadow-based models for each category, and significantly improved the accuracy of the traditional shadow-based method. As shown in Figure 10, the improved shadow-based method shows a good accuracy with an R² value of 0.63 and a MAE of 5.18 m. Clearly, the accuracy of estimated building heights obtained through the proposed method exceeds that of the improved shadow-based method. All these comparisons reveal that combining both the shadow and offset vector information makes for a significant improvement in building height estimation.

As previously indicated, we found that buildings with larger MAE values are mainly concentrated in densely populated areas. Building shadows and offset vectors in these areas are affected by occlusions between buildings, resulting in relatively large deviations. This is because in densely populated areas, the density of buildings is high, and the spacing between buildings is relatively small. At the same time, the building heights are relatively consistent in densely populated areas. In this situation, two main factors contribute to the inaccuracies in extracting offsets and measuring building shadow length. Firstly, severe occlusions between buildings makes it challenging to accurately extract offsets in remote sensing images. Secondly, the overlapping shadows between buildings and the presence of missing shadows result in the imprecise extraction of building shadow length. These two factors introduce significant errors in estimating building heights in such densely populated areas.

There are some limitations in our proposed method. Firstly, the high-precision extraction of shadow and offset vector information is crucial for building height estimation. It requires the input satellite or aerial images must meet strict criteria (i.e., high-resolution and clear) to ensure that the appropriate building shadows and offset vectors can be captured; and on the other hand, it also requires that the algorithms used for building information extraction must be sufficiently robust. The rapidly advancing deep learning technology provides opportunities for solving this problem. In the future, more advanced deep learning algorithms need to be tested to further improve the extraction of building information in this study. Secondly, the ATL03 data exhibit sparsity and non-uniformity due to the trajectory characteristics of the ICESat-2satellite. Specifically, a 3 km distance always exists between each beam and, consequently, not all building footprints can be intersected by ICESat-2 ATL03 photons, with only a few portions of building height samples being able to be calculated. In the future, a mixture of ICESat-2 and other space-borne LiDAR data (e.g., GEDI data) may be a solution to mitigate the impact of data sparsity. With more building height samples available, we expect that the performance of our proposed method can be further enhanced. Thirdly, the elevation of building roofs or ground from ICESat-2 photons may not be reliable since it can be influenced by surrounding objects. This is because the photon may actually represent the elevation of surrounding objects other than the buildings it falls on. Previous studies [43,66] used an iteration approach to calculate the offset by minimizing the height residuals between photons and reference DEM data, and discovered that the ATL03 photons had an offset of approximately 1–3 m. Therefore, we suggest adopting the approach of creating buffer zones to increase the number of photons or calculating the offset to correct the offsets of photons to obtain reliable results.

6. Conclusions

Understanding the vertical structure of urban areas is crucial for comprehending the impact of the built environment on the surroundings. Buildings play a significant role in shaping this vertical structure and are therefore essential in studying the relationship between the built environment and its effects on the environment. Consequently, the extraction of building height will offer new perspectives on the vertical composition of urban areas and the patterns of urban development. This study proposes a method for extracting building height based on ICESat-2 photons and high-resolution remote sensing data. Taking the building height samples calculated from ICESat-2 ATL03 photons, we developed a building height estimation model by combining building shadow and offset length information extracted from GES images. A representative urban area located in the central part of Shanghai city was selected for the experiment, and the results indicate that the proposed method has a high accuracy in estimating building heights, with a MAE of 4.70 m. Compared with the methods solely based on shadow length or offset vector length, our method performs better.

Our contributions include two aspects. Firstly, in response to the challenge of obtaining building heights in urban areas continuously using ICESat-2 data, we proposed a method that combines ICESat-2 ATL03 photons and high-resolution remote sensing imagery to estimate individual building heights within urban areas. Secondly, we have innovatively incorporated two height estimation features, namely the length of building shadows and building offset vectors, into the building height estimation process. By leveraging these additional features and employing optimization techniques, we have significantly improved the accuracy of building height estimation. Given that the proposed method can effectively extract building height with a high accuracy, it provides the possibility of extracting building heights at city level. Therefore, a possible future research direction is to produce a dataset of building heights at the individual building level using multi-source remote sensing data, including ICESat-2 data and high-resolution remote sensing data. Furthermore, this work can be enhanced by the integration of interdisciplinary knowledge. For example, knowledge from the field of urban planning can be introduced to provide valuable insights into the specific characteristics and requirements of different types of buildings in urban areas. This knowledge can help in refining the algorithms used for building height estimation and in interpreting the results in the context of urban development and planning.