Leveraging Open-Source Satellite-Derived Building Footprints for Height Inference

Populations are increasing at a global scale, and it is estimated that by 2050, 68% of the global population will live in urban environments [17, 20]. Buildings are a key component of the urban environment and their footprints have been used across a myriad of subjects such as population density estimation [31, 32], building energy usage [12, 39], disaster management [28], building type [1], building height [6, 25, 37] and urban heat islands (UHI’s) [10]. Being able to characterize the built environment is imperative to address these issues and information on building footprints, building height and urban morphology is critical in these efforts.

Fortunately, over the past decade, there has been a dramatic increase in the amount of open-source building footprint datum available from organizations such as Microsoft²²2https://www.microsoft.com/en-us/maps/bing-maps/building-footprints, Google³³3https://sites.research.google/open-buildings/, and Oak Ridge National Laboratory (ORNL)⁴⁴4https://gis-fema.hub.arcgis.com/pages/usa-structures. Between these products, there are over 3 billion footprints available for use. However, each of the aforementioned products is generated via differing methods for the pixel extraction/segmentation to identify buildings and the regularization process of the identified footprints. Furthermore, differences in imagery sources and resolution as well as environmental factors such as shadows or sun angle can also influence the footprint extraction and regularization process [13].

The processing workflow from Microsoft is described in their documentation as first leveraging a deep neural network (DNN) to identify buildings from aerial imagery and then converting the identified pixels into polygons representing building footprints⁵⁵5https://github.com/microsoft/GlobalMLBuildingFootprints. The best available information on Google’s footprint generation is from a paper in 2023 by Sirko et al. [35]. In their report, the authors describe utilizing a U-Net model, a common approach for segmenting satellite datum [2, 29, 38]. Once extracted, the building footprints are then processed through a contouring algorithm that realigns groups of adjacent polygons to regularize the building footprints [35]. The authors also provide a caveat that newer versions (v2 and v3) of the Google Open Buildings dataset underwent further improvements that are not documented. Of the three datasets, ORNL provides the highest level of detail in how building footprints were both extracted and regularized from satellite datum as they leverage a deep convolutional neural network (CNN) framework for pixel extraction and the ArcGIS proprietary building footprint regularization module ⁶⁶6https://pro.arcgis.com/en/pro-app/latest/tool-reference/3d-analyst/regularize-building-footprint.htm [41, 42]. However, the ORNL footprint dataset is only available publicly in the United States (U.S.), so it lacks the volume and spatial scale of building footprints that Microsoft and Google provide. While Microsoft, Google, and ORNL each utilize a deep learning framework to identify, delineate, and regularize the footprints, there are proven differences associated with the footprints provided by each entity [8, 14].

Chamberlain et al. found substantial differences between footprint patterns displayed by Microsoft and Google when comparing the products at a grid scale in Africa [8]. The authors noted that consideration is needed by users regarding the suitability of the specific building footprint dataset for its intended application. For example, in urban areas, Microsoft seemed to have better coverage in relation to the number of matching footprints, but this pattern was not universal. Also in Africa, Gonzales found patterns of irregularity when comparing Google and Microsoft, but at a building-by-building level [14]. When investigating urban areas, Microsoft tended to generate larger footprints which may encapsulate multiple buildings while Google seemed to have more, smaller buildings. When rural areas were investigated, the building counts were relatively similar, showing little discrepancy. Both at scale and at a building-by-building level, care must be taken when leveraging footprint datasets [8, 14].

Recently, research has shown that building height is obtainable based on building footprint information alone [37]. The authors showcased a novel method to infer height at a high accuracy using only information derived from an individual buildings footprint. Furthermore, they did so based on footprints extracted from both lidar and satellite datum. The ability to infer height from footprints extracted from satellite datum allows for this approach to generate building height maps at large scale. However, the authors only demonstrated their approach on building footprints developed by ORNL and the model inference may produce irregular results when exposed to a different footprint source. For example, in Stipek et. al [37] they discuss that the features with the highest impact on inferring building height were contextual (number of neighbors) and engineered (complexity of footprint shape). It has been proven that at both a grid [8] and building-by-building level [14], Microsoft and Google have differing shapes and sizes which would affect the contextual and engineered metrics generated at the building level. Therefore, it would be imprudent to assume the approach proposed by Stipek et al. [37] can be applied to other footprint datasets without further testing.

In this paper we demonstrate that it is possible to infer building height at a building-by-building level, agnostic to footprint source. We prove this by comparing the inferred heights from two distinct products, ORNL and Microsoft across 10 cities in the U.S., with figure 1 showcasing the spatial extent of our research. We show that regardless of differences associated with the extraction, regularization of the footprints and other factors, such as imagery date or environmental factors, our approach of inferring height from footprints can be applied to ORNL and Microsoft footprints. This signifies that it is possible to leverage the 1.2 billion footprints which Microsoft has made openly available to infer height at a global scale. Google footprints are not currently available in the U.S. and this research only focused on ORNL and Microsoft footprints.

While the authors acknowledge a deep field of literature in relation to leveraging deep learning on satellite datum, we would like to bring attention to works which relate to extracting building information. Secondly, we focus on studies that have inferred height from features derived from building footprints.

Here, we discuss the methodology used for our research which aims to address if it is possible to infer height based on footprints derived from satellite datum which have been identified and regularized using varying methods. For this work we leverage 3.09 million building records across the U.S. (Table 1). We selected 10 cities within the U.S. that had satellite derived footprints from ORNL and Microsoft which overlapped with lidar derived footprints, which had a height associated with the footprints (Figure 1).

The XGB model was chosen based on its superior performance in relation to the other models applied (LR, RF, TabNet). While we acknowledge that the RF outperformed the XGB in certain cities, the XGB was more consistent across both datasets (ORNL, Microsoft). All the results in the following sections are the inferred values from the XGB model. For the results associated with the LR, RF, and TabNet models please see Supplementary Table S3.

The expansion of open-source building footprint datasets has provided the possibility for leveraging these products to characterize the built environment. Our results show that, across 3.09 million buildings in the U.S., our method of inferring height from footprint information alone is effective for datasets produced by ORNL as well as the much larger and globally available dataset from Microsoft. Furthermore, our height prediction process is reliable and agnostic to building footprint source. This finding ensures that our approach of inferring height from footprint morphology features can be scaled to leverage other publicly available footprints, such as Microsoft footprints. By inferring building height, this method provides valuable contextual information for population density estimation, building energy, disaster management, and UHI’s [10, 12, 31, 32, 39].

While the main objective of this research is to test the efficacy of leveraging open-source footprints, we applied various models to ensure the best possible method was selected. It is important to note that the TabNet model did not outperform either the RF or XGB for any cities across both footprint sources. In some instances, such as in Phoenix, the difference was -18% in relation to $R^2$ results displayed by the XGB. However, in other cities (Houston), the TabNet outperformed the percent improvement displayed by the RF by +1% in relation to $R^2$. Regardless, the tree based approach consistently outperformed the TabNet model which signifies that while deep learning models developed for tabular data have made progress [21, 22], in this instance tree based models show higher accuracy.

While successful, our study does have limitations that need to be acknowledged. First, the area-of-interest is only within one country, the U.S., and more work is needed to expand this approach to additional countries. It is known that the built environment varies both spatially and temporally and a more diverse sample set is needed to further validate this approach [6, 25, 27, 34, 37]. Another limitation is that during our strict one-to-one conflation process, building footprints that don’t have a one-to-one match are removed and therefore not included in the morphology feature generation. Contextual features that look at a building’s neighbors have been found to be influential to the model’s behaviour and therefore, the model may not perform as well on the filtered dataset as it would on the unfiltered dataset [37]. For example, the generated feature n count measures the number of centroids within a defined radius surrounding a building. This was evidenced by the range of values displayed for the n count 500 for the ORNL when compared to Microsoft (Table S4). The range in values for n count 500 signifies that at a 500 m radius, there are differences associated with the total number of buildings, which can influence the model’s ability to infer an individual building’s height.

Additionally, there needs to be a formal analysis completed to understand if it is possible to train on one distinct footprint dataset and test on another. For example, due to the differences discussed between the Google and Microsoft datasets within Africa [8, 14], can it be possible to train on the Google footprints to then infer height on the Microsoft footprints. While the approach presented in this research shows the ability to infer height agnostic of footprint source, it does not test across the sources, i.e. training on ORNL and testing on Microsoft.

Furthermore, the differences in footprint shape based on the pixel identification and regularization process can lead to irregularities in predicted height (Fig. 3). For example, the complexity ratio, an engineered feature that is the shape length divided by the shape area which shows high significance in relation to inferring height, can vary depending on the footprint shape [37]. For example, in Boston, we display the Microsoft, ORNL and lidar footprint for one specific building where the footprint shapes are similar and the height prediction for the MS footprint is $8.52m$ and $7.82m$ for the ONRL footprint (Fig. 3). However, when there are differences displayed by the building’s footprint, there can be large differences associated with the predicted height, as evidenced in the example in Seattle where the height inferred from the Microsoft footprint is $18.11m$ and $50.86m$ on the ORNL footprint. Therefore, based on the shape and size of the footprint, the inferred height may vary, as displayed in figure 3. While the majority of the morphology features showed minimal differences in their distributions, it is important to note that some of the engineered features, such as complexity ps showed differences (Table S4). Therefore, while the approach presented in this research has proven it is possible to infer height from various footprint sources, it would be irresponsible to apply without additional testing if leveraging an additional footprint source, such as Google.

This research has highlighted the need for multiple avenues of future work. A comprehensive analysis in relation to the distributions displayed by the morphological features is necessary to truly understand the differences displayed between ORNL and Microsoft datasets. As the scope of this paper is to investigate if it is possible to infer height from both products, we do not fully investigate the differences displayed by the ORNL and Microsoft footprints in relation to the engineered and contextual features. Other potential work could explore the possibility of training on one homogeneous footprint data source and testing on another.

In this paper, we demonstrate the ability of our method to infer height from building footprints derived from different sources (ORNL, Microsoft). Our results show that, across over 3 million footprints in the U.S., we successfully infer building height within $3m$ of the ground truth height with 98% accuracy. More importantly, while previous work has proven that it is possible to infer height from footprints derived from satellite datum, this is the first time, to our knowledge, that a comparison study has been completed that indicates a machine-learning height inference method can be applied across multiple datasets. We believe our approach is successful due to the ability to learn from the distinct morphology features, regardless of the footprint dataset. This is a significant finding which displays the generalization of our method to inferring height regardless of how the building footprints are extracted and regularized. Furthermore, this opens the door to now leverage the over 1 billion Microsoft footprints to infer building height at a building-by-building level across the globe.