Document Open Access Logo

Semi-Supervised Learning from Street-View Images and OpenStreetMap for Automatic Building Height Estimation

Authors Hao Li, Zhendong Yuan, Gabriel Dax, Gefei Kong, Hongchao Fan, Alexander Zipf, Martin Werner

PDF
Thumbnail PDF

File

LIPIcs.GIScience.2023.7.pdf
  • Filesize: 4.62 MB
  • 15 pages

Document Identifiers

Author Details

Hao Li
  • Technical University of Munich, Germany
Zhendong Yuan
  • Utrecht University, The Netherlands
Gabriel Dax
  • Technical University of Munich, Germany
Gefei Kong
  • Norwegian University of Science and Technology, Trondheim, Norway
Hongchao Fan
  • Norwegian University of Science and Technology, Trondheim, Norway
Alexander Zipf
  • GIScience Chair, Heidelberg University, Germany
Martin Werner
  • Technical University of Munich, Germany

Cite As Get BibTex

Hao Li, Zhendong Yuan, Gabriel Dax, Gefei Kong, Hongchao Fan, Alexander Zipf, and Martin Werner. Semi-Supervised Learning from Street-View Images and OpenStreetMap for Automatic Building Height Estimation. In 12th International Conference on Geographic Information Science (GIScience 2023). Leibniz International Proceedings in Informatics (LIPIcs), Volume 277, pp. 7:1-7:15, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2023) https://doi.org/10.4230/LIPIcs.GIScience.2023.7

Abstract

Accurate building height estimation is key to the automatic derivation of 3D city models from emerging big geospatial data, including Volunteered Geographical Information (VGI). However, an automatic solution for large-scale building height estimation based on low-cost VGI data is currently missing. The fast development of VGI data platforms, especially OpenStreetMap (OSM) and crowdsourced street-view images (SVI), offers a stimulating opportunity to fill this research gap. In this work, we propose a semi-supervised learning (SSL) method of automatically estimating building height from Mapillary SVI and OSM data to generate low-cost and open-source 3D city modeling in LoD1. The proposed method consists of three parts: first, we propose an SSL schema with the option of setting a different ratio of "pseudo label" during the supervised regression; second, we extract multi-level morphometric features from OSM data (i.e., buildings and streets) for the purposed of inferring building height; last, we design a building floor estimation workflow with a pre-trained facade object detection network to generate "pseudo label" from SVI and assign it to the corresponding OSM building footprint. In a case study, we validate the proposed SSL method in the city of Heidelberg, Germany and evaluate the model performance against the reference data of building heights. Based on three different regression models, namely Random Forest (RF), Support Vector Machine (SVM), and Convolutional Neural Network (CNN), the SSL method leads to a clear performance boosting in estimating building heights with a Mean Absolute Error (MAE) around 2.1 meters, which is competitive to state-of-the-art approaches. The preliminary result is promising and motivates our future work in scaling up the proposed method based on low-cost VGI data, with possibilities in even regions and areas with diverse data quality and availability.

Subject Classification

ACM Subject Classification
  • Information systems
  • Information systems → Geographic information systems
Keywords
  • OpenStreetMap
  • Street-view Images
  • VGI
  • GeoAI
  • 3D city model
  • Facade parsing

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Supplementary Materials

References

  1. A Alobeid, K Jacobsen, and C Heipke. Building height estimation in urban areas from very high resolution satellite stereo images. In ISPRS Hannover Workshop, volume 5, pages 2-5, 2009. Google Scholar
  2. Joshua S Apte, Kyle P Messier, Shahzad Gani, Michael Brauer, Thomas W Kirchstetter, Melissa M Lunden, Julian D Marshall, Christopher J Portier, Roel CH Vermeulen, and Steven P Hamburg. High-resolution air pollution mapping with google street view cars: exploiting big data. Environmental science & technology, 51(12):6999-7008, 2017. Google Scholar
  3. Kuldip Singh Atwal, Taylor Anderson, Dieter Pfoser, and Andreas Züfle. Predicting building types using openstreetmap. Scientific Reports, 12(1):19976, 2022. Google Scholar
  4. Jérémy Bernard, Erwan Bocher, Elisabeth Le Saux Wiederhold, François Leconte, and Valéry Masson. Estimation of missing building height in OpenStreetMap data: A French case study using GeoClimate 0.0.1. Geoscientific Model Development, 15(19):7505-7532, October 2022. URL: https://doi.org/10.5194/gmd-15-7505-2022.
  5. Filip Biljecki and Yoong Shin Chow. Global building morphology indicators. Computers, Environment and Urban Systems, 95:101809, 2022. Google Scholar
  6. Filip Biljecki, Hugo Ledoux, and Jantien Stoter. Generating 3d city models without elevation data. Computers, Environment and Urban Systems, 64:1-18, 2017. Google Scholar
  7. Filip Biljecki, Hugo Ledoux, and JE Stoter. Height references of citygml lod1 buildings and their influence on applications. In Proceedings. 9th ISPRS 3DGeoInfo Conference 2014, 11-13 November 2014, Dubai, UAE,(authors version). Citeseer, 2014. Google Scholar
  8. Yinxia Cao and Xin Huang. A deep learning method for building height estimation using high-resolution multi-view imagery over urban areas: A case study of 42 Chinese cities. Remote Sensing of Environment, 264:112590, October 2021. URL: https://doi.org/10.1016/j.rse.2021.112590.
  9. Bumseok Chun and Jean-Michel Guldmann. Two- and three-dimensional urban core determinants of the urban heat island: A statistical approach. Journal of Environmental Science and Engineering B, 1(3):363-378, 2012. Google Scholar
  10. Thomas Esch, Julian Zeidler, Daniela Palacios-Lopez, Mattia Marconcini, Achim Roth, Milena Mönks, Benjamin Leutner, Elisabeth Brzoska, Annekatrin Metz-Marconcini, Felix Bachofer, et al. Towards a large-scale 3d modeling of the built environment - joint analysis of tandem-x, sentinel-2 and open street map data. Remote Sensing, 12(15):2391, 2020. Google Scholar
  11. Hongchao Fan and Liqiu Meng. A three-step approach of simplifying 3d buildings modeled by citygml. International Journal of Geographical Information Science, 26(6):1091-1107, 2012. Google Scholar
  12. Hongchao Fan and Alexander Zipf. Modelling the world in 3d from vgi/crowdsourced data. European handbook of crowdsourced geographic information, 435, 2016. Google Scholar
  13. Hongchao Fan, Alexander Zipf, and Qing Fu. Estimation of building types on openstreetmap based on urban morphology analysis. Connecting a digital Europe through location and place, pages 19-35, 2014. Google Scholar
  14. Marcus Goetz. Towards generating highly detailed 3d citygml models from openstreetmap. International Journal of Geographical Information Science, 27(5):845-865, 2013. Google Scholar
  15. Peng Gong, Zhan Li, Huabing Huang, Guoqing Sun, and Lei Wang. ICESat GLAS Data for Urban Environment Monitoring. IEEE Transactions on Geoscience and Remote Sensing, 49(3):1158-1172, March 2011. URL: https://doi.org/10.1109/TGRS.2010.2070514.
  16. Michael F. Goodchild. Citizens as sensors: The world of volunteered geography. GeoJournal, 69:211-221, August 2007. URL: https://doi.org/10.1007/s10708-007-9111-y.
  17. Krzysztof Janowicz, Song Gao, Grant McKenzie, Yingjie Hu, and Budhendra Bhaduri. Geoai: spatially explicit artificial intelligence techniques for geographic knowledge discovery and beyond. International Journal of Geographical Information Science, 34(4):625-636, 2020. Google Scholar
  18. Longlong Jing and Yingli Tian. Self-supervised visual feature learning with deep neural networks: A survey. IEEE transactions on pattern analysis and machine intelligence, 43(11):4037-4058, 2020. Google Scholar
  19. Thomas H Kolbe, Gerhard Gröger, and Lutz Plümer. Citygml-3d city models and their potential for emergency response. In Geospatial information technology for emergency response, pages 273-290. CRC Press, 2008. Google Scholar
  20. Gefei Kong and Hongchao Fan. Enhanced facade parsing for street-level images using convolutional neural networks. IEEE Transactions on Geoscience and Remote Sensing, 59(12):10519-10531, 2020. Google Scholar
  21. Julia Kubanek, Eike-Marie Nolte, Hannes Taubenböck, Friedemann Wenzel, and Martin Kappas. Capacities of remote sensing for population estimation in urban areas. Earthquake Hazard Impact and Urban Planning, pages 45-66, 2014. Google Scholar
  22. Dong-Hyun Lee et al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICML, volume 3(2), page 896, 2013. Google Scholar
  23. Hao Li, Benjamin Herfort, Wei Huang, Mohammed Zia, and Alexander Zipf. Exploration of openstreetmap missing built-up areas using twitter hierarchical clustering and deep learning in mozambique. ISPRS Journal of Photogrammetry and Remote Sensing, 166:41-51, 2020. Google Scholar
  24. Hao Li, Benjamin Herfort, Sven Lautenbach, Jiaoyan Chen, and Alexander Zipf. Improving openstreetmap missing building detection using few-shot transfer learning in sub-saharan africa. Transactions in GIS, 26(8):3125-3146, 2022. Google Scholar
  25. Xuecao Li, Yuyu Zhou, Peng Gong, Karen C. Seto, and Nicholas Clinton. Developing a method to estimate building height from Sentinel-1 data. Remote Sensing of Environment, 240:111705, April 2020. URL: https://doi.org/10.1016/j.rse.2020.111705.
  26. Chao-Jung Liu, Vladimir A. Krylov, Paul Kane, Geraldine Kavanagh, and Rozenn Dahyot. IM2ELEVATION: Building Height Estimation from Single-View Aerial Imagery. Remote Sensing, 12(17):2719, January 2020. URL: https://doi.org/10.3390/rs12172719.
  27. Nikola Milojevic-Dupont, Nicolai Hans, Lynn H Kaack, Marius Zumwald, François Andrieux, Daniel de Barros Soares, Steffen Lohrey, Peter-Paul Pichler, and Felix Creutzig. Learning from urban form to predict building heights. Plos one, 15(12):e0242010, 2020. Google Scholar
  28. Hui En Pang and Filip Biljecki. 3d building reconstruction from single street view images using deep learning. International Journal of Applied Earth Observation and Geoinformation, 112:102859, 2022. Google Scholar
  29. Yujin Park and Jean-Michel Guldmann. Creating 3D city models with building footprints and LIDAR point cloud classification: A machine learning approach. Computers, Environment and Urban Systems, 75:76-89, May 2019. URL: https://doi.org/10.1016/j.compenvurbsys.2019.01.004.
  30. Martin Raifer, Rafael Troilo, Fabian Kowatsch, Michael Auer, Lukas Loos, Sabrina Marx, Katharina Przybill, Sascha Fendrich, Franz-Benjamin Mocnik, and Alexander Zipf. Oshdb: a framework for spatio-temporal analysis of openstreetmap history data. Open Geospatial Data, Software and Standards, 4:1-12, 2019. Google Scholar
  31. Joseph Redmon and Ali Farhadi. Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767, 2018. Google Scholar
  32. Eirik Resch, Rolf André Bohne, Trond Kvamsdal, and Jardar Lohne. Impact of urban density and building height on energy use in cities. Energy Procedia, 96:800-814, 2016. Google Scholar
  33. Yao Sun, Yuansheng Hua, Lichao Mou, and Xiao Xiang Zhu. Large-scale Building Height Estimation from Single VHR SAR image Using Fully Convolutional Network and GIS building footprints. In 2019 Joint Urban Remote Sensing Event (JURSE), pages 1-4, May 2019. URL: https://doi.org/10.1109/JURSE.2019.8809037.
  34. Heike Tost, Markus Reichert, Urs Braun, Iris Reinhard, Robin Peters, Sven Lautenbach, Andreas Hoell, Emanuel Schwarz, Ulrich Ebner-Priemer, Alexander Zipf, et al. Neural correlates of individual differences in affective benefit of real-life urban green space exposure. Nature neuroscience, 22(9):1389-1393, 2019. Google Scholar
  35. Zhiyong Wang, Tessio Novack, Yingwei Yan, and Alexander Zipf. Quiet route planning for pedestrians in traffic noise polluted environments. IEEE Transactions on Intelligent Transportation Systems, 22(12):7573-7584, 2020. Google Scholar
  36. Abraham Noah Wu and Filip Biljecki. Roofpedia: Automatic mapping of green and solar roofs for an open roofscape registry and evaluation of urban sustainability. Landscape and Urban Planning, 214:104167, 2021. Google Scholar
  37. Michael Wurm, Hannes Taubenböck, Mathias Schardt, Thomas Esch, and Stefan Dech. Object-based image information fusion using multisensor earth observation data over urban areas. International Journal of Image and Data Fusion, 2(2):121-147, 2011. Google Scholar
  38. Xiongfeng Yan, Tinghua Ai, Min Yang, and Hongmei Yin. A graph convolutional neural network for classification of building patterns using spatial vector data. ISPRS journal of photogrammetry and remote sensing, 150:259-273, 2019. Google Scholar
  39. Yizhen Yan and Bo Huang. Estimation of building height using a single street view image via deep neural networks. ISPRS Journal of Photogrammetry and Remote Sensing, 192:83-98, October 2022. URL: https://doi.org/10.1016/j.isprsjprs.2022.08.006.
  40. Zhendong Yuan, Jules Kerckhoffs, Gerard Hoek, and Roel Vermeulen. A knowledge transfer approach to map long-term concentrations of hyperlocal air pollution from short-term mobile measurements. Environmental Science & Technology, September 2022. URL: https://doi.org/10.1021/acs.est.2c05036.
  41. Chaoquan Zhang, Hongchao Fan, and Gefei Kong. Vgi3d: an interactive and low-cost solution for 3d building modelling from street-level vgi images. Journal of Geovisualization and Spatial Analysis, 5(2):1-16, 2021. Google Scholar
  42. Chenni Zhang, Yunfan Cui, Zeyao Zhu, San Jiang, and Wanshou Jiang. Building Height Extraction from GF-7 Satellite Images Based on Roof Contour Constrained Stereo Matching. Remote Sensing, 14(7):1566, January 2022. URL: https://doi.org/10.3390/rs14071566.
  43. Yunxiang Zhao, Jianzhong Qi, and Rui Zhang. CBHE: Corner-based Building Height Estimation for Complex Street Scene Images. In The World Wide Web Conference, WWW '19, pages 2436-2447, New York, NY, USA, May 2019. Association for Computing Machinery. URL: https://doi.org/10.1145/3308558.3313394.
  44. Xiaojin Jerry Zhu. Semi-supervised learning literature survey. University of Wisconsin-Madison Department of Computer Sciences, 2005. Google Scholar
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2024 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact