Document Open Access Logo

U-Prithvi: Integrating a Foundation Model and U-Net for Enhanced Flood Inundation Mapping

Authors Vit Kostejn , Yamil Essus , Jenna Abrahamson , Ranga Raju Vatsavai

PDF

File

Thumbnail PDF
PDF
LIPIcs.GIScience.2025.18.pdf
  • Filesize: 2.29 MB
  • 17 pages

Document Identifiers

Subject Classification

ACM Subject Classification
  • Computing methodologies → Image segmentation
Keywords
  • GeoAI
  • flood mapping
  • foundation model
  • U-Net
  • Prithvi

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Abstract

In recent years, large pre-trained models, commonly referred to as foundation models, have become increasingly popular for various tasks leveraging transfer learning. This trend has expanded to remote sensing, where transformer-based foundation models such as Prithvi, msGFM, and SatSwinMAE have been utilized for a range of applications. While these transformer-based models, particularly the Prithvi model, exhibit strong generalization capabilities, they have limitations on capturing fine-grained details compared to convolutional neural network architectures like U-Net in segmentation tasks. In this paper, we propose a novel architecture, U-Prithvi, which combines the strengths of the Prithvi transformer with those of U-Net. We introduce a RandomHalfMaskLayer to ensure balanced learning from both models during training. Our approach is evaluated on the Sen1Floods11 dataset for flood inundation mapping, and experimental results demonstrate better performance of U-Prithvi over both individual models, achieving improved performance on out-of-sample data. While this principle is illustrated using the Prithvi model, it is easily adaptable to other foundation models.

Cite As Get BibTex

Vit Kostejn, Yamil Essus, Jenna Abrahamson, and Ranga Raju Vatsavai. U-Prithvi: Integrating a Foundation Model and U-Net for Enhanced Flood Inundation Mapping. In 13th International Conference on Geographic Information Science (GIScience 2025). Leibniz International Proceedings in Informatics (LIPIcs), Volume 346, pp. 18:1-18:17, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2025) https://doi.org/10.4230/LIPIcs.GIScience.2025.18

Author Details

Vit Kostejn
  • Charles University, Prague, Czech Republic
Yamil Essus
  • Industrial and Systems Engineering Department, North Carolina State University, Raleigh, NC, USA
Jenna Abrahamson
  • Center for Geospatial Analytics, North Carolina State University, Raleigh, NC, USA
Ranga Raju Vatsavai
  • Computer Science Department, North Carolina State University, Raleigh, NC, USA

Funding

  • Abrahamson, Jenna: Funded by a National Science Foundation Graduate Research Fellowship Grant No. DGE-2137100.

Supplementary Materials

References

  1. November 2024. URL: https://github.com/cloudtostreet/Sen1Floods11.
  2. Akhyar Akhyar, Mohd Asyraf Zulkifley, Jaesung Lee, Taekyung Song, Jaeho Han, Chanhee Cho, Seunghyun Hyun, Youngdoo Son, and Byung-Woo Hong. Deep artificial intelligence applications for natural disaster management systems: A methodological review. Ecological Indicators, 163:112067, June 2024. URL: https://doi.org/10.1016/j.ecolind.2024.112067.
  3. Reza Azad, Ehsan Khodapanah Aghdam, Amelie Rauland, Yiwei Jia, Atlas Haddadi Avval, Afshin Bozorgpour, Sanaz Karimijafarbigloo, Joseph Paul Cohen, Ehsan Adeli, and Dorit Merhof. Medical image segmentation review: The success of u-net. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12):10076-10095, 2024. URL: https://doi.org/10.1109/TPAMI.2024.3435571.
  4. Roberto Bentivoglio, Elvin Isufi, Sebastian Nicolaas Jonkman, and Riccardo Taormina. Deep learning methods for flood mapping: a review of existing applications and future research directions. Hydrology and Earth System Sciences, 26(16):4345-4378, August 2022. URL: https://doi.org/10.5194/hess-26-4345-2022.
  5. Amrendra Bhushan, Vikas Chandra Goyal, and Arun Lal Lal Srivastav. Greenhouse gas emissions from inland water bodies and their rejuvenation: a review. Journal of Water and Climate Change, page jwc2024561, October 2024. URL: https://doi.org/10.2166/wcc.2024.561.
  6. Derrick Bonafilia, Beth Tellman, Tyler Anderson, and Erica Issenberg. Sen1floods11: a georeferenced dataset to train and test deep learning flood algorithms for sentinel-1. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 835-845, June 2020. URL: https://doi.org/10.1109/CVPRW50498.2020.00113.
  7. Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L. Yuille. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4):834-848, April 2018. URL: https://doi.org/10.1109/TPAMI.2017.2699184.
  8. Martin Claverie, Junchang Ju, Jeffrey G. Masek, Jennifer L. Dungan, Eric F. Vermote, Jean-Claude Roger, Sergii V. Skakun, and Christopher Justice. The harmonized landsat and sentinel-2 surface reflectance data set. Remote Sensing of Environment, 219:145-161, December 2018. URL: https://doi.org/10.1016/j.rse.2018.09.002.
  9. Jesline Daniel, J. T. Anita Rose, F. Sangeetha Francelin Vinnarasi, and Venkatesan Rajinikanth. Vgg-unet/vgg-segnet supported automatic segmentation of endoplasmic reticulum network in fluorescence microscopy images. Scanning, 2022(1):7733860, 2022. URL: https://doi.org/10.1155/2022/7733860.
  10. Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248-255, June 2009. ISSN: 1063-6919. URL: https://doi.org/10.1109/CVPR.2009.5206848.
  11. Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, 2016. URL: http://www.deeplearningbook.org.
  12. Boran Han, Shuai Zhang, Xingjian Shi, and Markus Reichstein. Bridging Remote Sensors with Multisensor Geospatial Foundation Models, April 2024. arXiv:2404.01260. URL: https://doi.org/10.48550/arXiv.2404.01260.
  13. Ali Hatamizadeh, Yucheng Tang, Vishwesh Nath, Dong Yang, Andriy Myronenko, Bennett Landman, Holger Roth, and Daguang Xu. UNETR: Transformers for 3D Medical Image Segmentation, October 2021. arXiv:2103.10504. URL: https://doi.org/10.48550/arXiv.2103.10504.
  14. Xin He, Yong Zhou, Jiaqi Zhao, Di Zhang, Rui Yao, and Yong Xue. Swin Transformer Embedding UNet for Remote Sensing Image Semantic Segmentation. IEEE Transactions on Geoscience and Remote Sensing, 60:1-15, 2022. URL: https://doi.org/10.1109/TGRS.2022.3144165.
  15. Chang Huang, Yun Chen, Shiqiang Zhang, and Jianping Wu. Detecting, extracting, and monitoring surface water from space using optical sensors: A review. Reviews of Geophysics, 56(2):333-360, 2018. URL: https://doi.org/10.1029/2018RG000598.
  16. Intergovernmental Panel On Climate Change (Ipcc). Climate Change 2021 – The Physical Science Basis: Working Group I Contribution to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press, July 2023. URL: https://doi.org/10.1017/9781009157896.
  17. Johannes Jakubik, Sujit Roy, C. E. Phillips, Paolo Fraccaro, Denys Godwin, Bianca Zadrozny, Daniela Szwarcman, Carlos Gomes, Gabby Nyirjesy, Blair Edwards, Daiki Kimura, Naomi Simumba, Linsong Chu, S. Karthik Mukkavilli, Devyani Lambhate, Kamal Das, Ranjini Bangalore, Dario Oliveira, Michal Muszynski, Kumar Ankur, Muthukumaran Ramasubramanian, Iksha Gurung, Sam Khallaghi, Hanxi, Li, Michael Cecil, Maryam Ahmadi, Fatemeh Kordi, Hamed Alemohammad, Manil Maskey, Raghu Ganti, Kommy Weldemariam, and Rahul Ramachandran. Foundation models for generalist geospatial artificial intelligence. CoRR, November 2023. arXiv:2310.18660. URL: https://doi.org/10.48550/arXiv.2310.18660.
  18. A. Emily Jenifer and Sudha Natarajan. Deepflood: A deep learning based flood detection framework using feature-level fusion of multi-sensor remote sensing images. JUCS - Journal of Universal Computer Science, 28(33):329-343, March 2022. URL: https://doi.org/10.3897/jucs.80734.
  19. Wenwen Li, Hyunho Lee, Sizhe Wang, Chia-Yu Hsu, and Samantha T. Arundel. Assessment of a new geoai foundation model for flood inundation mapping. CoRR, November 2023. arXiv:2309.14500. URL: https://doi.org/10.48550/arXiv.2309.14500.
  20. Zhouyayan Li and Ibrahim Demir. U-net-based semantic classification for flood extent extraction using sar imagery and gee platform: A case study for 2019 central us flooding. Science of The Total Environment, 869:161757, April 2023. URL: https://doi.org/10.1016/j.scitotenv.2023.161757.
  21. Yuchi Ma, Shuo Chen, Stefano Ermon, and David B. Lobell. Transfer learning in environmental remote sensing. Remote Sensing of Environment, 301:113924, February 2024. URL: https://doi.org/10.1016/j.rse.2023.113924.
  22. Gonzalo Mateo-Garcia, Joshua Veitch-Michaelis, Lewis Smith, Silviu Vlad Oprea, Guy Schumann, Yarin Gal, Atılım Güneş Baydin, and Dietmar Backes. Towards global flood mapping onboard low cost satellites with machine learning. Scientific Reports, 11(1):7249, March 2021. URL: https://doi.org/10.1038/s41598-021-86650-z.
  23. Matias Mendieta, Boran Han, Xingjian Shi, Yi Zhu, and Chen Chen. Towards Geospatial Foundation Models via Continual Pretraining, August 2023. arXiv:2302.04476. URL: https://doi.org/10.48550/arXiv.2302.04476.
  24. Yohei Nakayama, Jiawei Su, and Luis M. Pazos-Outón. SatSwinMAE: Efficient Autoencoding for Multiscale Time-series Satellite Imagery, October 2024. arXiv:2405.02512. URL: https://doi.org/10.48550/arXiv.2405.02512.
  25. Maxim Neumann, Andre Susano Pinto, Xiaohua Zhai, and Neil Houlsby. In-domain representation learning for remote sensing, November 2019. arXiv:1911.06721. URL: https://doi.org/10.48550/arXiv.1911.06721.
  26. Keiller Nogueira, Otávio A. B. Penatti, and Jefersson A. dos Santos. Towards better exploiting convolutional neural networks for remote sensing scene classification. Pattern Recognition, 61:539-556, January 2017. URL: https://doi.org/10.1016/j.patcog.2016.07.001.
  27. Fernando Pech-May, Raúl Aquino-Santos, Omar Álvarez Cárdenas, Jorge Lozoya Arandia, and German Rios-Toledo. Segmentation and visualization of flooded areas through sentinel-1 images and u-net. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 17:8996-9008, 2024. URL: https://doi.org/10.1109/JSTARS.2024.3387452.
  28. Olivier Petit, Nicolas Thome, Clément Rambour, and Luc Soler. U-Net Transformer: Self and Cross Attention for Medical Image Segmentation, March 2021. arXiv:2103.06104. URL: https://doi.org/10.48550/arXiv.2103.06104.
  29. Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. CoRR, May 2015. arXiv:1505.04597 [cs]. URL: https://arxiv.org/abs/1505.04597.
  30. Rizwan Sadiq, Zainab Akhtar, Muhammad Imran, and Ferda Ofli. Integrating remote sensing and social sensing for flood mapping. Remote Sensing Applications: Society and Environment, 25:100697, January 2022. URL: https://doi.org/10.1016/j.rsase.2022.100697.
  31. Apoorva Shastry, Elizabeth Carter, Brian Coltin, Rachel Sleeter, Scott McMichael, and Jack Eggleston. Mapping floods from remote sensing data and quantifying the effects of surface obstruction by clouds and vegetation. Remote Sensing of Environment, 291:113556, June 2023. URL: https://doi.org/10.1016/j.rse.2023.113556.
  32. Nahian Siddique, Sidike Paheding, Colin P. Elkin, and Vijay Devabhaktuni. U-net and its variants for medical image segmentation: A review of theory and applications. IEEE Access, 9:82031-82057, 2021. URL: https://doi.org/10.1109/ACCESS.2021.3086020.
  33. Zhongbin Su, Wei Li, Zheng Ma, and Rui Gao. An improved u-net method for the semantic segmentation of remote sensing images. Applied Intelligence, 52(3):3276-3288, February 2022. URL: https://doi.org/10.1007/s10489-021-02542-9.
  34. Mirela G. Tulbure, Mark Broich, Vinicius Perin, Mollie Gaines, Junchang Ju, Stephen V. Stehman, Tamlin Pavelsky, Jeffrey G. Masek, Simon Yin, Joachim Mai, and Luc Betbeder-Matibet. Can we detect more ephemeral floods with higher density harmonized landsat sentinel 2 data compared to landsat 8 alone? ISPRS Journal of Photogrammetry and Remote Sensing, 185:232-246, March 2022. URL: https://doi.org/10.1016/j.isprsjprs.2022.01.021.
  35. Yuefei Wang, Xi Yu, Yixi Yang, Shijie Zeng, Yuquan Xu, and Ronghui Feng. FTUNet: A Feature-Enhanced Network for Medical Image Segmentation Based on the Combination of U-Shaped Network and Vision Transformer. Neural Processing Letters, 56(2):83, March 2024. URL: https://doi.org/10.1007/s11063-024-11533-z.
  36. Michael A. Wulder, Jeffrey G. Masek, Warren B. Cohen, Thomas R. Loveland, and Curtis E. Woodcock. Opening the archive: How free data has enabled the science and monitoring promise of landsat. Remote Sensing of Environment, 122:2-10, July 2012. URL: https://doi.org/10.1016/j.rse.2012.01.010.
  37. Qiming Yang, Zixin Wang, Shinan Liu, and Zizheng Li. Research on improved u-net based remote sensing image segmentation algorithm. In 2024 6th International Conference on Internet of Things, Automation and Artificial Intelligence (IoTAAI), pages 686-689, July 2024. URL: https://doi.org/10.1109/IoTAAI62601.2024.10692547.
  38. Bofei Zhao, Haigang Sui, and Junyi Liu. Siam-dwenet: Flood inundation detection for sar imagery using a cross-task transfer siamese network. International Journal of Applied Earth Observation and Geoinformation, 116:103132, February 2023. URL: https://doi.org/10.1016/j.jag.2022.103132.
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact