2026-10-07T14:03:58Z https://drops.dagstuhl.de/oai

oai:drops-oai.dagstuhl.de:23842 2025-11-12T12:20:26Z ddc:004 open_access

Assessing Map Reproducibility with Visual Question-Answering: An Empirical Evaluation Koukouraki, Eftychia Degbelo, Auriol Kray, Christian map comparison computational reproducibility visual question answering large language models GeoAI Reproducibility is a key principle of the modern scientific method. Maps, as an important means of communicating scientific results in GIScience and across disciplines, should be reproducible. Currently, map reproducibility assessment is done manually, which makes the assessment process tedious and time-consuming, ultimately limiting its efficiency. Hence, this work explores the extent to which Visual Question-Answering (VQA) can be used to automate some tasks relevant to map reproducibility assessment. We selected five state-of-the-art vision language models (VLMs) and followed a three-step approach to evaluate their ability to discriminate between maps and other images, interpret map content, and compare two map images using VQA. Our results show that current VLMs already possess map-reading capabilities and demonstrate understanding of spatial concepts, such as cardinal directions, geographic scope, and legend interpretation. Our paper demonstrates the potential of using VQA to support reproducibility assessment and highlights the outstanding issues that need to be addressed to achieve accurate, trustworthy map descriptions, thereby reducing the time and effort required by human evaluators. Schloss Dagstuhl – Leibniz-Zentrum für Informatik Eftychia Koukouraki and Auriol Degbelo and Christian Kray 2025 Is Part Of LIPIcs, Volume 346, 13th International Conference on Geographic Information Science (GIScience 2025) InProceedings Text doc-type:ResearchArticle publishedVersion application/pdf doi:10.4230/LIPIcs.GIScience.2025.13 urn:nbn:de:0030-drops-238426 https://drops.dagstuhl.de/entities/document/10.4230/LIPIcs.GIScience.2025.13 eng https://creativecommons.org/licenses/by/4.0/legalcode