Document
A Cross-Linguistic Evaluation of Cardinal Direction Reasoning in LLMs (Short Paper)
Authors
Panagiota Gyftou 
,
Robert E. Blackwell ,
Manolis Koubarakis ,
Anthony G. Cohn
-
Part of:
Volume:
17th International Conference on Spatial Information Theory (COSIT 2026)
Series: Leibniz International Proceedings in Informatics (LIPIcs)
Conference: Conference on Spatial Information Theory (COSIT) - License:
Creative Commons Attribution 4.0 International license
- Publication Date: 2026-09-10
File
PDF
LIPIcs.COSIT.2026.18.pdf
- Filesize: 1.08 MB
- 9 pages
Author Details
- Dept. of Informatics and Telecommunications, National and Kapodistrian University of Athens, Greece
- Dept. of Informatics and Telecommunications, National and Kapodistrian University of Athens, Greece
Supplementary Materials
-
Dataset (Dataset)
https://github.com/panagiotagyft/cosit2026
browse
archived version
References
- A. G. Cohn and R. E. Blackwell. Evaluating the ability of large language models to reason about cardinal directions (short paper). In COSIT 2024, 2024. URL: https://doi.org/10.4230/LIPIcs.COSIT.2024.28.
- A. G. Cohn and R. E. Blackwell. Evaluating the ability of large language models to reason about cardinal directions, revisited, 2025. Workshop QR@IJCAI 2025. URL: https://arxiv.org/abs/2507.12059.
- A. Petrov et al. Language model tokenizers introduce unfairness between languages, 2023. URL: https://arxiv.org/abs/2305.15425.
- Bryan Li et al. Eliciting better multilingual structured reasoning from LLMs through code, 2024. URL: https://arxiv.org/abs/2403.02567.
- C. Wang et al. Under the shadow of babel: How language shapes reasoning in llms, 2025. URL: https://arxiv.org/abs/2506.16151.
- D. Roussis et al. Krikri: Advancing open large language models for Greek, 2025. URL: https://arxiv.org/abs/2505.13772.
-
E. Gardelakos et al. Can large reasoning models reason about spatial relations? In ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery, 2025.
- F. Shi et al. Language models are multilingual chain-of-thought reasoners, 2022. URL: https://arxiv.org/abs/2210.03057.
- H. Huang et al. Not all languages are created equal in llms: Improving multilingual capability by cross-lingual-thought prompting, 2023. URL: https://arxiv.org/abs/2305.07004.
- J. Wei et al. Chain-of-thought prompting elicits reasoning in large language models, 2022. URL: https://arxiv.org/abs/2201.11903.
- K. Cobbe et al. Training verifiers to solve math word problems, 2021. URL: https://arxiv.org/abs/2110.14168.
- L. Voukoutis et al. Meltemi: The first open large language model for Greek, 2024. URL: https://arxiv.org/abs/2407.20743.
- Lachlan McPheat et al. Decompsr: A dataset for decomposed analyses of compositional multihop spatial reasoning, 2025. URL: https://arxiv.org/abs/2511.02627.
- Miguel Moura Ramos et al. Eurollm-22b: Technical report, 2026. URL: https://arxiv.org/abs/2602.05879.
- O. Gouvert et al. The Lucie-7B LLM and the Lucie Training Dataset: Open resources for multilingual language generation, 2025. URL: https://arxiv.org/abs/2503.12294.
- Pedro Henrique Martins et al. Eurollm-9b: Technical report, 2025. URL: https://arxiv.org/abs/2506.04079.
- R. E. Blackwell et al. Towards reproducible LLM evaluation: Quantifying uncertainty in LLM benchmark scores, 2024. URL: https://arxiv.org/abs/2410.03492.
- M. Müller and F. Laurent. Cedille: A large autoregressive french language model, 2022. URL: https://arxiv.org/abs/2202.03371.
- S. Skiadopoulos and M. Koubarakis. Composing cardinal direction relations. Artif. Intell., 152(2), 2004. URL: https://doi.org/10.1016/S0004-3702(03)00137-1.