DROPS

Document

Short Paper

DOI: 10.4230/LIPIcs.COSIT.2026.17

Limits of Spatial Reasoning Using LLMs (Short Paper)

Authors: Varis Ithivatana and Brandon Bennett

Published in: LIPIcs, Volume 393, 17th International Conference on Spatial Information Theory (COSIT 2026)

Abstract

This paper investigates the limits of spatial reasoning of LLMs in a grid-based warehouse environment, with ground-truth answers computed by simulation. We test two kinds of task: executing given movement instructions, and generating shortest action sequences for planning problems. In the execution tasks, an agent must follow egocentric and allocentric actions and report its final absolute direction relative to a target object. In the planning tasks, the model must find a shortest route, sometimes with object carrying, allocentric descriptions, or ordered pallet-pushing goals. We evaluate 100 instances per experiment using GPT-5-mini, and compare a smaller planning set with ChatGPT-5.2 (extended thinking), Gemini-Pro-3.1, and Grok-Expert. The results show high accuracy on short and moderately long instruction-following tasks, but reduced accuracy on long trajectories with a much sharper drop on planning tasks. Static landmark labels in the form of coloured walls improve robustness, compared with problems described in terms of cardinal directions, while planning with ordered object-pushing remains difficult even for stronger models.

Cite as

Varis Ithivatana and Brandon Bennett. Limits of Spatial Reasoning Using LLMs (Short Paper). In 17th International Conference on Spatial Information Theory (COSIT 2026). Leibniz International Proceedings in Informatics (LIPIcs), Volume 393, pp. 17:1-17:8, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2026)

Copy BibTex To Clipboard

@InProceedings{ithivatana_et_al:LIPIcs.COSIT.2026.17,
  author =	{Ithivatana, Varis and Bennett, Brandon},
  title =	{{Limits of Spatial Reasoning Using LLMs}},
  booktitle =	{17th International Conference on Spatial Information Theory (COSIT 2026)},
  pages =	{17:1--17:8},
  series =	{Leibniz International Proceedings in Informatics (LIPIcs)},
  ISBN =	{978-3-95977-438-3},
  ISSN =	{1868-8969},
  year =	{2026},
  volume =	{393},
  editor =	{Timpf, Sabine and Filomena, Gabriele and Kapaj, Armand and Zhu, Rui and Giudice, Nicholas A. and Manley, Ed},
  publisher =	{Schloss Dagstuhl -- Leibniz-Zentrum f{\"u}r Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{https://drops.dagstuhl.de/entities/document/10.4230/LIPIcs.COSIT.2026.17},
  URN =		{urn:nbn:de:0030-drops-275616},
  doi =		{10.4230/LIPIcs.COSIT.2026.17},
  annote =	{Keywords: Spatial Reasoning, Large Language Models, Frame of Reference, Path Integration, Planning}
}

Search Results

Documents authored by Ithivatana, Varis

Limits of Spatial Reasoning Using LLMs (Short Paper)

Abstract

Cite as

Thanks for your feedback!

Could not send message