Automating the Classification of Smart Tourism Tools with LLMs: A Taxonomy-Driven Approach (Short Paper)

Abstract

Smart Tourism Tools (STTs) are digital solutions that support tourism stakeholders in designing, managing, promoting, and enhancing tourism experiences. As the number and diversity of these tools increase, consistent classification becomes essential for organizing them in observatory-like repositories and making them easier to search, compare, and reuse. However, previous attempts to classify STTs using Large Language Models (LLMs) showed that broad taxonomy descriptions can lead to subjective and inconsistent interpretations. This paper proposes an improved STT classification framework that operationalizes the original taxonomy as a set of binary classification questions. Each taxonomy category is represented by a yes/no question, making the classification process more explicit, traceable, and easier to analyze. The taxonomy is also refined by renaming selected action domains and categories and removing one low-utility category. An exploratory evaluation was conducted using 100 randomly selected English-language STT descriptions and three state-of-the-art LLMs. A randomly selected subset of 20 examples was validated by a human expert. The results suggest that the proposed framework can support LLM-based classification with substantial alignment to expert judgment, while also revealing remaining challenges related to over-classification and ambiguous category boundaries. The paper contributes a more transparent and reproducible framework for classifying STTs and provides initial evidence of its feasibility for semi-automatic classification in smart tourism observatories.