What, When, and Where Do You Mean? Detecting Spatio-Temporal Concept Drift in Scientific Texts

The notion of concept has many different definitions across or even within domains, such as in linguistics, psychology, computer science, and cognitive science [35]. In this work, we adopt the definition by Stock [45] in information science, which defines a concept as a class containing objects that share certain properties.³³3It is worth noting that this definition of concept is closer to what cognitive scientists would call a category.

Concepts are fundamental units of meaning and serve as the building blocks of ontologies that help structure knowledge, enable reasoning, and facilitate interoperability. In terms of representation, previous work [49, 44] typically characterized a concept by its label (i.e., name), intension (i.e., defining properties), and extension (i.e., instances that fall under it), in the form ($label(C),int(C),ext(C)$) for a concept $C$. However, this would only apply to concepts that already existed in predefined ontologies. Verkijk et al. [48] proposed to use embedding techniques to derive vector representations of concepts. While their work focuses on knowledge-graph data, they showcased the ability of embedding techniques to capture flexible, context-aware representations of concepts for natural language data as well, in the form ($label(C),context(C)$).

In this work, we treat keywords in research publications as representations of underlying concepts. Unlike established ontological categories, research keywords are rapidly evolving as science and society change. This makes them particularly relevant for studying spatio-temporal concept drift. Here, we focus on research keywords also with the aim of developing a structured ontology that can capture their changing meanings over time and space. Such an ontology could contribute to RDM by improving metadata organization, literature retrieval, question answering, and knowledge graph construction in scientific databases.

Adding a temporal dimension to concept representation accounts for changes in their meaning over time. The study of concept drift, as defined by Wang et al. [49], aims to capture these changes in concept meaning over time. For example, the keyword global warming was once the dominant term in research publications, referring to the rise in Earth’s temperature. Over time, climate change became more widely used to capture broader climate-induced impacts [26] and account for the fact that warming is not uniform. To model a concept $C$ with a temporal component, it can be represented in the form ($labe{l}_t(C),in{t}_t(C),ex{t}_t(C)$) or ($labe{l}_t(C),contex{t}_t(C)$) at time $t$. Extending this notion to a spatio-temporal dimension means that a concept’s meaning may change both over time and space, e.g., at different rates. Here, we define this phenomenon as spatio-temporal concept drift. In this case, a concept can be represented as ($labe{l}_{t,s}(C),in{t}_{t,s}(C),ex{t}_{t,s}(C)$) or ($labe{l}_{t,s}(C),contex{t}_{t,s}(C)$) for a concept $C$ at time $t$ and region $s$.

Figure 1 illustrates how a concept moves in both time and geographic space. More abstractly, this can be thought of as the trajectory of a concept in a space-time prism [35]. The color gradient indicates (semantic) concept similarity [36, 39, 34, 22], thereby reflecting changes in its thematic dimension over time and space. Take the keywords global warming and climate change as an illustrative example. Region $s_1$ may have already adopted the use of climate change since time $t_1$, while $s_2$ still has a mixed use of both terms. Note that, here at $t_1$, the variation in the thematic dimension between $s_1$ and $s_2$ represents spatial variability, which differs from spatio-temporal concept drift, as it captures regional differences without a temporal dimension [25]. Later by $t_3$, $s_2$ adopts the distinct use of climate change and global warming, aligning its concept representation with $s_1$. Over time, as concepts evolve, their meanings may change gradually, showing a concept drift from $t_2$ to $t_3$ in $s_1$, or change so much that they diverge into two, showing a concept split⁴⁴4Definitions of concept drift and concept split are provided in our earlier work [40]. in region $s_2$. Ultimately, the two concepts may converge into a shared understanding for both regions (indicated by the semantic similarity between $C_1$ and $C_1^{\prime }$ at $t_3$).

Even with the advent of semantic search [18], which allows for more flexible query interpretation, concept drift remains a challenge, particularly in RDM and other archival systems. If the past and present keywords are not properly linked, search results may still be skewed toward more recent ones, simply because of their pertinence. This would lead to either incomplete retrieval results or misinterpretation of archival documents. Addressing spatio-temporal concept drift helps ensure that evolving knowledge remains accessible and meaningful across different time periods and regions. It could therefore enhance semantic interoperability overall and support the FAIR principles.

With the introduction of Word2Vec [29], word embeddings have revolutionized representation by converting words into dense vectors in a high-dimensional space,⁵⁵5Note that some literature uses the term “low-dimensional space” here when comparing the dimensionality to a one-hot encoding. We use the term here to signify that the resulting embeddings are in a, say, 300-dimensional vector space. where semantically similar words are close to each other. Such representation enables a more flexible study of temporal and spatial variations in lexical semantics, offering an advantage over directly comparing different ontology versions. Early pre-trained word embeddings, such as GloVe [32], provide static representations, where each word is assigned a single vector, independent of context. Later, more advanced models like BERT [10] provide context-aware embeddings that capture more variations in meaning based on surrounding text using attention mechanisms.

Several studies [3, 20, 37] have explored the enrichment of word embeddings with temporal and spatial information. For example, Zhang et al. [54] focused on temporal counterpart search that detects semantically similar terms over time. The authors later also investigated the geographic variations in lexical semantics [55], e.g., showing that typhoon in Japan would be the most similar term to hurricane in the United States. Gong et al. [16] further extended this idea and proposed a model that conditions word embeddings on time or location (i.e., generating time- and location-specific embeddings). Their findings included word similarities over time (e.g., bitcoin in 2015 and stocks in 1992) and locations (e.g., president in the United States and prime minister in Canada). A few other studies in GIScience explored the representation learning of places via word embeddings, thereby using spatial information alone. Yan et al. [52] applied word-embedding techniques to learn embeddings of places based on their types and distances. Later, Zhai et al. [53] extended this approach to the representation learning of functional regions.

While these approaches captured variations in lexical semantics along one dimension effectively, they did not jointly consider spatial and temporal dimensions. As a result, they would fail to capture spatial-temporal lexical similarity, such as chancellor in Germany in 2010 and prime minister in the United Kingdom in 1980. Such similarity is centered in our study on spatio-temporal concept drift, which accounts for both dimensions at the same time.

The Word Embedding Association Test (WEAT) [7], inspired by the Implicit Association Test in psychology, is a widely used method for quantifying semantic associations in word embeddings. It calculates association scores by comparing cosine similarities between two sets of target words (e.g., man and woman) and two sets of attribute words (e.g., doctor and gynecologist). For example, associations between $\overrightarrow{man}$ and $\overrightarrow{doctor}$ versus $\overrightarrow{woman}$ and $\overrightarrow{gynecologist}$ can be assessed using vector arithmetic [6, 30], expressed as $\overrightarrow{man}-\overrightarrow{woman}\approx \overrightarrow{doctor}-\overrightarrow{gynecologist}$. The resulting WEAT score indicates the degree of association between the two groups in the embedding space. WEAT provides a standardized measure and allows for statistical significance testing of observed changes. By adapting this method, we can compute the cosine similarities between different temporal snapshots and geographical regions, e.g., ($\overrightarrow{hurricane}$ to $\overrightarrow{Mexico}$, 2005) and ($\overrightarrow{typhoon}$ to $\overrightarrow{China}$, 2015). Note that 2005 and 2015 are not treated as vectors themselves but rather indicate the time periods associated with these concept-region pairs. If in a vector space, $\overrightarrow{hurricane}-\overrightarrow{typhoon}\approx \overrightarrow{Mexico}-\overrightarrow{China}$, this would allow us to quantify concept changes across space and time and reveal geographic prototypes underlying word embeddings. However, applying WEAT to spatio-temporal analysis also presents challenges, particularly in maintaining statistical power when data is sparse across certain regions or time periods.

As with the what, when, and where questions, we argue that each concept has thematic, temporal, and spatial dimensions. In this dataset, we treat each keyword as signifying an individual concept⁸⁸8The distinction between a symbol and a concept can be explained using the triangle of reference [31]. and represent these three dimensions accordingly.

To represent the (1) thematic dimension, we use the associated abstract, which provides contextual information of each concept. Since Scopus is an abstract and citation database without guaranteed full-text access, abstracts – being more consistently available across publications – are a practical choice for large-scale analysis. For the (2) temporal dimension, we use the publication year of the paper associated with each concept. Lastly, for the (3) spatial dimension, we use OpenStreetMap Nominatim⁹⁹9https://nominatim.org to geocode the identified locations and extract the corresponding country for sub-national locations (e.g., cities). For location mentions like “East African”, we retain them at the continent level. If multiple countries are mentioned in an abstract, we document all of them. After retrieving 2,112 publications with location mentions, we manually reviewed 16 unidentified cases, assigning them to the country level or removing them where necessary. This resulted in a final dataset of 2,107 publications.

Table 2 includes examples of abstracts with location mentions and the extracted countries (or regions). The distribution of the 10 most mentioned countries in publications within our dataset is visualized in the heatmap in Figure 2. From this heatmap, we can observe that the United States and China lead in the number of publications, followed by other English-speaking countries (e.g., the United Kingdom, Canada, and Australia) and several European countries. Along the temporal axis, we also observe a notable increase in publications since 2005 in this scientometric dataset.

With the defined spatial and temporal dimensions of each concept, we leverage word embeddings to capture their variations in the thematic dimension. We employ the pre-trained SciBERT model [5], which is designed for scientific texts, to compute embeddings. We use context-aware representations of concepts in natural language, i.e., ($label(C),context(C)$) for a concept $C$, as discussed in Section 2.1. Here, we compute these two types of embeddings for concepts (in this case, keywords): (1) label embedding, which is static and derived from the keywords themselves, and (2) context embedding, which is context-aware and based on their associated abstracts with location mentions.

Additionally, we investigate the sensitivity of context embeddings to location mentions by computing (3) context embedding without locations as well. This embedding is derived from associated abstracts of a concept, where each identified location mention is replaced with the placeholder “[Location]”. For instance, in the first example in Table 2, the sentence would become “We illustrate…based on the street pattern of a small [Location] town.” This helps reveal whether explicit geospatial references influence the context-aware representation of concepts.

For each keyword/concept $C$ in a given year $t$ and country (or region) $s$, we first average its context embeddings across all relevant abstracts, to ensure a single embedding for each unique keyword-year-country combination. We then integrate this with the label embedding through a convex combination to obtain the composite embedding $C_{t,s}$, formulated as:

where $label(C)$ is the embedding of the keyword itself through its label; $context(C,d)$ is the embedding of the abstract in document $d$ containing the keyword; $D_{t,s}$ is the set of documents that contain the keyword from year $t$ and country $s$; and $|D_{t,s}|$ is the number of such documents. The parameter $\alpha$ determines the weight assigned to the label versus context embeddings.

To quantify how a concept $C$ drifts across different space-time combinations, we use cosine similarity between their respective composite embeddings. Given two concept representations, e.g., $C_{t_1,s_1}$ and $C_{t_2,s_2}$, their similarity is computed as:

Understanding spatio-temporal concept drift in scientific texts requires linking keywords (and their underlying concepts) to geographic locations and time, but this process inherently introduces biases.

In our case study, we use publication year as the temporal dimension of a concept. However, the publication year could be different from the actual study period (e.g., a paper published in 2010 on East Africa in the 1970s). We attribute spatial dimensions of concepts (keywords) based on location mentions in their associated abstracts. However, not all location mentions correspond to the actual study area; some may appear as examples, comparisons, or even counterexamples. We then aggregate these locations to use the country as the spatial unit of concept drift, overlooking regional variations within the country level. Take the concept of urban planning as an example. Its interpretation could differ significantly for New York City (e.g., a walkable, transit-oriented city) versus Los Angeles (e.g., a car-centric city) over the years. These regional disparities would become particularly pronounced in larger countries (e.g., the United States) with diverse geographic and socioeconomic conditions. Our spatial aggregation approach implicitly assumes concept homogeneity at the country level, which introduces biases into the learned embeddings.

Future work could explore improved spatio-temporal scoping techniques to capture study periods and areas more accurately; it should also include different spatial levels – cities, countries, and continents – to measure variations.

Our sensitivity analysis reveals varying effects across countries when removing location mentions. For example, keywords associated with Japan show slightly larger differences in their embeddings, though the overall differences remained small. Here, several factors may complicate the interpretation of these results. First, the dataset has a substantial imbalance, with publications mentioning the United States far outnumbering those that mention other countries. When extracting unique country-year combinations, this imbalance leads to sparse samples for less represented countries, thus masking meaningful patterns. Second, we did not account for the proportion of location mentions within abstracts, e.g., some abstracts contain a list of study areas, while others mention one location briefly. The observed differences in embeddings with and without locations may be due to the removal of more contextual information rather than an inherent sensitivity to geographic reference. These factors need a more fine-grained analysis to quantify the impact of explicit location mentions on embedding representations in future work.

Since large language models (LLMs) become more commonly used for ontology learning tasks [27, 2, 41], we need to ensure geographic and temporal variations in knowledge are accounted for to have more context-aware representations. Current LLMs are trained on vast corpora of text that usually lack explicit spatial and temporal structuring [13], which would likely overlook the spatio-temporal variations in concept representations unless specified in the prompt. Our findings show that concepts in scientific texts evolve differently across geographic space and over time. This suggests that ontologies derived from LLMs may inherit hidden geographic biases. For instance, when an LLM processes the concept of smart city, its interpretation might be overly influenced (and represented) by temporally and regionally dominant implementations (e.g., the recent decade in Singapore) if the training data is dominated by publications from this region and time period.

Our observations show that concepts in scientific texts can vary across geographic space and time, and suggest the need for a more context-aware mechanism when using LLMs for ontology learning. This could be achieved with region-specific knowledge validation and/or the development of geographically aware prompting strategies. To capture the spatio-temporal dynamics in scientific concepts, future work could include the design of more few-shot learning approaches, where examples are carefully selected to represent diverse temporal and geographic interpretations of concepts.