Urban Mobility Analytics

The recent technological advances in location-based services and devices have been providing researchers with unprecedented amounts of data to study human mobility [3]. However, the same vast amounts of data that brought new research opportunities in urban mobility analytics have also created a series of new challenges that still need to be addressed by the research community. Our group discussion focused on trying to identify and propose potential solutions to these problems.

1. 1.

   Benchmark datasets: benchmark datasets are used for both training and testing of methodologies, but can also be helpful for evaluating the performance of unfamiliar datasets for implemented methods. The desire for benchmark datasets became clear during the discussions in breakout groups and the plenary. However, the multidisciplinary aspect of urban mobility analytics seems to call for not one but multiple benchmarking datasets that can attend the demands of specific applications within urban mobility analytics, such as traffic planning, routing or programming self-driving vehicles.

2. 2.

   Bias and representativeness of the data: the massive amounts of data provided by modern location-based technologies come along with an increasing public and legal concern with individual’s privacy. These privacy concerns have led to data anonymization, meaning that researchers do not have any metadata describing the population sample (e.g., gender, age, income) that generated the mobility datasets and therefore are not aware of how representative of the total population these data are. Understanding the bias in modern mobility datasets is especially critical for applied research on urban mobility analytics for solving real world problems.

3. 3.

   Data availability and access: Even though massive amounts of data on human mobility are produced daily, that does not mean that these data are accessible to researchers and policy makers. Many of these datasets are produced, owned and sold by private companies, such as Near and Strava. The high fees for such data may pose a barrier to many researchers, especially the ones working at countries with less advantaged economies. There are also datasets owned by governmental agencies and public companies, however, even when those are available, many of them are not widely known by or not readily accessible to the research community.

We believe that a unified international data catalog could be a good starting point for addressing the aforementioned issues. The development of a unified international data catalog has been a strategy used by movement ecology (see [4]), a discipline that has also recently benefited from the new opportunities and challenges brought by location-based services and devices [5]. In fact, a discipline that has been working for decades on challenges that urban movement analytics researchers are just now facing.

We believe that unified international data catalogue would support and foster a growing research community as well as potentially bring the following benefits:

1. 1.

   Federated cataloguing data sets on human and human-related movement as a centralised international index. The goal is to facilitate the access to and exploration of data sets that would otherwise be unknown, while supporting research collaboration at global, regional and local level. Moreover, this is also an opportunity to maximize the financial resources applied for data purchase by avoiding buying duplicate data from companies.

2. 2.

   Data reviewing across disciplines The catalogue should be a forum for the community from different fields to put data sets into perspective with first-hand experience and related contextual data such as census data, fostering transparency on openness of data sets. It supports best practices for cross-research work. Data sets come with papers and code they have been used in, linking to contextual and supplementary material.

3. 3.

   Data collection waste reduction The catalogue helps to foster research collaboration and to establish networks in order to reduce data collection redundancy and improve the use of the limited financial resources.

4. 4.

   Data fusion and multi-modality The catalogue is a channel for data compilation where data sets on different transportation modes can be combined to answer specific research questions, while also encouraging multi-modal transportation research and planning.

5. 5.

   Methodological development In its core, the catalogue is data and benchmark centered. However, it is supposed to become a central starting point for developing methodologies and state-of-the-art benchmarks and enhance reproducibility in human movement analytics studies across disciplines, in particular for bias detection. It supports discovering transferability opportunities from solutions derived from the data shared.

From the discussions held on how a mobility data catalog should look like, the overall motivation is to provide a mobility data platform which encourages open research and reproducibility.

Figure 2 show mock-up interfaces with different functionalities. Other than the typical search bar, a map with dataset locations and accessibility levels helps users to immediately have a sense of where mobility data is being collected, how to access them and the intensity of data collection activities in a certain location.

Upon clicking the dataset of interest, the next page offers a detailed description of the dataset which includes a summary, dataset attributes, benchmark, publications, comments and related datasets. In this way, users are able to understand how the dataset has been used for research and improvement opportunities from the publications that used the particular dataset. The related dataset offers possibility to validate findings and implement models or algorithms using other sources of data as well.

To encourage data contributions, contributor profiles include badges that acts as acknowledgements for offering the data and follow up badges for open research, etc.

Submitting data is easily done by a click of a button and a simple submission interface and finally an overview page provides a list of datasets with accessibility levels, number of views, number of publications, related datasets and a link to the contributor profile.

Obviously, a catalogue is only as good as its users and curators. If it does not provide any insights or does not suit the way people work, it will be a non-starter. However, data repository and engineering workflow (“github for datasets”) or an integrated analysis platform (see e.g. [6]) would mean a huge conceptual effort that requires engineering and platform resources, which is beyond our scope, hence, the reason why we envision the catalogue to be a more special-purpose combination of https://paperswithcode.com/datasets and https://openreview.net/ known in the ML community.

Today’s research and analytical methods, such as machine learning, rely heavily on tooling (software and scripts) capturing analysis as code, and on data. Any computational research poses strong challenges for effective communication and collaboration since both data and code are not well shared through the traditional method of scholarly communication, the scientific paper. If research outcomes cannot be shared, they can not transparently inform practices to improve mobility management in society.

The framework presented in Figure 3 framed the discussion in this group. Applicable to any computational research, this framework ties together the requirements on data (further discussed by a separate breakout group) and analysis.

Currently, many computational mobility analysis methods are de-facto black boxes. This may be either because the methods are only vaguely described in papers (often to an insufficient level of formality or because these methods are non-trivial to be then implemented in a computational environment) or because of the nature of the methods themselves, (i.e., the hard to explain and interpret computational procedures learned from specific datasets, such as in the case of deep neural networks). In cases where no human directly implemented assumptions and models, but a computer derived (learned) patterns from datasets, explainability must be especially considered to provide openness and transparency. Lack of explainability can result in erosion of trust from users who may question the results. This issue is also related to ethics because resulting recommendations may discriminate against certain groups due to biased training data or other issues. Furthermore, without explainability, theoretical frameworks cannot advance and therefore, scientific progress is slowed down.

We argued that a comprehensive publication of methods and their explainability is required to enable understanding, reuse, and extension of pieces of research so that societal challenges can be tackled effectively. Methods and the explainability of methods were discussed in this group along with possible avenues for improvement as a move towards better transparency: improvement in generalisability and robustness of methods, the ability to re-use the state of the art, and thus advance the discipline in a transparent manner.

The group collected topics broadly reflecting on current methods underpinning mobility analysis. This list naturally shows the breadth of the methodological toolkit needed to model and analyse mobility:

1. 1.

   Explainability and transferability in deep learning models for human mobility [1];

2. 2.

   Data integration, incl. geospatial encodings and embeddings for AI and challenges around vector vs. raster data;

3. 3.

   How to separate outlier patterns of interest in large datasets from large amounts of standard situations;

4. 4.

   Integrating physics-based models with data-driven models for analyzing mobility data to achieve more interpretable results;

5. 5.

   Investigation of trajectory analytical methods for modeling travel behaviours using network-based trajectory data rather than grid-based mobility data;

6. 6.

   The challenge of analysing interactions between modelled individuals, who are often assumed to be independent by mobility predictors and generators;

7. 7.

   Models that capture the complexity of mobility more comprehensively, such as the complexities of dynamic structures of transport networks and multi-modal routing;

8. 8.

   How to comprehensively model urban mobility across modalities (road traffic, public transport, pedestrian/bike), incl using deep learning (DL); and

9. 9.

   Multi-modal networks performance optimization.

To identify missing methods that need to be developed, domain knowledge and context information are of vital importance. Overall, the design of methods should be further improved to satisfy the theoretical constraints of mobility patterns, instead of just relying on training complex models and large amounts of data to lead to usable outcomes.

Participants with a background in scientific software development emphasized the need for a consolidation in tool development based on replicable and reusable software principles, thus enabling reference implementations of critical partial methods. This would enable us to compare, consolidate, and advance the discipline that is currently fragmented across tools and approaches¹¹¹¹11See a list at https://github.com/anitagraser/movement-analysis-tools.

A consolidation of tools and their open publishing is a prerequisite for more collaborative and effective development of tools, where tool extension and improvement to fix real world problems are valued more than marginal methodological improvements. Understanding and theory building should be the goal, i.e., scientific progress, not merely addressing an engineering challenge.

The group also discussed possible incentives to contribute to joint efforts enabling methodological explainability. Participants were interested in inspecting DL models and increasing the understanding of model behaviour. A key question is: what methods exist to explain the inner workings of DL models but also their inputs. Potential explainability approaches include: transfer of methods from computer vision, model comparison (especially for simple/simplified models), or visualisation of intermediate layers for understanding outputs.

Understanding and interpretability were also discussed on a more abstract level, that is: What kind of interpretability do we want for which type of analysis? Do we want to understand the mechanisms for generating the predictions, or make the actual model understandable to humans?

The participants share a quite critical view of the current state of methods in urban mobility analysis. There are concerns about the usefulness, usability, and relevance of methods, especially modern yet complex and possibly questionable methods based on AI and DL. These shared concerns motivate the following definition of challenges and recommendations.

• $\mathrm{■}$

  Challenge 1: defining appropriate objectives to evaluate machine learning models

  High accuracy alone is often inadequate to ensure accountable and reliable deployment of models in practice. We also need to offer explanations to end-users as to why the model produces a certain prediction (such as a routing suggestion) to guarantee transparency, fairness, and reliability [2]. In these cases, explainability should also be considered as one of the objectives besides accuracy. But explainability is a fuzzy concept and can mean various things in different contexts. This makes it an extremely challenging objective to optimize for [3]. Whether an explanation achieves its desired goal also depends on who receives it, how the person perceives it, and what actions a person can take based on the explanation.

• $\mathrm{■}$

  Challenge 2: bridging a theory-driven approach with a data-driven approach

  It is still unclear how to effectively combine the data-driven approach with the theory-driven approach in mobility analysis. A key barrier to the integration is a lack of shared data, tools, and terminologies across different research communities working on mobility analysis (e.g., city planning, transportation engineering, geography, computer science, etc.).

To address the above mentioned challenges, we have identified the following steps revolving around the development of shared resources for mobility analysis:

• $\mathrm{■}$

  Recommendation 1: joint terminology

  Fundamental terminological glossaries are needed to enable communication within the community. This terminological glossary would also relate to best practice about methodological translation of a mobility/transport concept (stop, home location, activity, behaviour) to the computational methods realising it, and an argument about their appropriateness in a given setting.

• $\mathrm{■}$

  Recommendation 2: reference implementations

  Standard, community-verified implementations ( cross-linguistic, possibly, to enable implementation in the main programming languages) would enable contributors to build robust solutions.

• $\mathrm{■}$

  Recommendation 3: standard dataset

  A set of simple to more complex shared datasets (e.g., basic trajectories, semantic trajectories, related urban structure data) are needed to test and evaluate computational methods – see recommendations of Group 2.

• $\mathrm{■}$

  Recommendation 4: standard data formats

  A definition of a common data representation for movement data would enable an interoperable (incl multi-language) interface to the data (see, e.g., the OGD MovingFeatures standard or the standardisation on input formats in geoparquet for python/geopandas and R).

• $\mathrm{■}$

  Recommendation 5: shared ML models

  Reference implementations of basic ML methods and a platform for sharing benchmarks or established models built on top of these methods could become a community convergence point (similar to Hugging Face¹²¹²12Hugging Face: https://huggingface.co/ in the ML community).

The goal of this discussion group was to generate ideas in which results of urban mobility analytics can be used in applications and how the respective application areas can be linked to urban mobility analytics. After a brainstorming phase the group concentrated on the fact that the knowledge of mobility behavior is of very much use in the domain of autonomous driving, in which autonomous vehicles interact with other traffic participant in urban traffic. To achieve an optimal behavior, the autonomous vehicles require knowledge about the behavior of other traffic participants to adapt their own behavior and to interact seamlessly with others. It is expected that for a long time human driven vehicles, autonomous vehicles, bicyclists, pedestrians and other traffic participants will share the same roads; therefore, models of pedestrians’ behavior, bicyclists’ behavior, and the behavior of human drivers is very much relevant. Furthermore, the behavior of traffic participants cannot be assumed to be the same worldwide but it differs from country to country. E.g. the driving style in Italy obviously differs from the driving style in Germany, the U.S. or in India or China. Similarly, pedestrians behave differently in those country. Hence, the autonomous vehicle must be able to understand the country specific behavior of other traffic participants, and also adapt their behaviour accordingly.

In the following, the aspects data, analysis methods, and necessary stakeholders were discussed, which would be needed to push forward such an application.

The country-specific adaptation of behavior models can be considered as a typical application of machine learning methods, which are based on revealing this information from large amounts of observed data. So the question is how these data can be generated. Several data sources have been discussed in the group, including

• $\mathrm{■}$

  the usage of existing datasets like highD¹³¹³13highD: https://www.highd-dataset.com/, INTERACTION¹⁴¹⁴14INTERACTION: http://interaction-dataset.com/, LUMPI¹⁵¹⁵15LUMPI: https://data.uni-hannover.de/cs_CZ/dataset/lumpi

• $\mathrm{■}$

  datasets for environmental perception (e.g. Kitti, nuScenes)

• $\mathrm{■}$

  recording own datasets from top view perspectives using drones or mounting cameras on high buildings

• $\mathrm{■}$

  recording own datasets from test fields, e.g. Testfeld Karlsruhe¹⁶¹⁶16Testfeld Autonomes Fahren Baden-Württemberg https://taf-bw.de/

Besides these open data sets, commercial data would be very beneficial. Data schould be available in different European cities and potentially in further cities outside of Europe.

Since it is impossible to record and model the behavior of traffic participants in all possible traffic situations it would be necessary to start with limiting the recordings to some most interesting, and critical, scenarios. These could include

• $\mathrm{■}$

  merging into highway ramps

• $\mathrm{■}$

  shared spaces

• $\mathrm{■}$

  crossing intersections

• $\mathrm{■}$

  vehicle following

To model these scenarios it would be necessary to extract relevant features from the data including

• $\mathrm{■}$

  trajectory data (position over time, velocity, acceleration)

• $\mathrm{■}$

  relationship data (relative distances, relative velocities between traffic participants, interactions)

• $\mathrm{■}$

  movement patterns (e.g. groups)

• $\mathrm{■}$

  typical patterns at certain locations

• $\mathrm{■}$

  geometric data (shape of traffic participants, layout of road infrastructure, lanes, curbs)

• $\mathrm{■}$

  intentional data (face and body orientation of pedestrians, roll angle of bicyclists, indicator lights)

The analysis of the data requires automatic methods and processes, e.g. the extraction of the features mentioned above from the sensory data, the creation of behavior models for traffic participants, and the integration of those models into the decision making and trajectory planning system of the autonomous vehicle. Urban mobility analytics comes in play especially for the second category of methods that analyze the recorded mobility data. Various methods might be suitable, including deep learning methods, transformers, recurrent neural networks, inverse reinforcement learning. In addition, also methods from computational geometry, e.g. trajectory pattern analysis, are relevant. An important aspect is to include the notion of cooperation into the process, which allows the autonomous vehicle to also exploit information from other traffic participants, which enlarges its own perception range.

For the integration of behavior models to decision making and trajectory planning of the autonomous vehicle it would be useful to have a powerful simulation environment available, that also should be adapted to country specific behavior. Finally, for a real live demonstration the implementation of the behavior models in a real autonomous vehicle would be desirable.

A joint project to extract and exploit country specific behavior patterns for autonomous vehicles could be relevant for various partners in research and industry. The following stakeholders could be included

• $\mathrm{■}$

  researchers in urban mobility analytics: analyze data, build behavior models, extract country specific parameters; develop models and concepts that allow for exploiting shared data, without the need to make it publicly available (e.g. using federated learning ([1])

• $\mathrm{■}$

  researchers in perception: record sensory data and extract relevant features

• $\mathrm{■}$

  researchers in traffic psychology: analyze and explain country specific behavior patterns

• $\mathrm{■}$

  researchers in autonomous driving: provide autonomous vehicle, integrate behavior models in decision making of autonomous vehicles

• $\mathrm{■}$

  automobile industry: integrate country specific behavior models into their vehicles, provide data from onboard sensors

• $\mathrm{■}$

  data and map companies: collect and analyze country specific data; specify map interfaces for country specific behavior patterns

• $\mathrm{■}$

  municipalities: provide access to infrastructure, equip certain areas with sensors, implement certain regulations (e.g. temporal speed limits)