Finding Feasible Routes with Reinforcement Learning Using Macro-Level Traffic Measurements (Short Paper)

Authors Mustafa Can Ozkan , Tao Cheng



PDF
Thumbnail PDF

File

LIPIcs.GIScience.2023.58.pdf
  • Filesize: 0.66 MB
  • 6 pages

Document Identifiers

Author Details

Mustafa Can Ozkan
  • SpaceTimeLab, University College London, UK
Tao Cheng
  • SpaceTimeLab, University College London, UK

Cite AsGet BibTex

Mustafa Can Ozkan and Tao Cheng. Finding Feasible Routes with Reinforcement Learning Using Macro-Level Traffic Measurements (Short Paper). In 12th International Conference on Geographic Information Science (GIScience 2023). Leibniz International Proceedings in Informatics (LIPIcs), Volume 277, pp. 58:1-58:6, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2023)
https://doi.org/10.4230/LIPIcs.GIScience.2023.58

Abstract

The quest for identifying feasible routes holds immense significance in the realm of transportation, spanning a diverse range of applications, from logistics and emergency systems to taxis and public transport services. This research area offers multifaceted benefits, including optimising traffic management, maximising traffic flow, and reducing carbon emissions and fuel consumption. Extensive studies have been conducted to address this critical issue, with a primary focus on finding the shortest paths, while some of them incorporate various traffic conditions such as waiting times at traffic lights and traffic speeds on road segments. In this study, we direct our attention towards historical data sets that encapsulate individuals' route preferences, assuming they encompass all traffic conditions, real-time decisions and topological features. We acknowledge that the prevailing preferences during the recorded period serve as a guide for feasible routes. The study’s noteworthy contribution lies in our departure from analysing individual preferences and trajectory information, instead focusing solely on macro-level measurements of each road segment, such as traffic flow or traffic speed. These types of macro-level measurements are easier to collect compared to individual data sets. We propose an algorithm based on Q-learning, employing traffic measurements within a road network as positive attractive rewards for an agent. In short, observations from macro-level decisions will help us to determine optimal routes between any two points. Preliminary results demonstrate the agent’s ability to accurately identify the most feasible routes within a short training period.

Subject Classification

ACM Subject Classification
  • Computing methodologies → Q-learning
Keywords
  • routing
  • reinforcement learning
  • q-learning
  • data mining
  • macro-level patterns

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads

References

  1. T Ahamed, B Zou, N P Farazi, and T Tulabandhula. Deep Reinforcement Learning for Crowdsourced Urban Delivery. Transportation Research Part B: Methodological, 152:227-257, 2021. URL: https://doi.org/10.1016/j.trb.2021.08.015.
  2. R Basso, B Kulcsár, I Sanchez-Diaz, and X Qu. Dynamic stochastic electric vehicle routing with safe reinforcement learning. Transportation Research Part E: Logistics and Transportation Review, 157, 2022. URL: https://doi.org/10.1016/j.tre.2021.102496.
  3. Edsger W. Dijkstra. A note on two problems in connexion with graphs. Numerische Mathematik, 1:269-271, 1959. Google Scholar
  4. Y Geng, E Liu, R Wang, Y Liu, W Rao, S Feng, Z Dong, Z Fu, and Y Chen. Deep Reinforcement Learning Based Dynamic Route Planning for Minimizing Travel Time. In 2021 IEEE International Conference on Communications Workshops, ICC Workshops 2021, Shanghai, China, 2021. URL: https://doi.org/10.1109/ICCWorkshops50388.2021.9473555.
  5. Peter Hart, Nils Nilsson, and Bertram Raphael. A formal basis for the heuristic determination of minimum cost paths. IEEE Transactions on Systems Science and Cybernetics, 4(2):100-107, 1968. URL: https://doi.org/10.1109/tssc.1968.300136.
  6. Y Hu, L Yang, and Y Lou. Path Planning with Q-Learning. In 2021 2nd International Conference on Internet of Things, Artificial Intelligence and Mechanical Automation, IoTAIMA 2021, volume 1948, North Carolina State University, Raleigh, NC 27695, United States, 2021. IOP Publishing Ltd. URL: https://doi.org/10.1088/1742-6596/1948/1/012038.
  7. F Jamshidi, L Zhang, and F Nezhadalinaei. Autonomous Driving Systems: Developing an Approach based on A* and Double Q-Learning. In 7th International Conference on Web Research, ICWR 2021, pages 82-85, East China Normal University, Moe International Joint Lab of Trustworthy Software, Shanghai, China, 2021. URL: https://doi.org/10.1109/ICWR51868.2021.9443139.
  8. E Liang, K Wen, W H K Lam, A Sumalee, and R Zhong. An Integrated Reinforcement Learning and Centralized Programming Approach for Online Taxi Dispatching. IEEE Transactions on Neural Networks and Learning Systems, 2021. URL: https://doi.org/10.1109/TNNLS.2021.3060187.
  9. T S Mostafa and H Talaat. An Intelligent Geographical Information System for Vehicle Routing (IGIS-VR): A modeling framework. In 13th International IEEE Conference on Intelligent Transportation Systems, ITSC 2010, pages 801-805, Intelligent Transportation Systems Program, Nile University, 2010. URL: https://doi.org/10.1109/ITSC.2010.5625095.
  10. P Tong, Y Yan, D Wang, and X Qu. Optimal route design of electric transit networks considering travel reliability. Computer-Aided Civil and Infrastructure Engineering, 36(10):1229-1248, 2021. URL: https://doi.org/10.1111/mice.12678.
  11. C Wei, Y Wang, X Yan, and C Shao. Look-Ahead Insertion Policy for a Shared-Taxi System Based on Reinforcement Learning. IEEE Access, 6:5716-5726, 2017. URL: https://doi.org/10.1109/ACCESS.2017.2769666.
  12. Jing Yuan, Yu Zheng, Xing Xie, and Guangzhong Sun. Driving with knowledge from the physical world. In Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '11, pages 316-324, New York, NY, USA, 2011. Association for Computing Machinery. URL: https://doi.org/10.1145/2020408.2020462.
  13. Y Zhang, R Bai, R Qu, C Tu, and J Jin. A deep reinforcement learning based hyper-heuristic for combinatorial optimisation with uncertainties. European Journal of Operational Research, 2021. URL: https://doi.org/10.1016/j.ejor.2021.10.032.
  14. M Zolfpour-Arokhlo, A Selamat, S Z Mohd Hashim, and H Afkhami. Modeling of route planning system based on Q value-based dynamic programming with multi-agent reinforcement learning algorithms. Engineering Applications of Artificial Intelligence, 29:163-177, 2014. URL: https://doi.org/10.1016/j.engappai.2014.01.001.
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail