Document Open Access Logo

Open the Chests: An Environment for Activity Recognition and Sequential Decision Problems Using Temporal Logic

Authors Ivelina Stoyanova, Nicolas Museux , Sao Mai Nguyen , David Filliat

PDF
Thumbnail PDF

File

LIPIcs.TIME.2024.5.pdf
  • Filesize: 1.22 MB
  • 19 pages

Document Identifiers

Author Details

Ivelina Stoyanova
  • U2IS, ENSTA Paris, Institut Polytechnique de Paris, Palaiseau, France
  • THALES, Palaiseau, France
Nicolas Museux
  • THALES, Palaiseau, France
Sao Mai Nguyen
  • U2IS, ENSTA Paris, Institut Polytechnique de Paris, Palaiseau, France
David Filliat
  • U2IS, ENSTA Paris, Institut Polytechnique de Paris, Palaiseau, France

Cite AsGet BibTex

Ivelina Stoyanova, Nicolas Museux, Sao Mai Nguyen, and David Filliat. Open the Chests: An Environment for Activity Recognition and Sequential Decision Problems Using Temporal Logic. In 31st International Symposium on Temporal Representation and Reasoning (TIME 2024). Leibniz International Proceedings in Informatics (LIPIcs), Volume 318, pp. 5:1-5:19, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2024)
https://doi.org/10.4230/LIPIcs.TIME.2024.5

Abstract

This article presents Open the Chests, a novel benchmark environment designed for simulating and testing activity recognition and reactive decision-making algorithms. By leveraging temporal logic, Open the Chests offers a dynamic, event-driven simulation platform that illustrates the complexities of real-world systems. The environment contains multiple chests, each representing an activity pattern that an interacting agent must identify and respond to by pressing a corresponding button. The agent must analyze sequences of asynchronous events generated by the environment to recognize these patterns and make informed decisions. With the aim of theoretically grounding the environment, the Activity-Based Markov Decision Process (AB-MDP) is defined, allowing to model the context-dependent interaction with activities. Our goal is to propose a robust tool for the development, testing, and bench-marking of algorithms that is illustrative of realistic scenarios and allows for the isolation of specific complexities in event-driven environments.

Subject Classification

ACM Subject Classification
  • Computing methodologies → Simulation environments
Keywords
  • Event-Based Decision Making
  • Activity Recognition
  • Temporal Logic
  • Reinforcement Learning
  • Dynamic Systems
  • Complex Event Processing
  • Benchmark Environment
  • Real-Time Simulation

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Supplementary Materials

References

  1. Rishabh Agarwal, Max Schwarzer, Pablo Samuel Castro, Aaron Courville, and Marc G Bellemare. Deep reinforcement learning at the edge of the statistical precipice. Advances in Neural Information Processing Systems, 2021. Google Scholar
  2. Elias Alevizos, Anastasios Skarlatidis, Alexander Artikis, and Georgios Paliouras. Probabilistic complex event recognition: A survey. ACM Computing Surveys (CSUR), 50(5):1-31, 2017. URL: https://doi.org/10.1145/3117809.
  3. James F Allen and George Ferguson. Actions and events in interval temporal logic. Journal of logic and computation, 4(5):531-579, 1994. URL: https://doi.org/10.1093/LOGCOM/4.5.531.
  4. Anindya Das Antar, Masud Ahmed, and Md Atiqur Rahman Ahad. Challenges in sensor-based human activity recognition and a comparative analysis of benchmark datasets: A review. In 2019 Joint 8th International Conference on Informatics, Electronics & Vision (ICIEV) and 2019 3rd International Conference on Imaging, Vision & Pattern Recognition (icIVPR), pages 134-139. IEEE, 2019. Google Scholar
  5. Fahiem Bacchus, Craig Boutilier, and Adam Grove. Structured solution methods for non-markovian decision processes. In AAAI/IAAI, pages 112-117. Citeseer, 1997. URL: http://www.aaai.org/Library/AAAI/1997/aaai97-018.php.
  6. Iyad Batal, Gregory F Cooper, Dmitriy Fradkin, James Harrison, Fabian Moerchen, and Milos Hauskrecht. An efficient pattern mining approach for event detection in multivariate temporal data. Knowledge and information systems, 46:115-150, 2016. URL: https://doi.org/10.1007/S10115-015-0819-6.
  7. Benjamin Beyret, José Hernández-Orallo, Lucy Cheke, Marta Halina, Murray Shanahan, and Matthew Crosby. The animal-ai environment: Training and testing animal-like artificial cognition. arXiv preprint arXiv:1909.07483, 2019. URL: https://arxiv.org/abs/1909.07483.
  8. Damien Bouchabou, Sao Mai Nguyen, Christophe Lohr, Benoit LeDuc, and Ioannis Kanellos. A survey of human activity recognition in smart homes based on iot sensors algorithms: Taxonomies, challenges, and opportunities with deep learning. Sensors, 21(18):6037, 2021. URL: https://doi.org/10.3390/S21186037.
  9. Craig Boutilier, Thomas Dean, and Steve Hanks. Decision-theoretic planning: Structural assumptions and computational leverage. Journal of Artificial Intelligence Research, 11:1-94, 1999. URL: https://doi.org/10.1613/JAIR.575.
  10. Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. Openai gym, 2016. URL: https://arxiv.org/abs/arXiv:1606.01540.
  11. Christos G Cassandras and Stéphane Lafortune. Introduction to discrete event systems. Springer, 2008. Google Scholar
  12. Wuhui Chen, Xiaoyu Qiu, Ting Cai, Hong-Ning Dai, Zibin Zheng, and Yan Zhang. Deep reinforcement learning for internet of things: A comprehensive survey. IEEE Communications Surveys & Tutorials, 23(3):1659-1692, 2021. URL: https://doi.org/10.1109/COMST.2021.3073036.
  13. Jean-René Coffi, Christophe Marsala, and Nicolas Museux. Adaptive complex event processing for harmful situation detection. Evolving Systems, 3:167-177, 2012. URL: https://doi.org/10.1007/S12530-012-9052-7.
  14. Gianpaolo Cugola and Alessandro Margara. Processing flows of information: From data stream to complex event processing. ACM Computing Surveys (CSUR), 44(3):1-62, 2012. URL: https://doi.org/10.1145/2187671.2187677.
  15. Ala’ Darabseh and Nikolaos M Freris. A software-defined architecture for control of iot cyberphysical systems. Cluster Computing, 22(4):1107-1122, 2019. URL: https://doi.org/10.1007/S10586-018-02889-8.
  16. Dario Della Monica, Valentin Goranko, Angelo Montanari, and Guido Sciavicco. Interval temporal logics: a journey. Bulletin of EATCS, 3(105), 2013. Google Scholar
  17. Florenc Demrozi, Graziano Pravadelli, Azra Bihorac, and Parisa Rashidi. Human activity recognition using inertial, physiological and environmental sensors: A comprehensive survey. IEEE access, 8:210816-210836, 2020. URL: https://doi.org/10.1109/ACCESS.2020.3037715.
  18. Derui Ding, Qing-Long Han, Yang Xiang, Xiaohua Ge, and Xian-Ming Zhang. A survey on security control and attack detection for industrial cyber-physical systems. Neurocomputing, 275:1674-1683, 2018. URL: https://doi.org/10.1016/J.NEUCOM.2017.10.009.
  19. Anton Dries and Luc De Raedt. Towards clausal discovery for stream mining. In Inductive Logic Programming: 19th International Conference, ILP 2009, Leuven, Belgium, July 02-04, 2009. Revised Papers 19, pages 9-16. Springer, 2010. URL: https://doi.org/10.1007/978-3-642-13840-9_2.
  20. Gabriel Dulac-Arnold, Nir Levine, Daniel J Mankowitz, Jerry Li, Cosmin Paduraru, Sven Gowal, and Todd Hester. Challenges of real-world reinforcement learning: definitions, benchmarks and analysis. Machine Learning, 110(9):2419-2468, 2021. URL: https://doi.org/10.1007/S10994-021-05961-4.
  21. Kevin Esslinger, Robert Platt, and Christopher Amato. Deep transformer q-networks for partially observable reinforcement learning. arXiv preprint arXiv:2206.01078, 2022. URL: https://doi.org/10.48550/arXiv.2206.01078.
  22. Ido Finder, Eitam Sheetrit, and Nir Nissim. Time-interval temporal patterns can beat and explain the malware. Knowledge-Based Systems, 241:108266, 2022. URL: https://doi.org/10.1016/J.KNOSYS.2022.108266.
  23. Frédérick Garcia and Emmanuel Rachelson. Markov decision processes. Markov Decision Processes in Artificial Intelligence, pages 1-38, 2013. Google Scholar
  24. Nikos Giatrakos, Elias Alevizos, Alexander Artikis, Antonios Deligiannakis, and Minos Garofalakis. Complex event recognition in the big data era: a survey. The VLDB Journal, 29:313-352, 2020. URL: https://doi.org/10.1007/S00778-019-00557-W.
  25. Ben Goertzel, Nil Geisweiller, Lucio Coelho, Predrag Janičić, and Cassio Pennachin. Real-World Reasoning: Toward Scalable, Uncertain Spatiotemporal, Contextual and Causal Inference, volume 2. Springer Science & Business Media, 2011. Google Scholar
  26. Shaogang Gong and Tao Xiang. Recognition of group activities using dynamic probabilistic networks. In Proceedings ninth IEEE international conference on computer vision, pages 742-749. IEEE, 2003. URL: https://doi.org/10.1109/ICCV.2003.1238423.
  27. Valentin Goranko, Angelo Montanari, and Guido Sciavicco. A road map of interval temporal logics and duration calculi. Journal of Applied Non-Classical Logics, 14(1-2):9-54, 2004. URL: https://doi.org/10.3166/JANCL.14.9-54.
  28. Assaf Hallak, Dotan Di Castro, and Shie Mannor. Contextual markov decision processes. arXiv preprint arXiv:1502.02259, 2015. URL: https://arxiv.org/abs/1502.02259.
  29. Derek Hao Hu, Sinno Jialin Pan, Vincent Wenchen Zheng, Nathan Nan Liu, and Qiang Yang. Real world activity recognition with multiple goals. In Proceedings of the 10th international conference on Ubiquitous computing, pages 30-39, 2008. URL: https://doi.org/10.1145/1409635.1409640.
  30. Frank Höppner. Learning temporal rules from state sequences. In IJCAI Workshop on Learning from Temporal and Spatial Data, volume 25. Citeseer, 2001. Google Scholar
  31. Michael Kaminski and Nissim Francez. Finite-memory automata. Theoretical Computer Science, 134(2):329-363, 1994. URL: https://doi.org/10.1016/0304-3975(94)90242-9.
  32. Nikos Katzouris, Evangelos Michelioudakis, Alexander Artikis, and Georgios Paliouras. Online learning of weighted relational rules for complex event recognition. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2018, Dublin, Ireland, September 10-14, 2018, Proceedings, Part II 18, pages 396-413. Springer, 2019. URL: https://doi.org/10.1007/978-3-030-10928-8_24.
  33. Khimya Khetarpal, Matthew Riemer, Irina Rish, and Doina Precup. Towards continual reinforcement learning: A review and perspectives. Journal of Artificial Intelligence Research, 75:1401-1476, 2022. URL: https://doi.org/10.1613/JAIR.1.13673.
  34. Karol Kurach, Anton Raichuk, Piotr Stańczyk, Michał Zając, Olivier Bachem, Lasse Espeholt, Carlos Riquelme, Damien Vincent, Marcin Michalski, Olivier Bousquet, et al. Google research football: A novel reinforcement learning environment. In Proceedings of the AAAI conference on artificial intelligence, pages 4501-4510, 2020. Google Scholar
  35. Niels Landwehr. Modeling interleaved hidden processes. In Proceedings of the 25th international conference on Machine learning, pages 520-527, 2008. URL: https://doi.org/10.1145/1390156.1390222.
  36. D Luckham. The power of events: An introduction to complex event processing in distributed enterprise systems. additions-wesley. Reading, 2001. Google Scholar
  37. Sultan Javed Majeed and Marcus Hutter. On q-learning convergence for non-markov decision processes. In IJCAI, volume 18, pages 2546-2552, 2018. URL: https://doi.org/10.24963/IJCAI.2018/353.
  38. Nijat Mehdiyev, Julian Krumeich, David Enke, Dirk Werth, and Peter Loos. Determination of rule patterns in complex event processing using machine learning techniques. Procedia Computer Science, 61:395-401, 2015. URL: https://doi.org/10.1016/J.PROCS.2015.09.168.
  39. Vlad I Morariu and Larry S Davis. Multi-agent event recognition in structured scenarios. In CVPR 2011, pages 3289-3296. IEEE, 2011. URL: https://doi.org/10.1109/CVPR.2011.5995386.
  40. Robert Moskovitch. Multivariate temporal data analysis-a review. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 12(1):e1430, 2022. URL: https://doi.org/10.1002/WIDM.1430.
  41. Robert Moskovitch. Mining temporal data. Machine Learning for Data Science Handbook: Data Mining and Knowledge Discovery Handbook, pages 469-490, 2023. Google Scholar
  42. Sindhu Padakandla. A survey of reinforcement learning algorithms for dynamically varying environments. ACM Computing Surveys (CSUR), 54(6):1-25, 2021. URL: https://doi.org/10.1145/3459991.
  43. Carlos F Pfeiffer, Veralia Gabriela Sánchez, and Nils-Olav Skeie. A discrete event oriented framework for a smart house behavior monitor system. In 2016 12th International Conference on Intelligent Environments (IE), pages 119-123. IEEE, 2016. URL: https://doi.org/10.1109/IE.2016.26.
  44. Emmanuel Rachelson. Temporal markov decision problems. PhD thesis, Citeseer, 2009. Google Scholar
  45. Emmanuel Rachelson, Gauthier Quesnel, Frédérick Garcia, and Patrick Fabiani. A simulation-based approach for solving generalized semi-markov decision processes. In ECAI 2008, pages 583-587. IOS Press, 2008. URL: https://doi.org/10.3233/978-1-58603-891-5-583.
  46. Matthijs TJ Spaan. Partially observable markov decision processes. In Reinforcement learning: State-of-the-art, pages 387-414. Springer, 2012. URL: https://doi.org/10.1007/978-3-642-27645-3_12.
  47. Richard S Sutton. The quest for a common model of the intelligent decision maker. arXiv preprint arXiv:2202.13252, 2022. URL: https://arxiv.org/abs/2202.13252.
  48. Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018. Google Scholar
  49. Kevin Tang, Li Fei-Fei, and Daphne Koller. Learning latent temporal structure for complex event detection. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, pages 1250-1257. IEEE, 2012. URL: https://doi.org/10.1109/CVPR.2012.6247808.
  50. Guy Tennenholtz, Nadav Merlis, Lior Shani, Martin Mladenov, and Craig Boutilier. Reinforcement learning with history dependent dynamic contexts. In International Conference on Machine Learning, pages 34011-34053. PMLR, 2023. URL: https://proceedings.mlr.press/v202/tennenholtz23a.html.
  51. Mark Towers, Ariel Kwiatkowski, Jordan Terry, John U Balis, Gianluca De Cola, Tristan Deleu, Manuel Goulão, Andreas Kallinteris, Markus Krimmel, Arjun KG, et al. Gymnasium: A standard interface for reinforcement learning environments. arXiv preprint arXiv:2407.17032, 2024. Google Scholar
  52. Aashma Uprety and Danda B Rawat. Reinforcement learning for iot security: A comprehensive survey. IEEE Internet of Things Journal, 8(11):8693-8706, 2020. URL: https://doi.org/10.1109/JIOT.2020.3040957.
  53. Antonio Vitale, Alpha Renner, Celine Nauer, Davide Scaramuzza, and Yulia Sandamirskaya. Event-driven vision and control for uavs on a neuromorphic chip. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 103-109. IEEE, 2021. URL: https://doi.org/10.1109/ICRA48506.2021.9560881.
  54. Michalis Vrigkas, Christophoros Nikou, and Ioannis A Kakadiaris. A review of human activity recognition methods. Frontiers in Robotics and AI, 2:28, 2015. URL: https://doi.org/10.3389/FROBT.2015.00028.
  55. Cuebong Wong, Erfu Yang, Xiu-Tian Yan, and Dongbing Gu. Autonomous robots for harsh environments: a holistic overview of current solutions and ongoing challenges. Systems Science & Control Engineering, 6(1):213-219, 2018. Google Scholar
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2024 Schloss Dagstuhl – LZI GmbH Imprint Privacy Contact