Document Open Access Logo

An Offline Neuro-Symbolic Football Pattern Retrieval Approach Using Constraint Programming

Authors Augustin Crespin , Pierre Schaus

PDF

File

Thumbnail PDF
PDF
LIPIcs.CP.2026.16.pdf
  • Filesize: 23.42 MB
  • 22 pages

Document Identifiers

Subject Classification

ACM Subject Classification
  • Software and its engineering → Constraint and logic languages
  • Information systems → Video search
  • Theory of computation → Constraint and logic programming
  • Theory of computation → Pattern matching
Keywords
  • Pattern Matching
  • Domain-Specific Language
  • Video Analysis

Metrics

Abstract

Using a single broadcast camera, modern deep learning methods can detect and label players and ball positions on a frame-by-frame basis. This work focuses on post-game analysis, where frame-level labels are available for the entire video sequence. Deep learning alone performs poorly when retrieving intervals of frames in which specific spatio-temporal conditions or tactical patterns occur involving players and ball positions. A loosely coupled neuro-symbolic approach is proposed, in which these precomputed frame-level detections are processed through an SQL-like domain-specific query language. Each query is compiled into a Constraint Programming (CP) model that retrieves intervals of frames satisfying the specified constraints. The method leverages well-established CP constructs, such as time intervals and regular constraints. Experiments on real football games demonstrate that this approach is simple and efficient, enabling expressive querying for post-game tactical analysis while remaining accurate and scalable.

Cite As Get BibTex

Augustin Crespin and Pierre Schaus. An Offline Neuro-Symbolic Football Pattern Retrieval Approach Using Constraint Programming. In 32nd International Conference on Principles and Practice of Constraint Programming (CP 2026). Leibniz International Proceedings in Informatics (LIPIcs), Volume 379, pp. 16:1-16:22, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2026) https://doi.org/10.4230/LIPIcs.CP.2026.16

Author Details

Augustin Crespin
  • ICTEAM, UCLouvain, Louvain-la-Neuve, Belgium
Pierre Schaus
  • ICTEAM, UCLouvain, Louvain-la-Neuve, Belgium

Supplementary Materials

References

  1. John OR Aoga, Tias Guns, and Pierre Schaus. An efficient algorithm for mining frequent sequence with constraint programming. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 315-330. Springer, 2016. URL: https://doi.org/10.1007/978-3-319-46227-1_20.
  2. John OR Aoga, Tias Guns, and Pierre Schaus. Mining time-constrained sequential patterns with constraint programming. Constraints, 22(4):548-570, 2017. URL: https://doi.org/10.1007/s10601-017-9272-3.
  3. Relja Arandjelovic, Petr Gronát, Akihiko Torii, Tomás Pajdla, and Josef Sivic. Netvlad: Cnn architecture for weakly supervised place recognition. CoRR, abs/1511.07247, 2015. URL: https://doi.org/10.48550/arXiv.1511.07247.
  4. Mohamed-Bachir Belaid, Christian Bessiere, and Nadjib Lazaar. Constraint programming for association rules. In Proceedings of the 2019 SIAM International Conference on Data Mining, pages 127-135. SIAM, 2019. URL: https://doi.org/10.1137/1.9781611975673.15.
  5. Quentin Cappart, John OR Aoga, and Pierre Schaus. Episodesupport: A global constraint for mining frequent patterns in a long sequence of events. In International Conference on the Integration of Constraint Programming, Artificial Intelligence, and Operations Research, pages 82-99. Springer, 2018. URL: https://doi.org/10.1007/978-3-319-93031-2_7.
  6. Augustin Crespin. crespina/TactiCP. Software, URL: https://github.com/crespina/TactiCP (visited on 2026-05-18).
    Software Heritage Logo archived version
  7. Tom Decroos, Jan Van Haaren, and Jesse Davis. Automatic discovery of tactics in spatio-temporal soccer match data. In Yike Guo and Faisal Farooq, editors, Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2018, London, UK, August 19-23, 2018, KDD '18, pages 223-232, New York, NY, USA, 2018. ACM. URL: https://doi.org/10.1145/3219819.3219832.
  8. Adrien Deliège, Anthony Cioppa, Silvio Giancola, Meisam J. Seikavandi, Jacob V. Dueholm, Kamal Nasrollahi, Bernard Ghanem, Thomas B. Moeslund, and Marc Van Droogenbroeck. Soccernet-v2 : A dataset and benchmarks for holistic understanding of broadcast soccer videos. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2021. URL: https://doi.org/10.1109/CVPRW53098.2021.00508.
  9. Silvio Giancola, Anthony Cioppa, Bernard Ghanem, and Marc Van Droogenbroeck. Deep learning for action spotting in association football videos. CoRR, abs/2410.01304:427-459, July 2024. URL: https://doi.org/10.48550/arXiv.2410.01304.
  10. Silvio Giancola and Bernard Ghanem. Temporally-aware feature pooling for action spotting in soccer broadcasts, 2021. URL: https://doi.org/10.48550/arXiv.2104.06779.
  11. Vladimir Golovkin, Nikolay Nemtsev, Vasyl Shandyba, Oleg Udin, Nikita Kasatkin, Pavel Kononov, Anton Afanasiev, Sergey Ulasen, and Andrei Boiarov. From broadcast to minimap: Achieving state-of-the-art soccernet game state reconstruction, 2025. URL: https://doi.org/10.48550/arXiv.2504.06357.
  12. Tias Guns, Anton Dries, Siegfried Nijssen, Guido Tack, and Luc De Raedt. Miningzinc: A declarative framework for constraint-based mining. Artificial Intelligence, 244:6-29, 2017. URL: https://doi.org/10.1016/j.artint.2015.09.007.
  13. Tias Guns, Siegfried Nijssen, and Luc De Raedt. Itemset mining: A constraint programming perspective. Artificial Intelligence, 175(12-13):1951-1983, 2011. URL: https://doi.org/10.1016/j.artint.2011.05.002.
  14. Laszlo Gyarmati and Xavier Anguera. Automatic extraction of the passing strategies of soccer teams, 2015. URL: https://doi.org/10.48550/arXiv.1508.02171.
  15. K. Kausalya, S. Kanaga Suba Raja, and S. Sudha. An optimized deep learning approach for detection and classification of player actions in football game. Entertain. Comput., 55:101003, 2025. URL: https://doi.org/10.1016/j.entcom.2025.101003.
  16. Amina Kemmar, Samir Loudni, Yahia Lebbah, Patrice Boizumault, and Thierry Charnois. A global constraint for mining sequential patterns with gap constraint. In International Conference on AI and OR Techniques in Constriant Programming for Combinatorial Optimization Problems, pages 198-215. Springer, 2016. URL: https://doi.org/10.1007/978-3-319-33954-2_15.
  17. Philippe Laborie and Jerome Rogerie. Reasoning with conditional time-intervals. In Proceedings of the 21th International Florida Artificial Intelligence Research Society Conference, FLAIRS-21, pages 555-560, January 2008. URL: http://www.aaai.org/Library/FLAIRS/2008/flairs08-126.php.
  18. Connor McGillick, Chris Towlson, Steve Barrett, and John Toner. Performance analysis in sport and soccer: Past, present and future-narrative review. Journal of sports research, 11(1):19-41, 2024. Google Scholar
  19. Benjamin Negrevergne and Tias Guns. Constraint-based sequence mining using constraint programming. In International Conference on Integration of Constraint Programming, Artificial Intelligence, and Operations Research, pages 288-305. Springer, 2015. URL: https://doi.org/10.1007/978-3-319-18008-3_20.
  20. Yin May Oo, Yewon Hwang, Muhammad Amrulloh Robbani, Vanyi Chao, Ankhzaya Jamsrandorj, Hoang Quoc Nguyen, Kyung-Ryoul Mun, and Jinwook Kim. Broadcast2pitch: Game state reconstruction from unconstrained soccer videos. URL: https://api.semanticscholar.org/CorpusID:286258420.
  21. Gilles Pesant. A regular language membership constraint for finite sequences of variables. In Mark Wallace, editor, Principles and Practice of Constraint Programming - CP 2004, pages 482-495, Berlin, Heidelberg, 2004. Springer Berlin Heidelberg. URL: https://doi.org/10.1007/978-3-540-30201-8_36.
  22. Pierre Schaus, John OR Aoga, and Tias Guns. Coversize: A global constraint for frequency-based itemset mining. In International Conference on Principles and Practice of Constraint Programming, pages 529-546. Springer, 2017. URL: https://doi.org/10.1007/978-3-319-66158-2_34.
  23. Pierre Schaus, Guillaume Derval, Augustin Delecluse, Laurent Michel, and Pascal Van Hentenryck. Maxicp: A constraint programming solver for scheduling and vehicle routing, 2024. URL: https://github.com/aia-uclouvain/maxicp.
  24. João V. B. Soares, Avijit Shah, and Topojoy Biswas. Temporally precise action spotting in soccer videos using dense detection anchors. In 2022 IEEE International Conference on Image Processing (ICIP), pages 2796-2800, 2022. URL: https://doi.org/10.1109/ICIP46576.2022.9897256.
  25. Vladimir Somers and et al. SoccerNet game state reconstruction: End-to-end athlete tracking and identification on a minimap. In 2024 IEEE/CVF Conf. Comput. Vis. Pattern Recognit. Work. (CVPRW), June 2024. URL: https://doi.org/10.1109/CVPRW63382.2024.00334.
  26. Manuel Stein, Halldór Janetzko, Tobias Schreck, and Daniel A. Keim. Tackling similarity search for soccer match analysis: Multimodal distance measure and interactive query definition. IEEE Computer Graphics and Applications, 39(5):60-71, 2019. URL: https://doi.org/10.1109/MCG.2019.2922224.
  27. Ferran Vidal-Codina, Nicolas Evans, Bahaeddine El Fakir, and Johsan Billingham. Automatic event detection in football using tracking data, 2022. URL: https://doi.org/10.48550/arXiv.2202.00804.
  28. Xiao Xie, Jiachen Wang, Hongye Liang, Dazhen Deng, Shoubin Cheng, Hui Zhang, Wei Chen, and Yingcai Wu. Passvizor: Toward better understanding of the dynamics of soccer passes. IEEE Transactions on Visualization and Computer Graphics, 27(2):1322-1331, 2021. URL: https://doi.org/10.1109/TVCG.2020.3030359.
  29. Hao Xu, Arbind Agrahari Baniya, Sam Well, Mohamed Reda Bouadjenek, Richard Dazeley, and Sunil Aryal. Deep learning for sports video event detection: Tasks, datasets, methods, and challenges, 2025. URL: https://doi.org/10.48550/arXiv.2505.03991.
  30. Mohammed J. Zaki. Spade: An efficient algorithm for mining frequent sequences. Machine Learning, 42(1):31-60, January 2001. URL: https://doi.org/10.1023/A:1007652502315.
  31. Boqiang Zhang, Kehan Li, Zesen Cheng, Zhiqiang Hu, Yuqian Yuan, Guanzheng Chen, Sicong Leng, Yuming Jiang, Hang Zhang, Xin Li, Peng Jin, Wenqi Zhang, Fan Wang, Lidong Bing, and Deli Zhao. Videollama 3: Frontier multimodal foundation models for image and video understanding. arXiv preprint arXiv:2501.13106, 2025. URL: https://doi.org/10.48550/arXiv.2501.13106.
  32. Xin Zhou, Le Kang, Zhiyu Cheng, Bo He, and Jingyu Xin. Feature combination meets attention: Baidu soccer embeddings and transformer based temporal detection, 2021. URL: https://doi.org/10.48550/arXiv.2106.14447.
Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2026 Schloss Dagstuhl – LZI GmbH Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact