Towards Streaming Evaluation of Queries with Correlation in Complex Event Processing

Authors Alejandro Grez, Cristian Riveros

Thumbnail PDF


  • Filesize: 0.56 MB
  • 17 pages

Document Identifiers

Author Details

Alejandro Grez
  • Pontificia Universidad Católica de Chile, Santiago, Chile
  • Millennium Institute for Foundational Research on Data, Santiago, Chile
Cristian Riveros
  • Pontificia Universidad Católica de Chile, Santiago, Chile
  • Millennium Institute for Foundational Research on Data, Santiago, Chile

Cite AsGet BibTex

Alejandro Grez and Cristian Riveros. Towards Streaming Evaluation of Queries with Correlation in Complex Event Processing. In 23rd International Conference on Database Theory (ICDT 2020). Leibniz International Proceedings in Informatics (LIPIcs), Volume 155, pp. 14:1-14:17, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2020)


Complex event processing (CEP) has gained a lot of attention for evaluating complex patterns over high-throughput data streams. Recently, new algorithms for the evaluation of CEP patterns have emerged with strong guarantees of efficiency, i.e. constant update-time per tuple and constant-delay enumeration. Unfortunately, these techniques are restricted for patterns with local filters, limiting the possibility of using joins for correlating the data of events that are far apart. In this paper, we embark on the search for efficient evaluation algorithms of CEP patterns with joins. We start by formalizing the so-called partition-by operator, a standard operator in data stream management systems to correlate contiguous events on streams. Although this operator is a restricted version of a join query, we show that partition-by (without iteration) is equally expressive as hierarchical queries, the biggest class of full conjunctive queries that can be evaluated with constant update-time and constant-delay enumeration over streams. To evaluate queries with partition-by we introduce an automata model, called chain complex event automata (chain-CEA), an extension of complex event automata that can compare data values by using equalities and disequalities. We show that this model admits determinization and is expressive enough to capture queries with partition-by. More importantly, we provide an algorithm with constant update time and constant delay enumeration for evaluating any query definable by chain-CEA, showing that all CEP queries with partition-by can be evaluated with these strong guarantees of efficiency.

Subject Classification

ACM Subject Classification
  • Information systems → Data streams
  • Theory of computation → Database query processing and optimization (theory)
  • Theory of computation → Formal languages and automata theory
  • Theory of computation → Automata extensions
  • Complex event processing
  • Query languages
  • Correlation
  • Constant delay enumeration.


  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    PDF Downloads


  1. S. Abiteboul, R. Hull, and V. Vianu. Foundations of databases: the logical level. Addison-Wesley, 1995. Google Scholar
  2. Yanif Ahmad, Oliver Kennedy, Christoph Koch, and Milos Nikolic. Dbtoaster: Higher-order delta processing for dynamic, frequently fresh views. Proceedings of the VLDB Endowment, 5(10):968-979, 2012. Google Scholar
  3. A. Aho and J. Hopcroft. The design and analysis of computer algorithms. Pearson Education India, 1974. Google Scholar
  4. E. Alevizos, A. Artikis, and G. Paliouras. Symbolic Automata with Memory: a Computational Model for CEP. arXiv preprint arXiv:1804.09999, 2018. Google Scholar
  5. A. Arasu, S. Babu, and J. Widom. The CQL Continuous Query Language: Semantic Foundations and Query Execution. The VLDB Journal, 2006. Google Scholar
  6. Guillaume Bagan. MSO queries on tree decomposable structures are computable with linear delay. In International Workshop on Computer Science Logic, pages 167-181. Springer, 2006. Google Scholar
  7. C. Berkholz, J. Keppeler, and N. Schweikardt. Answering conjunctive queries under updates. In PODS, pages 303-318, 2017. Google Scholar
  8. Stefano Ceri and Jennifer Widom. Deriving Production Rules for Incremental View Maintenance. In VLDB, 1991. Google Scholar
  9. Rada Chirkova, Jun Yang, et al. Materialized views. Foundations and Trendsregistered in Databases, 4(4):295-405, 2012. Google Scholar
  10. T. Cormen, C. Leiserson, R. Rivest, and C. Stein. Introduction to algorithms. MIT press, 2009. Google Scholar
  11. Bruno Courcelle. Linear delay enumeration and monadic second-order logic. Discrete Applied Mathematics, 157(12):2675-2700, 2009. Google Scholar
  12. G. Cugola and A. Margara. Processing flows of information: From data stream to complex event processing. ACM Computing Surveys, 2012. Google Scholar
  13. Nilesh N. Dalvi and Dan Suciu. The dichotomy of conjunctive queries on probabilistic structures. In Proceedings of the Twenty-Sixth ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems, June 11-13, 2007, Beijing, China, pages 293-302, 2007. Google Scholar
  14. James R Driscoll, Neil Sarnak, Daniel D Sleator, and Robert E Tarjan. Making data structures persistent. Journal of computer and system sciences, 38(1):86-124, 1989. Google Scholar
  15. Esper Enterprise Edition website. Accessed: 2018-12-21. URL:
  16. Opher Etzion, Peter Niblett, and David C Luckham. Event processing in action. Manning Greenwich, 2011. Google Scholar
  17. A. Grez, C. Riveros, and M. Ugarte. A formal framework for Complex Event Processing. In ICDT, 2019. Google Scholar
  18. A. Grez, C. Riveros, M. Ugarte, and S. Vansummeren. A Second-Order Approach to Complex Event Recognition. arXiv preprint arXiv:1712.01063, 2017. Google Scholar
  19. M. Groover. Automation, production systems, and computer-integrated manufacturing. Prentice Hall, 2007. Google Scholar
  20. Monika Henzinger, Sebastian Krinninger, Danupon Nanongkai, and Thatchaphol Saranurak. Unifying and strengthening hardness for dynamic problems via the online matrix-vector multiplication conjecture. In Proceedings of the forty-seventh annual ACM symposium on Theory of computing, pages 21-30. ACM, 2015. Google Scholar
  21. Martin Hirzel, Guillaume Baudart, Angela Bonifati, Emanuele Della Valle, Sherif Sakr, and Akrivi Akrivi Vlachou. Stream processing languages in the big data era. ACM SIGMOD Record, 47(2):29-40, 2018. Google Scholar
  22. M. Idris, M. Ugarte, and S. Vansummeren. The dynamic Yannakakis algorithm: Compact and efficient query processing under updates. In SIGMOD, 2017. Google Scholar
  23. M. Idris, M. Ugarte, S. Vansummeren, H. Voigt, and W. Lehner. Conjunctive queries with inequalities under updates. VLDB, 11(7):733-745, 2018. Google Scholar
  24. M. Kaminski and N. Francez. Finite-memory automata. Theoretical Computer Science, 134(2):329-363, 1994. Google Scholar
  25. Christoph Koch. Incremental query evaluation in a ring of databases. In Proceedings of the twenty-ninth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, pages 87-98. ACM, 2010. Google Scholar
  26. Paraschos Koutris and Dan Suciu. Parallel evaluation of conjunctive queries. In Proceedings of the thirtieth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, pages 223-234. ACM, 2011. Google Scholar
  27. B. Mukherjee, T. Heberlein, and K. Levitt. Network intrusion detection. IEEE network, 1994. Google Scholar
  28. B. Sahay and J. Ranjan. Real time business intelligence in supply chain analytics. Information Management & Computer Security, 2008. Google Scholar
  29. L. Segoufin. Automata and logics for words and trees over an infinite alphabet. In CSL, 2006. Google Scholar
  30. L. Segoufin. Enumerating with constant delay the answers to a query. In Proceedings of the 16th International Conference on Database Theory, pages 10-20. ACM, 2013. Google Scholar
  31. E. Wu, Y. Diao, and S. Rizvi. High-performance complex event processing over streams. In SIGMOD, 2006. Google Scholar
Questions / Remarks / Feedback

Feedback for Dagstuhl Publishing

Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail