Document Open Access Logo

CAWET: Context-Aware Worst-Case Execution Time Estimation Using Transformers

Authors Abderaouf N Amalou , Elisa Fromont , Isabelle Puaut

PDF
Thumbnail PDF

File

LIPIcs.ECRTS.2023.7.pdf
  • Filesize: 1 MB
  • 20 pages

Document Identifiers

Author Details

Abderaouf N Amalou
  • Univ. Rennes, INRIA, CNRS, IRISA, France
Elisa Fromont
  • Univ. Rennes, IUF, INRIA, CNRS, IRISA, France
Isabelle Puaut
  • Univ. Rennes, INRIA, CNRS, IRISA, France

Cite AsGet BibTex

Abderaouf N Amalou, Elisa Fromont, and Isabelle Puaut. CAWET: Context-Aware Worst-Case Execution Time Estimation Using Transformers. In 35th Euromicro Conference on Real-Time Systems (ECRTS 2023). Leibniz International Proceedings in Informatics (LIPIcs), Volume 262, pp. 7:1-7:20, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2023)
https://doi.org/10.4230/LIPIcs.ECRTS.2023.7

Abstract

This paper presents CAWET, a hybrid worst-case program timing estimation technique. CAWET identifies the longest execution path using static techniques, whereas the worst-case execution time (WCET) of basic blocks is predicted using an advanced language processing technique called Transformer-XL. By employing Transformers-XL in CAWET, the execution context formed by previously executed basic blocks is taken into account, allowing for consideration of the micro-architecture of the processor pipeline without explicit modeling. Through a series of experiments on the TacleBench benchmarks, using different target processors (Arm Cortex M4, M7, and A53), our method is demonstrated to never underestimate WCETs and is shown to be less pessimistic than its competitors.

Subject Classification

ACM Subject Classification
  • Computer systems organization → Real-time system architecture
Keywords
  • Worst-case execution time
  • machine learning
  • transformers
  • hybrid technique

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

References

  1. Abderaouf N. Amalou, Elisa Fromont, and Isabelle Puaut. Catreen: Context-aware code timing estimation with stacked recurrent networks. In 34rd IEEE International Conference on Tools with Artificial Intelligence, (ICTAI). IEEE, 2022. Google Scholar
  2. Abderaouf N Amalou, Isabelle Puaut, and Gilles Muller. We-hml: hybrid wcet estimation using machine learning for architectures with caches. In 2021 IEEE 27th International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA), pages 31-40. IEEE, 2021. Google Scholar
  3. Clément Ballabriga, Hugues Cassé, Christine Rochange, and Pascal Sainrat. Otawa: An open toolbox for adaptive wcet analysis. In Software Technologies for Embedded and Ubiquitous Systems: 8th IFIP WG 10.2 International Workshop, SEUS 2010, Waidhofen/Ybbs, Austria, October 13-15, 2010. Proceedings 8, pages 35-46. Springer, 2010. Google Scholar
  4. Adam Betts, Nicholas Merriam, and Guillem Bernat. Hybrid measurement-based wcet analysis at the source level using object-level traces. In 10th International Workshop on Worst-Case Execution Time Analysis (WCET 2010). Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik, 2010. Google Scholar
  5. Armelle Bonenfant, Denis Claraz, Marianne De Michiel, and Pascal Sotin. Early wcet prediction using machine learning. In 17th International Workshop on Worst-Case Execution Time Analysis (WCET 2017), pages 5-1. OASICs, Dagstuhl Publishing, 2017. Google Scholar
  6. Axel Brando, Isabel Serra, Enrico Mezzetti, Jaume Abella, and Francisco J Cazorla. Using quantile regression in neural networks for contention prediction in multicore processors. In 34th Euromicro Conference on Real-Time Systems (ECRTS 2022). Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2022. Google Scholar
  7. Cédric Courtaud, Julien Sopena, Gilles Muller, and Daniel Gracia Pérez. Improving prediction accuracy of memory interferences for multicore platforms. In 2019 IEEE Real-Time Systems Symposium (RTSS), pages 246-259. IEEE, 2019. Google Scholar
  8. Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc V Le, and Ruslan Salakhutdinov. Transformer-xl: Attentive language models beyond a fixed-length context. arXiv preprint, 2019. URL: https://arxiv.org/abs/1901.02860.
  9. Jean-François Deverge and Isabelle Puaut. Safe measurement-based wcet estimation. In 5th International Workshop on Worst-Case Execution Time Analysis (WCET'05). Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2007. Google Scholar
  10. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT, pages 4171-4186. Association for Computational Linguistics, 2019. Google Scholar
  11. Boris Dreyer, Christian Hochberger, Alexander Lange, Simon Wegener, and Alexander Weiss. Continuous non-intrusive hybrid wcet estimation using waypoint graphs. In 16th International Workshop on Worst-Case Execution Time Analysis (WCET 2016). Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik, 2016. Google Scholar
  12. Christof Ebert, James Cain, Giuliano Antoniol, Steve Counsell, and Phillip Laplante. Cyclomatic complexity. IEEE software, 33(6):27-29, 2016. Google Scholar
  13. Heiko Falk, Sebastian Altmeyer, Peter Hellinckx, Björn Lisper, Wolfgang Puffitsch, Christine Rochange, Martin Schoeberl, Rasmus Bo Sørensen, Peter Wägemann, and Simon Wegener. Taclebench: A benchmark collection to support worst-case execution time research. In 16th International Workshop on Worst-Case Execution Time Analysis, 2016. Google Scholar
  14. SEGGER Microcontroller GmbH. Ozone User Guide & Reference Manual. URL: https://www.segger.com/.
  15. Matthew R Guthaus, Jeffrey S Ringenberg, Dan Ernst, Todd M Austin, Trevor Mudge, and Richard B Brown. Mibench: A free, commercially representative embedded benchmark suite. In 4th IEEE international workshop on workload characterization, 2001. Google Scholar
  16. Damien Hardy, Benjamin Rouxel, and Isabelle Puaut. The heptane static worst-case execution time estimation tool. In 17th International Workshop on Worst-Case Execution Time Analysis (WCET 2017). Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik, 2017. Google Scholar
  17. Thomas Huybrechts, Thomas Cassimon, Siegfried Mercelis, and Peter Hellinckx. Introduction of deep neural network in hybrid wcet analysis. In Advances on P2P, Parallel, Grid, Cloud and Internet Computing: Proceedings of the 13th International Conference on P2P, Parallel, Grid, Cloud and Internet Computing (3PGCIC-2018), pages 415-425. Springer, 2019. Google Scholar
  18. Thomas Huybrechts, Siegfried Mercelis, and Peter Hellinckx. A new hybrid approach on wcet analysis for real-time systems using machine learning. In 18th International Workshop on Worst-Case Execution Time Analysis (WCET 2018). Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik, 2018. Google Scholar
  19. Richard Johnson, David Pearson, and Keshav Pingali. The program structure tree: Computing control regions in linear time. In Vivek Sarkar, Barbara G. Ryder, and Mary Lou Soffa, editors, Proceedings of the ACM SIGPLAN'94 Conference on Programming Language Design and Implementation (PLDI), Orlando, Florida, USA, June 20-24, 1994, pages 171-185. ACM, 1994. URL: https://doi.org/10.1145/178243.178258.
  20. Daniel Kästner, Markus Pister, Simon Wegener, and Christian Ferdinand. Obtaining worst-case execution time bounds on modern microprocessors. In Embedded World Conference, 2018. Google Scholar
  21. Daniel Kästner, Markus Pister, Simon Wegener, and Christian Ferdinand. Timeweaver: A tool for hybrid worst-case execution time analysis. In 19th International Workshop on Worst-Case Execution Time Analysis (WCET 2019). Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik, 2019. Google Scholar
  22. Raimund Kirner, Ingomar Wenzel, Bernhard Rieder, and Peter Puschner. Using measurements as a complement to static worst-case execution time analysis. Intelligent Systems at the Service of Mankind, 2(8):20, 2005. Google Scholar
  23. Taku Kudo and John Richardson. Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. arXiv preprint, 2018. URL: https://arxiv.org/abs/1808.06226.
  24. Vikash Kumar. Deep neural network approach to estimate early worst-case execution time. In 2021 IEEE/AIAA 40th Digital Avionics Systems Conference (DASC), pages 1-8. IEEE, 2021. Google Scholar
  25. Vikash Kumar. Estimation of an early wcet using different machine learning approaches. In Advances on P2P, Parallel, Grid, Cloud and Internet Computing: Proceedings of the 17th International Conference on P2P, Parallel, Grid, Cloud and Internet Computing (3PGCIC-2022), pages 297-307. Springer, 2022. Google Scholar
  26. Björn Lisper and Marcelo Santos. Model identification for wcet analysis. In 2009 15th IEEE Real-Time and Embedded Technology and Applications Symposium, pages 55-64. IEEE, 2009. Google Scholar
  27. Charith Mendis, Alex Renda, Saman Amarasinghe, and Michael Carbin. Ithemal: Accurate, portable and fast basic block throughput estimation using deep neural networks. In International Conference on machine learning, pages 4505-4515. PMLR, 2019. Google Scholar
  28. Ruchir Puri, David S Kung, Geert Janssen, Wei Zhang, Giacomo Domeniconi, Vladimir Zolotov, Julian Dolby, Jie Chen, Mihir Choudhury, Lindsey Decker, et al. Codenet: A large-scale ai for code dataset for learning a diversity of coding tasks. arXiv preprint, 2021. URL: https://arxiv.org/abs/2105.12655.
  29. Federico Reghenzani, Giuseppe Massari, William Fornaciari, and Andrea Galimberti. Probabilistic-wcet reliability: On the experimental validation of evt hypotheses. In Proceedings of the International Conference on Omni-Layer Intelligent Systems, pages 229-234, 2019. Google Scholar
  30. Federico Reghenzani, Luca Santinelli, and William Fornaciari. Dealing with uncertainty in pwcet estimations. ACM Transactions on Embedded Computing Systems (TECS), 19(5):1-23, 2020. Google Scholar
  31. Segger. J-Trace PRO – The Leading Trace Solution. URL: https://www.segger.com/products/debug-probes/j-trace/.
  32. Hardik Shah, Andrew Coombes, Andreas Raabe, Kai Huang, and Alois Knoll. Measurement based wcet analysis for multi-core architectures. In Proceedings of the 22Nd International Conference on Real-Time Networks and Systems, pages 257-266, 2014. Google Scholar
  33. Jun S Shim, Bogyeong Han, Yeseong Kim, and Jihong Kim. Deeppm: transformer-based power and performance prediction for energy-aware software. In 2022 Design, Automation & Test in Europe Conference & Exhibition (DATE), pages 1491-1496. IEEE, 2022. Google Scholar
  34. Robert Tarjan. Depth-first search and linear graph algorithms. SIAM journal on computing, 1(2):146-160, 1972. Google Scholar
  35. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. Google Scholar
  36. Reinhard Wilhelm, Jakob Engblom, Andreas Ermedahl, Niklas Holsti, Stephan Thesing, David Whalley, Guillem Bernat, Christian Ferdinand, Reinhold Heckmann, Tulika Mitra, et al. The worst-case execution-time problem—overview of methods and survey of tools. ACM Transactions on Embedded Computing Systems (TECS), 7(3):1-53, 2008. Google Scholar
  37. Tomofumi Yuki. Understanding polybench/c 3.2 kernels. In International workshop on polyhedral compilation techniques (IMPACT), pages 1-5, 2014. Google Scholar
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2024 Schloss Dagstuhl – LZI GmbH Imprint Privacy Contact