Document Open Access Logo

EMSBench: Benchmark and Testbed for Reactive Real-Time Systems

Authors Florian Kluge, Christine Rochange , Theo Ungerer

PDF
Thumbnail PDF

File

LITES-v004-i002-a002.pdf
  • Filesize: 0.66 MB
  • 23 pages

Document Identifiers

Author Details

Florian Kluge
  • University of Augsburg
Christine Rochange
  • IRIT, Université de Toulouse, CNRS
Theo Ungerer
  • University of Augsburg

Cite As Get BibTex

Florian Kluge, Christine Rochange, and Theo Ungerer. EMSBench: Benchmark and Testbed for Reactive Real-Time Systems. In LITES, Volume 4, Issue 2 (2017). Leibniz Transactions on Embedded Systems, Volume 4, Issue 2, pp. 02:1-02:23, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2017) https://doi.org/10.4230/LITES-v004-i002-a002

Abstract

Benchmark suites for real-time embedded systems (RTES) usually contain only pure computations that are often used in this domain. They allow to evaluate computing performance, but do not reproduce the complexity and behaviour that is typical for such systems. Actual RTES have to interact with the physical environment, which is often reflected by code that is executed concurrently. In this article, we present the software package EMSBench that mimics such complex behaviour, and highlight some of its use cases. The benchmark code ems of EMSBench is based on the open-source engine management system (EMS) FreeEMS. Additionally, EMSBench contains a trace generator (tg) that provides input signals for ems and enables to execute ems close to reality. We provide detailed descriptions of the ems's execution behaviour and of trace generation. EMSBench can be used as test or benchmark program to compare different hardware platforms, e.g. in terms of schedulability. Also, we use EMSBench as a benchmark for static worst-case execution time (WCET) analysis and compare these results to measurements performed on existing hardware. Our results based on the OTAWA WCET estimation tool show WCET overestimations by the static analysis from 11.9% to 41.1% depending on the complexity of the analysed functions.

Subject Classification

ACM Subject Classification
  • Computer systems organization
  • Computer systems organization → Real-time systems
  • Software and its engineering
  • Software and its engineering → Real-time systems software
Keywords
  • Real-time benchmark
  • WCET Analysis
  • Engine Management System

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

References

  1. Clément Ballabriga, Hugues Cassé, Christine Rochange, and Pascal Sainrat.OTAWA: An Open Toolbox for Adaptive WCET Analysis. In Sang Lyul Min, Robert G. Pettit IV, Peter P. Puschner, and Theo Ungerer, editors, Software Technologies for Embedded and Ubiquitous Systems - 8th IFIPWG 10.2 Int'l Workshop, SEUS 2010, Waidhofen/Ybbs, Austria, October 13-15, 2010. Proceedings, volume 6399 of Lecture Notes in Computer Science, pages 35-46. Springer, 2010. URL: http://dx.doi.org/10.1007/978-3-642-16256-5_6
  2. Benchmark program and test bed for reactive embedded systems. GitHub repository. URL: https://github.com/unia-sik/emsbench.
  3. Heiko Falk, Sebastian Altmeyer, Peter Hellinckx, Björn Lisper, Wolfgang Puffitsch, Christine Rochange, Martin Schoeberl, Rasmus Bo Sorensen, Peter Wägemann, and Simon Wegener.TACLeBench: A Benchmark Collection to Support Worst-Case Execution Time Research. In Martin Schoeberl, editor, 16th Int'l Workshop on Worst-Case Execution Time Analysis, WCET 2016, July 5, 2016, Toulouse, France, volume 55 of OASIcs, pages 2:1-2:10. Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik, 2016. URL: http://dx.doi.org/10.4230/OASIcs.WCET.2016.2
  4. Jan Gustafsson, Adam Betts, Andreas Ermedahl, and Björn Lisper.The Mälardalen WCET Benchmarks: Past, Present And Future. In Björn Lisper, editor, 10th Int'l Workshop on Worst-Case Execution Time Analysis, WCET 2010, July 6, 2010, Brussels, Belgium, volume 15 of OASIcs, pages 136-146. Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik, Germany, 2010. URL: http://dx.doi.org/10.4230/OASIcs.WCET.2010.136
  5. Jan Gustafsson, Andreas Ermedahl, Christer Sandberg, and Björn Lisper.Automatic Derivation of Loop Bounds and Infeasible Paths for WCET Analysis Using Abstract Execution. In Proceedings of the 27th IEEE Real-Time Systems Symposium (RTSS 2006), 5-8 December 2006, Rio de Janeiro, Brazil, pages 57-66. IEEE Computer Society, 2006. URL: http://dx.doi.org/10.1109/RTSS.2006.12
  6. Jeff Hartman.How to Tune and Modify Engine Management Systems. Motorbooks Workshop. MBI Publishing Company, 2004. Google Scholar
  7. Niklas Holsti, Jan Gustafsson, Linus Källberg, and Björn Lisper.Analysing Switch-Case Code with Abstract Execution. In Francisco J. Cazorla, editor, 15th Int'l Workshop on Worst-Case Execution Time Analysis, WCET 2015, July 7, 2015, Lund, Sweden, volume 47 of OASIcs, pages 85-94. Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik, 2015. URL: http://dx.doi.org/10.4230/OASIcs.WCET.2015.85
  8. Niklas Holsti, Thomas Langbacka, and Sami Saarinen.Using a worst-case execution time tool for real-time verification of the DEBIE software. European Space Agency Publications - ESA SP, 457:307-312, 2000. Google Scholar
  9. Florian Kluge.A Simple Capture/Compare Timer. Technical Report 2015-01, Department of Computer Science, University of Augsburg, June 2015. URL: http://dx.doi.org/10.13140/2.1.1251.2321
  10. Florian Kluge and Theo Ungerer.EMSBench: Benchmark und Testumgebung für reaktive Systeme. In Wolfgang A. Halang and Olaf Spinczyk, editors, Echtzeit 2015, Informatik Aktuell, pages 11-20. Springer, 2015. URL: http://dx.doi.org/10.1007/978-3-662-48611-5_2
  11. Yau-Tsun Steven Li and Sharad Malik.Performance Analysis of Embedded Software Using Implicit Path Enumeration. In Bryan Preas, editor, Proceedings of the 32st Conference on Design Automation, San Francisco, California, USA, Moscone Center, June 12-16, 1995., pages 456-461. ACM Press, 1995. URL: http://dx.doi.org/10.1145/217474.217570
  12. Marianne De Michiel, Armelle Bonenfant, Hugues Cassé, and Pascal Sainrat.Static Loop Bound Analysis of C Programs Based on Flow Analysis and Abstract Interpretation. In The Fourteenth IEEE Internationl Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2008, Kaohisung, Taiwan, 25-27 August 2008, Proceedings, pages 161-166. IEEE Computer Society, 2008. URL: http://dx.doi.org/10.1109/RTCSA.2008.53
  13. COUNCIL DIRECTIVE of 20 March 1970 on the approximation of the laws of the Member States on measures to be taken against air pollution by emissions from motor vehicles. Version from 01.01.2007. Google Scholar
  14. Fadia Nemer, Hugues Cassé, Pascal Sainrat, Jean Paul Bahsoun, and Marianne De Michiel.PapaBench: a Free Real-Time Benchmark. In Frank Mueller, editor, 6th Int'l Workshop on Worst-Case Execution Time (WCET) Analysis, July 4, 2006, Dresden, Germany, volume 4 of OASIcs. Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik, 2006. URL: http://dx.doi.org/10.4230/OASIcs.WCET.2006.678
  15. William H. Press, Saul A. Teukolsky, William T. Vetterling, and Brian P. Flannery.Numerical Recipes in C: The Art of Scientific Computing. Cambridge University Press, New York, NY, USA, 1992. 2nd edition. Google Scholar
  16. Wolfgang Puffitsch.Efficient Worst-Case Execution Time Analysis of Dynamic Branch Prediction. In 28th Euromicro Conference on Real-Time Systems, ECRTS 2016, Toulouse, France, July 5-8, 2016, pages 152-162. IEEE Computer Society, 2016. URL: http://dx.doi.org/10.1109/ECRTS.2016.23
  17. Jan Reineke, Daniel Grund, Christoph Berg, and Reinhard Wilhelm.Timing predictability of cache replacement policies. Real-Time Systems, 37(2):99-122, 2007. URL: http://dx.doi.org/10.1007/s11241-007-9032-3
  18. Christine Rochange and Pascal Sainrat.A Context-Parameterized Model for Static Analysis of Execution Times. Trans. HiPEAC, 2:222-241, 2009. URL: http://dx.doi.org/10.1007/978-3-642-00904-4_12
  19. STMicroelectronics.UM1472 User Manual - Discovery kit for STM32f407/417 lines. STMicroelectronics, November 2013. Google Scholar
  20. Trace Generation in EMSBench. URL: https://github.com/unia-sik/emsbench/blob/master/doc/tg/tg.pdf.
  21. Stephan Thesing.Safe and precise WCET determination by abstract interpretation of pipeline models. PhD thesis, Saarland University, Saarbrücken, Germany, 2004. URL: http://scidok.sulb.uni-saarland.de/volltexte/2005/466/index.html.
  22. Peter Wägemann, Tobias Distler, Timo Hönig, Volkmar Sieh, and Wolfgang Schröder-Preikschat.GenE: A Benchmark Generator for WCET Analysis. In Francisco J. Cazorla, editor, 15th Int'l Workshop on Worst-Case Execution Time Analysis, WCET 2015, July 7, 2015, Lund, Sweden, volume 47 of OASIcs, pages 33-43. Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik, 2015. URL: http://dx.doi.org/10.4230/OASIcs.WCET.2015.33
  23. Henning Wallentowitz and Konrad Reif, editors.Handbuch Kraftfahrzeugelektronik: Grundlagen, Komponenten, Systeme, Anwendungen. Vieweg, Wiesbaden, 2006. Google Scholar
  24. Reinhard Wilhelm, Jakob Engblom, Andreas Ermedahl, Niklas Holsti, Stephan Thesing, David B. Whalley, Guillem Bernat, Christian Ferdinand, Reinhold Heckmann, Tulika Mitra, Frank Mueller, Isabelle Puaut, Peter P. Puschner, Jan Staschulat, and Per Stenström.The worst-case execution-time problem - overview of methods and survey of tools.ACM Trans. Embedded Comput. Syst., 7(3):36:1-36:53, 2008. URL: http://dx.doi.org/10.1145/1347375.1347389
  25. Javad Zareei and Amir H. Kakaee.Study and the effects of ignition timing on gasoline engine performance and emissions. European Transport Research Review, 5(2):109-116, 2013. URL: http://dx.doi.org/10.1007/s12544-013-0099-8
  26. Jakob Zwirchmayr, Pascal Sotin, Armelle Bonenfant, Denis Claraz, and Philippe Cuenot.Identifying Relevant Parameters to Improve WCET Analysis. In Heiko Falk, editor, 14th Int'l Workshop on Worst-Case Execution Time Analysis, WCET 2014, July 8, 2014, Ulm, Germany, volume 39 of OASIcs, pages 93-102. Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik, 2014. URL: http://dx.doi.org/10.4230/OASIcs.WCET.2014.93
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2024 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact