The Safe and Effective Use of Learning-Enabled Components in Safety-Critical Systems

Authors Kunal Agrawal , Sanjoy Baruah , Alan Burns

Thumbnail PDF


  • Filesize: 0.57 MB
  • 20 pages

Document Identifiers

Author Details

Kunal Agrawal
  • Washington University in Saint Louis, MO, USA
Sanjoy Baruah
  • Washington University in Saint Louis, MO, USA
Alan Burns
  • The University of York, UK

Cite AsGet BibTex

Kunal Agrawal, Sanjoy Baruah, and Alan Burns. The Safe and Effective Use of Learning-Enabled Components in Safety-Critical Systems. In 32nd Euromicro Conference on Real-Time Systems (ECRTS 2020). Leibniz International Proceedings in Informatics (LIPIcs), Volume 165, pp. 7:1-7:20, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2020)


Autonomous systems increasingly use components that incorporate machine learning and other AI-based techniques in order to achieve improved performance. The problem of assuring correctness in safety-critical systems that use such components is considered. A model is proposed in which components are characterized according to both their worst-case and their typical behaviors; it is argued that while safety must be assured under all circumstances, it is reasonable to be concerned with providing a high degree of performance for typical behaviors only. The problem of assuring safety while providing such improved performance is formulated as an optimization problem in which performance under typical circumstances is the objective function to be optimized while safety is a hard constraint that must be satisfied. Algorithmic techniques are applied to derive an optimal solution to this optimization problem. This optimal solution is compared with an alternative approach that optimizes for performance under worst-case conditions, as well as some common-sense heuristics, via simulation experiments on synthetically-generated workloads.

Subject Classification

ACM Subject Classification
  • Computer systems organization → Embedded and cyber-physical systems
  • Computing methodologies → Machine learning
  • Software and its engineering → Real-time schedulability
  • Learning-enabled components (LECs)
  • Safety-critical systems
  • Typical analysis
  • Performance optimization
  • Run-time monitoring


  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    PDF Downloads


  1. Assuring autonomy international programme. Accessed: 2020-01-17. URL:
  2. Kunal Agrawal and Sanjoy Baruah. Adaptive real-time routing in polynomial time. In Real-Time Systems Symposium (RTSS), 2019 IEEE, December 2019. Google Scholar
  3. Kunal Agrawal, Sanjoy Baruah, Alan Burns, and Abhishek Singh. Minimizing execution duration in the presence of learning-enabled components. In Proceedings of the Second International Workshop on Autonomous Systems Design (ASD 2020), 2020. Google Scholar
  4. Sanjoy Baruah. Rapid routing with guaranteed delay bounds. In Real-Time Systems Symposium (RTSS), 2018 IEEE, December 2018. Google Scholar
  5. Sanjoy Baruah and Nathan Fisher. Choosing preemption points to minimize typical running times. In Proceedings of the Twenty-Fourth International Conference on Real-Time and Network Systems, RTNS '19, New York, NY, USA, 2019. ACM. Google Scholar
  6. J. Lee, A. Prajogi, E. Rafalovsky, and P. Sarathy. Assuring behavior of autonomous UxV systems. In S5: The Air Force Research Laboratory (AFRL) Safe and Secure Systems and Software Symposium, July 2016. Google Scholar
  7. Robert M. Nauss. The 0-1 knapsack problem with multiple choice constraints. European Journal of Operational Research, 2(2):125-131, 1978. URL:
  8. Dr. Sandeep Neema. Assurance for Autonomous Systems is Hard. Accessed: 2019-03-07. URL:
  9. Sophie Quinton, Matthias Hanke, and Rolf Ernst. Formal analysis of sporadic overload in real-time systems. In Proceedings of the Conference on Design, Automation and Test in Europe, DATE '12, pages 515-520, San Jose, CA, USA, 2012. EDA Consortium. URL:
  10. John A. Stankovic and Krithi Ramamritham. What is predictability for real-time systems? Real-Time Syst., 2(4):247-254, October 1990. URL:
  11. Reinhard Wilhelm, Jakob Engblom, Andreas Ermedahl, Niklas Holsti, Stephan Thesing, David Whalley, Guillem Bernat, Christian Ferdinand, Reinhold Heckmann, Tulika Mitra, Frank Mueller, Isabelle Puaut, Peter Puschner, Jan Staschulat, and Per Stenström. The worst-case execution-time problem - overview of methods and survey of tools. ACM Transactions on Embedded Computing Systems, 7(3):36:1-36:53, May 2008. Google Scholar
Questions / Remarks / Feedback

Feedback for Dagstuhl Publishing

Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail