On How to Identify Cache Coherence: Case of the NXP QorIQ T4240

Authors Nathanaël Sensfelder, Julien Brunel, Claire Pagetti

Thumbnail PDF


  • Filesize: 0.75 MB
  • 22 pages

Document Identifiers

Author Details

Nathanaël Sensfelder
  • ONERA, Toulouse, France
Julien Brunel
  • ONERA, Toulouse, France
Claire Pagetti
  • ONERA, Toulouse, France

Cite AsGet BibTex

Nathanaël Sensfelder, Julien Brunel, and Claire Pagetti. On How to Identify Cache Coherence: Case of the NXP QorIQ T4240. In 32nd Euromicro Conference on Real-Time Systems (ECRTS 2020). Leibniz International Proceedings in Informatics (LIPIcs), Volume 165, pp. 13:1-13:22, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2020)


Architectures used in safety critical systems have to pass certain certification standards, which require sufficient proof that they will behave as expected. Multi-core processors make this challenging by featuring complex interactions between the tasks they run. A lot of these interactions are made without explicit instructions from the program designers. Furthermore, they can have strong negative impacts on performance (and potentially affect correctness). One important such source of interactions is cache coherence, which speeds up operations in most cases, but can also lead to unexpected variations in execution time if not fully understood. Architecture documentations often lack details on the implementation of cache coherence. We thus propose a strategy to ascertain that the platform does indeed implement the cache coherence protocol its user believes it to. We also apply this strategy to the NXP QorIQ T4240, resulting in the identification of a protocol (MESIF) other than the one this architecture’s documentation led us to believe it was using (MESI).

Subject Classification

ACM Subject Classification
  • Computer systems organization → Multicore architectures
  • Computer systems organization → Real-time systems
  • Real-time systems
  • multi-core processor
  • cache coherence


  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    PDF Downloads


  1. Jyotika Athavale, Riccardo Mariani, and Michael Paulitsch. Flight safety certification implications for complex multi-core processor based avionics systems. In 25th IEEE International Symposium on On-Line Testing and Robust System Design, IOLTS 2019, Rhodes, Greece, July 1-3, 2019, pages 38-39, 2019. Google Scholar
  2. Ayoosh Bansal, Jayati Singh, Yifan Hao, Jen-Yang Wen, Renato Mancuso, and Marco Caccamo. Cache where you want! reconciling predictability and coherent caching, 2019. URL: http://arxiv.org/abs/1909.05349.
  3. Pedro Benedicte, Carles Hernandez, Jaume Abella, and Francisco J. Cazorla. HWP: Hardware Support to Reconcile Cache Energy, Complexity, Performance and WCET Estimates in Multicore Real-Time Systems. In 30th Euromicro Conference on Real-Time Systems (ECRTS 2018), volume 106 of Leibniz International Proceedings in Informatics (LIPIcs), pages 3:1-3:22, Dagstuhl, Germany, 2018. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik. URL: https://doi.org/10.4230/LIPIcs.ECRTS.2018.3.
  4. Jingyi Bin, Sylvain Girbal, Daniel Gracia Perez, Arnaud Grasset, and Alain Merigot. Studying co-running avionic real-time applications on multi-core cots architectures. In Embedded Real Time Software and System Conference (ERTS2), February 2014. Google Scholar
  5. Frédéric Boniol, Youcef Bouchebaba, Julien Brunel, Kevin Delmas, Thomas Loquen, Alfonso Mascarenas Gonzalez, Claire Pagetti, Thomas Polacsek, and Nathanaël Sensfelder. PHYLOG certification methodology: a sane way to embed multi-core processors. In 10th European Congress on Embedded Real Time Software and Systems (ERTS 2020), Toulouse, France, January 2020. URL: https://hal.archives-ouvertes.fr/hal-02441323.
  6. Jason F. Cantin, Mikko H. Lipasti, and James E. Smith. Dynamic Verification of Cache Coherence Protocols, pages 25-42. Springer New York, New York, NY, 2004. URL: https://doi.org/10.1007/978-1-4419-8987-1_3.
  7. Thomas Carle and Hugues Cassé. Reducing Timing Interferences in Real-Time Applications Running on Multicore Architectures. In 18th International Workshop on Worst-Case Execution Time Analysis (WCET 2018), volume 63 of OpenAccess Series in Informatics (OASIcs), pages 3:1-3:12, Dagstuhl, Germany, 2018. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik. URL: https://doi.org/10.4230/OASIcs.WCET.2018.3.
  8. Franck Cassez and Jean-Luc Béchennec. Timing analysis of binary programs with UPPAAL. In 13th International Conference on Application of Concurrency to System Design, ACSD 2013, pages 41-50. IEEE Computer Society, July 2013. URL: https://doi.org/10.1109/ACSD.2013.7.
  9. Certification Authorities Software Team. Multi-core Processors - Position Paper. Technical Report CAST 32-A, Federal Aviation Administration, November 2016. Google Scholar
  10. A. DeOrio, A. Bauserman, and V. Bertacco. Post-silicon verification for cache coherence. In 2008 IEEE International Conference on Computer Design, pages 348-355, October 2008. URL: https://doi.org/10.1109/ICCD.2008.4751884.
  11. M. Elver and V. Nagarajan. Mcversi: A test generation framework for fast memory consistency verification in simulation. In 2016 IEEE International Symposium on High Performance Computer Architecture (HPCA), pages 618-630, March 2016. URL: https://doi.org/10.1109/HPCA.2016.7446099.
  12. Hakan Forsberg and Andreas Schwierz. Emerging cots-based computing platforms in avionics need a new assurance concept. In the 38th Digital Avionics Systems Conference (DASC'19). IEEE Press, 2019. Google Scholar
  13. Freescale. e6500 core reference manual, rev 0, 2014. Google Scholar
  14. Freescale. T4240 QorIQ: Integrated multicore communications processor family reference manual, 2014. Google Scholar
  15. Michele Garetto, Emilio Leonardi, and Valentina Martina. A unified approach to the performance analysis of caching systems. ACM Trans. Model. Perform. Eval. Comput. Syst., 1(3), May 2016. URL: https://doi.org/10.1145/2896380.
  16. Sylvain Girbal, Jimmy le Rhun, and Hadi Saoud. METrICS: a measurement environment for multi-core time critical systems. In 9th European Congress on Embedded Real Time Software and Systems (ERTS'18), 2018. Google Scholar
  17. James Goodman and Hhj Hum. Mesif: A two-hop cache coherency protocol for point-to-point interconnects (2004), 2004. Google Scholar
  18. Daniel Grund and Jan Reineke. Toward Precise PLRU Cache Analysis. In 10th International Workshop on Worst-Case Execution Time Analysis (WCET 2010), volume 15 of OpenAccess Series in Informatics (OASIcs), pages 23-35, Dagstuhl, Germany, 2010. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik. The printed version of the WCET'10 proceedings are published by OCG (www.ocg.at) - ISBN 978-3-85403-268-7. URL: https://doi.org/10.4230/OASIcs.WCET.2010.23.
  19. Andreas Gustavsson, Andreas Ermedahl, Björn Lisper, and Paul Pettersson. Towards WCET analysis of multicore architectures using UPPAAL. In 10th International Workshop on Worst-Case Execution Time Analysis, WCET 2010, July 6, 2010, Brussels, Belgium, pages 101-112, 2010. URL: https://doi.org/10.4230/OASIcs.WCET.2010.101.
  20. M. Hassan and H. Patel. Mcxplore: Automating the validation process of dram memory controller designs. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 37(5):1050-1063, May 2018. URL: https://doi.org/10.1109/TCAD.2017.2705123.
  21. Mohamed Hassan, Anirudh M. Kaushik, and Hiren D. Patel. Predictable cache coherence for multi-core real-time systems. In 2017 IEEE Real-Time and Embedded Technology and Applications Symposium, RTAS 2017, Pittsburg, PA, USA, April 18-21, 2017, pages 235-246, 2017. URL: https://doi.org/10.1109/RTAS.2017.13.
  22. B. Kumar, A. K. Bhosale, M. Fujita, and V. Singh. Validating multi-processor cache coherence mechanisms under diminished observability. In 2019 IEEE 28th Asian Test Symposium (ATS), pages 99-995, 2019. Google Scholar
  23. Benjamin Lesage, David Griffin, Sebastian Altmeyer, Liliana Cucu-Grosjean, and Robert I. Davis. On the analysis of random replacement caches using static probabilistic timing methods for multi-path programs. Real-Time Syst., 54(2):307–388, April 2018. URL: https://doi.org/10.1007/s11241-017-9295-2.
  24. Mingsong Lv, Nan Guan, Jan Reineke, Reinhard Wilhelm, and Wang Yi. A survey on static cache analysis for real-time systems. Leibniz Transactions on Embedded Systems, 3(1):05-1-05:48, 2016. URL: https://doi.org/10.4230/LITES-v003-i001-a005.
  25. Claire Maiza, Hamza Rihani, Juan M. Rivas, Joël Goossens, Sebastian Altmeyer, and Robert I. Davis. A survey of timing verification techniques for multi-core real-time systems. ACM Comput. Surv., 52(3), June 2019. URL: https://doi.org/10.1145/3323212.
  26. Laurence Mutuel, Xavier Jean, Vincent Brindejonc, Anthony Roger, Thomas Megel, and E. Alepins. Assurance of Multicore Processors in Airborne Systems, 2017. Google Scholar
  27. Viet Anh Nguyen, Damien Hardy, and Isabelle Puaut. Cache-conscious Off-Line Real-Time Scheduling for Multi-Core Platforms: Algorithms and Implementation. Real-Time Systems, pages 1-37, 2019. URL: https://doi.org/10.4230/LIPIcs.ECRTS.2017.14.
  28. Jan Nowotsch and Michael Paulitsch. Leveraging multi-core computing architectures in avionics. In Proceedings of the 2012 Ninth European Dependable Computing Conference, EDCC '12, pages 132-143, Washington, DC, USA, 2012. IEEE Computer Society. Google Scholar
  29. Mark S. Papamarcos and Janak H. Patel. A low-overhead coherence solution for multiprocessors with private cache memories. SIGARCH Comput. Archit. News, 12(3):348-354, January 1984. URL: https://doi.org/10.1145/773453.808204.
  30. Nathanaël Sensfelder, Julien Brunel, and Claire Pagetti. Modeling Cache Coherence to Expose Interference. In 31st Euromicro Conference on Real-Time Systems (ECRTS 2019), volume 133 of Leibniz International Proceedings in Informatics (LIPIcs), pages 18:1-18:22, Dagstuhl, Germany, 2019. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik. URL: https://doi.org/10.4230/LIPIcs.ECRTS.2019.18.
  31. Daniel J. Sorin, Mark D. Hill, and David A. Wood. A Primer on Memory Consistency and Cache Coherence. Morgan & Claypool Publishers, 1st edition, 2011. Google Scholar
  32. Valentin Touzeau, Claire Maïza, David Monniaux, and Jan Reineke. Fast and exact analysis for lru caches. Proc. ACM Program. Lang., 3(POPL), January 2019. URL: https://doi.org/10.1145/3290367.
  33. Hui Wang, Sandeep Baldawa, and Rama Sangireddy. Dynamic error detection for dependable cache coherency in multicore architectures. 21st International Conference on VLSI Design (VLSID 2008), pages 279-285, 2008. Google Scholar
  34. Reinhard Wilhelm and Jan Reineke. Embedded systems: Many cores - many problems. In 7th IEEE International Symposium on Industrial Embedded Systems (SIES'12), pages 176-180, 2012. Google Scholar