Document Open Access Logo

Transformer-Based Signal Inference for Electrified Vehicle Powertrains (Short Paper)

Authors Stan Muñoz Gutiérrez , Adil Mukhtar , Franz Wotawa

PDF
Thumbnail PDF

File

OASIcs.DX.2024.29.pdf
  • Filesize: 1.85 MB
  • 14 pages

Document Identifiers

Author Details

Stan Muñoz Gutiérrez
  • Institute of Software Technology, TU Graz, Austria
Adil Mukhtar
  • Institute of Software Technology, TU Graz, Austria
Franz Wotawa
  • Institute of Software Technology, TU Graz, Austria

Funding

This study was conducted within the framework of the project ARCHIMEDES, receiving funding from the Key Digital Technologies Joint Undertaking (KDT JU) - the Public-Private Partnership for research, development, and innovation under Horizon Europe – and National Authorities under Grant Agreement No 101112295 funded by the European Union and the FFG under Grant Agreement No FO999899377.

Cite As Get BibTex

Stan Muñoz Gutiérrez, Adil Mukhtar, and Franz Wotawa. Transformer-Based Signal Inference for Electrified Vehicle Powertrains (Short Paper). In 35th International Conference on Principles of Diagnosis and Resilient Systems (DX 2024). Open Access Series in Informatics (OASIcs), Volume 125, pp. 29:1-29:14, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2024) https://doi.org/10.4230/OASIcs.DX.2024.29

Abstract

The scarcity of labeled data for intelligent diagnosis of non-linear technical systems is a common problem for developing robust and reliable real-world applications. Several deep learning approaches have been developed to address this challenge, including self-supervised learning, representation learning, and transfer learning. Due largely to their powerful attention mechanisms, transformers excel at capturing long-term dependencies across multichannel and multi-modal signals in sequential data, making them suitable candidates for time series modeling. Despite their potential, studies applying transformers for diagnostic functions, especially in signal reconstruction through representation learning, remain limited. This paper aims to narrow this gap by identifying the requirements and potential of transformer self-attention mechanisms for developing auto-associative inference engines that learn exclusively from healthy behavioral data. We apply a transformer backbone for signal reconstruction using simulated data from a simplified powertrain. Feedback from these experiments, and the reviewed evidence from the literature, allows us to conclude that autoencoder and autoregressive approaches are potentiated by transformers.

Subject Classification

ACM Subject Classification
  • Hardware → Fault models and test metrics
  • Computing methodologies → Machine learning algorithms
  • Hardware → Failure recovery, maintenance and self-repair
Keywords
  • Signal Inference
  • Deep Learning
  • Self-Supervised Learning
  • Multimodal Transformer Autoencoder
  • Electric Vehicle
  • Powertrain
  • Electric Motor

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Supplementary Materials

References

  1. Sheena Angra and Sachin Ahuja. Machine learning and its applications: A review. In 2017 international conference on big data analytics and computational intelligence (ICBDAC), pages 57-60. IEEE, 2017. Google Scholar
  2. ARCHIMEDES Consortium. ARCHIMEDES ensures lifetime with digital means. https://www.archimedesproject.eu/, 2024. Accessed on September 9, 2024.
  3. Ritik Argawal, Dattatraya Kalel, M Harshit, Arun D Domnic, and R Raja Singh. Sensor Fault Detection using Machine Learning Technique for Automobile Drive Applications. In 2021 National Power Electronics Conference (NPEC), pages 1-6, 2021. URL: https://doi.org/10.1109/NPEC52100.2021.9672546.
  4. Roman Bachmann, Oğuzhan Fatih Kar, David Mizrahi, Ali Garjani, Mingfei Gao, David Griffiths, Jiaming Hu, Afshin Dehghan, and Amir Zamir. 4M-21: An Any-to-Any Vision Model for Tens of Tasks and Modalities, 2024. URL: https://doi.org/10.48550/arXiv.2406.09406.
  5. Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. Beit: Bert pre-training of image transformers, 2022. URL: https://doi.org/10.48550/arXiv.2106.08254.
  6. Yong Chen, Siyuan Liang, Wanfu Li, Hong Liang, and Chengdong Wang. Faults and Diagnosis Methods of Permanent Magnet Synchronous Motors: A Review. Applied Sciences, 9(10):2116, May 2019. URL: https://doi.org/10.3390/app9102116.
  7. Zhuyun Chen and Weihua Li. Multisensor Feature Fusion for Bearing Fault Diagnosis Using Sparse Autoencoder and Deep Belief Network. IEEE Transactions on Instrumentation and Measurement, 66(7):1693-1702, 2017. URL: https://doi.org/10.1109/TIM.2017.2669947.
  8. Anurag Choudhary, Tauheed Mian, Shahab Fatima, and B K Panigrahi. Deep Transfer Learning Based Fault Diagnosis of Electric Vehicle Motor. In 2022 IEEE International Conference on Power Electronics, Drives and Energy Systems (PEDES), pages 1-6. IEEE, December 2022. URL: https://doi.org/10.1109/PEDES56012.2022.10080274.
  9. Yao Da, Xiaodong Shi, and Mahesh Krishnamurthy. A New Approach to Fault Diagnostics for Permanent Magnet Synchronous Machines Using Electromagnetic Signature Analysis. IEEE Transactions on Power Electronics, 28(8):4104-4112, 2013. URL: https://doi.org/10.1109/TPEL.2012.2227808.
  10. Yifei Ding, Minping Jia, Qiuhua Miao, and Yudong Cao. A novel time–frequency Transformer based on self–attention mechanism and its application in fault diagnosis of rolling bearings. Mechanical Systems and Signal Processing, 168:108616, 2022. URL: https://doi.org/10.1016/j.ymssp.2021.108616.
  11. Robert X Gao and Ruqiang Yan. Non-stationary signal processing for bearing health monitoring. International journal of manufacturing research, 1(1):18-40, 2006. URL: https://doi.org/10.1504/IJMR.2006.010701.
  12. Adam Glowacz. Acoustic based fault diagnosis of three-phase induction motor. Applied Acoustics, 137:82-89, 2018. URL: https://doi.org/10.1016/j.apacoust.2018.03.010.
  13. Geoffrey E Hinton and Richard Zemel. Autoencoders, minimum description length and helmholtz free energy. Advances in neural information processing systems, 6, 1993. Google Scholar
  14. Saidul Islam, Hanae Elmekki, Ahmed Elsebai, Jamal Bentahar, Nagat Drawel, Gaith Rjoub, and Witold Pedrycz. A comprehensive survey on applications of transformers for deep learning tasks. Expert Systems with Applications, 241:122666, 2024. URL: https://doi.org/10.1016/j.eswa.2023.122666.
  15. Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013. Google Scholar
  16. Sotiris B Kotsiantis, Ioannis Zaharakis, P Pintelas, et al. Supervised machine learning: A review of classification techniques. Emerging artificial intelligence applications in computer engineering, 160(1):3-24, 2007. URL: http://www.booksonline.iospress.nl/Content/View.aspx?piid=6950.
  17. Yogesh Kumar, Komalpreet Kaur, and Gurpreet Singh. Machine learning aspects and its applications towards different research areas. In 2020 International conference on computation, automation and knowledge management (ICCAKM), pages 150-156. IEEE, 2020. Google Scholar
  18. Xiang Li, Wei Zhang, and Qian Ding. Understanding and improving deep learning-based rolling bearing fault diagnosis with attention mechanism. Signal Processing, 161:136-154, 2019. URL: https://doi.org/10.1016/j.sigpro.2019.03.019.
  19. Siyuan Liang, Yong Chen, Hong Liang, and Xu Li. Sparse Representation and SVM Diagnosis Method Inter-Turn Short-Circuit Fault in PMSM. Applied Sciences, 9(2):224, January 2019. URL: https://doi.org/10.3390/app9020224.
  20. Shih-Lin Lin. Application Combining VMD and ResNet101 in Intelligent Diagnosis of Motor Faults. Sensors (Basel, Switzerland), 21(18):6065, 2021. URL: https://doi.org/10.3390/S21186065.
  21. Meng Ma, Chuang Sun, and Xuefeng Chen. Deep Coupling Autoencoder for Fault Diagnosis With Multimodal Sensory Data. IEEE Transactions on Industrial Informatics, 14(3):1137-1145, 2018. URL: https://doi.org/10.1109/TII.2018.2793246.
  22. S.S. Moosavi, A. Djerdir, Y. Ait-Amirat, and D.A. Khaburi. ANN based fault diagnosis of permanent magnet synchronous motor under stator winding shorted turn. Electric Power Systems Research, 125:67-82, 2015. URL: https://doi.org/10.1016/j.epsr.2015.03.024.
  23. Vladimir Nasteski. An overview of the supervised machine learning methods. Horizons. b, 4(51-62):56, 2017. Google Scholar
  24. Yaw Nyanteh, Chris Edrington, Sanjeev Srivastava, and David Cartes. Application of artificial intelligence to real-time fault detection in permanent-magnet synchronous machines. IEEE Transactions on Industry Applications, 49(3):1205-1214, 2013. URL: https://doi.org/10.1109/TIA.2013.2253081.
  25. Yaw D. Nyanteh, Sanjeev K. Srivastava, Chris S. Edrington, and David A. Cartes. Application of artificial intelligence to stator winding fault diagnosis in permanent magnet synchronous machines. Electric Power Systems Research, 103:201-213, 2013. URL: https://doi.org/10.1016/j.epsr.2013.05.018.
  26. YCAP Reddy, P Viswanath, and B Eswara Reddy. Semi-supervised learning: A brief review. Int. J. Eng. Technol, 7(1.8):81, 2018. Google Scholar
  27. Salah Rifai, Pascal Vincent, Xavier Muller, Xavier Glorot, and Yoshua Bengio. Contractive auto-encoders: explicit invariance during feature extraction. In Proceedings of the 28th International Conference on International Conference on Machine Learning, ICML'11, pages 833-840, Madison, WI, USA, 2011. Omnipress. URL: https://icml.cc/2011/papers/455_icmlpaper.pdf.
  28. Tunan Shen, Yuping Chen, Christian Thulfaut, and Hans-Christian Reuss. A Data Based Diagnostic Method for Current Sensor Fault in Permanent Magnet Synchronous Motors (PMSM). In IECON Proceedings (Industrial Electronics Conference), volume 2019-Octob, pages 5979-5985, 2019. URL: https://doi.org/10.1109/IECON.2019.8927667.
  29. Ah-Hyung Shin, Seong Tae Kim, and Gyeong-Moon Park. Time Series Anomaly Detection Using Transformer-Based GAN With Two-Step Masking. IEEE Access, 11:74035-74047, 2023. URL: https://doi.org/10.1109/ACCESS.2023.3289921.
  30. Zia Ullah and Jin Hur. A Comprehensive Review of Winding Short Circuit Fault and Irreversible Demagnetization Fault Detection in PM Type Machines. Energies, 11(12):3309, November 2018. URL: https://doi.org/10.3390/en11123309.
  31. Zia Ullah, Bilal Ahmad Lodhi, and Jin Hur. Detection and Identification of Demagnetization and Bearing Faults in PMSM Using Transfer Learning-Based VGG. Energies (Basel), 13(15):3834, 2020. Google Scholar
  32. Julio-César Urresty, Jordi-Roger Riba, and Luis Romeral. Diagnosis of interturn faults in pmsms operating under nonstationary conditions by applying order tracking filtering. IEEE Transactions on Power Electronics, 28(1):507-515, 2013. URL: https://doi.org/10.1109/TPEL.2012.2198077.
  33. Julio-César Urresty, Jordi-Roger Riba, and Luís Romeral. Application of the zero-sequence voltage component to detect stator winding inter-turn faults in PMSMs. Electric Power Systems Research, 89:38-44, 2012. URL: https://doi.org/10.1016/j.epsr.2012.02.012.
  34. Muhammad Usama, Junaid Qadir, Aunn Raza, Hunain Arif, Kok-Lim Alvin Yau, Yehia Elkhatib, Amir Hussain, and Ala Al-Fuqaha. Unsupervised machine learning for networking: Techniques, applications and research challenges. IEEE access, 7:65579-65615, 2019. URL: https://doi.org/10.1109/ACCESS.2019.2916648.
  35. Jesper E Van Engelen and Holger H Hoos. A survey on semi-supervised learning. Machine learning, 109(2):373-440, 2020. URL: https://doi.org/10.1007/S10994-019-05855-6.
  36. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. https://arxiv.org/abs/1706.03762, URL: https://doi.org/10.48550/arXiv.1706.03762.
  37. Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th International Conference on Machine Learning, ICML '08, pages 1096-1103, New York, NY, USA, 2008. Association for Computing Machinery. URL: https://doi.org/10.1145/1390156.1390294.
  38. Franz Wotawa, David Kaufmann, Adil Mukhtar, Iulia Nica, Florian Klück, Hermann Felbinger, Petr Blaha, Matus Kozovsky, Zdenek Havranek, and Martin Dosedel. Real-Time Predictive Maintenance - Model-Based, Simulation-Based and Machine Learning Based Diagnosis. In Artificial Intelligence for Digitising Industry – Applications. River Publishers, 2011. Google Scholar
  39. Zheng Yang, Binbin Xu, Wei Luo, and Fei Chen. Autoencoder-based representation learning and its application in intelligent fault diagnosis: A review. Measurement, 189:110460, 2022. URL: https://doi.org/10.1016/j.measurement.2021.110460.
  40. Hongkun Zhang and Wenjun Li. A New Method of Sensor Fault Diagnosis Based on a Wavelet Packet Neural Network for Hybrid Electric Vehicles. In 2016 9th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), pages 1143-1147, 2016. URL: https://doi.org/10.1109/CISP-BMEI.2016.7852886.
  41. Jiyu Zhang, Giorgio Rizzoni, and Qadeer Ahmed. Fault Modelling for Hierarchical Fault Diagnosis and Prognosis. In The American Society of Mechanical Engineers, editor, Dynamic Systems and Control Conference, volume 2, October 2013. URL: https://doi.org/10.1115/DSCC2013-3825.
  42. Chunheng Zhao, Yi Li, Matthew Wessner, Chinmay Rathod, and Pierluigi Pisu. Support-Vector Machine Approach for Robust Fault Diagnosis of Electric Vehicle Permanent Magnet Synchronous Motor. In Annual Conference of the PHM Society, volume 12(1), page 10, November 2020. URL: https://doi.org/10.36001/phmconf.2020.v12i1.1291.
  43. Bin Zhou, Shenghua Liu, Bryan Hooi, Xueqi Cheng, and Jing Ye. BeatGAN: Anomalous Rhythm Detection using Adversarially Generated Time Series. In IJCAI, volume 2019, pages 4433-4439, 2019. URL: https://doi.org/10.24963/IJCAI.2019/616.
  44. Ping Zou, Baocun Hou, Lei Jiang, and Zhenji Zhang. Bearing Fault Diagnosis Method Based on EEMD and LSTM. International Journal of Computers, Communications and Control, 15(1), February 2020. URL: https://doi.org/10.15837/IJCCC.2020.1.3780.
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2024 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact