Intelligent Security: Is “AI for Cybersecurity” a Blessing or a Curse

This talk covers two main topics relevant to how neural networks (NNs) have become a powerful tool to assess the security of cryptographic primitives and how NNs themselves have been targeted to extract their assets. The first part of the talk is devoted to NN-enabled side-channel analysis (SCA), in particular, profiled SCA that leverages leakage from cryptographic implementations to extract the secret key. It is known that when combined with advanced methods in NNs, profiled SCA can successfully attack even crypto-cores with protection devised to impair the effectiveness of SCA. Similar to other machine learning tasks, a range of questions have remained unanswered about NN-enabled SCA, namely: how to choose an NN with an adequate configuration, how to tune the NN’s hyperparameters, when to stop the training, etc. This talk introduces “InfoNEAT,” which tackles these issues in a natural way. InfoNEAT relies on the concept of neural structure search (NAS), enhanced by information-theoretic metrics to guide the evolution, halt it with novel stopping criteria, and improve time-complexity and memory footprint. Besides the considerable advantages regarding the automated configuration of NNs, InfoNEAT demonstrates significant improvements over other approaches for effective key recovery in terms of the number of epochs and the number of attack traces compared to both MLPs and CNNs, as well as a reduction in the number of trainable parameters compared to MLPs. Furthermore, through experiments, it is demonstrated that InfoNEAT’s models are robust against noise and desynchronization in traces.

In the second part of the talk, SCA against NNs has been taken into account. In fact, recent work has highlighted the risks of intellectual property (IP) piracy of deep learning (DL) models from the side-channel leakage of DL hardware accelerators. In response, fundamental cryptographic approaches, specifically built upon the notion of multi-party computation, could potentially improve the robustness against side-channel leakage. To examine this and weigh the costs and benefits, we introduce hardware garbled NN (HWGN²), a DL hardware accelerator implemented on FPGA. HWGN² also provides NN designers with the flexibility to protect their IP in real-time applications, where hardware resources are heavily constrained, through a hardware-communication cost trade-off. Concretely, we apply garbled circuits, implemented using a MIPS architecture that achieves up to $62.5\times$ fewer logical and $66\times$ less memory utilization than the state-of-the-art approaches at the price of communication overhead. Further, the side-channel resiliency of HWGN² is demonstrated by employing the test vector leakage assessment (TVLA) test against both power and electromagnetic side channels.

Designing a secure system requires a lot of expertise in the security domain. In that process, some of the tasks can be automated with the help of Artificial Intelligence (AI). The use of AI methods does not aim to replace the human designer; rather, they can help in the design optimization process, where standardized algorithms can be readily applied to increase the efficiency. As long as a complex system design task can be decomposed into simpler elements, AI methods can substantially facilitate the optimization of individual components. Furthermore, most methods can be used to optimize an arbitrary (set of) design criteria.

However, although there are problems that can be efficiently solved with AI techniques, it is not always obvious which AI technique or optimization algorithm should be applied. In practice, a bit of knowledge in both domains is needed to select the appropriate method and to efficiently apply it to the problem at hand. Even then, for many AI methods there are no formal guarantees of efficiency, which is especially evident for obscure machine learning models such as deep neural networks.

Ideally, the AI component should provide explainability, so the decision making process can be justified at each step. We may even employ less efficient but explainable models to evaluate obscure models which bring performance. There are use cases in which a part of a black-box model may be replaced with an equivalent white-box component offering the same level of performance. Additionally, different optimization algorithms may be used to prune “fat” models, either to provide insight into their functionality or to reduce application complexity. In this regard, neuroevolution methods may be used to design and optimize the structure and hyperparameters of deep neural models.

The application of the above techniques can be found in model building efforts in various domains; the usual goals are knowledge representation, model parameter optimization, feature extraction and selection, etc. Some of the efficient examples of this paradigm are already evident in cryptology and security where different AI techniques, most notably evolutionary algorithms, have been applied. Here, the focus was mainly on the design of different cryptography primitives, such as Boolean functions, S-boxes and pseudo-random number generators. Successful applications also include fault injection, intrusion detection, hyper-parameter optimization etc. Recently, evolutionary algorithm methods have also been applied to fuzzing, where they obtained competitive performance in a target-based comparison with commonly used solutions.

In this presentation, we provided the discussion on two main direction on the area of hardware security and deep learning (DL). First, we discussed about the use of feature extraction or pre-processing techniques, which could help improving the performance of DL based side-channel attacks (SCA). In most of the research works done, the main goal is towards the direction of designing an efficient network that can provide the best attacks against each side-channel trace dataset. On the other hand, little work has been done to investigate the possibility of strengthening DL architecture with the capability of integrating existing side-channel pre-processing or filtering techniques, which have been thoroughly investigated over the past decades. As such, one of the aim is to minimize the necessity for architecture adjustments while enabling seamlessly integration of pre-processing method for attack. In our work, we propose to incorporate feature extraction and classification in a single framework by using a multi-branch model. The experimental results indicated that the model can perform better than the benchmark model even though it is not specifically tailored for the dataset. These show that it is an inherent property of MCNN which allows it to learn more feature representations and result in better attacks. As for the potential future direction, we discussed the possibility of using other DL based approach as a way to further automate the feature pre-processing method.

Next, we discussed about the vulnerability of DL implementation on physical device against side-channel and fault attacks. Due to the rapid growth of DL application, more and more efforts are being allocated to build and train critical DL models. These DL models have then become valuable Intellectual Properties (IPs) that cost companies lots of time and resources, which inadvertently attract malicious parties to steal them. We presented the work on model extraction and reverse engineering of the neural networks model through electromagnetic (EM) side-channel leakage. We also presented alternative work for reverse engineering of neural network models through cold boot attacks. The work is then conducted targeting edge AI hardware accelerators, Intel Neural Compute Stick 2 (NCS2). It is based on the observation that the model architecture and parameters have to be loaded to Intel NCS2 before the inference, and thus, by performing cold boot attack on host device, it is possible to recover the information, albeit with correction required. As for potential future direction, we proposed to investigate different target devices or more complex architectures. We also discussed on possible countermeasures for the implementation as well as the security evaluation of these countermeasures.

Fault injection types such as laser FI, electromagnetic FI, or voltage glitching have different parameters to define. Nevertheless, the parameter search space becomes large for all types because of many parameters and possibilities. Since the search space is large, commonly used methods like grid and random search lead to suboptimal performance/results. We use AI techniques discussed in this talk to improve the efficiency of the search. Specifically, genetic and memetic algorithms from evolutionary computation were shown to find more parameter combinations that lead to erroneous outputs compared to random search [1]. Additionally, hyperparameter tuning methods like successive halving and reinforcement learning from the machine learning domain were also shown to be quite successful [2, 3]. On the other hand, machine learning can be helpful for transferability issues in fault injection. As discussed during the talk, we can use prior knowledge from tested devices and parameter combinations generalized with decision trees to find more vulnerabilities on a new target or bench in the same amount of tested parameters.

Despite the topic of AI (cyber)security has received lots of academic and industrial attention in recent years, these communities have started to realize the need for a holistic approach related to this topic. We do not mean only from a system perspective, where the different implementation layers (e.g., cloud) also contribute to the security (and even to the attack surface) of the AI-application, but also from equally important features like privacy, transparency/explainability, bias, and safety (to name just a few). Extrapolating relevant security research to this much needed holistic approach is critical for the uptake of trusted AI system. This talk discusses some relevant industrial and regulation-related aspects on the field of AI trustworthiness, along with few identified challenges which are being tackled from an EU perspective. One of the main points relates to the need of developing a framework for supporting the assessment of AI systems for cybersecurity certification purposes. The referred framework should be able to leverage realistic levels of automation which can pave the road for continuous (automated) certification. It is expected that such a framework might provide support for accelerating the uptake of relevant standards and regulations like the EU AI Act.

Abstract. In this talk, we went through different use cases of side- channel analysis for different security purposes. The first use case was the analysis of micro-architectural leakage, in order to address the gap between leakage and unknown micro-architectures. The second use case was the usage of electromagnetic leakage to classify and/or detect malware and rootkits[1, 2]. Then the talk quickly tackled some problems of securing PQ-cryptography from side-channel point of view. From a more general point of view, side-channel analysis could be viewed as a subpart of data analysis for security. How to extract or quantify sensitive information present in huge amounts of noise data, this where IA (or machine learning) can solve existing issues.

In this talk, we give a general overview of AI methods and computational models to design cryptographic primitives. These include the use of bio-inspired optimization techniques (particularly evolutionary algorithms) to construct symmetric primitives with good cryptographic properties, like Boolean functions and S-boxes. The approach leverages also on the use of AI computational models like Cellular Automata (CA) as an efficient representation technique for such primitives. In the second part of the talk, new directions of research are illustrated based on the experience gained with regard to the above AI methods and models. In particular, we focus on the use of evolutionary algorithms to design algebraic constructions of symmetric primitives, to evolve differential distinguishers for small symmetric ciphers, and to explore the space of adversarial examples in machine learning models. Particular emphasis is given to the inherent interpretability and explainability of the solutions provided by evolutionary algorithms, specifically in the case of Genetic Programming (GP).

The evolution of our digital society relies on networks that can handle an increasing amount of data, exchanged by an increasing number of connected devices at an increasing communication speed. With the growth of the online world, criminal activities also extend onto the Internet. Network Intrusion Detection Systems (NIDSs) detect malicious activities by analyzing network data. While neural network-based solutions can effectively detect various attacks in an offline setting, it is not straightforward to deploy them in high-bandwidth online systems. This talk elucidates why Field-Programmable Gate Arrays (FPGAs) are the preferred platforms for online network intrusion detection, and which challenges need to be overcome to develop FPGA-based NIDSs for Terabit Ethernet networks.

Fuzzing – testing software and hardware with randomly generated inputs – has gained significant traction due to its success in exposing program vulnerabilities automatically. Machine learning has increasingly been applied to different parts of the fuzzing loop, with the goal of improving fuzzing efficiency. In this talk, we examine neural program smoothing for fuzzing, a family of methods that approximate the tested program with a neural network for novel test case generation. We uncover fundamental and practical limitations of neural program smoothing, which prevent it from reaching its advertised performance and limit its practical interest.

Deep learning-based side-channel analysis is an extremely powerful option as it can work without feature engineering and defeats various hiding and masking countermeasures. Still, from the evaluator’s perspective, even after a successful evaluation (attack), a crucial detail is missing: how did the neural network break the target? Thus, the explainability of deep learning-based side-channel analysis becomes an important issue. Unfortunately, up to now, there are only sporadic attempts to understand how neural network defeats countermeasures and none that gives the complete answer. Some early explored techniques include SVCCA [1] and ablation [2]. While good first steps, these techniques do not provide enough information to understand how countermeasures are circumvented. This talk concentrated on a recent approach to explaining the deep learning-based side-channel attack: layer-wise explainability and its comparative advantages over previous approaches.

Cybersecurity implementations, in hardware or software, are created from engineering models, not from scientific models. Scientific models reflect the laws of nature in formulae, while engineering models aim at the opposite: we use the laws of nature to mimic an abstraction.

The observations of secure implementations in the real world are noisy distortions from the ideal, noiseless engineering models. However, we know that the ground truth corresponds to the engineering model, which is noiseless and undistorted.

This has an important consequence on machine learning applications. We can use simulation (of engineering models) to create a ground truth to improve inference on measured, distorted implementation. For example, using simulated data, we can build attacks on real-world systems that outperform real-world measurements [1].

Field-programmable gate arrays (FPGAs) have made their way into the cloud, allowing users to gain remote access to the state-of-the-art reconfigurable fabric and implement their custom accelerators. As FPGAs are large enough to accommodate multiple independent designs, the multi-tenant user scenario may soon be prevalent in cloud computing environments. However, shared FPGAs are vulnerable to remote power-side channel and fault-injection attacks [1, 3, 4]. Machine learning (ML) further broadens the attack space: (1) ML accelerators may be the targets of remote attacks, (2) ML techniques can be used to infer the type of workloads or the computations the FPGA is running [2], and (3) ML can help detecting malicious circuits in FPGA bitstreams. This talk has two parts: In the first, the techniques enabling remote electrical-level attacks on cloud FPGAs are explained. In the second, the opportunities for using ML for detecting and locating malicious activity, or for guiding the cloud hypervisors in managing the FPGA users in a security-aware manner are discussed.

To mount physical attacks adversaries might need to place probes in the proximity of the integrated circuits (ICs) package, create physical connections between their probes/wires and the system’s PCB, or physically tamper with the PCB’s components, chip’s package, or substitute the entire PCB to prepare the device for the attack. In this talk, inspired by methods known from the field of power integrity analysis, we show how the impedance characterization of the system’s power distribution network (PDN) using an on-chip circuit-based network analyzer can detect various categories of tamper events. By analyzing the frequency response of the system different classes of tamper events from board to chip level are revealed. Using the Wasserstein Distance as a metric, we demonstrate that we can confidently detect tamper events. We demonstrate that even environment-level tampering activities, e.g., proximity of contactless EM probes to the IC package or slightly polished IC package, can be detected using on-chip impedance sensing.

Deep neural networks (DNN) have become a significant threat to the security of cryptographic implementations with regards to side-channel analysis (SCA), as they automatically combine the leakages without any preprocessing needed, leading to a more efficient attack. However, these DNNs for SCA remain mostly black-box algorithms that are very difficult to interpret. Benamira et al. recently proposed an interpretable neural network called Truth Table Deep Convolutional Neural Network (TT-DCNN), which is both expressive and easier to interpret. In particular, a TT-DCNN has a transparent inner structure that can entirely be transformed into SAT equations after training. This talk gives a brief outline of why we need explainability, and on what TT-DCNN is. The talk also presented a way to analyse the SAT equations of TT-DCNN and show some results. Furthermore, we give a possible direction to analyse this paper.

Implementation attacks aim at the weaknesses of the implementation and not the algorithm. The most common options for implementation attacks are side-channel attacks and fault injection attacks. In both domains, AI is used extensively. In side-channel attacks, it is common to use machine learning in the profiling attack scenario. There, the attacker has a copy of the device to be attacked under control and uses it to build a model of a device. Later, the model is used to attack the target and obtain secret information. Machine learning attacks in such a setup have been used for over a decade and show excellent attack performance. More recent results with deep learning provide even better attack performance against targets protected with countermeasures and with no need to conduct feature engineering [3]. Still, multiple open issues need to be resolved. For instance, the attacks assume that the attacker has access to a copy of a device to be attacked, which is often not a realistic assumption. As such, one of the big challenges to be solved is how to mount non-profiling deep learning-based attacks [2]. Next, leakage assessment is important as it provides the first information on whether the target has secure implementation or if there is some leakage. The results with deep learning are promising but sparse [4]. Mounting an attack once the device is produced is a common setup but results in large expenses for manufacturers once security vulnerabilities are detected. As such, it is important to understand whether we can use various simulation-based approaches and techniques to construct synthetic measurements to assess the security of devices even before they are produced [5, 8]. Finally, as previously discussed, the explainability perspective is important for side-channel attacks. While most of the AI-based approaches for side-channel analysis use machine (deep) learning, there are also some efforts in feature engineering or hyperparameter tuning [1, 7]. More open challenges discussed during the workshop can be found in [6].

On the other hand, in fault injection, AI is mostly used to allow fast characterization of the target (cartography). In that context, various evolutionary and local search algorithms are used [11, 9]. More recently, deep learning is also used to predict if a point on a target will result in a faulty response [10]. We identified the research gaps in making the approaches more stable and maintaining the balance between exploring various regions of the target and fast convergence to a region with many faulty responses. Finally, research rarely explores how to use the located faults in mounting the attacks (which could help understand if all located faults are equally important).

Finally, implementation attacks can be used to attack machine learning, connecting this topic with the security of AI [12, 13].

Vulnerabilities caused by programming errors are a major threat to today’s programs. For instance, memory corruption vulnerabilities can lead to uncontrolled behavior in the program, which attackers can often abuse. A modern strategy to uncover such programming errors is automated software testing using fuzz testing (fuzzing). Fuzzing automatically generates inputs from testcases and feeds them to the program under test while monitoring it. If a programming error has been reached, the fuzzer notices that the program hangs or crashes. Mutational fuzzing requires a set of program inputs (seeds) that can be obtained from testcases or real inputs. The process of mutation can be influenced by 1) the location in the input that gets mutated and 2) the mutation that is applied, with the selection done randomly or guided by a heuristic. A common option is to use evolutionary algorithms for such goals [1]. While the approach works well, there are issues. Due to a wide number of available evolutionary algorithms, selecting what algorithm to use and how to customize it for the task is not trivial. Moreover, since evolutionary algorithms are guided through an objective function, appropriate evaluations should be done. Machine learning is also used in fuzzing for various tasks like seed file generation, testcase generation, or mutation operator selection [2]. It is important to understand whether evolutionary algorithms or machine learning produce better results for tasks that can be achieved by both (e.g., mutation operator selection) and in what scenarios to select a specific AI technique. For instance, finding the states in stateful fuzzing is not easy, and machine learning could be used for this task.

Machine (deep) learning found its place in various real-world applications, where many applications have security requirements. Unfortunately, as these systems become more pervasive, understanding how they fail becomes more challenging. There are several failure modes in machine learning, but one category received significant attention in the last few years: backdoor attacks. Backdoor attacks aim to make a model misclassify some of its inputs to a preset-specific label while other classification results behave normally. This misclassification is activated when a specific property is included in the model input. This property is called the trigger and can be anything the targeted model understands. Deep learning is evaluated in either a centralized or distributed setting. While the centralized one is simpler, it poses privacy concerns due to the need to have the training data available (and, for instance, shared in the case of online training). Then, a common option is to use federated learning as a distributed learning paradigm that works on isolated data. In federated learning, clients can collaboratively train a shared global model under the orchestration of a central server while keeping the data decentralized. Multiple backdoor attacks and defenses exist on machine learning systems (centralized and distributed) and for diverse data types: computer vision (e.g., images, video), sound, text, and graph data. While many observations can be transferred from one setup to another, unique characteristics also require detailed experimentalism [1, 2]. We need more systematic evaluations of diverse attack factors in different domains and with larger (more realistic) datasets and neural network models. Finally, more effort must be given to designing powerful, transferable, and efficient defenses [4, 3].