scispace - formally typeset
Search or ask a question

Showing papers on "Trojan published in 2020"


Proceedings ArticleDOI
07 Dec 2020
TL;DR: To the best of the knowledge, Februus is the first backdoor defense method for operation at run-time capable of sanitizing Trojaned inputs without requiring anomaly detection methods, model retraining or costly labeled data.
Abstract: We propose Februus; a new idea to neutralize highly potent and insidious Trojan attacks on Deep Neural Network (DNN) systems at run-time. In Trojan attacks, an adversary activates a backdoor crafted in a deep neural network model using a secret trigger, a Trojan, applied to any input to alter the model’s decision to a target prediction—a target determined by and only known to the attacker. Februus sanitizes the incoming input by surgically removing the potential trigger artifacts and restoring the input for the classification task. Februus enables effective Trojan mitigation by sanitizing inputs with no loss of performance for sanitized inputs, Trojaned or benign. Our extensive evaluations on multiple infected models based on four popular datasets across three contrasting vision applications and trigger types demonstrate the high efficacy of Februus. We dramatically reduced attack success rates from 100% to near 0% for all cases (achieving 0% on multiple cases) and evaluated the generalizability of Februus to defend against complex adaptive attacks; notably, we realized the first defense against the advanced partial Trojan attack. To the best of our knowledge, Februus is the first backdoor defense method for operation at run-time capable of sanitizing Trojaned inputs without requiring anomaly detection methods, model retraining or costly labeled data.

147 citations


Proceedings ArticleDOI
14 Jun 2020
TL;DR: This work proposes a novel Targeted Bit Trojan method, which can insert a targeted neural Trojan into a DNN through bit-flip attack, and demonstrates that flipping only several vulnerable bits identified by the method can transform a fully functional DNN model into a Trojan-infected model.
Abstract: Security of modern Deep Neural Networks (DNNs) is under severe scrutiny as the deployment of these models become widespread in many intelligence-based applications. Most recently, DNNs are attacked through Trojan which can effectively infect the model during the training phase and get activated only through specific input patterns (i.e, trigger) during inference. In this work, for the first time, we propose a novel Targeted Bit Trojan(TBT) method, which can insert a targeted neural Trojan into a DNN through bit-flip attack. Our algorithm efficiently generates a trigger specifically designed to locate certain vulnerable bits of DNN weights stored in main memory (i.e., DRAM). The objective is that once the attacker flips these vulnerable bits, the network still operates with normal inference accuracy with benign input. However, when the attacker activates the trigger by embedding it with any input, the network is forced to classify all inputs to a certain target class. We demonstrate that flipping only several vulnerable bits identified by our method, using available bit-flip techniques (i.e, row-hammer), can transform a fully functional DNN model into a Trojan-infected model. We perform extensive experiments of CIFAR-10, SVHN and ImageNet datasets on both VGG-16 and Resnet-18 architectures. Our proposed TBT could classify 92 of test images to a target class with as little as 84 bit-flips out of 88 million weight bits on Resnet-18 for CIFAR10 dataset.

145 citations


Proceedings ArticleDOI
30 Oct 2020
TL;DR: In this article, a more flexible and stealthy trojan attack that eludes backdoor scanners using trojan triggers composed from existing benign features of multiple labels is introduced, which can achieve accuracy comparable to its original version on benign data and misclassifies when the composite trigger is present in the input.
Abstract: With the prevalent use of Deep Neural Networks (DNNs) in many applications, security of these networks is of importance. Pre-trained DNNs may contain backdoors that are injected through poisoned training. These trojaned models perform well when regular inputs are provided, but misclassify to a target output label when the input is stamped with a unique pattern called trojan trigger. Recently various backdoor detection and mitigation systems for DNN based AI applications have been proposed. However, many of them are limited to trojan attacks that require a specific patch trigger. In this paper, we introduce composite attack, a more flexible and stealthy trojan attack that eludes backdoor scanners using trojan triggers composed from existing benign features of multiple labels. We show that a neural network with a composed backdoor can achieve accuracy comparable to its original version on benign data and misclassifies when the composite trigger is present in the input. Our experiments on 7 different tasks show that this attack poses a severe threat. We evaluate our attack with two state-of-the-art backdoor scanners. The results show none of the injected backdoors can be detected by either scanner. We also study in details why the scanners are not effective. In the end, we discuss the essence of our attack and propose possible defense.

114 citations


Book ChapterDOI
23 Aug 2020
TL;DR: A data-limited TrojanNet detector (TND) is proposed, which can detect a TrojanNet without accessing any data samples, and it is shown that such a TND can be built by leveraging the internal response of hidden neurons, which exhibits the Trojan behavior even at random noise inputs.
Abstract: When the training data are maliciously tampered, the predictions of the acquired deep neural network (DNN) can be manipulated by an adversary known as the Trojan attack (or poisoning backdoor attack). The lack of robustness of DNNs against Trojan attacks could significantly harm real-life machine learning (ML) systems in downstream applications, therefore posing widespread concern to their trustworthiness. In this paper, we study the problem of the Trojan network (TrojanNet) detection in the data-scarce regime, where only the weights of a trained DNN are accessed by the detector. We first propose a data-limited TrojanNet detector (TND), when only a few data samples are available for TrojanNet detection. We show that an effective data-limited TND can be established by exploring connections between Trojan attack and prediction-evasion adversarial attacks including per-sample attack as well as all-sample universal attack. In addition, we propose a data-free TND, which can detect a TrojanNet without accessing any data samples. We show that such a TND can be built by leveraging the internal response of hidden neurons, which exhibits the Trojan behavior even at random noise inputs. The effectiveness of our proposals is evaluated by extensive experiments under different model architectures and datasets including CIFAR-10, GTSRB, and ImageNet.

97 citations


Proceedings ArticleDOI
Ruixiang Tang1, Mengnan Du1, Ninghao Liu1, Fan Yang1, Xia Hu1 
23 Aug 2020
TL;DR: TrojanNet as discussed by the authors proposes a training-free attack approach which is different from previous work, in which trojaned behaviors are injected by retraining model on a poisoned dataset.
Abstract: With the widespread use of deep neural networks (DNNs) in high-stake applications, the security problem of the DNN models has received extensive attention. In this paper, we investigate a specific security problem called trojan attack, which aims to attack deployed DNN systems relying on the hidden trigger patterns inserted by malicious hackers. We propose a training-free attack approach which is different from previous work, in which trojaned behaviors are injected by retraining model on a poisoned dataset. Specifically, we do not change parameters in the original model but insert a tiny trojan module (TrojanNet) into the target model. The infected model with a malicious trojan can misclassify inputs into a target label when the inputs are stamped with the special trigger. The proposed TrojanNet has several nice properties including (1) it activates by tiny trigger patterns and keeps silent for other signals, (2) it is model-agnostic and could be injected into most DNNs, dramatically expanding its attack scenarios, and (3) the training-free mechanism saves massive training efforts comparing to conventional trojan attack methods. The experimental results show that TrojanNet can inject the trojan into all labels simultaneously (all-label trojan attack) and achieves 100% attack success rate without affecting model accuracy on original tasks. Experimental analysis further demonstrates that state-of-the-art trojan detection algorithms fail to detect TrojanNet attack. The code is available at https://github.com/trx14/TrojanNet.

58 citations


Posted Content
Ruixiang Tang1, Mengnan Du1, Ninghao Liu1, Fan Yang1, Xia Hu1 
TL;DR: This paper proposes a training-free attack approach to trojan attack, which aims to attack deployed DNN systems relying on the hidden trigger patterns inserted by malicious hackers, and inserts a tiny trojan module (TrojanNet) into the target model.
Abstract: With the widespread use of deep neural networks (DNNs) in high-stake applications, the security problem of the DNN models has received extensive attention. In this paper, we investigate a specific security problem called trojan attack, which aims to attack deployed DNN systems relying on the hidden trigger patterns inserted by malicious hackers. We propose a training-free attack approach which is different from previous work, in which trojaned behaviors are injected by retraining model on a poisoned dataset. Specifically, we do not change parameters in the original model but insert a tiny trojan module (TrojanNet) into the target model. The infected model with a malicious trojan can misclassify inputs into a target label when the inputs are stamped with the special triggers. The proposed TrojanNet has several nice properties including (1) it activates by tiny trigger patterns and keeps silent for other signals, (2) it is model-agnostic and could be injected into most DNNs, dramatically expanding its attack scenarios, and (3) the training-free mechanism saves massive training efforts comparing to conventional trojan attack methods. The experimental results show that TrojanNet can inject the trojan into all labels simultaneously (all-label trojan attack) and achieves 100% attack success rate without affecting model accuracy on original tasks. Experimental analysis further demonstrates that state-of-the-art trojan detection algorithms fail to detect TrojanNet attack. The code is available at this https URL.

57 citations


Journal ArticleDOI
TL;DR: In this article, the authors present the first data release of photometric analysis of TESS observations of small Solar System Bodies, focusing on the bright end of the observed main-belt asteroid and Jovian Trojan populations.
Abstract: Compared with previous space-borne surveys, the Transiting Exoplanet Survey Satellite (TESS) provides a unique and new approach to observe Solar System objects. While its primary mission avoids the vicinity of the ecliptic plane by approximately six degrees, the scale height of the Solar System debris disk is large enough to place various small body populations in the field-of-view. In this paper we present the first data release of photometric analysis of TESS observations of small Solar System Bodies, focusing on the bright end of the observed main-belt asteroid and Jovian Trojan populations. This data release, named TSSYS-DR1, contains 9912 light curves obtained and extracted in a homogeneous manner, and triples the number of bodies with unambiguous fundamental rotation characteristics, namely where accurate periods and amplitudes are both reported. Our catalogue clearly shows that the number of bodies with long rotation periods are definitely underestimated by all previous ground-based surveys, by at least an order of magnitude.

44 citations


Proceedings ArticleDOI
25 Mar 2020
TL;DR: A survey of neural Trojan attack and defense techniques can be found in this article, where a variety of techniques have been proposed over the last few years, such as detecting neural Trojans, erasing the Trojan's functionality from the neural network model and bypassing the Trojan.
Abstract: Neural networks have become increasingly prevalent in many real-world applications including security critical ones. Due to the high hardware requirement and time consumption to train high-performance neural network models, users often outsource training to a machine-learning-as-a-service (MLaaS) provider. This puts the integrity of the trained model at risk. In 2017, Liu et al. found that, by mixing the training data with a few malicious samples of a certain trigger pattern, hidden functionality can be embedded in the trained network which can be evoked by the trigger pattern [33]. We refer to this kind of hidden malicious functionality as neural Trojans. In this paper, we survey a myriad of neural Trojan attack and defense techniques that have been proposed over the last few years. In a neural Trojan insertion attack, the attacker can be the MLaaS provider itself or a third party capable of adding or tampering with training data. In most research on attacks, the attacker selects the Trojan's functionality and a set of input patterns that will trigger the Trojan. Training data poisoning is the most common way to make the neural network acquire the Trojan functionality. Trojan embedding methods that modify the training algorithm or directly interfere with the neural network's execution at the binary level have also been studied. Defense techniques include detecting neural Trojans in the model and/or Trojan trigger patterns, erasing the Trojan's functionality from the neural network model, and bypassing the Trojan. It was also shown that carefully crafted neural Trojans can be used to mitigate other types of attacks. We systematize the above attack and defense approaches in this paper.

34 citations


Posted Content
TL;DR: This paper surveys a myriad of neural Trojan attack and defense techniques that have been proposed over the last few years and systematizes the above attack anddefense approaches.
Abstract: Neural networks have become increasingly prevalent in many real-world applications including security critical ones. Due to the high hardware requirement and time consumption to train high-performance neural network models, users often outsource training to a machine-learning-as-a-service (MLaaS) provider. This puts the integrity of the trained model at risk. In 2017, Liu et al. found that, by mixing the training data with a few malicious samples of a certain trigger pattern, hidden functionality can be embedded in the trained network which can be evoked by the trigger pattern [33]. We refer to this kind of hidden malicious functionality as neural Trojans. In this paper, we survey a myriad of neural Trojan attack and defense techniques that have been proposed over the last few years. In a neural Trojan insertion attack, the attacker can be the MLaaS provider itself or a third party capable of adding or tampering with training data. In most research on attacks, the attacker selects the Trojan's functionality and a set of input patterns that will trigger the Trojan. Training data poisoning is the most common way to make the neural network acquire the Trojan functionality. Trojan embedding methods that modify the training algorithm or directly interfere with the neural network's execution at the binary level have also been studied. Defense techniques include detecting neural Trojans in the model and/or Trojan trigger patterns, erasing the Trojan's functionality from the neural network model, and bypassing the Trojan. It was also shown that carefully crafted neural Trojans can be used to mitigate other types of attacks. We systematize the above attack and defense approaches in this paper.

33 citations


Proceedings ArticleDOI
01 Nov 2020
TL;DR: TABOR as discussed by the authors proposes a new objective function to guide optimization to identify a trojan backdoor more correctly and accurately and prune the restored triggers, which can not only facilitate the identification of intentionally injected triggers but also filter out false alarms.
Abstract: A trojan backdoor is a hidden pattern typically implanted in a deep neural network (DNN). It could be activated and thus forces that infected model to behave abnormally when an input sample with a particular trigger is fed to that model. As such, given a DNN and clean input samples, it is challenging to inspect and determine the existence of a trojan backdoor. Recently, researchers design and develop several pioneering solutions to address this problem. They demonstrate that the proposed techniques have great potential in trojan detection. However, we show that none of these existing techniques completely address the problem. On the one hand, they mostly work under an unrealistic assumption of assuming the availability of the contaminated training database. On the other hand, these techniques can neither accurately detect the existence of trojan backdoors, nor restore high-fidelity triggers, especially when infected models are trained with high-dimensional data, and the triggers pertaining to the trojan vary in size, shape, and position. In this work, we propose TABOR, a new trojan detection technique. Conceptually, it formalizes the detection of a trojan backdoor as solving an optimization objective function. Different from the existing technique which also models trojan detection as an optimization problem, TABOR first designs a new objective function that could guide optimization to identify a trojan backdoor more correctly and accurately. Second, TABOR borrows the idea of interpretable AI to further prune the restored triggers. Last, TABOR designs a new anomaly detection method, which could not only facilitate the identification of intentionally injected triggers but also filter out false alarms (i.e., triggers detected from an uninfected model). We train 112 DNNs on five datasets and infect these models with two existing trojan attacks. We evaluate TABOR by using these infected models, and demonstrate that TABOR has much better performance in trigger restoration, trojan detection, and elimination than Neural Cleanse, the state-of-the-art trojan detection technique.

32 citations


Proceedings ArticleDOI
02 Nov 2020
TL;DR: This approach is the first attempt in applying reinforcement learning for efficient test generation to detect Trojans using delay-based analysis, and can significantly improve both side-channel sensitivity and test generation time compared to state-of-the-art test generation techniques.
Abstract: Reliability and trustworthiness are dominant factors in designing System-on-Chips (SoCs) for a variety of applications. Malicious implants, such as hardware Trojans, can lead to undesired information leakage or system malfunction. To ensure trustworthy computing, it is critical to develop efficient Trojan detection techniques. While existing delay-based side-channel analysis is promising, it is not effective due to two fundamental limitations: (i) The difference in path delay between the golden design and Trojan inserted design is negligible compared with environmental noise and process variations. (ii) Existing approaches rely on manually crafted rules for test generation, and require a large number of simulations, making it impractical for industrial designs. In this paper, we propose a novel test generation method using reinforcement learning for delay-based Trojan detection. This paper makes three important contributions. 1) Unlike existing methods that rely on the delay difference of a few gates, our proposed approach utilizes critical path analysis to generate test vectors that can maximize the side-channel sensitivity. 2) To the best of our knowledge, our approach is the first attempt in applying reinforcement learning for efficient test generation to detect Trojans using delay-based analysis. 3) Our experimental results demonstrate that our method can significantly improve both side-channel sensitivity (59% on average) and test generation time (17x on average) compared to state-of-the-art test generation techniques.

Proceedings ArticleDOI
09 Mar 2020
TL;DR: Experimental results demonstrate that the proposed automated test generation algorithm significantly outperforms state-of-the-art approaches that rely on ATPG or random test patterns for delay-based side-channel analysis.
Abstract: Side-channel analysis is widely used for hardware Trojan detection in integrated circuits by analyzing various side-channel signatures, such as timing, power and path delay Existing delay-based side-channel analysis techniques have two major bottlenecks: (i) they are not suitable in detecting Trojans since the delay difference between the golden design and a Trojan inserted design is negligible, and (ii) they are not effective in creating robust delay signatures due to reliance on random and ATPG based test patterns In this paper, we propose an efficient test generation technique to detect Trojans using delay-based side channel analysis This paper makes two important contributions (1) We propose an automated test generation algorithm to produce test patterns that are likely to activate trigger conditions, and change critical paths Compared to existing approaches where delay difference is solely based on extra gates from a small Trojan, the change of critical paths by our approach will lead to significant difference in path delay (2) We propose a fast and efficient reordering technique to maximize the delay deviation between the golden design and Trojan inserted design Experimental results demonstrate that our approach significantly outperforms state-of-the-art approaches that rely on ATPG or random test patterns for delay-based side-channel analysis

Journal ArticleDOI
TL;DR: The authors attempt to make a review of the hardware Trojan design and implementations in the last decade and also provide an outlook, focusing on the attacker's methods, capabilities, and challenges when the attacker designs and implements a hardware Trojan.
Abstract: Hardware Trojan detection techniques have been studied extensively However, to develop reliable and effective defenses, it is important to figure out how hardware Trojans are implemented in practical scenarios The authors attempt to make a review of the hardware Trojan design and implementations in the last decade and also provide an outlook Unlike all previous surveys that discuss Trojans from the defender's perspective, for the first time, the authors study the Trojans from the attacker's perspective, focusing on the attacker's methods, capabilities, and challenges when the attacker designs and implements a hardware Trojan First, the authors present adversarial models in terms of the adversary's methods, adversary's capabilities, and adversary's challenges in seven practical hardware Trojan implementation scenarios: in-house design team attacks, third-party intellectual property vendor attacks, computer-aided design tools attacks, fabrication stage attacks, testing stage attacks, distribution stage attacks, and field-programmable gate array Trojan attacks Second, the authors analyse the hardware Trojan implementation methods under each adversarial model in terms of seven aspects/metrics: hardware Trojan attack scenarios, the attacker's motivation, feasibility, detectability (anti-detection capability), protection and prevention suggestions for the designer, overhead analysis, and case studies of Trojan implementations Finally, future directions on hardware Trojan attacks and defenses are also discussed

Posted Content
TL;DR: A novel defensive technique is proposed, in which DNNs are taught to disregard the styles of inputs and focus on their content only to mitigate the effect of triggers during the classification, which reduces the attack success rate significantly and improves the initial accuracy of the models when processing both benign and adversarial data.
Abstract: Deep Neural Networks (DNNs) have been applied successfully in computer vision. However, their wide adoption in image-related applications is threatened by their vulnerability to trojan attacks. These attacks insert some misbehavior at training using samples with a mark or trigger, which is exploited at inference or testing time. In this work, we analyze the composition of the features learned by DNNs at training. We identify that they, including those related to the inserted triggers, contain both content (semantic information) and style (texture information), which are recognized as a whole by DNNs at testing time. We then propose a novel defensive technique against trojan attacks, in which DNNs are taught to disregard the styles of inputs and focus on their content only to mitigate the effect of triggers during the classification. The generic applicability of the approach is demonstrated in the context of a traffic sign and a face recognition application. Each of them is exposed to a different attack with a variety of triggers. Results show that the method reduces the attack success rate significantly to values < 1% in all the tested attacks while keeping as well as improving the initial accuracy of the models when processing both benign and adversarial data.

Journal ArticleDOI
01 Sep 2020
TL;DR: A landing or sample return mission to directly explore a Jupiter Trojan asteroid using solar power sail-craft OKEANOS (Oversize Kite-craft for Exploration and AstroNautics in the Outer Solar System), which includes analyses of the latest mission.
Abstract: The solar power sail is an original Japanese concept in which electric power is generated by thin-film solar cells attached on the solar sail membrane. Japan Aerospace Exploration Agency (JAXA) successfully demonstrated the world’s first solar power sail technology through IKAROS (Interplanetary Kite-craft Accelerated by Radiation of the Sun) mission in 2010. IKAROS demonstrated photon propulsion and power generation using thin-film solar cells during its interplanetary cruise. Scaled up, solar power sails can generate enough power to drive high specific impulse ion thrusters in the outer planetary region. With this concept, we propose a landing or sample return mission to directly explore a Jupiter Trojan asteroid using solar power sail-craft OKEANOS (Oversize Kite-craft for Exploration and AstroNautics in the Outer Solar System). After rendezvousing with a Trojan asteroid, a lander separates from OKEANOS to collect samples, and perform in-situ analyses in three proposed mission sequences, including sending samples back to Earth. This paper proposes a system design for OKEANOS and includes analyses of the latest mission.

Journal ArticleDOI
TL;DR: In this paper, the authors explore the possibility that Hercules is made of trojans, stars captured at L4, one the Lagrangian points of the stellar bar, and demonstrate that the retrograde nature of trojan orbits causes the asymmetry in the radial velocity distribution, typical of Hercules in the solar vicinity.
Abstract: About 20% of stars in the solar vicinity are in the Hercules stream, a bundle of stars that move together with a velocity distinct from the Sun. Its origin is still uncertain. Here, we explore the possibility that Hercules is made of trojans, stars captured at L4, one the Lagrangian points of the stellar bar. Using GALAKOS--a high-resolution N-body simulation of the Galactic disk--we follow the motions of stars in the co-rotating frame of the bar and confirm previous studies on Hercules being formed by stars in co-rotation resonance with the bar. Unlike previous work, we demonstrate that the retrograde nature of trojan orbits causes the asymmetry in the radial velocity distribution, typical of Hercules in the solar vicinity. We show that trojans remain at capture for only a finite amount of time, before escaping L4 without being captured again. We anticipate that in the kinematic plane the Hercules stream will de-populate along the bar major axis and be visible at azimuthal angles behind the solar vicinity with a peak towards L4. This test can exclude the OLR origin of the Hercules stream and be validated by Gaia DR3 and DR4.

Proceedings ArticleDOI
09 Mar 2020
TL;DR: It is argued that currently deployed protections still leave room for constructing attacks, by presenting transient trojans, software modules that conceal their malicious activity within transient execution mode.
Abstract: Transient execution is one of the most critical features used in CPUs to achieve high performance. Recent Spectre attacks demonstrated how this feature can be manipulated to force applications to reveal sensitive data. The industry quickly responded with a series of software and hardware mitigations among which microcode patches are the most prevalent and trusted. In this paper, we argue that currently deployed protections still leave room for constructing attacks. We do so by presenting transient trojans, software modules that conceal their malicious activity within transient execution mode. They appear completely benign, pass static and dynamic analysis checks, but reveal sensitive data when triggered. To construct these trojans, we perform a detailed analysis of the attack surface currently present in today's systems with respect to the recommended mitigation techniques. We reverse engineer branch predictors in several recent x86_64 processors which allows us to uncover previously unknown exploitation techniques. Using these techniques, we construct three types of transient trojans and demonstrate their stealthiness and practicality.

Journal ArticleDOI
TL;DR: In this paper, the authors investigate the stability of the two Trojan swarms, with a particular focus on these collisional families, and find that the members of Trojan swarm escape the population at a linear rate, with the primordial L4 (23.35 per cent escape) and L5 (24.89 per percent escape) population sizes likely 1.31 and 1.35 times larger than today.
Abstract: The Jovian Trojans are two swarms of objects located around the L4 and L5 Lagrange points. The population is thought to have been captured by Jupiter during the Solar system’s youth. Within the swarms, six collisional families have been identified in previous work, with four in the L4 swarm, and two in the L5. Our aim is to investigate the stability of the two Trojan swarms, with a particular focus on these collisional families. We find that the members of Trojan swarms escape the population at a linear rate, with the primordial L4 (23.35 per cent escape) and L5 (24.89 per cent escape) population sizes likely 1.31 and 1.35 times larger than today. Given that the escape rates were approximately equal between the two Trojan swarms, our results do not explain the observed asymmetry between the two groups, suggesting that the numerical differences are primordial in nature, supporting previous studies. Upon leaving the Trojan population, the escaped objects move on to orbits that resemble those of the Centaur and short-period comet populations. Within the Trojan collisional families, the 1996 RJ and 2001 UV209 families are found to be dynamically stable over the lifetime of the Solar system, whilst the Hektor, Arkesilos and Ennomos families exhibit various degrees of instability. The larger Eurybates family shows 18.81 per cent of simulated members escaping the Trojan population. Unlike the L4 swarm, the escape rate from the Eurybates family is found to increase as a function of time, allowing an age estimation of approximately 1.045 ± 0.364 × 109 yr.

Posted Content
TL;DR: The TrojAI software framework is introduced, an open source set of Python tools capable of generating triggered (poisoned) datasets and associated deep learning models with trojans at scale, and can be used to rapidly and comprehensively test new trojan detection methods.
Abstract: In this paper, we introduce the TrojAI software framework, an open source set of Python tools capable of generating triggered (poisoned) datasets and associated deep learning (DL) models with trojans at scale. We utilize the developed framework to generate a large set of trojaned MNIST classifiers, as well as demonstrate the capability to produce a trojaned reinforcement-learning model using vector observations. Results on MNIST show that the nature of the trigger, training batch size, and dataset poisoning percentage all affect successful embedding of trojans. We test Neural Cleanse against the trojaned MNIST models and successfully detect anomalies in the trained models approximately $18\%$ of the time. Our experiments and workflow indicate that the TrojAI software framework will enable researchers to easily understand the effects of various configurations of the dataset and training hyperparameters on the generated trojaned deep learning model, and can be used to rapidly and comprehensively test new trojan detection methods.

Journal ArticleDOI
Chen Dong1, Yulin Liu1, Chen Jinghui1, Ximeng Liu1, Wenzhong Guo1, Chen Yuzhong1 
TL;DR: An unsupervised hardware Trojans detection approach by combined the principal component analysis (PCA) and local outlier factor (LOF) algorithm, called PL-HTD, which shows the feasibility and efficiency of hardware TroJans detecting by employing a method without the guidance of class label information.
Abstract: With the booming development of the cyber-physical system, human society is much more dependent on information technology. Unfortunately, like software, hardware is not trusted at all, due to so many third parties involved in the separated integrated circuit's (IC) design and manufacturing stages for the high profit. The malicious circuits (named Hardware Trojans) can be implanted during any stage of the ICs' design and manufacturing process. However, the existing pre-silicon approaches based on machine learning theory have good performance, they all belong to supervised learning methods, which have a key prerequisite that is numerous already known information. Meanwhile, hardware Trojans are even more unimaginable because today's ICs are becoming more complicated. The known information is even harder to gain. Furthermore, the training process for supervised learning methods tends to be time-consuming and generally requires a huge amount of balanced training data. Therefore, this paper proposes an unsupervised hardware Trojans detection approach by combined the principal component analysis (PCA) and local outlier factor (LOF) algorithm, called PL-HTD. We firstly visualize the distribution features of normal nets and Trojan nets, and then reveal the differences between the two types of nets to reduce the dimension of the feature set. According to the outliers of each net, the abnormal nets are selected and verified by professionals later to confirm whether it is a true Trojan relative to the host circuit to realize the detection. The experiments show that the proposed method can detect hardware Trojans effectively and reduce the cost of manual secondary detection. For the Trust-HUB benchmarks, the PL-HTD achieves up to 73.08% TPR and 97.52% average TNR, moreover, it achieves average 96.00% accuracy, which shows the feasibility and efficiency of hardware Trojans detecting by employing a method without the guidance of class label information.

Proceedings ArticleDOI
02 Nov 2020
TL;DR: Cleann as discussed by the authors proposes an end-to-end framework that enables online mitigation of Trojans for embedded deep neural network (DNN) applications by leveraging dictionary learning and sparse approximation to characterize the statistical behavior of benign data and identify Trojan triggers.
Abstract: We propose Cleann, the first end-to-end framework that enables online mitigation of Trojans for embedded Deep Neural Network (DNN) applications. A Trojan attack works by injecting a backdoor in the DNN while training; during inference, the Trojan can be activated by the specific backdoor trigger. What differentiates Cleann from the prior work is its lightweight methodology which recovers the ground-truth class of Trojan samples without the need for labeled data, model retraining, or prior assumptions on the trigger or the attack. We leverage dictionary learning and sparse approximation to characterize the statistical behavior of benign data and identify Trojan triggers. Cleann is devised based on algorithm/hardware co-design and is equipped with specialized hardware to enable efficient real-time execution on resource-constrained embedded platforms. Proof of concept evaluations on Cleann for the state-of-the-art Neural Trojan attacks on visual benchmarks demonstrate its competitive advantage in terms of attack resiliency and execution overhead.

Posted Content
TL;DR: A novel framework hides the existence of a Trojan network with arbitrary desired functionality within a benign transport network and proves theoretically that the Trojan network's detection is computationally infeasible and empirically that the transport network does not compromise its disguise.
Abstract: The complexity of large-scale neural networks can lead to poor understanding of their internal details. We show that this opaqueness provides an opportunity for adversaries to embed unintended functionalities into the network in the form of Trojan horses. Our novel framework hides the existence of a Trojan network with arbitrary desired functionality within a benign transport network. We prove theoretically that the Trojan network's detection is computationally infeasible and demonstrate empirically that the transport network does not compromise its disguise. Our paper exposes an important, previously unknown loophole that could potentially undermine the security and trustworthiness of machine learning.

Journal ArticleDOI
TL;DR: This article presents a mechanism to introduce an extremely stealthy hardware Trojan into cryptographic primitives equipped with provably-secure first-order side-channel countermeasures and applies it to a threshold implementation of the PRESENT block cipher realized in two different CMOS technologies and shows that triggering the Trojan makes the ASIC prototypes vulnerable.
Abstract: Hardware Trojans have drawn the attention of academia, industry, and government agencies. Effective detection mechanisms and countermeasures against such malicious designs can only be developed when there is a deep understanding of how hardware Trojans can be built in practice, in particular, Trojans specifically designed to avoid detection. In this article, we present a mechanism to introduce an extremely stealthy hardware Trojan into cryptographic primitives equipped with provably-secure first-order side-channel countermeasures. Once the Trojan is triggered, the malicious design exhibits exploitable side-channel leakage, leading to successful key recovery attacks. Generally, such a Trojan requires neither addition nor removal of any logic which makes it extremely hard to detect. On ASICs, it can be inserted by subtle manipulations at the subtransistor level and on FPGAs by changing the routing of particular signals, leading to zero logic overhead. The underlying concept is based on modifying a securely masked hardware implementation in such a way that running the device at a particular clock frequency violates one of its essential properties, leading to exploitable leakage. We apply our technique to a threshold implementation of the PRESENT block cipher realized in two different CMOS technologies and show that triggering the Trojan makes the ASIC prototypes vulnerable.

Journal ArticleDOI
TL;DR: A new low overhead and high speed design for trust methodology for increasing both full activation and side channel sensitivity of Trojan is proposed, where the main idea is that the increase in transition probability of individual nets does not necessarily increase the transition probabilities of the succeeding nets of the circuit.
Abstract: Hardware Trojan attacks have emerged as a major security issue for hardware at different level of abstractions, which relate to malicious tampering of a hardware during design or fabrication process. In this paper, a new low overhead and high speed design for trust methodology for increasing both full activation and side channel sensitivity of Trojan is proposed. The main idea is that the increase in transition probability of individual nets does not necessarily increase the transition probability of the succeeding nets of the circuit. Accordingly, the rules and conflicts of the propagation of maximum transition probability for individual gates have been presented to ensure that a full transition path is constructed between each low transition probability net and primary inputs of the circuit. The results show that the proposed methodology achieves superior efficiency in Trojan full activation by more than $4 \times $ through logic testing approach besides higher sensitivity averagely around $20 {\times }$ for power-based side channel analysis compared to existing methods.

Journal ArticleDOI
TL;DR: In this article, the authors show that large vortices generated at the Lagrangian points are responsible for dust accumulation, where the leading Lagrangians trap a larger amount of submillimeter (submm) particles than the trailing ones.
Abstract: Trojans are defined as objects that share the orbit of a planet at the stable Lagrangian points $L_4$ and $L_5$. In the Solar System, these bodies show a broad size distribution ranging from micrometer($\mu$m) to centimeter(cm) particles (Trojan dust) and up to kilometer (km) rocks (Trojan asteroids). It has also been theorized that earth-like Trojans may be formed in extra-solar systems. The Trojan formation mechanism is still under debate, especially theories involving the effects of dissipative forces from a viscous gaseous environment. We perform hydro-simulations to follow the evolution of a protoplanetary disk with an embedded 1--10 Jupiter-mass planet. On top of the gaseous disk, we set a distribution of $\mu$m--cm dust particles interacting with the gas. This allows us to follow dust dynamics as solids get trapped around the Lagrangian points of the planet. We show that large vortices generated at the Lagrangian points are responsible for dust accumulation, where the leading Lagrangian point $L_4$ traps a larger amount of submillimeter (submm) particles than the trailing $L_5$, which traps mostly mm--cm particles. However, the total bulk mass, with typical values of $\sim M_{\rm moon}$, is more significant in $L_5$ than in $L_4$, in contrast to what is observed in the current Solar System a few gigayears later. Furthermore, the migration of the planet does not seem to affect the reported asymmetry between $L_4$ and $L_5$. The main initial mass reservoir for Trojan dust lies in the same co-orbital path of the planet, while dust migrating from the outer region (due to drag) contributes very little to its final mass, imposing strong mass constraints for the in situ formation scenario of Trojan planets.

Journal ArticleDOI
TL;DR: In this paper, four RTL Trojan features related to branching statement are proposed and the Minimum Redundancy Maximum Relevance (mRMR) feature selection is applied to the proposed Trojan features to determine the recommended feature combinations.
Abstract: Register-transfer-level (RTL) information is hardly available for hardware Trojan detection. In this paper, four RTL Trojan features related to branching statement are proposed. The Minimum Redundancy Maximum Relevance (mRMR) feature selection is applied to the proposed Trojan features to determine the recommended feature combinations. The feature combinations are then tested using different machine learning concepts in order to determine the best approach for classifying Trojan and normal branches. The result shows that a Decision Tree classification algorithm with all the four proposed Trojan features can achieve an average true positive detection rate of 93.72% on unseen test data.

Proceedings ArticleDOI
07 Dec 2020
TL;DR: In this paper, the authors proposed a novel golden-chip-free clustering method using backscattering side-channel to divide ICs into groups of Trojan-free and Trojan-infected boards.
Abstract: Over the past few years, malicious hardware modifications, a.k.a. hardware Trojans (HT), have emerged as a major security threat because integrated circuit (IC) companies have been fabricating chips at offshore foundries due to various factors including time-to-market, cost reduction demands, and the increased complexity of ICs. Among proposed hardware Trojan detection techniques, reverse engineering appears to be the most accurate and reliable one because it works for all circuits and Trojan types without a golden example of the chip. However, because reverse engineering is an extremely expensive, time-consuming, and destructive process, it is difficult to apply this technique for a large population of ICs in a real test environment. This paper proposes a novel golden-chip-free clustering method using backscattering side-channel to divide ICs into groups of Trojan-free and Trojan-infected boards. The technique requires no golden chip or a priori knowledge of the chip circuitry, and divides a large population of ICs into clusters based on how HTs (if existed) affect their backscattered signals. This significantly reduces the size of test vectors for reverse engineering based detection techniques, thus enables deployment of reverse engineering approaches to a large population of ICs in a real testing scenario. The results are collected on 100 different FPGA boards where boards are randomly chosen to be infected or not. The results show that we can cluster the boards with 100% accuracy and demonstrate that our technique can tolerate manufacturing variations among hardware instances to cluster all the boards accurately for 9 different dormant Trojan designs on 3 different benchmark circuits from Trusthub. We have also shown that we can detect dormant Trojan designs whose trigger size has shrunk to as small as 0.19% of the original circuit with 100% accuracy as well.

Journal ArticleDOI
Jiang Xie1, Shuhao Li1, Xiaochun Yun1, Yongzheng Zhang1, Chang Peng1 
TL;DR: This paper proposes an HTTP-based Trojan detection model via the Hierarchical Spatio-Temporal Features of traffics (HSTF-Model) based on the formalized description of traffic spatio-temporal behavior from both packet level and flow level that has a huge advantage over other related methods in generalization ability.

Proceedings ArticleDOI
17 Jan 2020
TL;DR: The proposed solution, unlike the prior art, does not require a Golden IC and trains a Neural Network to act as a process tracking watchdog for correlating the static timing data to the delay information obtained from clock frequency sweeping for Hardware Trojan detection.
Abstract: In this paper, we introduce a Learning Assisted Side Channel delay Analysis (LASCA) methodology for Hardware Trojan detection. Our proposed solution, unlike the prior art, does not require a Golden IC. Instead, it trains a Neural Network to act as a process tracking watchdog for correlating the static timing data (produced at design time) to the delay information obtained from clock frequency sweeping (at test time) for the purpose of Trojan detection. Using the LASCA flow, we detect close to 90% of Hardware Trojans in the simulated scenarios.