说明分析machine learning2019s课件attack v

1、Attack and DefenseAttack and DefenseHung-yi LeeSource of image: Motivation We seek to deploy machine learning classifiers not only in the labs, but also in real world. The classifiers that are robust to noises and work “most of the time” is not sufficient. We want the classifiers that are robust the

2、 inputs that are built to fool the classifier. Especially useful for spam classification, malware detection, network intrusion detection, etc. 光是強還不夠應付來自人類的惡意Attack medieval-weapon-1352-3.jpgWhat do we want to do?Something ElseTiger Cat0.64NetworkOriginal ImageAttacked ImagecloseLoss Function for At

3、tack Training: Non-targeted Attack: Targeted Attack: Constraint: Network不要被發現closefarclosee.g. cate.g. fish fixed fixed?Constraint L2-norm L-infinity Change one pixel muchChange every pixel a little bitsame L2 small L- large L- How to Attack Gradient Descent (Modified Version)Just like training a ne

4、ural network, but network parameter is replaced with input For t = 1 to T Start from original image If How to Attack Gradient Descent (Modified Version)Just like training a neural network, but network parameter is replaced with input def For all fulfill Return the one closest to For t = 1 to T Start

5、 from original image If How to Attack def For all fulfill Return the one closest to L2-norm L-infinityExample Tiger Cat0.64Original ImageAttacked ImageStar Fish1.00ResNet-50True = Tiger catFalse = Star FishExample Tiger Cat0.64Original ImageAttacked ImageStar Fish1.00=-50 xExample Tiger Cat0.64Origi

6、nal ImageAttacked ImageKeyboard0.98ResNet-50True = Tiger catFalse = Keyboardtabby catPersiancatfire screen tiger catWhat happened?Random Specific DirectionAttack Approaches FGSM ( ) Basic iterative method ( ) L-BFGS ( ) Deepfool ( ) JSMA ( ) C&W ( ) Elastic net attack ( ) Spatially Transformed (

7、 ) One Pixel Attack ( ) only list a fewAttack Approaches Fast Gradient Sign Method (FGSM)Different optimization methodsDifferent constraintsonly have +1 or -1 Attack Approaches Fast Gradient Sign Method (FGSM)Different optimization methodsDifferent constraints gradientonly have +1 or -1L-infinityAtt

8、ack Approaches Fast Gradient Sign Method (FGSM)Different optimization methodsDifferent constraints gradientonly have +1 or -1L-infinityvery large learning rateWhite Box v.s. Black Box In the previous attack, we fix network parameters to find optimal . To attack, we need to know network parameters Th

9、is is called White Box Attack. Are we safe if we do not release model? You cannot obtain model parameters in most on-line API. No, because Black Box Attack is possible. Black Box AttackIf you have the training data of the target networkTrain a proxy network yourselfUsing the proxy network to generat

10、e attacked objectsOtherwise, obtaining input-output pairs from target network NetworkBlackNetworkProxyTraining DataAttacked ObjectBlack Box AttackIf you have the training data of the target networkTrain a proxy network yourselfUsing the proxy network to generate attacked objectsOtherwise, obtaining

11、input-output pairs from target network BlackProxyUniversal Adversarial Attack Black Box Attack is also possible!Adversarial Reprogramming Gamaleldin F. Elsayed,Ian Goodfellow,Jascha Sohl-Dickstein, “Adversarial Reprogramming of Neural Networks”, ICLR, 2019Attack in the Real World Black Box AttackAtt

12、ack in the Real World rs/face-rec-ccs16.pdf1. An attacker would need to find perturbations that generalize beyond a single image.2. Extreme differences between adjacent pixels in the perturbation are unlikely to be accurately captured by cameras. 3. It is desirable to craft perturbations that are co

13、mprised mostly of colors reproducible by the printer. s/1707.08945Beyond Images You can attack audio You can attack text 07.07328.pdfDefenseDefense Defense Adversarial Attack cannot be defended by weight regularization, dropout and model ensemble. Two types of defense: Passive defense: Finding the a

14、ttached image without modifying the model Special case of Anomaly Detection Proactive defense: Training a model that is robust to adversarial attackPassive Defense KeyboardNetworkFiltere.g. Smoothing Attack signalOriginal Do not influence classificationLess harmfulTiger Cattiger cat0.64Smoothing tig

15、er cat0.45Keyboard0.98Smoothing tiger cat0.37Passive Defense Feature Squeeze s/1704.01155Randomization at Inference Phase s/1711.01991Proactive Defense 精神：找出漏洞、補起來For t = 1 to T Given training data Find adversarial input given by an attack algorithm Using both to update your model For n = 1 to N We have new training data Data Augmentationdifferent in each iteration找出漏洞Using to train your model 把洞補起來Using algorithm AThis method would stop algorithm A, but is still vulnerable for algorithm B. Concluding Remarks Attack: given the network parameters