CAAD VILLAGE  GeekPwn  The Uprising Geekpwn AI/Robotics Cybersecurity Contest U.S. 2018  High Frequenzy Targeted Attacks
Video in TIB AVPortal:
CAAD VILLAGE  GeekPwn  The Uprising Geekpwn AI/Robotics Cybersecurity Contest U.S. 2018  High Frequenzy Targeted Attacks
Formal Metadata
Title 
CAAD VILLAGE  GeekPwn  The Uprising Geekpwn AI/Robotics Cybersecurity Contest U.S. 2018  High Frequenzy Targeted Attacks

Alternative Title 
Adversarial^2 Training

Title of Series  
Author 

License 
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. 
Identifiers 

Publisher 

Release Date 
2018

Language 
English

Content Metadata
Subject Area  
Abstract 
Targeted attacks of image classifiers are difficult to transfer from one model to another. Only strong adversarial attacks with the knowledge of the classifier can bypass existing defenses. To defend against such attacks, we implement an “adversarial^2 training” method to strengthen the existing defenses. Yao Zhao is an applied scientist at Microsoft AI & Research working on natural language understanding/generation and search ranking. During his Ph.D. at Yale University, he worked in the field of computuer vision and optics. Yuzhe Zhao is a software engineer in Google Research, working on natural language understanding. He recently earned his Ph.D. from Yale University. Previously, he received his undergraduate degree in mathematics and physics from Shanghai Jiao Tong University.

00:00
Frequency
Presentation of a group
Artificial neural network
Multiplication sign
00:37
Predictability
Artificial neural network
Computergenerated imagery
.NET Framework
Computer network
Perturbation theory
Function (mathematics)
Mereology
Neuroinformatik
Frequency
Medical imaging
Software
Personal digital assistant
output
Physical system
Social class
01:51
Medical imaging
Type theory
Artificial neural network
Computergenerated imagery
Gradient
output
Perturbation theory
Reverse engineering
03:00
Curve
Artificial neural network
Weight
Gradient
Model theory
Multiplication sign
Weight
Gradient
Computer network
Black box
Message passing
Bit rate
Iteration
Personal digital assistant
Musical ensemble
Cuboid
Iteration
Musical ensemble
Endliche Modelltheorie
Physical system
04:16
Functional (mathematics)
Artificial neural network
Gradient
Insertion loss
Heat transfer
Medical imaging
Different (Kate Ryan album)
Personal digital assistant
Singleprecision floatingpoint format
Musical ensemble
Moving average
Endliche Modelltheorie
Row (database)
05:26
Medical imaging
Server (computing)
Personal digital assistant
Model theory
Combinational logic
Set (mathematics)
Right angle
Endliche Modelltheorie
Musical ensemble
Physical system
Neuroinformatik
06:52
Dataflow
Greatest element
Thread (computing)
Multiplication sign
Combinational logic
Menu (computing)
Computer programming
2 (number)
Medical imaging
Strategy game
Different (Kate Ryan album)
Semiconductor memory
Queue (abstract data type)
Endliche Modelltheorie
Multiplication
Graph (mathematics)
Electric generator
Key (cryptography)
Weight
Model theory
Tensor
Process (computing)
Befehlsprozessor
output
Right angle
Iteration
Musical ensemble
Physical system
Resultant
Spacetime
09:32
Strategy game
Bit rate
Multiplication sign
Moving average
Right angle
Endliche Modelltheorie
10:04
Presentation of a group
Graph (mathematics)
Key (cryptography)
Model theory
Black box
Instance (computer science)
2 (number)
Neuroinformatik
Medical imaging
Befehlsprozessor
Different (Kate Ryan album)
Musical ensemble
Endliche Modelltheorie
Simulation
00:00
okay the next presentation is from wincing ciao and yeah hotel okay they they bring us high frequency targeted attacks to use this method to win the car CTF yesterday thank you happy hi my name is Jojo this is my friend Vince in jail were both NLP researchers and in our spare time we work on like adversary attack and defenses for neural networks and today we're going to talk about our
00:39
method high frequency targeted attacks that we used in yesterday's CA D computation so in the first 1/2 I'm
00:50
going to introduce some like basic concepts of adversary attacks and defend and then the second part we're going to talk about our techniques in the computation so neural networks are becoming more a lot more popular in image classification and are deployed in a lot of a commercial system in this case when the image is given to you in your networks the network takes in the raw pixel and calculate the activation through a lot of hidden layers and then output a final label for image in the popular case like image net there can be a thousand labels for a image and a adversary attack against a neural network is that we apply some small perturbation on the input image and make the prediction of the neural network fail to another class in this case we changed a correct label from snow to Fox
01:52
there are generally two types of attacks at adversary texts the first one is non target attacks basically change the label correct label to any non Traktor label with our specific target the other one is the target attacks that is to given a target and perturb the image to have it to classify wrong in the other target so the method of constructing
02:24
adversarial images are the most popular method is a gradient based attack when given an input image we can calculate and the neural network we can calculate the loss by through the neural network and calculate the gradients back to the image and then if we add a perturb the image in the way that is opposite to the gradients then we can get a reverse aerial image that can fool the original neural network a more powerful attack is
03:03
a iterative attack is apply the same gradient method again and again over over many iterations and as you can see in the curve the more T iterations we we apply this message the higher successful rate the attack can be so in in the real
03:29
realistic system there can be blackbox attacks and white box attacks so for white box attacks the tacker have access to the model weights in this case gradient attack can be applied and the gradients can be accurately calculated and usually the attack success rate is very very high in the black box case model weights are not accessible to attackers says to successfully attack a neural network we need to either guess the neural net neural network the defender is using or ensemble a lot of neural networks and attack them at the same time for those ensemble attacks is
04:22
like single like single neural network attack we add the loss function of many different kind of neural networks together and calculate the great inspect through all of the neural network at the same time and apply the same gradient based attack as the previous step so in
04:46
this competition we focused on the targeted attack and the target attack has some specific behavior that when you attack a used attack method or model it usually doesn't transfer to a different to a different model in this case we have a lot of a different attack method on the rows and columns and you can only attack the model we can only attack the defender using the same model and really the attack image is rarely applied to new defenders so instantly it's going to
05:29
talk about this computation and the method and system were using okay so thank you yellow for that introduction so something I want to add this for example
05:39
here so you see that so for especially for a targeted attack the image is not transferable so which means you have to guess so first thing is it's really expensive to train your model so we think that in practice so people usually I mean if you work on the image that's data sets usually people will use the preexisting pretrained model right instead of between their own model so they only have like couple of thousands of models out there so the question is if we can attack them all so we can with hyper which attack any system so because we we assume that people are using all assemble ensembles of those models any some combination of them to do the defense so that's so that's our assumption and it's it's basically the case Asche and then the other thing is it's not transferable so that means if you if people are if some somebody is using like see inception v3 and we don't have that model so it's really hard for us to build a tank model at the server image to attack that model without using that model to generate the image so what's for competition what's
06:56
what's important here so we we are allowed to submit our check every six second so that's uh that's a budget we have so we the competition runs for like thirty minutes so that means we can try three hundred times so maybe two hundred times so the key here is we want to try different combination of ensembles and to generate the image but we want to do that really fast so how do you do that
07:24
so basically it's it's quite simple I mean that we run a multi thread program so there is a one thread which do the submission so the submission is controlled by the one thread and image is pointing to the doubleended queue and we have a automatically generate generator to generate a self image as well as manually a menu generator so automatically generators that we have some prefix that ensembles combinations we have like 250 of them and we will try them all so it's fully automatic and that you make it run fast so you have to I mean the technical details as a base a we use the tensor flow and the tensor flow is pretty slow to build a graph okay the blue graph takes like 30 seconds so you don't want to build graph for for each iteration so we want to reuse the graph but which once you change the ensemble's so for each song example you have some weights right so weight says that as an input so another graph and the best thing for tensor flow is like for them pole F in this patch if you want to use that model so you don't want to evaluate the bottom right but testable didn't support that so so there's some space you can improve that but basically right now it's like if you have ten five models in your example so no matter you use it that not so tense of a while Oh evaluate that model so it takes time but but still good enough and so that's basic for our automatic generator and the phone menu generator so we will look at the results of the feedbacks and come up with some combinations we think in my work and submit that job to CPU so automatic generator is run on a GPU and menu menu as generators on run a CPU so so that they will not compete for memories but definitely a menu generators store so that's our strategy and so yeah so we can see yesterday's
09:33
like we we attack everybody a crazy right so I mean the success rate rate is not high if as long as we can get this time squared so that's fine so that's what we do yeah that's what we did yesterday yeah our strategy yeah thank
09:52
you question it's tremendous yeah yeah so it's like if you use our model so our biggest
10:05
ensembles that we use seven models so the to build a graph that takes like 30 seconds and then to do the computation I mean we calculate we compute 10 images two at once so 10 names takes like 20 seconds but for for CPU if you do that same thing for just one image it takes like 4 minutes something like that yeah so different skill yeah yeah yeah so we yeah we heavily rely on the automatic so that those predefined ensembles the way we would get them you know we get so because people make my utrom that's that's a strata so yeah because nowadays we still believe that so for the black box tax or the key is to guess what model the opponent is using right cool
11:05
thank you [Applause] thanks for instance the yeah so this is a last presentation yeah we finish this this money thanks everyone