CAAD VILLAGE - GeekPwn - The Uprising Geekpwn AI/Robotics Cybersecurity Contest U.S. 2018 - High Frequenzy Targeted Attacks

Video thumbnail (Frame 0) Video thumbnail (Frame 932) Video thumbnail (Frame 1169) Video thumbnail (Frame 2787) Video thumbnail (Frame 3532) Video thumbnail (Frame 4489) Video thumbnail (Frame 5109) Video thumbnail (Frame 6396) Video thumbnail (Frame 7033) Video thumbnail (Frame 8144) Video thumbnail (Frame 8466) Video thumbnail (Frame 10289) Video thumbnail (Frame 11047) Video thumbnail (Frame 14297) Video thumbnail (Frame 14756) Video thumbnail (Frame 15110) Video thumbnail (Frame 16589)
Video in TIB AV-Portal: CAAD VILLAGE - GeekPwn - The Uprising Geekpwn AI/Robotics Cybersecurity Contest U.S. 2018 - High Frequenzy Targeted Attacks

Formal Metadata

CAAD VILLAGE - GeekPwn - The Uprising Geekpwn AI/Robotics Cybersecurity Contest U.S. 2018 - High Frequenzy Targeted Attacks
Alternative Title
Adversarial^2 Training
Title of Series
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Release Date

Content Metadata

Subject Area
Targeted attacks of image classifiers are difficult to transfer from one model to another. Only strong adversarial attacks with the knowledge of the classifier can bypass existing defenses. To defend against such attacks, we implement an “adversarial^2 training” method to strengthen the existing defenses. Yao Zhao is an applied scientist at Microsoft AI & Research working on natural language understanding/generation and search ranking. During his Ph.D. at Yale University, he worked in the field of computuer vision and optics. Yuzhe Zhao is a software engineer in Google Research, working on natural language understanding. He recently earned his Ph.D. from Yale University. Previously, he received his undergraduate degree in mathematics and physics from Shanghai Jiao Tong University.
Frequency Presentation of a group Artificial neural network Multiplication sign
Predictability Artificial neural network Computer-generated imagery .NET Framework Computer network Perturbation theory Function (mathematics) Mereology Neuroinformatik Frequency Medical imaging Software Personal digital assistant output Physical system Social class
Medical imaging Type theory Artificial neural network Computer-generated imagery Gradient output Perturbation theory Reverse engineering
Curve Artificial neural network Weight Gradient Model theory Multiplication sign Weight Gradient Computer network Black box Message passing Bit rate Iteration Personal digital assistant Musical ensemble Cuboid Iteration Musical ensemble Endliche Modelltheorie Physical system
Functional (mathematics) Artificial neural network Gradient Insertion loss Heat transfer Medical imaging Different (Kate Ryan album) Personal digital assistant Single-precision floating-point format Musical ensemble Moving average Endliche Modelltheorie Row (database)
Medical imaging Server (computing) Personal digital assistant Model theory Combinational logic Set (mathematics) Right angle Endliche Modelltheorie Musical ensemble Physical system Neuroinformatik
Dataflow Greatest element Thread (computing) Multiplication sign Combinational logic Menu (computing) Computer programming 2 (number) Medical imaging Strategy game Different (Kate Ryan album) Semiconductor memory Queue (abstract data type) Endliche Modelltheorie Multiplication Graph (mathematics) Electric generator Key (cryptography) Weight Model theory Tensor Process (computing) Befehlsprozessor output Right angle Iteration Musical ensemble Physical system Resultant Spacetime
Strategy game Bit rate Multiplication sign Moving average Right angle Endliche Modelltheorie
Presentation of a group Graph (mathematics) Key (cryptography) Model theory Black box Instance (computer science) 2 (number) Neuroinformatik Medical imaging Befehlsprozessor Different (Kate Ryan album) Musical ensemble Endliche Modelltheorie Simulation
okay the next presentation is from wincing ciao and yeah hotel okay they they bring us high frequency targeted attacks to use this method to win the car CTF yesterday thank you happy hi my name is Jojo this is my friend Vince in jail were both NLP researchers and in our spare time we work on like adversary attack and defenses for neural networks and today we're going to talk about our
method high frequency targeted attacks that we used in yesterday's CA D computation so in the first 1/2 I'm
going to introduce some like basic concepts of adversary attacks and defend and then the second part we're going to talk about our techniques in the computation so neural networks are becoming more a lot more popular in image classification and are deployed in a lot of a commercial system in this case when the image is given to you in your networks the network takes in the raw pixel and calculate the activation through a lot of hidden layers and then output a final label for image in the popular case like image net there can be a thousand labels for a image and a adversary attack against a neural network is that we apply some small perturbation on the input image and make the prediction of the neural network fail to another class in this case we changed a correct label from snow to Fox
there are generally two types of attacks at adversary texts the first one is non target attacks basically change the label correct label to any non Traktor label with our specific target the other one is the target attacks that is to given a target and perturb the image to have it to classify wrong in the other target so the method of constructing
adversarial images are the most popular method is a gradient based attack when given an input image we can calculate and the neural network we can calculate the loss by through the neural network and calculate the gradients back to the image and then if we add a perturb the image in the way that is opposite to the gradients then we can get a reverse aerial image that can fool the original neural network a more powerful attack is
a iterative attack is apply the same gradient method again and again over over many iterations and as you can see in the curve the more T iterations we we apply this message the higher successful rate the attack can be so in in the real
realistic system there can be blackbox attacks and white box attacks so for white box attacks the tacker have access to the model weights in this case gradient attack can be applied and the gradients can be accurately calculated and usually the attack success rate is very very high in the black box case model weights are not accessible to attackers says to successfully attack a neural network we need to either guess the neural net neural network the defender is using or ensemble a lot of neural networks and attack them at the same time for those ensemble attacks is
like single like single neural network attack we add the loss function of many different kind of neural networks together and calculate the great inspect through all of the neural network at the same time and apply the same gradient based attack as the previous step so in
this competition we focused on the targeted attack and the target attack has some specific behavior that when you attack a used attack method or model it usually doesn't transfer to a different to a different model in this case we have a lot of a different attack method on the rows and columns and you can only attack the model we can only attack the defender using the same model and really the attack image is rarely applied to new defenders so instantly it's going to
talk about this computation and the method and system were using okay so thank you yellow for that introduction so something I want to add this for example
here so you see that so for especially for a targeted attack the image is not transferable so which means you have to guess so first thing is it's really expensive to train your model so we think that in practice so people usually I mean if you work on the image that's data sets usually people will use the pre-existing pre-trained model right instead of between their own model so they only have like couple of thousands of models out there so the question is if we can attack them all so we can with hyper which attack any system so because we we assume that people are using all assemble ensembles of those models any some combination of them to do the defense so that's so that's our assumption and it's it's basically the case Asche and then the other thing is it's not transferable so that means if you if people are if some somebody is using like see inception v3 and we don't have that model so it's really hard for us to build a tank model at the server image to attack that model without using that model to generate the image so what's for competition what's
what's important here so we we are allowed to submit our check every six second so that's uh that's a budget we have so we the competition runs for like thirty minutes so that means we can try three hundred times so maybe two hundred times so the key here is we want to try different combination of ensembles and to generate the image but we want to do that really fast so how do you do that
so basically it's it's quite simple I mean that we run a multi thread program so there is a one thread which do the submission so the submission is controlled by the one thread and image is pointing to the double-ended queue and we have a automatically generate generator to generate a self image as well as manually a menu generator so automatically generators that we have some prefix that ensembles combinations we have like 250 of them and we will try them all so it's fully automatic and that you make it run fast so you have to I mean the technical details as a base a we use the tensor flow and the tensor flow is pretty slow to build a graph okay the blue graph takes like 30 seconds so you don't want to build graph for for each iteration so we want to reuse the graph but which once you change the ensemble's so for each song example you have some weights right so weight says that as an input so another graph and the best thing for tensor flow is like for them pole F in this patch if you want to use that model so you don't want to evaluate the bottom right but testable didn't support that so so there's some space you can improve that but basically right now it's like if you have ten five models in your example so no matter you use it that not so tense of a while Oh evaluate that model so it takes time but but still good enough and so that's basic for our automatic generator and the phone menu generator so we will look at the results of the feedbacks and come up with some combinations we think in my work and submit that job to CPU so automatic generator is run on a GPU and menu menu as generators on run a CPU so so that they will not compete for memories but definitely a menu generators store so that's our strategy and so yeah so we can see yesterday's
like we we attack everybody a crazy right so I mean the success rate rate is not high if as long as we can get this time squared so that's fine so that's what we do yeah that's what we did yesterday yeah our strategy yeah thank
you question it's tremendous yeah yeah so it's like if you use our model so our biggest
ensembles that we use seven models so the to build a graph that takes like 30 seconds and then to do the computation I mean we calculate we compute 10 images two at once so 10 names takes like 20 seconds but for for CPU if you do that same thing for just one image it takes like 4 minutes something like that yeah so different skill yeah yeah yeah so we yeah we heavily rely on the automatic so that those predefined ensembles the way we would get them you know we get so because people make my utrom that's that's a strata so yeah because nowadays we still believe that so for the black box tax or the key is to guess what model the opponent is using right cool
thank you [Applause] thanks for instance the yeah so this is a last presentation yeah we finish this this money thanks everyone