AI VILLAGE - Adversarial Stickers

Video thumbnail (Frame 0) Video thumbnail (Frame 928) Video thumbnail (Frame 3966) Video thumbnail (Frame 6134) Video thumbnail (Frame 6622) Video thumbnail (Frame 7874) Video thumbnail (Frame 9197) Video thumbnail (Frame 9811) Video thumbnail (Frame 11356) Video thumbnail (Frame 11988) Video thumbnail (Frame 13060) Video thumbnail (Frame 13977) Video thumbnail (Frame 16184) Video thumbnail (Frame 18137) Video thumbnail (Frame 22849) Video thumbnail (Frame 24040) Video thumbnail (Frame 26606) Video thumbnail (Frame 28010)
Video in TIB AV-Portal: AI VILLAGE - Adversarial Stickers

Formal Metadata

Title
AI VILLAGE - Adversarial Stickers
Alternative Title
Adversarial Patches
Title of Series
Author
License
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
2018
Language
English

Content Metadata

Subject Area
Abstract
Adversarial examples fooling machine learning field are a burgeoning field. We propose applications to fool self driving cars or facial recognition systems but most of the techniques are purely academic. They require minute manipulations to the bit values of the pixels entering a system. Adversarial patches are an attack that could actually work. This talk will cover how to make them and further applications I got my Ph.D. in algebraic topology in 2016 and immediately moved into machine learning to work on something useful to people. I then completed a post-doc in mathematical machine learning where I worked on medical data. I now work at endgame.
Patch (Unix) Planning Game theory
Topology Pixel Computer-generated imagery Function (mathematics) 2 (number) Neuroinformatik Medical imaging Sign (mathematics) Machine learning Different (Kate Ryan album) Computer hardware Integrated development environment Absolute value Social class Noise (electronics) Algebraic number Bit Perturbation theory Mathematics Type theory Video game Endliche Modelltheorie Quicksort Object (grammar) Information security
Pixel Link (knot theory) Connectivity (graph theory) Patch (Unix) Computer-generated imagery Range (statistics) Shape (magazine) Parallel port Mereology Wave packet Neuroinformatik Data model Medical imaging Tensor Different (Kate Ryan album) Core dump Spacetime Pixel Noise (electronics) Patch (Unix) Range (statistics) Bit Wave packet Shape (magazine) Tensor Component-based software engineering Process (computing) Endliche Modelltheorie Object (grammar)
Rotation Random number Patch (Unix) Transformation (genetics) Scaling (geometry) Connectivity (graph theory) Computer-generated imagery Range (statistics) Sampling (statistics) Translation (relic) Shape (magazine) Transformation (genetics) Translation (relic) Affine space Rotation Wave packet Degree (graph theory) Component-based software engineering Medical imaging Tensor Sample (statistics) Auditory masking
Point (geometry) Transformation (genetics) Field (computer science) Wave packet Medical imaging Mathematics Military operation Cuboid Endliche Modelltheorie Loop (music) Mathematical optimization Graphics processing unit Continuous track Patch (Unix) Bit Component-based software engineering Personal digital assistant Auditory masking Endliche Modelltheorie Right angle Thetafunktion Mathematical optimization Data structure Gradient descent Booting
Trail Artificial neural network Transformation (genetics) Insertion loss Drop (liquid) Flow separation Entire function Wave packet 10 (number) 2 (number) Medical imaging Message passing Loop (music) Personal digital assistant Musical ensemble Endliche Modelltheorie Musical ensemble Game theory Social class Gradient descent
Data model Message passing Software Gradient Resonator Musical ensemble Endliche Modelltheorie Bit rate Endliche Modelltheorie Mereology Resultant Sequence
Covering space Point (geometry) Data model Medical imaging Scaling (geometry) Musical ensemble Endliche Modelltheorie Bit rate Drop (liquid) Sequence
Covering space Rotation Curve Scaling (geometry) Transformation (genetics) Code 40 (number) Orientation (vector space) Range (statistics) Parallel port Bit Unit testing Parameter (computer programming) Parallel port Wave packet Degree (graph theory) Data model Medical imaging Musical ensemble Endliche Modelltheorie Software testing Endliche Modelltheorie
Point (geometry) Functional (mathematics) Code Patch (Unix) Letterpress printing Virtual machine Set (mathematics) Inclined plane Black box Mereology Wave packet Medical imaging Sign (mathematics) Gastropod shell Cuboid Endliche Modelltheorie Computer architecture Weight Projective plane Data management Process (computing) Software Repository (publishing) Blog Video game
Data model Amenable group Medical imaging Software Multiplication sign Musical ensemble Endliche Modelltheorie Parallel computing
Open source Transformation (genetics) Augmented reality Parameter (computer programming) Limit (category theory) Affine space Wave packet Data model Medical imaging Software Operator (mathematics) Endliche Modelltheorie Computer architecture
Rotation Point (geometry) Greatest element Scaling (geometry) Multiplication sign 1 (number) Control flow Field (computer science) Wave packet Degree (graph theory) Medical imaging Arithmetic mean Coefficient of determination Message passing Loop (music) Military operation Blog Musical ensemble Data structure Physical system
cool so the first stop we have is by spam Cattell data scientist at endgame and he's here to talk about a bursary of Hatch's basically the plan for the stork was I was going to make feature with ever serial passion and the volunteer t-shirts all have this stuff promise I've tried for about two months to do and eventually it's working so well and I'm going to talk about how to make these patches how they work why the thing and then like reasons we'll think so so I'm a data scientist named game
we've really heard about me so that's
going to be doing today so the
background of epic you'll examples you've got two things basic classes of resumes one where you modify the pixels coming in for images you modify the pixels of an image so that a classifier screwdrive this is the classic thing we've across creating assignments it's a creature could be saving MCU or various different absolute act and we most hope most of us have seen Ian's panda misclassifies it as given with some noise these sorts of attacks are good for understanding how neural deep learning works but they don't act there I'm actually attacked they can't do anything in the real world because you'd have to grab begin without the output of the camera and modify the pixels on the fly and it would you know it's not a realistic attack because you leave acquirable Hardware computation then you probably can get into assisting other type of attack against machine learning is the physical modification so what you can do is you put us to the stickers on a object ahead of misclassify when the first paper that did this was a one with their attacking the classifier they do stop signs because you know self-driving cars stop stop signs and they came up with a way of producing pixel modifications that they can put stickers on the stop sign and causes the hots hard to get mr. you work badly there's some evidence that their attack does not work very well in real life and you have to have a week of poor classifier and a few other things to make it look like it works but these sorts of attacks are much harder to do so adversarial stickers are the second type of attack you actually can't put a thing on a person and have a bit okay so
the way you build the attack is there
are three parts of the attack base so you have the actual sticker so this move like I've built this is real object they contained the image the floating lead pencil that holds the sticker I also hold a mosque so you can have big sticker be different shapes then we have a placer that puts a sticker on a bunch of images so that you can train against those images or with the back spine and then we have a trainer that actually does the training process so it's very easy bill bum just you've got computers go to build these 3d components and go look at the original paper there's a link to a github account which has basically these things laid out so you got a sticker well the way I built the sticker is you can take image a PNG image with the mosque or a mosque and it can be whatever you want as long as it fits within your training target thing or you can take an umpire array and then randomly initialize the patch around 50% gray with a bit of noise and that's the start of the secret the initialize of that when you pull the sticker it gives you back the tensor from the sticker this is I did this in a torch and so you can start applying it to images and it does a few things very clips ascetism as the pista core is a tactical image so it stays that I think the pixel range and then it multiplied by the mosque so it looks like that much an example is we got the patch with
their logo and fits with it within the masker produced it and then we have banging em over and the master produced it so this is the simplest component but like the thing that we're actually training this is the further the rest is decorations around this the probably the
most the most important component is the Placer so what this does is you can take you give three ranges so a range of rotations that you're allowing so anything any rotation in this range so plus or minus 15 degrees 30 degrees a range of translations you're allowing and a range of scaling that you're not you can scale it up by 30% or down by 30% or whatever and you randomly select it when you call it is what it does is it makes samples a bunch of affine transformations with those within the ranges you've provided copies the sticker into as much as many images you because you have and then transforms the those stickers to be the right shape also does this with masks and then applies it to your stick so for how it
works so we've got stickers on the side and the masks on the side and here is an affine transformation have to randomly select a s theta T X and the T Y and a little select in this case 25 of those and then apply that feature those stickers and then you get the transformation on out there so see on the left you have the untransformed stickers and on the right you have a bunch of transform stickers and so then you have you stole that and then put a little bit of math and you apply the stickers to these images and you get sorry for the image quality here you get a bunch of stuck images like stuck the images and that at that point you can be to a classifier and then we get to the last component of
trainer this thing you can take in several models to train again so these will be your white box models that you're training against your attack again it also takes an optimizer and a field of things you need to do gradient descent and this thing places the models and the sticker is our energy few and does all the stuff the juggling that you don't want to think about and actually what it does it just performs the training we have a basic train the basic
training loop here this is for C far that we have our sticker on the left and so the sticker goes through the place room with some images and so get the gift place on the place on these things so we have a 125 images here that have been how to sticker attach them then this goes into a classifier whatever classifier we have that the other thing and then we take the loss with respect to our target class so in this case how is going for the ninth pass the 10th class hobbsy bar so we take loss with respect to what class which I'm a gift and then we do a back drop grading descent against the original sticker is this entire track thing from the original sticker to the loss is smooth so there's a smooth transformation so we can do gradient descent with respect to the sticker itself so this is the basic training loop and if you want you to do a single neural network if you wanted to train it against a single classifier you just do this and you're good but if you
want to train a game several classifiers so what happens is we want to train your games to well this you can train it against you by going okay in my loop I'm just going to do it twice basically the same the same basically twice something to have you're going to do the first classifier and then the second classifier and I'm going to loop until I get a sticker that works so this is for an ensemble training against not a lot and then the other way of doing it which it wouldn't do is you instead of feeding
into one pass of our you view the to pass part you get a loss add add it up take the gradient all the way back and you get a result so there's two things work and you can train up a classifier and here we go results
so if we have I did this with like the only thing I could get this working on was see farm and this will happily talk to people about like null results here so what I did was I got those full models inside so vgg style network with nine layers vgu 13 layers of resonance between two and a deep neck which doesn't perform that well with ten layers and then we target target of the
game okay
I targeted against a resonant ten and the drop on the left is for that scale for zero scale we had no performance so this is a this would be the image scale down to a point obviously it's not going to do anything useful and then when it's twenty covers twenty percent of the image it's kind of useful and then at about forty percent the image it kind of it saturates the convolutional layers and add poses mister pacification for this for this sticker here you need to
cover forty percent of the image and then it will close in this classifier notes classification
and the youth this was trained within this eventual model and then if you train it in parallel it doesn't perform as well and reasons for that but so I went built this out test it against see far really gonna see far was the unit test and then I went okay I'm going to get this working on imagenet and so went through the original paper and we'll see far you can with this stuff who was trained with a certain range of rotations scalings and populations and if you take the rotation to be within with upper minus 90 degrees it doesn't it doesn't train well it doesn't work so the in the original paper as I mentioned oh you can do complete rotations it can be stuck in any orientation it's fine it'll work but what they when you actually look at the code they're only translated by plus or minus 15 degrees in that range which that's the first thing that I was like this is a little bit not people but so would see far this thing is with their parameters and if you go for bigger parameters more manipulation of the affine transformations is that curve drops off rapidly with more movie trivia so for these this is how the gifts of
the training through an image net so what I did was I grabbed for image net for management models of the PI port repository I stuck I pointed my trainer to train against those four and the same set that I trained see far again so if you have a Resnick resident 50-some to be gg's and a deep neck so I trained that again against that and I targeted a resident to be three or bedico for one of the VG's and the performance I got out of this was about ten percent five so it doesn't actually function as well as the paper that they have says if you work at the same as the so see part first and I go I'm flattered I try I went back to they paper they're only using five thousand examples they're using models that may or may not I don't I don't know what their models look like could not get the thing to work after like two months of it but so basically like welcome to AI village we have a normal job so references like I only use the
original paper then we've got Ian Panda and the stop sign so you guys have any questions yeah yeah I would be able to then do a white box attack and these were designed to be black box attacks and it's more effective with white box attacks that I if I wanted this to be a realistic attack against machine learning models which I shouldn't and that should be robust enough to be black box also one of the things that makes that make the image more robust to try and against several models it was supposed to be an ox it classifies as a conch shell but that's partially because of t-shirt printing so it's not the t-shirt printing is invented enough to get the fine details required for this to overwhelm a pacifier so I started this project with like yeah these adversarial patches that looks the paper look awesome I'm going to do it and then end up Roger good like yeah I don't remember at this point I know it might be that I made a mistake somewhere so the code that I wrote is are up on github and I will be writing a blog post with the AI village and describing exactly why I think this was an old result and with all the code and it's designed to be easily readable so you can see what's going on but yeah yeah yeah 40% I'm you well it we come visit like from making a few thousand of these I have inclination to say you're going to get situations where is the deeper the network the more effective the attack part I think is because it can diffuse out more before it gets to the final fully connected layer in the combinational layers I didn't yeah I trained it on my desktop that has 210 ATT is and like it 670 6700 K thing yeah I also train I actually trained OC for things on my razor blade with the 1060 so the this architecture the model
parallel architecture is about seven times faster than the sequential thing
because you can put it on each one in
each GPU and you don't have to juggle things I so it can't rain in about half an hour for images against a fortune of images you only want to train it again on maybe 10,000 images so you don't need to go all the way also when you actually go to decide okay I'm going to do this I take a very very aggressive step size so there step lied in the paper is five and I found that if you make that too low it just software doesn't so the these gifts were made with the
step size of 0.3 to make them pretty and
make them actually look like they're doing something but they're yeah yeah you want to try that out because open source on the Reaper see the heart so the way I built this was with the that the a fine grid transformation and Python which was designed for spatial sequential networks so in those they have a training parameter from two undeterred the network's you know you put an affine transformation on an image to do data limitation so these networks were designed to learn how to undo data augmentation as well as doing pacification so they're the architecture that think is if there's a pair of operations and high-torque that do that step and they take in parameters like that like the scaling thing that affine transformation is just a grid transformation and you could be it's a smooth transformation you can feed data into that thing to fix your transformation to be useful so there's actually a paper for doing a manipulating the affine transformations yeah I would just if I was a specific network
that I was targeting I just do this I wouldn't try to do ensemble training and stuff just focus on the neckline yeah no I wouldn't have to do anything you might be able to be able to pop your resume your training too mm-hmm yeah yes that's the thing the when you do this when you take this if you allow the
rotation to be sampled from within plus or minus 90 degrees the whole system break down so the rotation that you is actually effective is possible minus 30 degrees or 15 degrees so they already pretty much say these stickers have to be sitting straight up and nine yeah yeah message yeah but they they learn how to recognize dogs that are upside down yeah you you yeah maybe so I tried yes but sorry um a question even with the do GG point do they think they kill into there so just kidding is low in so the scaling is the receptive field at the bottom is 3x3 but then you go up one layer the receptive field up think think when you go all the way back down to the image is nine by nine and then one layer up it's like 27 by 27 sorry it's not a knockout by 9 is 5 by 5 and then 6 by 6 it's because I separate my tenants so the receptive field for this pickup picks up the top of the vge 22 is a free time yeah 22 times 2 plus 1 so it's massive so this picks will cease a lot yeah I wanted to make yeah okay hi we also with red measures and that this loop tried also to also you can try it against their the architrave inception is also one of the ones of you training is so so I think we're going to mean mark maker as the next guy oh one quick question and then um I don't know is I'm going to write a blog post these and then off to that I'll feel if I feel like competing to work on this is fine but I think we need to move on our next speaker so I'm mark [Applause]
Feedback