Bestand wählen
Merken

Deep Learning Blindspots

Zitierlink des Filmsegments
Embed Code

Automatisierte Medienanalyse

Beta
Erkannte Entitäten
Sprachtranskript
to it and
the and a uh and kind and the and by going cover from phytoestrogens often with thank you so much for the introduction and thank you so much for being here taking your time I know that Congress is really exciting so I really appreciate you spending some time with me today it's my 1st ever Congress so I'm also really excited and I want to meet new people so if you want come say hi to me later I'm I'm somewhat friendly so we can maybe be friends later on today on and talk about is deep learning blind spots or had a fool artificial intelligence I like to put artificial intelligence in quotes because the i will talk about that but I think it should be in quotes and that they were gonna talk a little bit about deep learning how it works
and how you can maybe fool so I ask us is a Iot becoming more intelligent and I ask this because when I open a browser and of course often it's Chrome and Google is already prompting me for what I should look at and I know that I work with the machine learning right and these are the headlines that I see every day are computers are already smarter than humans if so I think that it is pack up and go home right but we fix computers right of computer is smarter than me that IRT fixed it we can go home there's no need to talk about computers anymore blisters move on with life but so his budget that's not true right so we know because we work with computers and we know how stupid computers are sometimes they're pretty bad computers you only what we tell them to do generally so I don't think the computer can think and be smarter than me so with the same kinds of headlines that you see this then you also see the and yet
so Apple recently released their face ID and this unlocks your phone with your fakes and as I did a great idea right you have a unique faces you have a face nobody else can take your face but unfortunately what we find out about computers is that they're awful sometimes ends for these women for this Chinese woman that owned an iPhone her co-worker was able to unlock her and I think handwritten Karen talked about if you're here for the last talk we have a lot of problems in machine learning and 1 of them is stereotypes and prejudices that are within our training data or within our minds that we into our models and perhaps they didn't do adequate training data on determining different features of Chinese folks and perhaps it's other problems with the model or the training data or whatever they're trying to do but they clearly have some issues right so when somebody asks me is a agonistic over the world and is the Super Robot that's gonna come and you 9 new you know leader or so to speak I tell them we can even figure out the stuff that we already have in production so of we can even figure out the stuff we already have production I'm a little bit less worried then of the Super Robot coming to kill me that said unfortunately the
powers that be the powers that be a lot of times we believe in this and they believe strongly in artificial intelligence and machine learning and they're collecting data every day about you and me and everyone else and they're going to use this data to build even better models and this is because the revolution that we're seeing now in machine learning has really not much to do with new algorithms architectures it has a lot more to do with heavy computer and with massive massive datasets and the more that we have training data of petabytes per 24 hours or even lasts the more were able to essentially fix up the parts that don't work so well and the companies that we see here are companies are investing heavily in machine learning in a AI and part of how they're investing heavily is the collecting more and more data about you and me and everyone else Google and Facebook more than 1 billion active users I was surprised to know that in Germany the desktop search traffic for Google is higher than most of the rest of the world and 4 by do their growing with the speed that broadband is available and so what we see is these people are collecting this data and they also are using new technologies like GP use NTP use in new ways to paralyze workflows and read this they're able to mess up right they're still messing up but they mess up slightly less and they're not going to get on interested in this topic so we need to kind of start to
prepare how we respond to this type of behavior the and so 1 of the things that has been a big area of research actually also for a lot of these companies is what will talk about today and that's adversarial machine learning but the 1st thing that will start with is what is behind what we
call AI so most the time when you think of AI or something like URI and so forth you are actually potentially talking about what an old school rule-based system this is a rule like you see a particular thing and then series like yes I know how to respond to this and we even hard program these types of things in the right that is 1 version of a i is essentially it's been pre programmed to do and understand certain things another form that usually like for example for the people they're trying to build a i robots and the people that are trying to build what we call general AI so this is something that can maybe were like a human still use reinforcement learning I don't specialize in reinforcement learning but what it does is is it it essentially tries to reward you for behavior that you're expected to do so the completed task you gate of cookie you complete to other task you get 2 or 3 more cookies depending on how important the task it's and this will help you learn how to behave to get more points and as user robots in gaming and so forth but not really gonna talk about that today because most of that is still not really something that you were I interactive what I am going to talk about today is neural networks or as some people like to call them deep learning rate I'm so deep learning 1 the neural network forces deep learning battle while ago itself here is an example neural network we have an input layer and that's where we essentially make a quantitative version of whatever our data it's so we need to make it into numbers then we have a hidden layer and we might have multiple hidden layers and depending on how deep a network is or network inside a network right which is possible we might have very much different layers there and they may even act cyclical weights and then that that's for all the weights and the variables and the warning happens so that has a holds a lot of information and data that we eventually want to train their and finally we have an output layer and depending on the net and the network to what we're trying to do the output layer can vary between something I look for the input like for example the 1 to machine translate then I want the output to look like input right I wanted to just in a different language or the output could be a different class a can be here this is a call for word this is you know trade and so forth so really depends where you're trying to solve but the output layer gives us the answer and how we train this is we use backpropagation and bought back propagation is nothing new and neither is 1 of the most popular methods to do so which is called stochastic gradient descent and what we deal when we go through that part of the training is we go from the output layer we go backwards through the network size the propagation great and as we go backwards to the network we up so and and in the most simple way we upload and download what's working and what's not working so we say 0 you got it right you get a little bit more importance of you got wrong you get a little bit less importance and eventually we hope over time that they essentially correct each other's errors enough that we get a right answer so that's how I a very general overview of how works and the cool thing is is because it works that way we can
fool it and people have been researching ways to fool it for quite some time so give you a brief overview of the history of this field so we can kind of know where working from and maybe hopefully then were going to the in 2005 was 1 of the 1st most important papers to our approach adversarial learning it was written by a series of researchers and they wanted to see if they could act as an informed attacker and attack the linear classifier so this is just a spam filter and they're like can I can spend my friends I don't know what it wants you this but can I send stand my friends if I tried testing out a few ideas and what they were able to show is yes rather than just you know trial and error which anybody can do a brute-force attack of just and because e-mails and see what happens they were able to craft a few algorithms that they could use to try and find important words to change to make it go through the spam filter the In 2007 NIPS which is a very popular machine learning conference had 1 of their 1st all day workshops on computer security and when they did so they had a bunch of different people there working on machine learning in the computer security from our detection to network intrusion detection to of course spam and they also had a few talks on this type of adversarial learning to how you act as an adversary to your own model and then how do you learn how to counter that adversary in 2013 there was a really great paper that a lot of people's attention called poisoning attacks against support vector machines now support vector machines are essentially usually a linear classifier and we use them a lot to say on this is member of this class that another we pertain to text so I have a text and I wanna know what the text is about I wanna know if it's a positive or negative sentiment a lot of times so user support vector machine and we call ASEAN's as well and Batista video was the main researcher and he's actually written quite a lot about these poisoning attacks and it's the poison and the training data so for a lot of this is the system sometimes they have active learning and this means you I will classifier e-mails as spam were helping train the network and so he poisoned the training data and was able to show that by poisoning it in a particular way that he was able to then send spam e-mail because he knew what words were then benign essential he went on to study a few other things about biometric data if the interest in biometrics but then in 20 thousand 14 and Chris Getty in Goodfellow and a few other main researchers at Google brain released intrigue old intriguing properties of neural networks and that really became the explosion of overseeing today in adversarial learning and what they were able to do is they were able to say we believe there is linear properties of these neural networks even if they're not necessarily linear networks and we believe we can exploit them to fool them and they 1st introduced then the fast gradient sign method which will talk about later today the so how does
it work for someone has to get a little bit of an intuition around how this works the here's a graph it graphic of gradient descent In gradient descent arm where this vertical access is our cost function the and what we're trying to do is we're trying to minimize cost we want to minimize the air and so when we start out we just chose random weights and variables so all of our hidden layers they just have maybe random weights or random distribution and then we want to get to a place for the weights have meaning right we want our of networks who knows something even if it's just a mathematical pattern right so we start arm in the high area of the graph are more the reddish area and that's where we started we have high error there and then we try to get to the lowest area of the graph or here the dark blue that is right about here yeah but sometimes what happens and so as we learn as we go through epochs and training were moving slowly down and hopefully were optimizing but what we might end up in instead of this global minimum we might end up in the local minimum which is the other this fine because it still 0 error right so we're so probably going to be able to succeed but we might not gets the best answer all the time whatever still tries to do in the most basic of ways it is it is essentially tries to push the error rate back up the hill for as many units as it can so essentially tries to increase the error slowly through perturbations and by disrupting the let's say the weakest links like the 1 that did not find the global minimum but instead found a local minimum we can hopefully fool the network because we're finding those weak spots and were capitalizing on them essentially the this so what is an adversarial
example actually look like you may have already
seen this because it is very popular on the Twitter-sphere and a few other places but this is a series of researches at MIT and it was debated whether you could do add
various adversarial learning in the real world a lot of the research has just been a
still image and what they are able to show is they created a 3 D printed turtle amylose the turtle to you as well correct and this 3 D printed turtle by the inception network which is a very popular computer vision networks is a right and
it is a rifle in every angle that you can see and the way they were able to do this and I don't know the next time goes round as you
could see perhaps and is a little bit easier on the video which I have personal shared and you could see perhaps that there is a slight
discoloration of the shell and they messed with the texture and my messing with this texture and the colors they were able to fool the neural network variable to activate different neurons that were not supposed to be activated you I and so what we
see here is that it can be done in the real world and when I saw this I started getting really excited permits video surveillance is a real thing right so if
you start fooling 3 D objects we can perhaps start fooling other things in the real
world that we would like to full
and thank so why do adversarial examples exist the we're gonna talk a little bit about some things that are approximations of what's actually happening so please forgive me for not being always exacts but I would rather it's all have a general understanding of what's happening across the top row we have an input layer and these images to the left we can see or the source images and the source image is like a piece of farming equipment or something and on the right we have a died image this is over trying to get the network to see wanted to misclassified this farm equipment as a pink bird so what these researchers did is they targeted different layers of the network and they said OK we're going to use this method charted this particular and we'll see what happens and so as they targeted these different layers you can see what's happening on the internal visualization on neural networks can see right they're looking at matrices of numbers but what we can do is we can use those internal values to try and see with our human eyes what they are learning and we can see here clearly inside the network we no longer see the farming equipment right we see a thing Burke and you did it this is not visible for humanize he really study and if you enlarge the image you can start to see OK there's a little bit of pink here are greens I don't know what's happening but we can still see it in the neural network we have now people don't exactly know yet why these these blind exist so still an area of active research exactly why we can fool neural networks so easily there are some prominent researchers that believe that neural networks are essentially varied linear and that we can use this simple linearity to misclassified to jump into another area but there are others that believe that there's these pockets are blind spots and that we can then find these blind spots for these neurons really odd the weakest links and they may be even haven't learned anything as we change the activation then we can fool the network easily so this is still an area of active research and let's say you're looking for your thesis is to be a pretty neat
thing to work out so we'll get into just a brief overview of some of the math behind the most popular methods 1st here the fast gradient sign method and that is was used in the initial paper and numbers in many iterations on it what we do is we have our same cost function busses so this is the same way that were trying to train on network and it's trying to learn and we take the subgradient sign of that and it if you can think it's OK if you're not used to doing vector calculus and especially not with a pen and paper in front of you but what what you think we're doing is we're essentially trying to calculate some approximation of a derivative of the function and this can kind of tell us where the going and if we know where it's going we can maybe anticipate that and change and done but then for to create ever still images we then take the original input plus a small number epsilon times that gradient sign for the Jacobian finishing up this is a new were method and it's a little bit more effective but it takes a little bit more compute and so this jacobian saliency map uses a Jacobian matrix and if if you remember also it's OK if you don't have a Jacobian matrix will look at the 4 derivative of the function so you take the 4 derivative of a cost function and it gives you of nature at that at that vector and it gives you a matrix that is a point wise approximation if the function is differential at that input vector story you can review this later to but the Jacobian matrix then we used to create the saliency map the same way were trying to essentially find some sort of linear approximation or pointwise approximation and we could then want to find 2 pixels that we can perturb that cause the most disruption and then we continue to the next and unforeseen this isn't currently in O n square problem but there's a few people that are trying to essentially find ways that we can approximate this and make it faster so maybe
now you want to fool and networks you and I hope you do cutset forward and talk 1st you
need to pick a problem or network type so you may already know and but you may want to investigate what perhaps is this company using what perhaps is this method using and do a little bit of research because that's going to help you then you want to research state of the art methods and this is like a typical research statement that you have a new state of the art method but the good news is that the state of the art 2 to 3 years ago is most likely in production or in systems today so once they find ways to speed it up and they usually some approximation of that is deployed and a lot of times these are then publicly available models so a lot of times if you're already working with the deep learning framework they'll come prepackaged with a few of the different popular models so you can even use that if your body building networks of course you can build your own and optional step but 1 that might be recommended is to fine tune your model and what this means is to essentially take a new training dataset maybe data that you think this company using or that you think this network is using and you're going to remove the last few layers of the neural network going to retrain the essentially odd nicely piggybacking on the work of the pre-trained model then you're using the final layers to create finesse this essentially means makes your model better at the task that you have for the finally then you use a library and we'll go through a few of them but some of the ones that I had use myself is clever Hans the full and deep owning and these all come with a nice built-in features for you to use for let's say the fast gradient sign method the Jacobian saliency map and a few other methods are available finally if not can always work so depending on your source and the target you won't always necessarily find a match where researchers have shown is it's a lot easier to fall a network that a cat is a dog but there is a full network that a cat is an airplane is just like we can make these intuitive so you might wanna pick input that's not super dissimilar from where you want to go but is dissimilar enough the and you want test it locally and then finally test the ones with the highest of misclassification rates on the target network the and you might say
Catherine are you can contagion circuit are you might say I don't know why it's the person is using I don't know what the company is using and I'll it it's OK because what's the improve it is you can attach a black box model you do not have to know what they're using you do not have to know exactly how it works you don't even have to know the training data because what you can do is if it has OK addendum it has to have some API you can interface but if it has an API you can interface with or even any API you can interact with that uses the same type of learning you can collect training data by querying the and then your training your local model on that data that you're collecting so you're collecting the data you're training your local model and as your local model gets more accurate and more similar to the deployed a black box that you don't know how works you were then still able to fool it and what's are this paper prove Nicholas Popper not and a few other great researchers is that with the schools usually less than 6 thousand queries were able the full the network between 84 and 97 % certainty the and what that same group of
researchers also studied is the ability to transfer the ability of 1 1 network into another network and they call that transferability so I can take a certain type of network and I can use adversarial examples against this network to fool a different type of machine learning technique and here we have that the matrix the heat maps that shows us exactly what they were able to fool so we have a cross on the left-hand side here the source machine learning technique we have deep learning logistic regression as the arms like we talked about decision trees and k-nearest neighbors and across the bottom we have the target machine learning so what were the targeting the created adversaries with the left hand side and the targeted across the bottom we finally have an ensemble model at the end and what the older shows like for example as the arms and decision trees are quite easy to fool but logistic regression and a little bit less so but still strong for deep learning and k nearest neighbors if you train a deep learning model or a k-nearest neighbor model then that performs fairly well against itself and so what they're able to show is that you don't necessarily need to know the target machine you you don't even have to get it right even if you do know you can use a different type of machine learning technique to target the network yeah the so
we'll look at 6 lines of pipelined here and in the 6 lines of Python I'm using the clever hands library and in 6 lines of Python I can both generate my adversarial input and I can't even predict on so if you don't code Python our it's pretty easy to learn and pick up and but for example if you we have Carson cares is a very popular deep learning library in Python it usually works with this they are nowhere tensor flow back end and we can use wrapper model pass to the fast gradient method plants and it then set up some parameters so here's our epsilon and a few extra parameters this is to tune our adversary and finally we can generate or adversarial examples and then predict on them so in a very small amount of Python were able to target and a trick as network and if you're already using tensor flow or it can it already works with those libraries yeah
and deep owning is 1 of the 1st libraries heard about in this space there was presented at DEF CON in 20 16 and when it comes with this element of a tensor flow built-in code it even comes with the way that you can train the model yourself so it has a few different models a few different convolutional networks and these are predominantly used in computer vision we also however has a semantic model and I normally work in NLP it and I was pretty excited to try it out and when it comes adults with the is the rotten tomatoes sentiment so this is rotten Tomatoes movie reviews that try to learn is positive or negative so the original text i input in when i was generating may have a sale networks was more trifle then triumph which is the real reviews and the adversarial text that it gave me was Jonah refreshing hauntingly key yeah so I was able to form my network but I lost any type of meaning and this is really the problem or we think about how we apply adversarial learning to different tasks is is easy for an image if we make a few changes for it to retain its image right it's many many pixels but when we start going into language if we change 1 word and then another word and another word or maybe we change all the words we no longer understand as humans and I would say this is garbage in garbage out this is not actual adversarial learning so we have a long way to go when it comes to Language tasks and being do adversarial learning and there is some research in this but it's not really advance yet so hopefully this is something that we can continue to work on and advance further and if so we need to support a few different types of networks that are more common in NLP than they are in computer vision there's some other
notable open source libraries that are available to you and I'll cover just a few here there's the vendor Koch Computational Economics Research land that has had and this allows you to poisoning attacks so if you want to target training data and poison it then you can do so with that any users cycle I deep fault allows you to do the fast gradient sign method but it tries to do smaller pertubations it tries to be less detectable to us humans it is based on the honor which is another a library they believe uses Lua as well as Python full box is kind of neat because they only heard about last week that it collects a bunch of different techniques all-in-one library and you can use it with 1 interface 51 1 experiment with a few different ones at once I would recommend taking a look at that and finally for something I will talk about briefly in a short period of time we have evolving AI Lab which release of fooling our library and this fooling library is able to generate images that you or I can't tell what it is but that the neural network is convinced it is something so this will talk about maybe some applications of this in a moment but they also open source all of the code and the researches you open source their code which is always very exciting
as you may have known from some of the reach of the research area cited most of the studies and the research in this area has been on malicious attacks so there's very few people try to figure out how to do this for what I would call benevolent purposes most of them are trying to act as an adversary in the traditional computer security sense there perhaps studying spam filters and how spammers can get by them there perhaps looking at network intrusion a botnet attacks and so forth the perhaps looking at self-driving cars so and I know that was referenced earlier solid handwritten can't talk to perhaps trying to make a yield sign look like a stop sign or a stop sign look like a yield sign or speed limit and so forth and it's scarily they're quite successful at this the or perhaps a looking at data poisoning so how do we poison the model so we render it useless in a particular context so we can utilize SAP and finally for malware so what a few researchers were able to show is by just changing a few things in the malware they were able to upload their malware to Google mail and send it to someone and this is still fully functional now are in that same sense there's the noun Gan project which uses a generative adversarial network to create malware that works I guess so there's a lot of research of these kind of malicious attacks been adversarial learning but what
I wonder is how might we use this for good and evil good in quotation marks because we all have different ethical and moral systems we use and which you may decide is ethical for you might be different but I think as a community especially at a conference like this hopefully we can converge on some ethical privacy concerns version of using these networks so I have composed
a few ideas and I hope that this is just the starting list of a longer conversation 1 idea is that we can perhaps to use this type of adversarial learning to fool surveillance so as surveillance effects you and I it even disproportionally affects people that most likely can't be here and so whether or not we're personally affected we can care about the many lines that are affected by this type of surveillance and we can try and build ways to fool surveillance systems stenography so we could potentially in a world where more and more people have less of a private we have sending messages to 1 another we can perhaps use adversarial learning to send private messages that were fully so again where I might have quite a lot of privilege and I don't actually see ads that are predatory on me as much there's a lot of people in the world that she's predatory advertising and so how can we help those problems by developing adversarial techniques poisoning your own private data so this depends on whether you actually use the surface and whether you like how the services helping you with the machine learning but if you don't care or if you need it essentially have the burn box of your data then potentially you could poison your own private data and finally I want us to use it to investigate deployed models so even if we don't actually need to use for fooling this particular network the more we know about was deployed and how we can fool at the more able to keep up with this technology as it continues to evolve so the more they're practicing the more they're ready for whatever might happen next and finally I really want to hear your ideas as well so I'll be here that the whole Congress and of course you can share during a 2 and a time if you have great ideas I really want to see them the so I decided to
play around a little bit with that some of my ideas and highly was convinced perhaps that I could make Facebook think I was a cat this is my goal can Facebook of endemic I can see nobody really likes community on straight but I have to be on it because my mom messages in their engineers e-mailing more self on Facebook I know it's so I use a pre-trained inception model and careless and I fine the layers and I'm not a computer vision person really but it to me like a day of figuring out how computer vision people to transfer the data into something like a put inside of the network figure that out and I was able to quickly train a model and the model could only distinguish between people cats that all the modern you have to do I give a picture is it's a person not a cat I have no idea I actually didn't try just giving an image of something else values just as a person or a cat maybe 50 50 minutes and so what I did was I use an image of myself and eventually I had my task reassignment method I use clever once and I was able to slowly increase epsilon and so the epsilon as a slow are move you you and I can't see the perturbations but also the Napa consider the perturbations so we need increase it and course as we increase the roma using a technique like f GSM we're also increasing the noise that we see and when I got to point 2 1 epsilon and I can't of learning interface for investigating and you wanna tag no at all and just testing finally if I at a point to 1 and so on in the no longer knew I was if the size is the book has a cat book maybe and thank so some firstly as initiating actually become attached to them pretty neat to come by I did you I was able to fool it I spoke with the computer vision specialists that I know injection works in this and I was like what methods think Facebook is using like that I really full all network or whatever I do and she's convinced most likely that they're actually using a statistical method called Viola-Jones which takes a look at the statistical distribution of their face and tries to guess if there isn't really a face there but what I was able to show transferability is that I can use my neural network even to fool this statistical model self and now I have a very noisy but happy photo on Facebook another use case
potentially is adversarial stenography and I was really excited reading this paper what this paper covered and they actually release a library as I mentioned is the study the ability of a neural network to be convinced that something's there that's not actually there and what they use of the NIPS training set I'm sorry if that's a good trigger-word a file if you've used and this 2 million times then I'm sorry for this but with the use and missed which is 0 through 9 of digits and what they were able to show using evolutionary networks if they were able to generate things that tasks look maybe like part and they actually use it on the sea for a dataset to which has colors and it was quite beautiful some of what they created effect they showed it in a gallery and what the network sees here is the digits across the top of the CAE bad digit there are more than 99 % convinced that that digit is there and what we see is pretty patterns or just noise and when I was reading this paper I was thinking how can we use this to send messages to each other that nobody else will know is their understanding and really nice man man artists in this my art and sharing it with my friend ends in a world where I'm afraid to go home because there is a crazy person in charge and I'm afraid that they might look at my phone in my computer and a million other things and I just want to make sure that my friend has been can number or this or that whatever I see use case for my life but again I leave a fairly produce privileged life there are other people were the actual life and livelihood insecurity might depend on using a technique like this and I think we could use still learning to create a new form of stenography the finally I
can not impressed enough that the more information we have about the systems that we interact with every day that a machine learning systems the AI systems or whatever you wanna call it that a deep networks the more information we have the better we can fight but we don't need perfect knowledge but the more knowledge that we have the better encompasses we can be if I think fully now live in Germany and if you're also European resident we have GP on which is the General Data Protection Regulation and it goes into effect in May of 20 18 and we can use GDP or how to make requests about our data we can use GDP are to make requests about machine learning systems that we interact with this is a right that we have and a recital 71 of the GDP or it states the data subject should have the right to not be subject to a decision which may include a measure for evaluating personal aspects related to him or her which is based solely on automated processing and which produces legal effects concerning him or her or similarly significant effects him or her such as automatic refusal of an online credit application or recruiting practices without any human intervention and I'm a lawyer and I don't know how this will be implemented and it's a recital so we don't even know it will reinforce the same way but the good news is pieces of this same sentiment or in the actual amendments and if they're in the amendments then we can legally use that and what he also says is we can ask companies to port our data other places we can ask companies to do meet our data we can ask for information about a half hour data is processed we can ask for information about what different automated decisions are being made and the more we all here ask for that data the more we can also share the same information with people worldwide because you know the systems that we interact with the not special to us the the same types of systems that are being deployed everywhere in the world so we can help our fellow humans outside of Europe by being good caretakers and using our right to make more information available to the entire world and use this information to find ways to use adversarial learning to full these types of systems and if it so how else might we be able to harness forget I cannot focus enough on GDP or and a right to collect more information about the information there already collecting about us and everyone else to use it splits find ways to share the information gained from it so I don't want to just be that 1 person requested and they learn something we have to find ways to share this information 1 test low-tech ways so I'm begin to you know I'm so excited that the makers face here in maker culture and other low tech human crafted ways to full networks we can use adversarial learning perhaps to get good ideas on how to form networks still lower-tech wastes where painted red pickles pixels all over my face west of you recognize what I'm not experiment with things that we learned from Comercio learning and try to find other lower-tech solutions to the same problem finally or nearly finally we need to increase the research beyond just computer vision quite a lot of adversarial learning has been only in computer vision and why I think that's important and has also been very practical because we can start to see how we can fool something we need to figure out natural language processing we need to figure out other ways that machine learning systems are being used for me to come up with clever ways to for the finally spread the word so I don't want the conversation and here I don't want the conversation and that Congress I want you to go back to your hacker collective your local CCC and the people that you talk with your co-workers and I want you to spread the word I want to do workshops on Adversarial learning I want more people to not treat this AI as something mystical and powerful because unforeseen is powerful but it's not mystical so we need to demystify the space we need to experiment we need to hack on it and we need to find ways to play with it and spread the word to other people finally I really wanna hear your other ideas and before I leave today I have to say a little bit about why I decided to join the resiliency track this year I read about the resiliency track and I was really excited it spoke to me and I said I want to live in a world where even if there's an entire burning trash fire around me I know that there are other people that I care about that I can count on that I can work with to try at least protect portions of our world to try and protect ourselves to trample protects people that do not have as much privilege so what I wanna be a part of it is something that can use maybe the skills I have the skills you have to do something with that and your data is the biggest source of value for everyone any free service you use they are selling your data get I donor for a fact but it is very sorted I feel very certain about the fact that their most likely selling your data if they're selling your data that they might also be buying your data and there's a whole market that's legal that's freely available to buy and sell your data and they make money off of that and the mind more information and make more money off of that and so forth so how will read ahead a little bit of my opinions put forth on this determine who you share your data and for what reasons GPR and data portability give us European residents stronger rights than most of the world with use them let's choose privacy concerns and ethical data companies over corporations are entirely built on selling ads was build start organisations open-source tools and systems that we can be truly proud of and let's port are dated to do so and some of those we have time for a few months some unintended sorry it's on the 0 the no big deal the show of machine learning that closing remarks as brief brief round-up of close arises that machine learning is not very intelligent I think artificial intelligence is a misnomer and a lot of ways but this doesn't mean that people are going to start using it in fact there's very smart powerful and rich people that are investing more than ever in it so it's not going anywhere and it's going to be something that potentially becomes more dangerous over time because as we hand over more of these 2 systems are moving could potentially control more more of our life we can use however adversarial machine learning techniques to find ways to fool box networks so we can use these and we know we don't have to have perfect knowledge however the information is powerful and the more information that we do have 2 more able to become a good GPA are the adversary so please use GPR unless discuss ways we can share information finally please support open source tools and research in the space was we need to keep up with where the state of the art is so we need to keep ourselves moving in open in that way and please support ethical data companies or start 1 if you come to me say Catherine I'm going to charge you this much money but I will never sell your data and I will never buy your data I would much rather you handle media so I want us especially those within the EU to start a new economy around trust and privacy and ethical duties and thank you very much because
of this and what could I use you know from 1 of the new 911 come up so 1 of the mind and the observations and the kind of thing was might want it is very interesting talking to some 1 impression that I got during the talk was with the this sorry learning approach on we just doing and testing and quality assurance of full of AI companies and it is going to build better machines that's a very good question and costs most of this research right now is coming from those companies because they're worried about this what however they have shown is they don't really have a good way to flew to learn how to fool this come most likely they will need to use a different type of network eventually so probably whether it's the blind spots on the linearity of these networks they are easy to fool and I'll have to come up with a different method for generating something that is robust enough to not be tracked around so he to some degree yes is a cat-and-mouse game right arm but that's why I want the research and open source to continue as well and I would be highly suspect if they also and figure out a way to make a neural network which has proven linear relationships that we can exploit the nonlinear and so it's usually a different type of networks as a lot more expensive to train in that doesn't actually generalize well so we're going to really hit them in a way were they going to have to be more specific try harder and I would rather do that than just kind of give up and so I do the look of singular so talk about as you have a variety of looking at it from the other direction like uh just trying to remove the 2 companies falsely classified data uh and um just through so many with so many of them also of data so that they learn from it from NASA these poisoning attacks so many times about policing in tax essentially ceding bad training data and were trying to get them to learn about the source of I when the bad things that we're trying to get them to learn false information and uh that only happens on accident all the time so I think the more to we can if we share information and they have a publicly available API where they're actually actively learning from our information yes I would say poisoning is a great attacks way and we can also share information of maybe how that works so especially I would be freed if we could do poisoning for hardware and malicious ad targeting on acutely 1 more question from the internet and then we run out of time so I know that I often feel that you want to do and so that's candidate to harm with against the residues of sorry what can do to harm someone model against that was real samples not much on what they have shown is that if you train on a mixture of real training data an adversarial data it's a little bit harder to fool by that just means that you have to try more iterations of adversarial input so right now the recommendation is to train on a mixture of adversarial and real training data and to continue to do that over time I'm and I would argue that you need to maybe do data validation on input and if you do data validation input maybe you can recognize abnormalities but that's because I come from mainly like productions levels not theoretical and I think maybe she just testing from the if they look weird you should maybe not take them into the system and that's all for the presence of this in mind but the the owners please do for the and and thank was it that
and and a and a and the it took place in the it it but
Bit
Künstliche Intelligenz
Toter Winkel
Neuronales Netz
Überlagerung <Mathematik>
Videospiel
Subtraktion
Bit
Wellenpaket
Browser
Biprodukt
Unternehmensmodell
Roboter
Virtuelle Maschine
Informationsmodellierung
Soft Computing
Künstliche Intelligenz
Datenverarbeitungssystem
Rechter Winkel
Vererbungshierarchie
Benutzerführung
Algorithmische Lerntheorie
Virtuelle Maschine
Facebook
Soft Computing
Algorithmus
Wellenpaket
Flächeninhalt
Datentyp
Mereologie
Statistische Analyse
Ruhmasse
Facebook
Algorithmische Lerntheorie
Unternehmensmodell
Leistung <Physik>
Neuronales Netz
Bit
Punkt
Ausbreitungsfunktion
Formale Sprache
Versionsverwaltung
Computer
Gradient
Videokonferenz
Gradient
Algorithmus
Vorzeichen <Mathematik>
Computersicherheit
Gradientenverfahren
Neuronales Netz
E-Mail
Funktion <Mathematik>
Kategorie <Mathematik>
Computersicherheit
Reihe
Systemaufruf
Programmierumgebung
Ein-Ausgabe
Bitrate
Support-Vektor-Maschine
Soft Computing
Datenfeld
Verknüpfungsglied
Forcing
Rechter Winkel
Stochastik
Ein-Ausgabe
Gradientenverfahren
Information
Biostatistik
Fehlermeldung
Stochastik
Lineare Abbildung
Subtraktion
Gewicht <Mathematik>
Wellenpaket
Klasse <Mathematik>
Vektorraum
Zahlenbereich
Task
Virtuelle Maschine
Informationsmodellierung
Bildschirmmaske
Variable
Eindringerkennung
Spieltheorie
Datentyp
Optimierung
Algorithmische Lerntheorie
Logische Programmierung
Backpropagation-Algorithmus
Schlussregel
Physikalisches System
Vektorraum
Roboter
Mereologie
Cookie <Internet>
Wort <Informatik>
Bestärkendes Lernen <Künstliche Intelligenz>
Neuronales Netz
Diskrete Wahrscheinlichkeitsverteilung
Bit
Wellenpaket
Gewicht <Mathematik>
Graph
Mathematik
Extrempunkt
Datenmodell
Bitrate
Binder <Informatik>
Hill-Differentialgleichung
Arithmetisches Mittel
Turtle <Informatik>
Variable
Einheit <Mathematik>
Flächeninhalt
Rechter Winkel
Kostenfunktion
Mustersprache
Gradientenverfahren
Randomisierung
Festplattenlaufwerk
Neuronales Netz
Fehlermeldung
Turtle <Informatik>
Drall <Mathematik>
Soft Computing
Winkel
Turtle <Informatik>
Datenmodell
Reihe
Unrundheit
Festplattenlaufwerk
Maschinelles Sehen
Bildgebendes Verfahren
Neuronales Netz
Bit
Nabel <Mathematik>
Datenmodell
Winkel
Videokonferenz
Objekt <Kategorie>
Textur-Mapping
Turtle <Informatik>
ROC-Kurve
Rechter Winkel
Reelle Zahl
Kantenfärbung
Festplattenlaufwerk
Neuronales Netz
Matrizenrechnung
Bit
Punkt
Natürliche Zahl
Mathematisierung
Matrizenrechnung
Iteration
Toter Winkel
Zahlenbereich
Derivation <Algebra>
Gradient
Statistische Hypothese
Gradient
Open Source
Vektoranalysis
Datensatz
Vorzeichen <Mathematik>
Kostenfunktion
Visualisierung
Bildgebendes Verfahren
Lineares Funktional
Approximation
Matrizenring
Pixel
Green-Funktion
Vorzeichen <Mathematik>
Vektorraum
Quellcode
Ein-Ausgabe
Binder <Informatik>
Quick-Sort
Linearisierung
Mapping <Computergraphik>
Quadratzahl
Funktion <Mathematik>
Flächeninhalt
Rechter Winkel
Derivation <Algebra>
Pixel
Grenzwertberechnung
Neuronales Netz
Sichtbarkeitsverfahren
Bit
Wellenpaket
Unternehmensmodell
Framework <Informatik>
Eins
Gradient
Task
Informationsmodellierung
Softwaretest
TUNIS <Programm>
Unternehmensmodell
Vorzeichen <Mathematik>
Datentyp
Neuronales Netz
Softwaretest
Befehl <Informatik>
Approximation
Matching <Graphentheorie>
Gebäude <Mathematik>
Stichprobe
Datenmodell
Ähnlichkeitsgeometrie
Physikalisches System
Quellcode
Ein-Ausgabe
Bitrate
Biprodukt
Mapping <Computergraphik>
Programmbibliothek
Computerunterstützte Übersetzung
Neuronales Netz
Aggregatzustand
Matrizenrechnung
Bit
Subtraktion
Wellenpaket
Logistische Verteilung
Blackbox
Maschinelles Lernen
Wärmeübergang
Netzwerktopologie
Virtuelle Maschine
Informationsmodellierung
Lineare Regression
Gruppe <Mathematik>
Datentyp
Minimum
Logistische Verteilung
Meta-Tag
Lineare Regression
Abfrage
Systemplattform
Quellcode
Entscheidungstheorie
Mapping <Computergraphik>
Zahlenbereich
Digitaltechnik
Neuronales Netz
Subtraktion
Formale Sprache
Mathematisierung
Element <Mathematik>
Unternehmensmodell
Code
Raum-Zeit
Gradient
Task
Spezialrechner
Informationsmodellierung
Unternehmensmodell
Tensor
Randomisierter Algorithmus
Datentyp
Programmbibliothek
Neuronales Netz
Maschinelles Sehen
Gerade
Bildgebendes Verfahren
Parametersystem
Pixel
Ein-Ausgabe
Datenfluss
Arithmetisches Mittel
Soft Computing
Rechter Winkel
Tensor
Wort <Informatik>
Message-Passing
Neuronales Netz
Grenzwertberechnung
Support-Vektor-Maschine
Soft Computing
Stereometrie
Subtraktion
Wellenpaket
Quader
Momentenproblem
Randwert
Kartesische Koordinaten
Code
Eins
Gradient
Spezialrechner
Open Source
Informationsmodellierung
Vorzeichen <Mathematik>
Programmbibliothek
Inverser Limes
Neuronales Netz
Störungstheorie
Bildgebendes Verfahren
Schnittstelle
Beobachtungsstudie
Filter <Stochastik>
Computersicherheit
Open Source
Malware
Digitalfilter
Malware
Bayes-Netz
Kontextbezogenes System
Frequenz
Soft Computing
Flächeninhalt
Gmail
Gmail
Dreiecksfreier Graph
Projektive Ebene
Versionsverwaltung
Neuronales Netz
Soundverarbeitung
Datenmissbrauch
Umsetzung <Informatik>
Quader
Versionsverwaltung
Mailing-Liste
Physikalisches System
Automatische Differentiation
Unternehmensmodell
Virtuelle Maschine
Dienst <Informatik>
Unternehmensmodell
Datentyp
Versionsverwaltung
Instant Messaging
Message-Passing
Gerade
Neuronales Netz
Facebook
Bit
Wellenpaket
Punkt
Existenzaussage
Taupunkt
Zahlenbereich
Geräusch
Wärmeübergang
Task
Bildschirmmaske
Informationsmodellierung
Digitale Photographie
Klon <Mathematik>
Mustersprache
Programmbibliothek
Neuronales Netz
Maschinelles Sehen
Figurierte Zahl
Bildgebendes Verfahren
Gerade
Metropolitan area network
Schnittstelle
Beobachtungsstudie
Soundverarbeitung
Diskrete Wahrscheinlichkeitsverteilung
Videospiel
Statistik
Statistische Analyse
Störungstheorie
Elektronische Publikation
Soft Computing
Digitalisierer
Mereologie
Injektivität
Kantenfärbung
Computerunterstützte Übersetzung
Mustererkennung
Message-Passing
Lesen <Datenverarbeitung>
Grenzwertberechnung
Neuronales Netz
Bit
Umsetzung <Informatik>
Prozess <Physik>
Iteration
Kartesische Koordinaten
Twitter <Softwareplattform>
Information
Raum-Zeit
Entscheidungstheorie
Richtung
Internetworking
Übergang
Spezialrechner
Softwaretest
Rechenschieber
Hacker
Neuronales Netz
Maschinelles Sehen
Einflussgröße
Nichtlineares System
Softwaretest
Hardware
Korrelation
Prozess <Informatik>
Gebäude <Mathematik>
Quellcode
Biprodukt
Ein-Ausgabe
Natürliche Sprache
Web log
Entscheidungstheorie
Linearisierung
Zusammengesetzte Verteilung
Arithmetisches Mittel
Dienst <Informatik>
Soft Computing
Rechter Winkel
Information
Varietät <Mathematik>
Aggregatzustand
Subtraktion
Wellenpaket
Kontrollstruktur
Quader
Zustandsmaschine
Selbst organisierendes System
Soundverarbeitung
Toter Winkel
Maschinelles Lernen
Datenmissbrauch
Systemprogrammierung
Open Source
Virtuelle Maschine
Informationsmodellierung
Weg <Topologie>
Unternehmensmodell
Reelle Zahl
Spieltheorie
Randomisierter Algorithmus
Datentyp
Stichprobenumfang
Luenberger-Beobachter
Algorithmische Lerntheorie
Mobiles Endgerät
Ganze Funktion
Leistung <Physik>
Soundverarbeitung
Videospiel
Datenmissbrauch
Raum-Zeit
Open Source
Validität
Automatische Differentiation
Physikalisches System
Fokalpunkt
Singularität <Mathematik>
Minimalgrad
Mereologie
Hypermedia
Wort <Informatik>
Neuronales Netz
Hypermedia
Medianwert
Systemprogrammierung

Metadaten

Formale Metadaten

Titel Deep Learning Blindspots
Untertitel Tools for Fooling the "Black Box"
Serientitel 34th Chaos Communication Congress
Autor Jarmul, Katharine
Lizenz CC-Namensnennung 4.0 International:
Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen.
DOI 10.5446/34794
Herausgeber Chaos Computer Club e.V.
Erscheinungsjahr 2017
Sprache Englisch

Inhaltliche Metadaten

Fachgebiet Informatik
Abstract In the past decade, machine learning researchers and theorists have created deep learning architectures which seem to learn complex topics with little intervention. Newer research in adversarial learning questions just how much “learning" these networks are doing. Several theories have arisen regarding neural network “blind spots” which can be exploited to fool the network. For example, by changing a series of pixels which are imperceptible to the human eye, you can render an image recognition model useless. This talk will review the current state of adversarial learning research and showcase some open-source tools to trick the "black box."
Schlagwörter Resilience

Zugehöriges Material

Video wird in der folgenden Ressource zitiert

Ähnliche Filme

Loading...
Feedback