CAAD VILLAGE - GeekPwn - The Uprising Geekpwn AI/Robotics Cybersecurity Contest U.S. 2018 - Practical adversarial attacks against challenging models environments

Video thumbnail (Frame 0) Video thumbnail (Frame 720) Video thumbnail (Frame 1509) Video thumbnail (Frame 2225) Video thumbnail (Frame 2915) Video thumbnail (Frame 3611) Video thumbnail (Frame 4650) Video thumbnail (Frame 5433) Video thumbnail (Frame 6267) Video thumbnail (Frame 7644) Video thumbnail (Frame 9368) Video thumbnail (Frame 11322) Video thumbnail (Frame 12723) Video thumbnail (Frame 14024) Video thumbnail (Frame 15027) Video thumbnail (Frame 15900) Video thumbnail (Frame 16845) Video thumbnail (Frame 17618) Video thumbnail (Frame 18501) Video thumbnail (Frame 19621) Video thumbnail (Frame 22458) Video thumbnail (Frame 23361) Video thumbnail (Frame 25254) Video thumbnail (Frame 26478) Video thumbnail (Frame 28238) Video thumbnail (Frame 29420) Video thumbnail (Frame 30051) Video thumbnail (Frame 32866) Video thumbnail (Frame 34779) Video thumbnail (Frame 36385) Video thumbnail (Frame 37287) Video thumbnail (Frame 39106) Video thumbnail (Frame 39955) Video thumbnail (Frame 41444) Video thumbnail (Frame 43037) Video thumbnail (Frame 44207) Video thumbnail (Frame 45898) Video thumbnail (Frame 47552)
Video in TIB AV-Portal: CAAD VILLAGE - GeekPwn - The Uprising Geekpwn AI/Robotics Cybersecurity Contest U.S. 2018 - Practical adversarial attacks against challenging models environments

Formal Metadata

Title
CAAD VILLAGE - GeekPwn - The Uprising Geekpwn AI/Robotics Cybersecurity Contest U.S. 2018 - Practical adversarial attacks against challenging models environments
Title of Series
Author
License
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
2018
Language
English

Content Metadata

Subject Area
Abstract
Moustafa Alzantot is a Ph.D. Candidate in Computer Science at UCLA. His research interests include machine learning, privacy, and mobile computing. He is an inventor of two US patents and the recipient of several awards including the COMESA 2014 innovation award. He worked as an intern at Google, Facebook, and Qualcom. Yash Sharma is a visiting scientist at Cornell who recently graduated with a Bachelors and Masters in Electrical Engineering. His research has focused on adversarial examples, namely pushing the state-of-the-art in attacks in both limited access settings and challenging domains. He is interested in finding more principled solutions for resolving the robustness problem, as well as studying other practical issues which are inhibiting us from achieving AGI.
Collaborationism Integrated development environment Different (Kate Ryan album) Chemical equation Integrated development environment Cartesian coordinate system Row (database)
Machine learning Algorithm Artificial neural network Multiplication sign Virtual machine Artificial intelligence Computer simulation Video game Endliche Modelltheorie Cartesian coordinate system Product (business)
Machine learning Group action Algorithm Forcing (mathematics) Surface Order (biology) Bit
Medical imaging Virtual machine Bus (computing) Object (grammar) Musical ensemble Address space Machine vision 2 (number) Neuroinformatik
Point (geometry) Arithmetic mean Machine learning Virtual machine Computer simulation Measurement Machine vision Neuroinformatik Task (computing)
Medical imaging Game controller Virtual machine Bus (computing) Division (mathematics)
Endliche Modelltheorie output Arc (geometry)
Domain name Noise (electronics) Ferry Corsten State of matter Confidence interval Direction (geometry) Weight Gradient Quantification Commutator Computer simulation Computer network Insertion loss Axiom Number Wave packet Medical imaging Bus (computing) Right angle Game theory Musical ensemble Object (grammar) output
Group action Stapeldatei Image resolution Gradient Electronic mailing list Shared memory Virtual machine Plastikkarte Physicalism Insertion loss Water vapor Benchmark Medical imaging Arithmetic mean Digital photography Software Software Energy level Electric current Library (computing) Gradient descent Physical system Condition number
Cue sports Pattern recognition Source code Virtual machine Physicalism Limit (category theory) Formal language Formal language Medical imaging Machine learning Internet forum Business model Speech synthesis Endliche Modelltheorie Endliche Modelltheorie
Medical imaging Noise (electronics) Closed set Computer-generated imagery Maxima and minima Error message
Medical imaging Gradient Weight Gradient Order (biology) Maxima and minima Water vapor
Asynchronous Transfer Mode Group action Ferry Corsten State of matter Weight Gradient Heat transfer Computer simulation Bit Black box Computer simulation Substitute good Wave packet Recurrence relation Substitute good Medical imaging Query language Endliche Modelltheorie Resultant
Virtual machine Computer simulation Parameter (computer programming) Black box Computer simulation Number Orbit Architecture Query language Function (mathematics) Query language Natural language Endliche Modelltheorie
Group action Algorithm Randomization Algorithm Gradient Gradient Workstation <Musikinstrument> Parameter (computer programming) Computer simulation Neuroinformatik Number Architecture Query language Internetworking Function (mathematics) Query language
Complex (psychology) Empennage Randomization State of matter Combinational logic Computer simulation Product (business) Medical imaging Mathematics Chromosomal crossover Selectivity (electronic) Endliche Modelltheorie Arc (geometry) Social class Area Algorithm Scaling (geometry) Inheritance (object-oriented programming) Prisoner's dilemma Gradient Fitness function Perturbation theory Vector space Personal digital assistant Natural language Spacetime
Scaling (geometry) Image resolution Computer-generated imagery Gradient Expression Valuation (algebra) Computer simulation Bit rate Streaming media Computer simulation Graph coloring Number Medical imaging Arithmetic mean Performance appraisal Different (Kate Ryan album) Term (mathematics) Personal digital assistant Query language Social class
Multiplication sign Direction (geometry) Channel capacity Computer simulation Power (physics) Wave packet Web 2.0 Medical imaging Performance appraisal Different (Kate Ryan album) Single-precision floating-point format Query language Reduction of order Software testing Feature space Endliche Modelltheorie Mathematical optimization Hydraulic jump Distribution (mathematics) Key (cryptography) Channel capacity Sampling (statistics) Computer simulation Demoscene Query language
Standard deviation Boss Corporation Electric generator Validity (statistics) Channel capacity Computer-generated imagery Sampling (statistics) Computer simulation Insertion loss Computer simulation Flow separation Wave packet Process (computing) Machine learning Query language Musical ensemble Endliche Modelltheorie Musical ensemble
Transformation (genetics) Differential (mechanical device) Gradient Connectivity (graph theory) 1 (number) Code Number Product (business) Medical imaging Goodness of fit Different (Kate Ryan album) Green's function Office suite Mathematical optimization Differentiable function Randomization Greedy algorithm Key (cryptography) Gradient Sampling (statistics) Computer simulation Bit Line (geometry) Instance (computer science) Limit (category theory) Stochastic Flux
Pixel Building Transformation (genetics) Differential (mechanical device) Gradient Multiplication sign Drop (liquid) Number Medical imaging Inference Mathematics Different (Kate Ryan album) Natural number Software testing Mathematical optimization Texture mapping Computer simulation Maxima and minima Limit (category theory) Formal language Word Personal digital assistant Infinite conjugacy class property Endliche Modelltheorie Natural language Table (information) Bounded variation
Context awareness Pixel Constraint (mathematics) Direction (geometry) Computer-generated imagery Formal language Product (business) Time domain Word Medical imaging Mathematics Bit rate Pixel Mathematical optimization Formal grammar Algorithm Constraint (mathematics) Scaling (geometry) Discrete group Gradient Fitness function Continuous function Formal language Word Personal digital assistant Uniformer Raum Natural number Natural language
Ocean current Point (geometry) Context awareness 1 (number) Maxima and minima Computer simulation Mereology Semantics (computer science) Wave packet Formal language Number Googol Spacetime Einbettung <Mathematik> Context awareness Addition Constraint (mathematics) Computer simulation Formal language Word Prediction Einbettung <Mathematik> Speech synthesis Right angle Quicksort Resultant
Predictability Boss Corporation Pattern recognition Confidence interval Computer simulation Set (mathematics) Mathematical analysis Open set Line (geometry) Semantics (computer science) Hypothesis Mathematics Average Network topology Speech synthesis Metropolitan area network Vulnerability (computing)
Pattern recognition Pattern recognition Algorithm Virtual machine Plastikkarte Set (mathematics) Database transaction Black box Revision control Googol Computer configuration Iteration Infinite conjugacy class property Speech synthesis Cuboid Codec Endliche Modelltheorie Mathematical optimization God Speech synthesis
Area Source code Noise (electronics) Pattern recognition Observational study Closed set Computer-generated imagery Source code Computer simulation Water vapor Bit rate Open set Plastikkarte Limit (category theory) Medical imaging Performance appraisal Average Computer configuration Personal digital assistant Speech synthesis Condition number Form (programming)
Source code Infinity Endliche Modelltheorie Quicksort Formal language
a lawyer for one minute themselves onto and beach you kind of it at u.c.l.a.. i mean he was young sherman a visitor record no. although they about but back to a set text in challenging environments is a joint walk with my advice of before so many of the stuff of from usually should who shot forty from i.b.m. research and some swami from the r l and or all of collaborators. of course everyone here intelligence most that artificial intelligence is being a big role in our last day a disused in different applications like silver driving cars.
and he has made its appearance and production. you know just move the public many of these applications are so critical and fifty clinical applications so we need to have an accurate picture of indulgence more more this.
and most of these models of artificial intelligence to life in what's called machine learning which starts with having. sake of getting data the new obligating i dressed or lebanon. to any data eleven a model and says more than a post on the stand was the buttons that exist in securing data and then jobless little too distant a data and france time.
this is a little bit more than is often do it as a surface who would like use of skin feet some new with this data and this data will reduce and i would but action from the mother but if we have an eye doctor or a malicious user he can manipulate the new data. a lot of two forces in order to produce some to apples.
here's an example of course who know that all humans are put in.
but the. i was out but i recognise and objects so it's easy but i don't want to tell which image the first images a band and second one is a school bus even a small market can do this task and do it very well but not only humans are good at english about now machines are also getting better.
so the existing computer vision address can also tell like the first images of banter and second one is a school bus and the gang do this was very high accurately it was even.
two thousand and fourteen that the best measure leveling the model for computer vision. said it could achieve the human activity with the same that is so it can have a super human activity. but does this mean that we are now at the point at which we can rely on machine leavening in doing or tasks given that the doctors are mostly fifty and critical patients and. machine learning can do it even more accurate and human.
let's get this example is a the same image we have shown before the human and machine both of them can recognize it correctly. and if there is a malicious user or the like are caught they can add some small blue cleverly commuted the division and oysters image such as that there isn't it still looks for me as a school bus so anyone who looks as i would imagine someone i knew was that i would say ok still us.
the school bus but then she let me more than who previously recognized the original image correctly was very high accurate to see it will now be fooled i could say this isn't a stretch and sex a serious threat because i talked and even have control over what would be the albert.
misclassification label boys on by some she lending models so he can force is a mother to produce that i would he want to say to get it. the idea is known as i say the attacks what i personally examples.
was first discovered in two thousand and fourteen bicycle which have not just that. instead we were called and growing but what's on your not what they have much is that small butted nation is and what can lead to significant i would change his art.
and for example when the exits more noise to this school but image it will now be but it could take as an ostrich state of the school bus and axiom year i could follow he started studying this phenomena more and more in his book called explaining and to harnessing to set an example. and he came up with a missile called if g.m. or foster games i mistook which is an efficient most of whom you what would be the actor said annoys what we do is a small british and we can act to the original image who was their mother whom misclassified or miss but it.
and so idea is very simple so given a number of image he commutes the last was a model for this and what image and then he concludes the gradient of the loss was this because and what and then he takes a scion and exist to the original image after scaling down so the exhibition would be small and the intuition. here is that the the loss age of quantification of the arab reduced was an airport and the gradient those a loss is the direction of which it will maximise this or eight and during the training you add to the weights. some update is over the direction of their of the great and those the last to minimize the air but if you want them makes the model misclassified or miss protect you need to act on our because and what in the direction of the gradient of the loss. and by applying this instead of having the mother but it means the least damage as a band that correctly it will now say it is at the bone was ninety nine percent accurate or just not only wrong but it is wrong with very high confidence. and this happens four years ago but the right now after four years of research we have a lot of objects in this domain like a little three sessions studying this phenomena.
and that have been many other. i tech methods for the age of five percent attacks resembles is a projected gradient a descent that is that the pool carlini and a list of network and with me and to me as i was about all of these share a common thing the rely on the same idea and could use to buy it for themselves and fourteen which. commuting the loss of the network findings a gradient and then adding some but conditions of that action that maximizes the loss. but the not only in such should have been studying at texas you have been also studying like defense and there is a software library cards i.b.m. actor said abbas to get and systems of software library called the clever hands which is like a benchmark of attacks i wasn't implemented. for sexual abuse and there is competition is organized around today this to mean like the next two thousand to thinking competition commission's that we had just a day in this conference.
and yet one of them. you mean is that the city a better work which is the likes of alone. the can just how phones that the can have a but in to visit i'll bet that if you bothered like nick is to inaugurate the emission levels more than looking to add the image of this obvious it will miss but it did so for example of that original image if you have a banana and did a photo vet and feed it was a machine letting mother. it was a good husband and i was ninety nine but think what if you for if you bought this small be so step old stick be able not just quit taken as a food then it was a light it is a poster with the ninety nine percent accuracy and this is a physical water that batch is not and much because one of the images is something you would like its design. and in the visible war. so so cool does this mean are we done ways that is a problem like is everything solved.
it will believe it is not all because there are a lot of remaining open challenges.
and our bed was over the top with a it's going to use to your like a few of these open it challenges like not all of them because that exist many own problems but we won a light highlight something. fuel and challenges and what a piece of chalk will have done in sydney. source the first one is still like how to attack machine eleven months while you are having only limited access business model and only limited knowledge about the more the second one is how to attack machine learning models that's not doing only images are other things like much of language. understanding moderates and said well as to how a few physical warwick attacks against the speech recognition models.
so to start was the first one a little music and give your are reminded of how can we trust the examples. remember what we have saved a lot we need to come with a small but admission that will maximise the error this more but the patient if we do with it as our.
it just desire for the subject of was a doctor is to have a four x. plus are the big picture of the modern it will do exactly what will the arctic label. such as access more the image after adding the noise it still remains. close was already the image.
and most of the water and doing this to rely on i get a ticket agent competition so you can with the gradient of the last with this because there and what could be the british n you act was already an image but commuting the gradient it's only boastful if your first you need to know like what is the model arctic.
the chart and unique to more what is the modern weights and by amateurs in order to come with this great yet so this approach is our only successful in our books are sitting where you have complete knowledge about the market and its in tennis.
but black book sex are more practical because in practice you would be willing to attack a more than your only have access to use it but they don't have access to its intelligence and you don't know its fiscal action or its weight so they have the research walk about this including that they would not tell and thousand sixteen that principles to. you'd models. for using train to using a there's a lot of quality our schools the victim models and then the exit on substitute model and the hope that this attack was concept was as a more than that.
there is a war by chin it all in south and seventeen this one of state of art what is in black book that debt instead of congo's exec the gradient by knowledge of them with an income of the to do is to make an approximate gradient so on the black box the quickest isn't what it saw the door a different equation is to mission to change every big. in his image a little bit and the east made what is the result of changing this big said how much does it affect that article scoreboard guilty.
but the problem with this misses the are not efficient the uk were doing to the huge number of where it is a victim of but if you haven't him to access to some other. then you want to like to an act that was a small number of ways to prevent his act the mother on a funnel knowing that something bad is happening.
and that's why when biggest or only genetic soldier tech is a black box attack against machine living models it can be done without knowing some intent on modern architecture orbiters and it assumes that soccer only has the ability to queries the model as a black box from so you didn't and what and today it's of addiction scores are.
of their mother without knowing anything about what's happening inside the mother.
as i do is the following its did offer lying on the second computing which are requires knowledge of the more the internet's or graduate approximately station which was which a number of queries we use a good idea to feel the magician so we don't even equal is to make or approximate the gradient so and. who was in that many to position we can conclude our actor said exempt and got no action is going to tell you much details about how does work. the thinking stuff or so how does or genentech work so it's basically importation genetic algorithms is based upon that soldier to get genetic algorithms work so first to initialize the population and the way you do as make a bunch of random provisions of the input and that becomes your population you then evaluate the film.
this of each member you want to see how good they are and we value the fitness in this case is look at the target production probability how good are they again the mall to most costly how good they are being adversarial home if the example is already ever serial you're done you have put it if not in use are interesting use or optimizing so you then to random selection of home. there remain pretty beaches of the image so the way genetic algorithms work is that because you don't have access to the gradient you don't know exactly what direction go in you try to search this piece as efficiently as possible to generate only surgeon hit on all possible places so you show the publisher the random perturbations you go. but the fitness some are good some or you then if you're done you're done if not the to select a few of these are the parents was why they call genetic algorithms and the selection is done it will probably be proportional to thin the score we would rather have selected areas which are good way to cross over his you just randomly combined his parents go children which is why. was called genetic algorithms home. of not sure what you mean well. the. i. on the way we did is just so we have the actor these are back to the sickly we just a random combination of the vector yes yes yes. so we have that we do crossover and then with a small probability a mutation probability the mutation any teaches just a random change and so in this way we try to explore the space very very well oem.
so now we're going prisoners ultz so we're expecting this attack could be good but we're expecting that will probably more querrey efficient than the zoo attack which the current state of your because that's to be the greedy is so costly but jankovic a note of high competition complexity so we were expecting got much improvement how we were very surprised it was a tremendous improvement so for example here the. see for ten example so on the on the left you can see the original label the top easy to predict a little clearly these look like the original example how we are all classified as ever serial to each of the corresponding classes will succeed against all the classes and were also able to take images models which are the more realistic images these are the larger scale.
higher resolution images really tack image and also for example that example is a is a bird a car to go gallery go and go able to get the costs for the troubles and it looks basically that's all i'm so in terms of valuation here we found a big difference so the christie of the resume which is a stream the gradient.
so for aniston see for ten do and genentech do pretty well and missed is a film small scale images are black and white and their numbers see for ten or small scale images but the color images image images are larger scale high resolution images so the features piece is way bigger is hard to attack interesting lee zoo because it's still competition and efficient. it took so many queries to estimate the greedy and is not able to succeed on image and the target case targeted case meaning its edges getting the attack to have that the model to miss cost flight was very easy an image that because so many classes you could get to his cost far from like an arctic fox to white folks and that qualifies of the ever so example but for example targeted case. were you want to be a cost was a truly of us do actually filled in this case relatively frequently or genetic didn't basically due to the fact that was actually truck trouble on the image so in this really express inquiry fission see so we just measure the mean number of queries it took to computer successful was an example for them is the fourteen image in it.
what do we do is a trial run iterative optimisation and most are successful they would abort so you're expecting that for and it's easy for ten will be much less queries however was it was basically the same only about five hundred thousand more images which is a lot but seeing how inefficient do is not that much in the relatively power for genentech for the smallest images cause. it easier and the features be a smaller we will succeed in a drastic reduction of the of queries so about two thousand times more querrey efficient and for image that which are drastically larger feature space we're twenty seven times more cost efficient which is very good however there is more important to be done just as a much much harder keys so now.
are we going to talk defenses so there's been a lot of defense is proposed of the one difference was just kind of stood the test of time is ever sultry ever so training is very very natural on so what is his own were we couldn't ever so examples let's train them and maybe will be robust as others have examples so if you call the arts in this you can't just take a butcher's of samples that train with. some expect to be extremely robust so majeed was poured is a mighty web of them they notice that the way you get what you can get this is you increase amount of capacity so assuming that every so examples are also distribution these russian at the of the model a scene which is pretty possible for the fact that the models miss costs flying them you need more capacity to feel the hand. will this warrior distribution so for example for and misses the relatively easy data set that increase or modify ten times to be robust to like small probation attackers and also instead of using at just some was a single step attacker in the direction the greedy and use era detectors in recent years of tech reserve better is because now instead of making a big jump. when you add to actually make like occurred jump a non-linear jump you make small initiative when you're jumps and that's approximately normally year.
so there's a few method so the standard way is the exact which you think so while you're training use aplenty with ever so examples in key going however there's a problem with that in that you generally ever so examples of its weak model because the models currently training you want the most of your boss to strong ever so examples also the ever so example generation process is. right in with the model training so you might learn some the generous solution where the actual adverse examples are weak and i'm it's because they'll just minimize the last because the only opposition check here is a loss of so ensemble ever so training separates these two things you generate ever so examples a guess on sample that's already been trained and use that to train your his role model and they found. much better results as actual valid in the next competition was we talked about before owns all the defenses are based upon this model so now we took this model which is found to be sickly be the most robust defence and we will succeed and it only took a few more queries than attacking clean model was stable succeed and i think it was about hundred thousand for all.
for inception the three clean model those two hundred thousand for the ensemble of researching defense to a little longer but way more tractable than zoo and see the consoles that's a sea anemone and now it's a parking meter home next week then tackle the big problem in machine learning or ever so much in winning so and he should tell you who's a also at mit introduce.
is this some this problem office good ingredients and does this actually a worthy i still twenty two best paper showing that's a big problem the community and three found those with a bunch of defences which are published likely twenty teen he took the ghetto codes any attack them he found that all all of these differences are really more robust their line up on a phenomenon.
which is why the high cost us to remember that before jan attack basically all attacks were greedy and based so what they did was go instead of becoming actually robust to provide patients risk a mess up the gradients so attackers phil and they found three instance of the so shattered radiance is one the defense renders agreed to be non-existent or incorrect to cast the gradients. is one the just make it a randomized so big redirection has become very random because very hard to optimize and exploding are vanishing gradients work the greens just store exploding and again these are all just harming optimisation attacks which are based on the greed of so they had a bunch of methods circumvent so the p.t.a. was just old what's analyze the model by. and none differential component replace it with something that's approximately equal but different people now we succeed or ego tea which was four and succeeding its accounts a greedy is where we instead of just optimizing apologise one random sample of the defense use keep randomly separate from the defense you optimise the the ran a transformation he called the be robust. to the whole all the renovations introduces however this only works in the way books keys need to know what's in on differential component to use b.p.a. aid to know that all of this is a randomized defense to actually do to put in the competition budget to actually optimised for this transformation however as we discuss janet eichengreen three this to me the greedy. it doesn't need the greedy and so it should work against these offences we found that to be true so but differed we attacked a bunch of non difference will into transformations which are the normal ones there are used so bitter production is on his the images are represented using a limited number of bits you cut off the last few bits which is not different trouble these greenpeace to.
its fill of we will succeed column verjee pick impression you just take your image received as jupiter which compresses the image is of the non differential transformation artek succeeded who filled in t.v. and the ism is a very costly tech one its randomised because the way it works its total various minimization they do random drop out on the pixels. and then after they minimize the total variation across the example so one it's very costly as are solving optimisation problem every time you win friends and to his randomized so is much harder zoo because it's so competition costly a little tidbit on the to actually runs to one hundred image images it takes five days on the table next to you to grow two weeks. months of so when the actual inference is a little opposition problem is you becomes completely intractable however algorithm wasn't and we will succeed about seventy percent the time so we targeted basically the number one defences are still changes to the test of time and obviously agrees with is what almost all the difference is currently are like upon so.
so now we've discussed attacking malls limited access now we're going to discuss attacking natural language model on so why can't we just do the same thing we do images and natural in which case all natural language different first words in texture discreet unlike image pixels which are continues so you might say oh but were to the wording buildings metsys words he's backed representations something.
you take the gradient you go in the direction the greedy and you pick a word replacement which is in that direction that greeted however with pixels and with images you can make a worst case probation and you can make imperceptible up to a very large scale predator with natural language if you change a single word a just change a single word so it maximize the target label is going to probably. we have nothing to do the substance is as can be easily detected is going to completely ruin semantics as and does so you can't really do that secondly that word change he says that despite the language is grimmer constraints if you change and now into a verb it is related to the context of the sentence it's going to make the sentence making sense as can be easily detected so these are all the difficulties with. is why it's very hard to buy greenpeace optimisation so how will you janitor genetic algorithms all so again years the same algorithm and again this is really the only to our knowledge the only algorithm the literature was able to actually make ever so changes and get universal examples which are some men to clinton talked to be similar and all.
also his books which makes a profit. all so the thing is we did the exact same thing as before the fitness as the toward a little production probability all these things except the mutation is different in the mutation is a just making a rate of change we now want to make a change such that the word or placement is magically to talk to be similar to how do we do that so first to compute the end nearest neighbours have selected word.
in the bidding cities so these are supposed to be similar however all we did use a counterfeited embedding because actually in the inventing species good and bad or very similar because these warning betting mall the training corps currents going bad appearance in what context however if you're pleased with that you completely ruined the semantics a sentence so the counterfeited in betting. is it takes is one of betting inject synonym an end to sort of constraints so the only words they're close to this particular word or ones which appears in context anderson's so the additives are from far away then after that we have earned years the numbers but we don't know if they've been the context we don't have the this it right part of speech all these things that we tickling was model which is used to produce. if the next word in a sentence so if the next word is something which doesn't appear in that context the probably be low so we use the language small to predict the probability of that word given the previous word in the sentence and we felt hours which don't in the context so now we have words which are symmetrically similar and syntactically similar at that point we know from the police by maximizing the target. globally so we have this isn't a synonym constraints semantic as a tactic constraints and then we try to make the replacement would be so the target probability we actually fell and very good results so for example this the i.d.b. desert was the classic this that percent of the houses and other and you can see hear this review is very negative on its negative and the result expedition was.
negative with seventy percent confidence however all the opening day with a change terrible to her if that which is a synonym and it changed regarded to consider guarded which is another some of them and engines kids to youngsters can only sit in them and now costs was as positive which is a vulnerability it's actually true ever so example because they retain the semantics and to die. in some tactics next we tech textron tillman so it takes one to one is the of some promise some situation you could say and then you have some hypothesis and dumb and the model predicts why this hypothesis and and deposits is until the promised contradicts the premise was mutual to the promised so for example is the original what in this original prediction.
of a runner wearing purple sets tries to the finish line that causes the runner wants to head to the finish line says doesn't tell the premise which is correct however the average for example was just replace runner with research was a synonym so it should still until the promise of over the model predicts contradiction so that's different problem of the second example. it's now one is originally contradiction the premises a man and woman stand in front of a christmas tree contemplating a single thought that bosses to people talk loudly in front of the characters which is does not entail the promise of the contradiction however if you change people talked to humans chitchat now as it says it entails the problems which is clearly wrong. so now going to cause a reduced offering a talk about the open problems in speech recognition a succession of.
that's the kind of commodities that we ate the machine letting mothers to be accepted or the nor this would let a speech that between a devotee and of course the lagoon of us happens at home does but it's much the smarts because they think your voice those comments and of light for example a cd and google. a good home you can ask them to do online transactions for you so they could lend off i say the attacks in a speech recognition what is the following your like to do some water such as a human with the whole essence to at the lake will say that this war is yes while them should letting more they will. conflict with this was seen or and is what was it was an all white it. the holy to still silence for the human as a s.
and with its war was published in its mashing deception workshops and seventeen is a black box at tech own speech recognition modern using again or you know i was more position. approach sos a victim or they'll be used and buying and so was another war by year calling from u.c. berkeley of which was going what books ect ect against which to measure models out what was against speech comments a data set which recognize like a single common to watch. but like yes nor are now an eyesore but god knows what it is a white box but it can perform second is a whole as a half hour sentence to this fixation no more.
so the have fun who can actually very high success eight saw and its for example if a reward in the usa and sixty s. can be converted to be ms close fight as on or was ninety six percent accuracy. to say it and you can be as close to getting the we must close fight as and up with eighty percent. the same limit of the condition like to the and put all your form. and we qualified what is the human best option of the warts after you at this a trustee of noise so we have found that on average like ninety percent of human lives not suited for this study this did organize a war. the as a source labels origin of the original labeled even so the more they will miss classified as a duck label and the. the nine percent of them will say that is what is a mom was for me or like is a corrupt what and only one of them one percent of them. as a top of the water.
but the challenges right here in this problem right now is that this work an artist ai take against the speech recognition model the assume you have a direct. access to the model so you can feed your or you directly in more than as an ambush but there is no equivalent yet for a visit a lot like attacks against which admission would for example in case and images there is this discovery attacks against image of the mission more they like that for see a bet we have seen book was a we have shown before which is like. the physical water to speak out or is it is this work by an explosion and so much a team is like taking a picture of access to the arab rented a image and with its give him an actress area but invisible or what like to say back against the spiritual mission if you really the first year old you'll from must be cut. and to record on the other side from a microphone and then it will no longer become active said. so that the men's and open challenge. as how to me develop robust over the air actor said a joint tax.
so as a reminder that there are still many many many open challenge us we just like to highlight a few of them and sort often have for each one of them. we'll be happy to accept a question.
i.
the ace is exactly we have a constraint on that. infiniti on. no we develop our own. actually most of all what is available as a one source womb. another questions. thank you for.
Feedback