AI VILLAGE - Towards a framework to quantitatively assess AI safety – challenges, open questions and opportunities.
This is a modal window.
Das Video konnte nicht geladen werden, da entweder ein Server- oder Netzwerkfehler auftrat oder das Format nicht unterstützt wird.
Formale Metadaten
| Titel |
| |
| Serientitel | ||
| Anzahl der Teile | 322 | |
| Autor | ||
| Lizenz | CC-Namensnennung 3.0 Unported: Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen. | |
| Identifikatoren | 10.5446/39786 (DOI) | |
| Herausgeber | ||
| Erscheinungsjahr | ||
| Sprache |
Inhaltliche Metadaten
| Fachgebiet | ||
| Genre | ||
| Abstract |
|
00:00
Produkt <Mathematik>MereologieInternetworkingPerfekte GruppeOffene MengeFramework <Informatik>Algorithmische LerntheorieCybersexComputeranimation
00:41
ComputerTelekommunikationTablet PCDatenmissbrauchUmwandlungsenthalpieComputersicherheitGesetz <Physik>AggregatzustandFramework <Informatik>CybersexMathematikSpieltheorieGesetz <Physik>AggregatzustandPhysikalisches SystemRobotikFramework <Informatik>Weg <Topologie>NeuroinformatikStandardabweichungRechter WinkelGamecontroller
01:54
RechnernetzE-MailVerschiebungsoperatorAussage <Mathematik>SpeicherabzugVirtuelle MaschineAlgorithmische LerntheorieBetafunktionBitrateTwitter <Softwareplattform>DokumentenserverPotenz <Mathematik>PackprogrammComputeranimation
02:35
Gesetz <Physik>AggregatzustandLeistungsbewertungFramework <Informatik>ComputersicherheitTwitter <Softwareplattform>RechenschieberAlgorithmusPerspektiveSelbst organisierendes SystemVirtuelle MaschineAlgorithmische LerntheorieStandardabweichung
03:20
Offene MengeFramework <Informatik>FlächeninhaltComputersicherheitClient
03:50
Virtuelle MaschineQuick-SortFramework <Informatik>SoftwareentwicklerMereologiePhysikalisches System
04:26
MereologieMereologiePhysikalisches SystemVirtuelle MaschineOpen SourceSoftwareRechter Winkel
05:00
Gebäude <Mathematik>Gesetz <Physik>Virtuelle MaschineQuick-SortSystemaufrufStrategisches SpielRobotikDatenfeldOffene MengeFramework <Informatik>Algorithmische LerntheorieMinkowski-MetrikStandardabweichung
05:48
StandardabweichungDienst <Informatik>Service providerDienst <Informatik>BruchrechnungProdukt <Mathematik>Physikalisches SystemVirtuelle MaschinePunktwolkeSoftwareschwachstellePerspektiveComputeranimation
06:39
Prozessfähigkeit <Qualitätsmanagement>ComputersicherheitStandardabweichungPCI-ExpressGesetz <Physik>PlastikkarteGesetz <Physik>PlastikkartePunktwolkeProzessfähigkeit <Qualitätsmanagement>Stochastische Abhängigkeit
07:08
ICC-GruppeMereologieWinkelFramework <Informatik>Computeranimation
07:33
Zellulares neuronales NetzSupport-Vektor-MaschineNatürliche ZahlTopologieEntscheidungstheorieForcingLineare RegressionPhysikalisches SystemVirtuelle MaschinePunktLogistische VerteilungMaskierung <Informatik>FreewareBestärkendes Lernen <Künstliche Intelligenz>ProgrammierparadigmaSchnittmengeDifferenteComputeranimation
08:34
InformationAlgorithmusAlgorithmusWellenpaketKategorie <Mathematik>NP-vollständiges ProblemInverser LimesResultanteKontextbezogenes SystemRechter WinkelGewicht <Ausgleichsrechnung>MultiplikationsoperatorComputeranimation
09:26
StörungstheorieGebundener ZustandZellulares neuronales NetzProgrammierungLineare AbbildungSpieltheorieDistributionenraumFehlermeldungSchlussregelDatenmodellBitrateSystemprogrammierungExistenzsatzKugelWellenpaketBereichsschätzungPhysikalisches SystemVirtuelle MaschineQuick-SortFehlermeldungFormation <Mathematik>PunktBefehl <Informatik>SoftwaretestMereologieCoxeter-GruppeGebundener ZustandSchnittmengeComputeranimation
10:36
FehlermeldungVIC 20ATMBeschreibungskomplexitätGebundener ZustandNatürliche ZahlWellenpaketSoftwaretestUnendlichkeitFunktionalFehlermeldungGebundener ZustandAdditionMultiplikationsoperatorRelativitätstheorieNichtlineares GleichungssystemRandwertComputeranimation
11:38
Bayes-NetzPerspektiveBitMereologieVirtuelle MaschineAutomatische HandlungsplanungDatenfeldPunktPackprogrammEinfache GenauigkeitComputeranimation
12:46
QuellcodeBenutzerfreundlichkeitInterpretiererLineare RegressionMereologiePhysikalisches SystemVirtuelle MaschineE-MailQuick-SortLinearisierungDatenfeldComputersicherheitFramework <Informatik>Endliche ModelltheoriePerfekte GruppeReflektor <Informatik>Computeranimation
14:08
Vorzeichen <Mathematik>Data MiningWellenpaketSoftwaretestVirtuelle MaschineReelle ZahlQuaderVorzeichen <Mathematik>Reflektor <Informatik>Computeranimation
15:08
Physikalisches SystemComputersicherheitKategorie <Mathematik>Deterministischer ProzessPhysikalisches SystemBasis <Mathematik>ComputersicherheitPunktMaskierung <Informatik>MultiplikationsoperatorSpider <Programm>DatenfeldEinfache GenauigkeitComputeranimation
15:47
Physikalisches SystemFramework <Informatik>SoftwareschwachstelleRechter WinkelComputeranimation
16:20
AbfrageFramework <Informatik>StichprobeArchitektur <Informatik>Orakel <Informatik>ProgrammverifikationOffene MengeSystemprogrammierungProzess <Informatik>InternetworkingBildgebendes VerfahrenCodeMaschinenschreibenMustererkennungComputerarchitekturWellenpaketMAPInverser LimesLeistung <Physik>MereologiePhysikalisches SystemRang <Mathematik>Virtuelle MaschineWinkelQuick-SortAbfrageInternetworkingParametersystemProzess <Informatik>RobotikComputersicherheitFormation <Mathematik>RankingQuaderGradientOffene MengeUmsetzung <Informatik>Framework <Informatik>BitrateKanalkapazitätBlackboxDifferenteAlgorithmische LerntheorieMinimalgradKontextbezogenes SystemMultiplikationsoperatorDienst <Informatik>Demo <Programm>OrtsoperatorComputerunterstützte ÜbersetzungMechanismus-Design-TheorieMathematische LogikPerspektiveTransformation <Mathematik>FrequenzProdukt <Mathematik>ProgrammverifikationInhalt <Mathematik>Service providerUbiquitous ComputingTVD-VerfahrenARPANetComputeranimation
21:06
SystemprogrammierungStrom <Mathematik>SpeicherabzugStabilitätstheorie <Logik>MathematikEntscheidungstheorieGesetz <Physik>Inhalt <Mathematik>InternetworkingBasis <Mathematik>HypermediaCASE <Informatik>SoundverarbeitungLesezeichen <Internet>Physikalisches SystemGrundsätze ordnungsmäßiger DatenverarbeitungInstantiierung
22:27
E-MailGerichtete MengeService providerCybersexKategorie <Mathematik>Bildgebendes VerfahrenGesetz <Physik>Physikalisches SystemVirtuelle MaschineE-MailCASE <Informatik>RichtungIn-System-ProgrammierungAlgorithmische LerntheorieCybersexPerspektiveKategorie <Mathematik>Metropolitan area networkFreier LadungsträgerChord <Kommunikationsprotokoll>Computeranimation
23:47
Kontextbezogenes SystemPhysikalisches SystemVirtuelle Maschinet-TestRechter WinkelPASS <Programm>Computeranimation
24:30
InternetworkingMini-DiscKontextbezogenes SystemSystemprogrammierungDatenverarbeitungssystemOrdnung <Mathematik>DatenverarbeitungssystemGesetz <Physik>Physikalisches SystemVirtuelle MaschineInternetworkingBasis <Mathematik>CASE <Informatik>EinfügungsdämpfungMini-DiscNeuroinformatikKontextbezogenes SystemCybersexDesign by ContractAusnahmebehandlungComputeranimation
26:06
Physikalisches SystemAusnahmebehandlungMini-DiscDatenverarbeitungssystemKontextbezogenes SystemBildgebendes VerfahrenCodeMustererkennungPhysikalisches SystemVirtuelle MaschineAbstandOpen SourceSoftwareentwicklerSoftwareSpieltheorieSymboltabelleRechter WinkelComputeranimation
27:00
Produkt <Mathematik>AlgorithmusQuellcodeService providerOffene MengePhysikalisches SystemAlgorithmusProdukt <Mathematik>BetragsflächePhysikalisches SystemCASE <Informatik>Service providerOpen SourceNeuroinformatikDienst <Informatik>KontrollstrukturComputeranimation
27:37
BildverstehenStrategisches SpielDatenmissbrauch
28:03
FokalpunktKünstliche IntelligenzTaskARPANetTaskForcingTermNeuronales NetzSystemverwaltungFokalpunktVerkehrsinformationAlgorithmische LerntheorieCybersexStandardabweichungUltraschallKurvenanpassungAutonomic ComputingComputeranimation
29:23
BetragsflächePhysikalisches SystemVirtuelle MaschineCASE <Informatik>Framework <Informatik>StandardabweichungDigitales Zertifikat
29:52
SoundverarbeitungInternetworkingForcingPhysikalisches SystemInternetworkingRichtungSoundverarbeitungAlgorithmische LerntheorieAnwendungsspezifischer Prozessor
30:31
Befehl <Informatik>Trojanisches Pferd <Informatik>CodeProgrammMinkowski-MetrikSoftwareEndliche ModelltheorieCodeSoftwareProgrammierungMaßerweiterungVirtuelle MaschineDatenfeldBefehl <Informatik>Endliche ModelltheorieAlgorithmische LerntheorieTuring-TestMultiplikationsoperatorAnwendungsspezifischer ProzessorComputeranimation
Transkript: Englisch(automatisch erzeugt)
00:00
Alright, so our next talk is Ram here and he's going to speak about a framework to quantitatively assess AI safety, which includes challenges, open questions and opportunities. So give it up for him and I'm going to let him take it away. So a quick show of hands, how many of you are currently using machine learning
00:22
in as part of your product or your company wants you to? Okay, perfect. How many of you want to? Okay, sweet. Okay, so that's great. You know, let's go back to a little bit of early 90s. The internet is just starting out, you know, things are fine.
00:40
And all of a sudden, you know, we see a spate of cyber attacks, like, you know, and I don't have to explain this to you guys. And then what happened was the regulatory landscape kind of quickly changed. You know, federal laws started coming into place, like, you know, the Computer Fraud and Abuse Act. Apparently, Reagan saw the War Games movies, got really inspired.
01:02
And you know, within like a couple of years, this act was formed. Hopefully people are now watching Mr. Robot and we get like stronger acts like that. State laws started cropping up to fill in the gaps. Anybody wants to take a wild guess which was the first state to actually make ransomware illegal?
01:21
Wisconsin. You're in the right track. Closer. It starts with a W. Oh my gosh, you're right. Wyoming. Yeah, Wyoming, interesting enough, is the first state. I think 2015 legislated ransomware is illegal. And we have like a spate of like standardized cybersecurity frameworks. If you are a company and you want to convince your customers that your system is secure,
01:45
you would adopt one of these frameworks. And that's how you would convince your customers that, hey, I pass all these controls and this is pretty much, I'm a secure system. Let's get a shift to like machine learning today. This is like the machine learning landscape by Siobhan Zillis, a venture capitalist at
02:02
Bloomberg Beta. If you can't see anything, which I'm sure you can, a lot of companies out there who are actually investing in machine learning as their core value proposition. And the interesting thing is the attacks on machine learning is now grown at an exponential
02:21
rate. Archive, which kind of documents how many, you know, it's a public repository of documents, has close to like a hundred papers on adversarial machine learning just in the past two years. And this is kind of like a timeline, sorry you can't see it, but I know I'll tweet
02:40
the slides out, by Batista Bijio, kind of like mapping out extensively like what are the different attacks that are possible? You know, starting all the way back from 2004. But today our regulatory landscape with respect to machine learning looks pretty empty. You know, there's no way, there's no guidance from the government, there's no guidance from
03:03
standards organizations to tell you which ML algorithm is safe. And this kind of is a problem because we've now all bought into this Kool-Aid of machine learning and we have no way to tell our customers, hey, what I'm doing is safe from my machine learning perspective. So that's what we're going to talk about today.
03:23
You know, my name is Ram. I work in the Azure Security Data Science team. Our promise to customers is, you know, we'll keep Azure and you secure. And this is, this work is done as, I'm also a research fellow at Berkman Client Center and this is what I'm going to be working on there.
03:41
So a lot of this is ongoing work. So it's rough around the edges and if you're working on this area, I'd love to learn, talk to you more about. And this talk basically has two parts. So the first half is going to be about how do I assess the safety rating of a machine learning system? So, you know, your development team, say, you know, let's say that they built a machine
04:04
learning system, you know, can we pass it through some sort of framework and get a safety rating? Just like, you know, how you would expect of like the health of a food or, you know, the safety of your car seats, which you can very much look at. What kind of guarantees can you get about the safety of your machine learning system?
04:23
So that's going to be the first half of the talk. The second part is going to be about the legal underpinning of a machine learning system. You know, we all have, we all heard about the doomsday scenario, right? Like, there's a drone, you know, using an open source system and instead of landing
04:41
in your backyard, what if it kind of like, you know, chops your hand off? What do you do then? Like, who do you sue? Do you sue the drone maker? Do you sue the open source software? What kind of like legal underpinning is there for like machine learning safety? So those are the two things.
05:00
So first of all, like the call for like machine learning safety is not new. New-ish. So Urs Gasser, you know, wrote a fantastic paper on AI governance and said, hey, the first thing you want to be tackling is some sort of standard, some sort of data governance with respect to machine learning safety. Ryan Calo, the robotics professor, I was a law professor at UW, has pretty much laid
05:24
a lot of open questions on this field and proposing a lot of fantastic ideas. And things are getting real. You know, Europe's AI strategy specifically says by 2019, you're going to come up with a standard framework and a liability framework for ML safety.
05:42
So things are picking up at a great space. So it's not like I am, all of these are new ideas. I'm building on top of them. So first off, why do you, why should he care? Like, you know, what is, why is this an important topic for us to discuss at DEF CON? I think the most important thing is we all, we all want to hold our service providers
06:03
to some accountability. Like your trust, just like how you trust your data to say the cloud, and you assume that the cloud is secure. If you're going to use a machine learning system, like, you know, you want to know that the machine learning system is secure as well. Kangli gave a fantastic talk just like an hour before about the software
06:24
vulnerabilities that come in, that you inherit when you build machine learning systems. And that's kind of like unacceptable because if you are running a production system, you want your customers to feel safe from that perspective. And from a regulatory standpoint, just like, you know, if you want to take the cloud
06:44
as an example, you have a bunch of laws that, you know, Congress, you know, regulatory agencies and even councils, independent councils have created. So, you know, if you want to host, say the credit card data, you would have to be PCI compliant. And if you fail, you pretty much get fines, you lose the ability to operate,
07:04
and you can be criminally prosecuted for this. So, hopefully, we don't want any of you guys to go to jail. So, the first half is going to be about like, you know, how to assess safety ratings. So, before like, you know, we go to the most important part about how to assess,
07:21
a bulk of this talk is going to talk about why this is a very difficult problem to solve that. So, end goal, I'm not going to have like one framework, one definitive answer. It's just going to be a proposal. So, what are the challenges? First off, you cannot escape adversarial machine learning. There's no vendor who can claim that, you know, my machine learning system is adversarial ML free
07:46
because it's ubiquitous among all ML paradigms, whether you take supervised learning, whether you take like reinforcement learning. Adversarial examples are just a force of nature. And Papernot kind of figured out that adversarial examples are also transferable.
08:02
So, if you have a system, say a logistic regression system, and you train on that, it's pretty much those attacks are transferable to say, decision trees as well. So, not only are you kind of like guaranteed, but it's also transferable. Scagetti in like 2016 said that, hey, it's not only transferable,
08:25
but even if you have two completely different data sets, you will still have like adversarial machine learning. So, you're pretty much caught in like a triple whammy at this point. And the second challenge is verifying if an algorithm is safe is practically difficult.
08:43
In fact, like it's an NP, it's computationally intractable. That was like one of the stellar results that we got this year that even if you can, it might be theoretically possible, but computationally, it's impossible because of limited training data.
09:01
And the biggest kicker is it's an NP-complete problem, especially in the context of deep neural nets, which is like really big popular right now. So, if somebody gives you a deep neural net, and if they ask you to verify a property about it, hey, is it safe? It's an NP-complete problem. There's no like exact, you can come up with a solution,
09:23
but in polynomial time, I can't really verify it. I mean, there's been some good news. I don't want to like damper the whole system. You can get some sort of upper bounds, some confidence bounds, but this work is super nascent at this point. The third challenge is, you know, completely safe ML systems
09:45
can only occur, you know, if you have a non-zero test error. So, just to like, you know, what this statement means is, if you, when you train your machine learning system to get like, you know, two things that you want to really care about, like how does it perform in the training set,
10:02
and how does it perform in the test set? And even if you get like a machine learning system that performs really well on a training test, on a training data, there's no guarantee it's going to perform well on a test data. So, imagine that you built like a, it's not me, thank you. Yeah, there's no music as far as this presentation.
10:24
So, the only way, one of the insights that two recent papers had is, the only way that you can kind of assume like a system is safe is if it has non-zero test error. And non-zero test error is really not possible. We'll see why. This is kind of like a relation.
10:41
It's called a VC-bound. It bounds your test error as like, you know, your training error and as a function of something called a VC-bound. We won't go into that. But just like, you know, keep in mind that test error is kind of like proportional to some, you know, some additive nature of VCs.
11:02
The problem is these bounds are really loose. Like, if you have a linear classifier, it is like, you know, and if you have like a million features, you know, your VC-bound is kind of like million plus one. So, if you think, if you go back here and you put like a million, you know, the bound is really loose. And for some learners, it's kind of infinity.
11:21
So, imagine like, you know, you've always wanted as a kid to put an infinity in an equation, now is your time. So, it's really, these bounds are very loose and they're getting like a zero test error is impossible, which means that has big safety implications. And this being DEFCON, you know, I found this like,
11:41
oh, now you can see the picture, but this is Vapnik, who kind of like, you know, found the VC-bound and it, if you could read it, it says all your Bayes are belong to us, but Bayes spelled with a Y, like Bayesian learning, like nerds, right? So, of course there's a lot of like defenses
12:03
that are being published and archive, you know, scores and scores of those, but there is no single defense, unfortunately, from a machine learning perspective that you can use to protect yourself against attackers. There's a great paper, which actually I think won the best paper at ICLR, a top-notch machine learning academic conference.
12:21
What the paper showed was like, they took like the nine defenses that were published as part of the conference and they broke seven of them. So it's very easy, you know, to kind of like get over these defenses. Goodfellow is kind of like the father of this field at this point, kind of like ranked
12:41
some of these defenses along certain axes and you'll see that a little bit. The next thing is like, the next big challenge of constructing a framework and, you know, guaranteeing safety is there's a big tension between adversarial examples and interpretability. How many of you know what GDPR is? Okay, perfect.
13:01
You have all gotten those annoying emails, right? Because if a company didn't, then awesome, you know, they're probably gonna get slapped big fines. So one of the things that GDPR articles say is that they want to support explanation. So, you know, which means like some sort of like explainability should be in your model.
13:21
So this way, if your mortgage gets denied, you know, you know exactly why it's been denied if there's a machine learning system behind it. So a lot of people are now using like explainable models as, you know, part of, you know, to get like explainability and the crux of most explainable models
13:41
is that they're linear learners and linear learners are extremely vulnerable to tampering. So if you have this tension wherein like you want to bring explainability, so you use linear models and if you use linear ML models, you're, you know, you kind of have this adversarial tampering. So this is kind of like a big tension,
14:02
just like how we face in security fields where we have a tension between usability and security as well. And finally, like ML safety is a lot more about adversarial tampering. Justin Gilmer who wrote a paper this time about like what it means to do adversarial example
14:21
in real world gave this anecdote. If you think of like, you know, the big poster child for adversarial examples is autonomous cars. You know, hey, what if, you know, somebody puts a sticker on my stop sign and now a stop sign is now like recognized as speed limit? Well, they said, hey, let's be more realistic here.
14:41
What if there's no stop sign in the beginning in the first place? What if a stop sign gets like knocked over by a gust of wind? What if like a guy is wearing a stop sign and like stop sign t-shirt and walking on the road? Does that mean the car's gonna just come to a halt? So a lot of these things in a practical sense kind of dictate what's still like bigger
15:01
than the scope of, you know, training or testing within like a dev machine under your box. So these are kind of like the challenges at this point. You know, you cannot escape attacks. There is no single defense. And it's extremely difficult to verify the security properties. And really, you know, the safety is like
15:22
more than one predominant technique, existential dread. Let's all crawl under a bed and now like, you know, wait for the doom to get over. But you guys are the perfect audience. Like in a security field, we face this every single time. Like everything that makes ML systems kind of insecure
15:42
is what we face in a security setting on a day-to-day basis. And we tackle this risk. We tackle this by minimizing risk, right? You know, this is just one framework, like, you know, the dread framework of like, you know, you try to see how much an attacker can cause damage. Now, how hard is it for an attacker
16:01
to kind of reproduce other attackers to kind of reproduce it? Are you always worrying about like APT or is it gonna take a strip kitty can bring your system down? How easy is it for somebody to discover this particular vulnerability? So with that in mind, like, you know,
16:20
here's like one possible framework that, you know, this just came to my mind is the goal is to kind of score the kind of attackers, score different parts of like your ML pipeline. So the first thing is like, you wanna score the attacker's capability. Hey, does the attacker have access to my architecture and training data is very different from like,
16:42
if the attacker can just like query my API. So you wanna have two different, like, you know, just like how you'd calculate a dread score, you wanna have a score for that. Hey, what are like, you know, some of the possible attacks? You know, is it like a white box attacks or is it like, you know, black box attacks with some sort of limited querying?
17:00
And have some, you know, a score for defense, kind of rank them in some aspect. If your AI, if your ML engineer is just using like rate decay as a defense technique, it's really, you're really not in a good position. Whereas, you know, if they're using some sort of, if they're using strong adversarial training with strong attacks,
17:20
or they're doing something with logit pairing, you're probably in a better place. So the same framework that you use in the context of assessing security risk, making an argument that you wanna do the same thing for ML risk. And once you get that, you can kind of like, you know, from a regulatory perspective,
17:41
it becomes a very easy conversation. Let's assume that you get a score at the end of it, right? Hey, then you can make a claim that all ML, you know, if there's a medical device that uses machine learning, that must have a safety rating of nine or above. Or, you know, if you're using like military grade drones versus civilian grade drones,
18:01
should have like different levels of safety ratings. But of course, there's like a lot of open questions, right, we saw like the challenges, we saw like one possible framework. Like who's gonna certify, you know, that ML system is safe? If you go to like a CDC or like an,
18:20
is there gonna be like a FAA that takes care of the safety of airplanes? Who's gonna take care of the safety of like ML systems, especially that they're so pervasive? Ryan Kalo kind of argues for, I mean, mentions like a federal robotics commission. And that idea is not too far fetched. Like, you know, given the fact that ML is so pervasive,
18:42
we still don't know what the process of certifying it looks like. If anybody's been through like the GDPR process, you kind of know how an immense amount of burden is put on folks to kind of like get that GDPR safe systems. And how do you ensure verifiability? If somebody, you know, if you go to a customer,
19:02
if you buy a service from a service provider, and they say that their machine learning is safe, and we have a safety rating of X, how do you verify that the safety rating is still the same? And there are also like very practical questions, right? When do you calculate the safety ratings? You calculate it every day, every time you push a feature? Well, tough luck,
19:20
because we're all now in the DevOps world, and we're all like, you know, shipping features every single day. So does that mean every time somebody touches code, do you calculate this ML safety rating? A lot of the mechanics of ML safety has not been nailed yet. So lots of open questions, and if you have ideas, I would love to hear more about it.
19:42
So that kind of like brings our first half of the talk, you know, to a kind of a close. We'll quickly kind of like look at, you know, what are the legal underpinnings of machine learning safety? And we're gonna look at it from the lens of internet safety, just because like, you know,
20:00
internet and machine learning seem to have a lot in common. Like, you know, both are transformative technologies. Nobody could have imagined that people can send cat pictures when they had DARPA put out DARPA net. And just like how like machine learning today is used for very similar purposes, you also have the idea of like transforming it from so many different angles.
20:22
And Jonathan's the train called internet a generative technology, which means that it has the capacity to produce new content for broad and varied audience. And the same thing is with machine learning. Yesterday, there was this amazing demo of creating music with code from this DJ,
20:41
which I thought was very interesting, really shows the power of machine learning and the internet. And both have like great degrees of instability. You know, in a loft talk, they said that how you could take the internet in 30 minutes or less. The same thing is with the machine learning. Hang Lee said how he was basic, very easily able to open a shell
21:00
in like a top like image recognition system in very short period of time. And from a legal aspect, you know, it makes sense to look at ML in the lens of internet, just because of precedent. And precedent is really, really core to common law legal system, right?
21:21
My favorite quote is, you know, in one of the verdicts from like Vasquez and Hillary, they said that the court really relies on precedent because it establishes a bedrock of principles, as opposed to some individuals, like, you know, making up law as they go. And the cases that got decided in like 1997
21:43
is still having an effect today in the internet. For instance, like in Reno versus ACLU is essentially why you can have profane content on the internet because on the legal basis of like freedom of the First Amendment. And in 2018, there was a case,
22:00
Packingham versus North Carolina, where North Carolina said, you know, sex offenders do not have the ability, should not be on social media anymore. But the court said no, on a unanimous decision. And they cited Reno as one of their legal precedents. And really, precedents do not get overturned by the court so easily. They wait for societal changes.
22:21
So really looking at it from the lens of internet safety makes a lot of sense. And why are we doing this from a legal perspective? So imagine like poisoning attacks in machine learning. Poisoning attacks are essentially when an attacker can send chaff data to your machine learning system
22:41
with the goal of subverting it. So to either misclassify it or even completely change the goal. A relevant cyber law case could be like CompuServe v. Cyber Promotions. So this was the case in 1997. So the basic gist was CompuServe was an ISP and Cyber Promotions was a direct marketing email.
23:04
And they were basically sending spam via CompuServe. And CompuServe customers were obviously like super pissed off. They were like, hey man, like, you guys need to stop doing this because my customers are gonna bail out. And Cyber Promotions says, hey no, that's kind of like, you know, this is your common carrier,
23:20
this is our first amendment. And the court basically said, no, it's not the case. And they established precedents that even for an electronic property, you can pretty much have like trespassing. And this might be possible, this might be relevant for poisoning attacks when an attacker is sending like spam images
23:43
to kind of derail your machine learning system. Or let's think of liability, right? Imagine that, you know, you go to a scenario where you get some sort of like a robotic system to detect art forgery. Like, you know, you're expecting this top notch
24:01
like ML system to detect forgery, but you basically get a lemon. And now you lost a bunch of money because instead of Picasso, you've gotten some other like weird art student painting. Now, can you take the person who sold you this machine learning system,
24:20
who said that it can like identify fake art to court? Because you suffered like millions of dollars in damage. It's a very interesting system because liability in the context of machine learning safety is still a nascent system. And irrelevant cyber law cases Transcorp the IBM.
24:43
So essentially Transcorp America bought like a lot of equipment back in 1994 from IBM and those disks failed and they suffered economic loss. So obviously TCA sued IBM for economic loss.
25:01
And in the initial, and this is like back when the internet was very nascent and the court basically tried to limit the liability because the courts basically said, you can sue them for contractual, you know, for them reneging on the contract, but it cannot sue them on the basis of economic loss. You cannot sue them for all the money that you've lost.
25:23
And it's kind of crazy because today when we buy computers and say your OS failed and you lost all your documents, we do not think of going and suing our computer, you know, the people who wrote the OS for economic loss
25:43
because we kind of implicitly assume computer systems are faulty. And today, you know, we kind of learned that machine learning systems can be attacked and machine learning systems can be implicitly faulty. So because of the changing landscape, it is very much possible that the courts will also try to limit the liability
26:02
because in order to foster innovation. I mean, there's a big exception. Like if you got, you know, if there's a drone that came in, you know, chop your hand off, there might be, you know, this probably doesn't apply. But here's a very interesting thing that Ryan Kalo pointed out.
26:20
If you have the scenario wherein there was a bodily harm caused by, you know, like a machine learning system that used open source software, what do you do then? Who do you assign proximate cost to? The people who built the drones or, you know, the thousands of developers
26:41
who contributed code to the open source systems. And these are again, like, very not easy questions to answer. Oh, this by the way, cafes like, if you're wondering why this was an open source symbol, cafe is a very popular like image recognition like system right there. So yeah, to kind of like, you know,
27:02
to wrap up with the open questions here, like when an ML system breaks down, how do you get relief? Like where the damage is foreseeable? Do you go after the company that made the product, the open source toolkit the company use? Or if the company kind of hosted it on a different service, the service provider,
27:20
or the researchers who kind of like, whose algorithm the computer use? And all of these are important questions for answers we do not really know because there is no case that's been tested so far. So to wrap up, if you were to think of like all the countries,
27:40
you know, there's been a spate of them releasing AI strategy so far. Like, you know, and these are some of the handful of companies, countries who have, you know, pretty much put AI safety and privacy as a big strategy, you know, in their vision. Of course, you might see our red,
28:01
white and blue is not here. And that's because we do not have one yet. The US' approach to machine learning safety has been very different. Us being like us, our focus is a lot on autonomous weapons. And the Obama administration back in the 2016, created like, you know,
28:21
the US Artificial Intelligence R&D Task Force. They produced two reports and recommendations. Obviously with, take it back, with this new administration, they disbanded the task force and kind of like, you know, push those reports aside. And if you're wondering where we are,
28:41
they created like a select committee of which it's co-chaired by, wait for it, our president, and a bunch of other people. So we don't, very recently there was a bill called the Future of Cyber Artificial Intelligence Act that's been proposed.
29:01
And nowhere are these, as Congress even mentioned about AI safety. So we're not really in a great place in terms of like, even thinking about this problem yet. And it's a shame because a lot of the companies have taken this on themselves to kind of define standards
29:20
and try to get ahead of the curve. So to kind of conclude, safety frameworks are super critical for machine learning systems. And that will help us like create robust standards and certifications that we can use to our advantage, whether to convince customers or to add value.
29:42
And this instead will help us get meaningful relief should there be or when there is a case that involves ML safety. And with that like you know, I want to end with these things. The court is still grappling with the effects of internet. This was Justice Anthony Kennedy
30:02
in Packingham v. North Carolina in 2018. And he said that the forces and directions of the internet are so new, so protean and so far reaching that courts must be conscious of what they say today might be obsolete tomorrow. And they're talking about internet which was started in the 1980s.
30:21
And think about like a feel like machine learning and what effects and how the legal system is gonna handle it. And this is a big open question. And if I were to wear my engineer hat on, you know, Ken Thompson, you know, a Turing Prize Award winner, you know, asks this question.
30:41
You know, to what extent like you know, should you trust a statement that a program is free from Trojan horses? And the answer is like you can't. You can't trust any code that you did not write yourself. And that's shocking, but think of like you know, how ML is being viewed.
31:00
Karpathy, you know, one of open as rector for machine learning and probably a big tight in this field compared machine learning as software 2.0. You know, that machines will now be able to write better programs than humans. And I'm gonna end with this unsettling note, which is how much are you gonna be trusting ML models
31:21
that we did not build ourselves? And that's pretty much, I feel is gonna define the field in the next coming years. So that's about it.