Merken

Consequences of an Insightful Algorithm

Zitierlink des Filmsegments
Embed Code

Automatisierte Medienanalyse

Beta
Erkannte Entitäten
Sprachtranskript
what are the and the and the and and the and the mass of the union of Michael check it does work in OK so right so have this talk is meant to be a tool kit for empathetic coding and we're gonna be delving into some pre specific examples of uncritical programming so the 1st thing I want to do because really talking about painful results of doing things in ways that are perfectly benign intended and then sites like people is I can do that to so so the
content warning here I'm worry delving into examples that include a bunch of things including grief PTSD depression miscarriage infertility racial profiling concentration camp surveillance sexual history consent and assault a model be dwelling on those that is topics that are going to come up are right so algorithms impose consequences on people all the time we were able to the extract such remarkably precise insights about any individual but we haven't really after source enough to have a right to know what they didn't consent to share with us even when they willingly share other data that and up leading us there and have to be asking ourselves more about how we mitigate against unintended consequences of other stuff so it's helpful is step back a little and started asking really basic question what is an algorithm and generically speaking it's just a step by step set of operations for predictively arriving at a conclusion and unpredictably is really operative word in there so obviously 1 which of algorithms usually need in the computer science and mathematics and patterns of instructions articulated in cold or formulas but you can also think of algorithms as being part of everyday life all the time these patterns of instructions articulated in other ways such as for instance a
recipe but there or 8
lovely shawl but you know if you're worse code nightmare uh looks like this you doing well I
OK so did learning is a particular new actually new old the style of algorithms so the technology goes back to in theory at least the 19 fifties but there's been some recent breakthroughs us since 2012 and 2013 that are really reshape the landscape of what's possible with machine learning it's really just the new hotness because of this arm oversimplified even think of machine that deep learning as of algorithms for fast Trainable Artificial Neural Networks it's a technology that you know like I said it's been around in academia for instance for a long time but really mostly theoretical scale and breakthroughs in peril zation but in the availability of processes of mesoscale these things are making it much more viable to actually take this knowledge production use and so because of that deep learning has become realistically able to extract insights out of vast Big Data in ways that we never really even been able to ImageNet up before deep learning is a particular approach to building and training artificial neural networks you think of them as decision black boxes I was so was is that meanwhile OK so we have some inputs you giving array of numbers representing words concepts objects in some sense and then executing it by running a series of functions that you write and executing against the array and the outputs you get are of the machine learning and predicting properties that it things will be useful in the future for drawing intuitions about similar datasets so you get this training datasets ASM miners patterns here this is probably how you can figure out similar results next time and then you controlled much larger datasets that it and it's able to come up with similar um predictions the so what would that look like for instance OK so there's a few problems here black box I the look at back 2nd so because deep learning relies on artificial neural networks automated discovery of patterns within the training dataset against supply those discoveries to draw intuitions about the future inputs a however that means that every floor assumption in the training dataset or in those original functions that you've written are gonna have unrecognized influence on the algorithms and on the outcomes of the generates so that's something that we can revisit but let's look at a really basic practical example of deep
learning Myo it's an artificial neural network is that teachers itself how to play Super Mario World it starts with actually no clue what so ever is a great you to the video of this by the way in action it spends 24 hours simply manipulating numbers and seeing what happens when a change is like pixel locations for instance and discovers that after 24 hours by golly it can play the whole game of I told you it was amazing How so it's
learning it it is not being given rules is not being given information it's exploring its world harm so can imagine on the datasets that we have access to what we could do with that learning to identify patterns and then using patterns to draw insights this is something we try to do all the time so speaking of Games
char
this well my punch line here Mario would you like to play in scale you global to learn plays global thermonuclear war and all of that word I'm going to play a different game
a and will call it bingo it is the US carriers bigger that you'll ever played as there deathly i'd angles but some were good we'll cover the sport real source said look at some of the ways that we have serious pitfalls deal with of will but we need to appreciate how many of those exist why presented challenges why we have to build considering dealer them in the very near future so that place around using some case studies as examples target making of her to this 1 right so they had uh figured out that hey you need people to radically change their buying habits if you catch them in this get trimester of pregnancy something that specific but how do you find out that you have a customer who is in her 2nd month of pregnancy the somatic muscle trimester and you know that certainly not something I ask on the visitor cards so they found that there were a few data points that in fact reliably predict that this and so this sending ad circulars and as
an circulars were very focused on the things that you need in your 2nd trimester as you're planning your little nest our met when they make and then he's actually furious wide my teenage daughter get this this is outrageous I you trying to encourage her to have sex I'm and supposedly came back adhere to later is that while I'm so sorry I don't know what's going on in my household my daughter is in fact right arm so we have this you know really useful predictability our own target to the lesson from that conversation the lesson they took was we really should disguise the intent of these ads better so now what you get is you get diaper but you also get like lawnmower had an API like Auto Oil had so you don't know that in fact only 1 of these ads is the 1 that's being actually targeted at you so here's here's a lesson about you know mining data is to be honest about what we're doing this was not actually the way to solve this problem that we knew too much there are lots of different ways to deal with it Shutterfly on this 1 is somewhat
less known but it's somewhat similar territory they sent out an ad
and are an e-mail saying congratulations new pair at the time to take out of odds dishonest thank you notes unfortunately they did not
really predict well the that survivor the congratulations of my new bundle of july I'm horribly Confero but hey I'm adopting cats I lost a baby in november who would have been due this week it was like you know that all all over again yeah Shutterfly responded the intent of the e-mail was to target customers who have recently had a baby yeah I get your intent some false positives are a problem we can treat them as edge cases what is the impact of a false positive was the impact of a false negative they didn't really apologize for having targeted wrong just 3 having received something Mark Zuckerberg of all people has little something to say about this a couple months ago he announced that he is the father soon but he also wrote about are the 3 miscarriages that they experienced it before that and he said he feels so hopeful when you learn you're going to have a child he start imagining who they'll become and dreaming of hopes for the future you start making plans and then the gun it's a lonely experience the I can imagine that the mark sucker Berger of 10 years ago would have understood and I think of all the feces to focus on hiring the marker Zuckerberg's of 10 years ago who believe that people who had had not nearly enough experiences of hard things in life are the best people to develop apps for people who have the this is the lexical burkas far more interesting to me he's got a much better insight on all the possible ways we can screw up or at least 1 of them the yeah
Facebook here and view of 4 years they have been doing this but I'm sure the current tweet how just servicing some posts from the past year that they feel were kind of a highlights reel of year year I problem is that not everything that had a lot like the past year it's something that necessarily was positive for you not everything that people commented on was something you wanna remember and sometimes the things that were back then exactly that exciting and memorable and wonderful have change over the course of a year we have changes in our relationships in our jobs and our location in our feelings having the belief that we can predict how people are feeling now about something happened in the past is so absurdly arrogant from the Eric Raymond
coined the term inadvertent algorithm cruelty and he defines it as the result of code that works in the overwhelming majority of cases but doesn't take into account other use cases and the reason that he gets to claim this is
because he's won the people it happened to his daughter died and you're view but that back and capturing it back over and over there's no obvious way to stop that and this is an algorithm to this was a predictable result right some and predicts that not everyone wants to have this come forth in unwillingly still his recommendation is
that we need to increase awareness and consideration of failure modes the edge cases worst-case scenarios and so I'm trying do that today but what we can do is I hope accomplish a little something a little bit moving forward that conversation and with that in mind my 1st recommendation here for avoiding this particular specific pitfalls along with a few others is be humble
we cannot into it the inter state emotions and private subjectivity of our users not yet anyway which raises issues of consent especially when Dean itemizing private data yeah
Fitbit so you can think of it as the attractor of all sorts of activities straight out 1 of the things that use track is your sex life a home it doesn't anymore and there's a reason for that
that our yeah so it it was in your profile then the profiles Republic end yeah the profiles are
pretty interesting to I I can't decide whether we really have to really sad for this person but everything had people feel about the Ashley in exposure OK so people would want to brag and the all competitive you know so for other people this is even horrible violation of their privacy and their expectation of privacy this was something crucially what's matter here is default you need to be thinking through what is default private what is default public and what are the consequences of making a port default decision I'm so hilariously how they dealt with this you
know deep technical problem there is a change robots . org a dot . txt so in a problem solved I'm sure you all you see if it fits and yeah and so this is why they no longer do sex tracking because they really were willing to put in the effort to do it well and you know what I actually think that that is a fair decision makers were not prepared to handle data really thoughtfully and well especially that something can harm them then yes after doing all together that's a result product decision
and so it over had died view if you hadn't heard of this arm in many ways is similar anyone else's admin tools you know we always need stuff for monitoring analytics while while they're making sure that the service has that is doing what you expect and burst is happening held got the well and happily was called at the on the problem here was not that they
had some sort of monitoring system but the way they used it and the way they use that words for funds but they use it to show potential investors for instance at dinner parties Hey look let's track of all the gas coming to the party and so they don't need to know I have some of the people were not OK with that and they use it for things like an executive was patients that a reporter was on the way and he guilty long so the tractor but in order to meet her at the lobby and let her know and share data with her and these are not necessary uses of data we need to be thinking about how much access is actually made to private data and as a really be need to be used in the way that were using it we make tools and then we spent too much on how they're used and Deuba was certainly the case that but I think it's a
little bit telling that they do really evil things like this in the code this is actual God you code and look at that and say true and I you remember OkCupid does blog they they don't really do any more but the research group at the dating site OkCupid these a blog about things that they're learning from the aggregate data trends and they would just focus on sharing central insight into simple ways that an OkCupid user could use their dating site patter right arm blueberries to blog about their data to crucially different a few things there's was not about improving customers experience of the Service there's was about invading people's privacy in order to pass judgment on their sex lives this is not a predictable consequence of signing up for a transport service you don't expect that you're gonna have someone being drawing conclusions about your private life and again you know it's so easy to look at something like Ashley somatosensory OK with those people have some sort of plausible deniability of abiding do what you said I did who were saying we know for sure we think our data it definitely predicts that you the worth what this is again a lot of opportunity for consequences for the user did they expected no the the don't do
concept like Cooper architect for real concern architect for informed consent so what does that mean informed
consent is permission freely granted where you know is a totally consequence free alternative and is the default value and it's giving informed appreciation understanding ahead of time of the facts implications and consequences involved in giving a yes and if that sounds overly burdensome come on look at the TOS we can set aside a little bit of information a jangly say like yo we set out to do this really fun stuff we look at your data and we say interesting things about you OK with that this doesn't have to be tremendously hard but we do have to make some sort of effort to give people better disclosure of what they're getting into go
add words is another really interesting example and there was Harvard research into a but basically it took 2 sets of names 1 that's highly correlated with people or block 1 that is highly correlated people who are white and then found real names of working academics who had those 1st names and some sort a last name so if we say that uh what toyour is a new highly correlated with black women then we say OK so little Adams is a real academic and we do center center and what they found were that when you search for names of white people versus black people you have
ads that are more likely to show up that implies that the black person has an arrest record but think about what that means because added algorithm is focused on predicting behavior and that's it it doesn't care about content it just has a template to fill and select which 1 its job is just to figure out what is going to make us click what based on what it knows about us and what it knows about others behavior with those same that's somewhat is reflecting back on us
and of the flames also really useful for doing photo analysis for RTE scenes in is I've
photo recognition facial recognition this actually was not using deep learning but here we're familiar with that some the interesting trends in possibilities that you can see right arm it's getting a lot better you know I I I thought was more helpful now but um but we're also seeing
the even with far more sophisticated implementations that are using deep learning this is from just 2 months ago so this is not in fact a jungle gym it's specialists this is not an
animal this is flicker J. tagging as well this is Google doing
auto-tagging To their credit they acted on it really quickly once told notch their credit this was a month after flicker had had that happen with black people being apes so learn to pay attention to the news they should have been acting on that a month sooner and they did get taken care of within 24 hours but that says that 29 days earlier this could never happen so why does something like happen at all and the answer goes back to the 19 fifties when Kodak was 1st developing ways to have really calibrated photos and so in order to make sure that voters were consistently rendered with nice accurate colors they created where culturally parts and these were cards
used to color balance out the printer and you might notice something about them Shirley cards were of white women film stock was optimized to accurately reproduced details in white skin people the digital sensors of today are still trying to mimic the results of film processing if we saw something radically different we be complaining with the sensor such the problem is that that means we have a a decade after decade of lessons legacy data and current data that is all really poor data it's of dark skinned peoples we think we have objective vast datasets but the dataset is polluted noise might have to
skip a few because running well in times of the jump
ahead a bit FIL so
rationale sort algorithm can only be seen from inside of that black box I talked about and so that's great what so the direction of that's a picture of the inside of the black box it's in there it's where thing about making decisions inside the black box is is no accountability there's no way to check our work and there's no recourse for people when they tell us we've got it wrong so there are some
things we can do is take some lessons from that and they're actually are professional at the CIS including ones are over fashion Aisha had heard about them and probably you haven't either I adaptive few rules from various sources including the Association for Computing Machinery so let's take a look at a few of those 1 avoid harm to others this might sound a lot like save medical ethics rules no had we do that well the our decisions potential impact on others while we're doing development while making decisions projector likelihood consequences contribute to human well-being minimize negative consequences to others it's not enough to just say I got this is a problem minimize minimize what is the least invasive solution we need to be really honest and trustworthy even more so because we are going to screw up at some point when they need to be able to sincerely apologize and be believed we need to be gendered enough trust that we can keep moving forward after we've made a failed so we need to provide others with full disclosure of the limitations of what we do and call attention to signs of risk we also need to actively counter bias inequality
because code is made by people it's not objective it reflects our tunnel vision it replicates or flies algorithms always have
underlying assumptions about meaning about accuracy about the world in which the generated by help code should assign meaning to data underlying assumptions influence outcomes and consequences being generated all weights so we need to do things like
order outcomes of your Member adage trust but verify them the apply
here but that's insufficient because photo of black
box instead what we have to do
is not trust but audit constantly this half the part of our practices of checking what's coming out is it what it should be is it what's expected is an unbiased so were
in this arms race right now because Google Microsoft Apple Facebook they're all deeply investing in these technologies and the doubling down in there already rolling out implementations of fascinating but they're moving forward very quickly and the problem with that is that they are both getting more precise in their correctness and more damaging in the wrongness so we need to be really thoughtful about how we move forward this to make sure that we are focused on being more precise in our correctness and using correct me information very carefully so we have
consistent injury because we want be and that right so this is what we need to do we have
have decision-making authority in the hands of highly diverse teams who can anticipate diverse
ways to screw up and finally we
have to commit to transparency we have to
be able to apply our own expertise to saying no to things they're gonna harm users at all we have to be will say
no over over again 2 pitfalls like these as much as is necessary because we understand the consequences refused to play along with that
thank you
thank and the this is and
Umwandlungsenthalpie
Resultante
Algorithmus
Web Site
Ruhmasse
Optimierung
Code
Software Development Kit
Algorithmus
Videospiel
Nichtlinearer Operator
Mathematik
Mathematisierung
Computer
Quellcode
Menge
Computeranimation
Vorhersagbarkeit
Ausdruck <Logik>
Konzentrizität
Informationsmodellierung
Algorithmus
Menge
Rechter Winkel
Code
Mereologie
Mustersprache
Wort <Informatik>
Informatik
Instantiierung
Objekt <Kategorie>
Resultante
Prozess <Physik>
Wellenpaket
Blackbox
Zahlenbereich
Gebäude <Mathematik>
Physikalische Theorie
Code
Computeranimation
Entscheidungstheorie
Virtuelle Maschine
Algorithmus
Prognoseverfahren
Datennetz
Mustersprache
Algorithmische Lerntheorie
Neuronales Netz
Funktion <Mathematik>
Algorithmus
Lineares Funktional
Zentrische Streckung
Kategorie <Mathematik>
Gebäude <Mathematik>
Reihe
Prognostik
Ähnlichkeitsgeometrie
Biprodukt
Ein-Ausgabe
Entscheidungstheorie
Objekt <Kategorie>
Reihe
Funktion <Mathematik>
Zahlenbereich
Ein-Ausgabe
Verbandstheorie
Wort <Informatik>
Neuronales Netz
Instantiierung
Pixel
Gruppenoperation
Mathematisierung
Zahlenbereich
Fitnessfunktion
Schlussregel
Computeranimation
Videokonferenz
Unendlichkeit
Spieltheorie
Super Mario World
Mustersprache
ART-Netz
Information
URL
Instantiierung
Neuronales Netz
Zentrische Streckung
Lochstreifen
Spieltheorie
Wort <Informatik>
Gerade
Computeranimation
Beobachtungsstudie
Subtraktion
Umsetzung <Informatik>
Punkt
Freier Ladungsträger
Winkel
Automatische Handlungsplanung
Vorzeichen <Mathematik>
Kolmogorov-Komplexität
Automatische Differentiation
Quellcode
Information
Fastring
Datenmissbrauch
Computeranimation
Data Mining
Chipkarte
Überlagerung <Mathematik>
Prognoseverfahren
Rechter Winkel
Rechter Winkel
Reelle Zahl
Identifizierbarkeit
NP-hartes Problem
Videospiel
App <Programm>
Ortsoperator
Automatische Handlungsplanung
PASS <Programm>
Digitalfilter
Information
Zuckerberg, Mark
Computeranimation
Entscheidungsmodell
Einheit <Mathematik>
Softwareentwickler
Computerunterstützte Übersetzung
E-Mail
Term
Brennen <Datenverarbeitung>
Faserbündel
Identifizierbarkeit
Resultante
Facebook
Sichtenkonzept
Mathematisierung
Dialekt
Information
Term
Code
Datenmissbrauch
Computeranimation
Algorithmus
Twitter <Softwareplattform>
Prozess <Informatik>
Rechter Winkel
Code
URL
Identifizierbarkeit
Resultante
ATM
Bit
Algorithmus
Zeitkomplexität
Kontextbezogenes System
Computeranimation
Algorithmus
Videospiel
Weg <Topologie>
Datenmissbrauch
Rechter Winkel
Mixed Reality
Digitalfilter
Repellor
Information
Quick-Sort
Computeranimation
Aggregatzustand
Erwartungswert
Datenmissbrauch
Default
Profil <Aerodynamik>
Default
Computeranimation
Entscheidungstheorie
Resultante
Roboter
Sichtenkonzept
MIMD
Mathematisierung
Systemverwaltung
Kolmogorov-Komplexität
Information
Biprodukt
Computeranimation
Entscheidungstheorie
Roboter
Weg <Topologie>
Dienst <Informatik>
Rechter Winkel
Einheit <Mathematik>
URL
Zentralisator
Vektorpotenzial
Bit
Web Site
Web log
Weg <Topologie>
Gruppenkeim
Code
Computeranimation
Datenhaltung
Weg <Topologie>
Videospiel
Äquivalenzklasse
Datenmissbrauch
Konvexe Hülle
Physikalisches System
Sichtenkonzept
Quick-Sort
Modallogik
Dienst <Informatik>
Token-Ring
Twitter <Softwareplattform>
Rechter Winkel
Grundsätze ordnungsmäßiger Datenverarbeitung
Wort <Informatik>
Ordnung <Mathematik>
Verkehrsinformation
Instantiierung
Bit
Freeware
Reelle Zahl
Default
Besprechung/Interview
Äußere Algebra eines Moduls
Information
Default
Quick-Sort
Computeranimation
Systemidentifikation
Template
Kolmogorov-Komplexität
p-Block
Automatische Differentiation
Information
Datenmissbrauch
Quick-Sort
Computeranimation
Datensatz
Datensatz
Algorithmus
Menge
Prozess <Informatik>
Wort <Informatik>
Inhalt <Mathematik>
Identifizierbarkeit
Mustererkennung
Datenmissbrauch
Computeranimation
Digitale Photographie
Demoszene <Programmierung>
Twitter <Softwareplattform>
Digitale Photographie
Rechter Winkel
Rechter Winkel
Korrelation
Benutzerführung
Analysis
Identifizierbarkeit
Analogieschluss
Implementierung
Summengleichung
Resultante
Abstimmung <Frequenz>
Digitale Photographie
Mereologie
Besprechung/Interview
Geräusch
Strömungsrichtung
Kantenfärbung
Ordnung <Mathematik>
Chipkarte
Algorithmus
Bit
Sortierverfahren
Datenanalyse
Blackbox
E-Mail
Information
Datenmissbrauch
Computeranimation
Richtung
Entscheidungstheorie
Quellcode
Wechselsprung
Zahlzeichen
Kommunalität
Mobiles Internet
Identifizierbarkeit
Vektorpotenzial
Punkt
Extremwert
Likelihood-Funktion
Besprechung/Interview
Schlussregel
Quellcode
Code
Service provider
Computeranimation
Eins
Entscheidungstheorie
Algorithmus
Ungleichung
Vorzeichen <Mathematik>
Anpassung <Mathematik>
Inverser Limes
Softwareentwickler
Maschinelles Sehen
Beamer
Gewicht <Mathematik>
Ordnung <Mathematik>
Code
Hilfesystem
Computeranimation
Quader
Mereologie
Computeranimation
Systemidentifikation
Programmierparadigma
Vektorpotenzial
Information
Computeranimation
Autorisierung
Programmierparadigma
Information
Computeranimation
Transinformation
Weg <Topologie>
Körpertheorie
Ereignishorizont
Computeranimation

Metadaten

Formale Metadaten

Titel Consequences of an Insightful Algorithm
Serientitel DjangoCon US 2015
Teil 45
Anzahl der Teile 46
Autor Zona, Carina C.
Mitwirkende Confreaks, LLC
Lizenz CC-Namensnennung 3.0 Unported:
Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen.
DOI 10.5446/32756
Herausgeber DjangoCon US
Erscheinungsjahr 2015
Sprache Englisch

Inhaltliche Metadaten

Fachgebiet Informatik
Abstract We have ethical responsibilities when coding. We’re able to extract remarkably precise intuitions about an individual. But do we have a right to know what they didn’t consent to share, even when they willingly shared the data that leads us there? A major retailer’s data-driven marketing accidentially revealed to a teen’s family that she was pregnant. Eek. What are our obligations to people who did not expect themselves to be so intimately known without sharing directly? How do we mitigate against unintended outcomes? For instance, an activity tracker carelessly revealed users’ sexual activity data to search engines. A social network’s algorithm accidentally triggered painful memories for grieving families who’d recently experienced death of their child and other loved ones. We design software for humans. Balancing human needs and business specs can be tough. It’s crucial that we learn how to build in systematic empathy. In this talk, we’ll delve into specific examples of uncritical programming, and painful results from using insightful data in ways that were benignly intended. You’ll learn ways we can integrate practices for examining how our code might harm individuals. We’ll look at how to flip the paradigm, netting consequences that can be better for everyone.

Ähnliche Filme

Loading...