Despicable machines: how computers can be assholes
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 160 | |
| Author | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/33745 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
EuroPython 201779 / 160
10
14
17
19
21
32
37
39
40
41
43
46
54
57
70
73
85
89
92
95
98
99
102
103
108
113
114
115
119
121
122
130
135
136
141
142
143
146
149
153
157
158
00:00
Virtual machineConvolutionBitComputer-assisted translationMedical imaging1 (number)View (database)NeuroinformatikAlgorithmLogic gateMathematical analysisPoint (geometry)Right angleBlogArtificial neural networkInheritance (object-oriented programming)Category of beingConnected spaceElectronic program guideQuicksortFlow separationDifferent (Kate Ryan album)Physical systemCASE <Informatik>Similarity (geometry)Universe (mathematics)Multiplication signVideo gameMetreCycle (graph theory)Traffic reportingAreaMassSoftware testingMachine visionBit rateService-oriented architectureComputer fontPerspective (visual)Presentation of a groupSummierbarkeitProcess (computing)Electronic mailing listRobotTheorySoftwareEvent horizonSpherical capParameter (computer programming)2 (number)Computer animation
09:08
WordVector spaceMetropolitan area networkComputer programWave packetSet (mathematics)Three-dimensional spaceNatural languageRepresentation (politics)Einbettung <Mathematik>Process (computing)CodeEndliche ModelltheoriePhysical systemMathematicsSpacetimeSimilarity (geometry)CASE <Informatik>Constraint (mathematics)PreprocessorQuicksortDistanceMetric systemMorphingResultantPoint (geometry)Arithmetic meanDimensional analysisNatural numberSign (mathematics)GenderSound effectEuler anglesWeightRight angleDiagram
14:09
Error messageWell-formed formulaTerm (mathematics)Physical systemHypermediaLikelihood functionGoodness of fitProcess (computing)TwitterPoint (geometry)Multiplication signBlack boxContext awarenessFitness functionLogicRepresentation (politics)Sound effectInterpreter (computing)Endliche ModelltheorieError messageDifferent (Kate Ryan album)Arrow of timeBuildingSet (mathematics)AreaWave packetGenderMathematical analysisPerspective (visual)Sampling (statistics)Task (computing)Parameter (computer programming)ResultantSpherical capComputer-assisted translationEvent horizonQuicksortPersonal identification numberMedical imagingNoise (electronics)Computer animationMeeting/Interview
19:10
Greatest elementRow (database)WeightSet (mathematics)Medical imagingQuicksortBitWaveFile archiverPredictabilityLatin squareHypermediaParameter (computer programming)Computer animation
20:33
NeuroinformatikQuicksortVirtual machinePhysical systemBitDecision theoryProcess (computing)CatastrophismPoint (geometry)Multiplication signState of matterMereologySelf-organizationRight angleRootComputer animation
22:35
Multiplication signComputer animation
23:02
Endliche ModelltheorieEstimatorDecision theoryDataflowPoint (geometry)TheoryMereologyAreaContext awarenessDirection (geometry)Position operatorMeeting/Interview
24:39
Pattern languageState of matterPhysical systemEndliche ModelltheorieSampling (statistics)Bit rateLatent heatPrisoner's dilemmaRight angleContext awarenessLecture/ConferenceMeeting/Interview
25:36
CASE <Informatik>Multiplication signPower (physics)Context awarenessState of matterPhysical systemBitCompass (drafting)Lecture/Conference
26:27
Reflection (mathematics)Perspective (visual)Context awarenessDifferent (Kate Ryan album)BitEndliche ModelltheorieSet (mathematics)SimulationPoint (geometry)AreaMereologyCross-correlationState of matterRule of inferenceCausalityPerfect groupContent (media)Negative numberData compressionMultiplication signCASE <Informatik>StatisticsWord1 (number)Computer animation
29:26
Multiplication signLecture/Conference
30:01
Multiplication signEndliche ModelltheorieObject (grammar)Computer programmingLocal ringLecture/Conference
30:52
Mathematical analysisVirtual machineComputer-generated imageryObservational studyGame controllerNumberTwitterLikelihood functionDeterminantWhiteboardOffice suiteVirtual machineSet (mathematics)Standard deviationWebsiteSpectrum (functional analysis)Process (computing)Software bugSoftwareEndliche ModelltheorieQuicksortSoftware testingData modelSoftware industryBitMereologyDependent and independent variablesCartesian coordinate systemData storage devicePoint (geometry)Product (business)Student's t-testLecture/Conference
34:09
Information securityRight angleNeuroinformatikWritingExpert systemMultiplication signDigital photographyTheoryQuicksortLecture/Conference
35:07
Observational studyCASE <Informatik>ForestTwitterLecture/Conference
Transcript: English(auto-generated)
00:05
But hopefully that the aim is already fulfilled. So I'll just let's just reinforce it and Just as a side note. I know it's the sort of almost the end of the fourth day So I have some gratuitous references to British pop culture in the presentation and if you spot anything shout
00:23
And you'll get a sweet I'll try to throw it at you if you're close enough Okay, so before we dive in I just want to credit some people who have really inspired me to Think about that and I kind of learned a lot from the first one is Zainab Tufekci, she's a researcher really interested in technology and
00:44
She thinks really deeply about About these things and I can really recommend her. She's got lots of articles not only about this topic, but also many things related She's got a book and our online talks that you can see Kathy O'Neill wrote a book on the topic. It's called weapons of mass destruction
01:05
We probably heard about it already, but you know It's kind of the Bible of of this area and finally Catherine who gave a amazing keynote yesterday morning You know a member of our own community and she also gave a Talk on pretty much the same topic. I'm going to talk about today at pi data Amsterdam a couple of months ago
01:24
So I try to make sort of talk about things from slightly different perspective, but the areas we cover are kind of similar So, you know I do my best but I can't hope to cover everything As well as they did and I just a note I think it's also kind of interesting that these are all women with different backgrounds and I think it also goes to show how
01:44
Important diversity is and I know that our community Cares about it a lot, but there is still work that we need to do Okay, so Does anyone know what that is? There you go. Okay, sweet
02:01
So this is deep thoughts. It's a computer from hitchhiker's guide to the galaxy and it was built by super intelligent multi-dimensional beings who got fed up with existential angst at some point and Decided to build a computer to to give them the answer to life universe and everything
02:23
and you know They build this computer and then they asked for the answer and the computer told them to come back in seven millionaires But you know, what is a bit of waiting for such an important answer? They waited seven million years they came back and then the computer told them that
02:41
The answer is 42, but it doesn't know what the question is So, you know, this is just a funny science fiction novel, so what does it have to do with reality and I think there is actually a connection and one of the reasons that hitchhiker's guide to the galaxy is so funny is that it has some very pointed criticism of
03:02
Our society and the way we think And I think it brings up some really good points so first of all, if your question is rubbish, you will not get a useful answer and That might seem pretty obvious, but keep that in mind because a little bit later we will come back to that It's not always obvious to everyone
03:23
But also secondly, let's think about for a second Why did? Super intelligent multi-dimensional beings Decide to build a computer to answer this question Like why didn't they instead decide to invest in humanities and let a well-funded
03:41
philosophy department sort this out, right so I Think one of the reasons it was that is because we think computers are objective, you know a philosopher Has their own personal biases They have their own points of view and basically they cannot be trusted like not really right
04:02
They are definitely biased in some human way Whereas computers are you know governed by ones and zeros and algorithms and logic gates and have no morals. So whatever answers they give us are Unbiased and objective and therefore true, right?
04:21
Does everyone agree with that that the answers that computer give us computers give us are always true Awesome so either I don't have lots of work to do to convince you or I'm very good at posing questions in a very suggestive way Okay, so let's say I was exaggerating a little bit computers aren't always giving us the right answers but
04:44
When they give us the wrong answer It's pretty easy to tell like you might have a convolution neural network that is you know Classifying images and it might misclassify an image of a cat as a penguin So, you know you look at that and you think it's an obvious mistake It's kind of absurd. It's very easy to spot. So I'm not really worried about it, right?
05:04
It will not trip you up in any dangerous way Do you agree that you know, you can one computer gives you an answer if it's wrong You can tell whether it's wrong or not Okay, at least one of you agree with it, but probably not most okay, so no one agrees awesome
05:28
Okay, so Even if you can't always put mistakes And I guess I know already what you're going to answer But do we agree that at least things like racism and sexism are you know, not computer things. There are human
05:43
Prejudices and computers don't have them Right, no one agrees as well. That's great. Sure, but when a computer gives you an answer, you know
06:00
The question is whether it can't contain biases like this or not So, you know, no one I guess no one falls into this trap already, which is awesome so Do you know who these people are? So there was an article published last year last year by ProPublica. It's an it's a publication and
06:22
Basically They explained how Predictive policing algorithms work and how they get things wrong First of all, when you say predictive policing a few years ago I thought you know It would be a joke like minority report style thing was a joke phrase for a long time for me
06:42
And when I read about that, I was really kind of shocked but turns out it's not a joke It's an actual thing. Both of these people were arrested in Florida in 2014 the person on the left His name is Bresha and she was arrested for stealing a kid's bike sort of Cycling on it for a few meters throwing it away and then running but she was caught and arrested
07:05
The guy on the right. His name is Vernon. He was arrested for shoplifting some stuff That was of similar value to the bike. So they're both arrested more or less at the same time independently And when they were arrested their risk of recidivism So risk of reoffending was assessed by a system called compass. So it's a thing that
07:25
that the police and Florida uses so they put data about the Other person into the system and the system says this person is high risk of reoffending This person is low risk and so on in this case Bresha was given high risk of reoffense and Vernon was given a low
07:42
risk assessment Now the article was published in 2016 So two years after all this happened and by that point we already knew that the system got it wrong So after two years Bresha didn't reoffend while Vernon broke it to some Warehouse so much more valuable things and is actually in prison. So, you know the system got it wrong
08:04
But you know shit happens. We know that computers are not perfect. So sometimes get they get get things wrong and Let's put aside for now the you know, the question whether any kind of algorithmic inaccuracy is acceptable for such an important system because there are even bigger problems that the journalists found and
08:24
Namely there is a racial bias in the system. So people of different races Tend to be misclassified at more or less the same rates. However, white people when they are misclassified are given too low Risk assessments while black people when they're misclassified are given too high risk assessments
08:44
So there is racial bias in there and it's it actually it's actually impacting people and there is much more to that There is a the article itself is really interesting. There is a separate blog post detailing the analysis that they did There are also some rebuttals to it. I really encourage you to look into it. It's it's a very interesting topic
09:03
So, let's see how it might happen that a system like this can be biased Do you know what these are? There's no sweets for that These are word vectors. So I'm just going to quickly explain it for those of you who don't know in 2013 there was a paper published by Google on natural language processing and
09:24
They introduced the system called words 2vec and the idea is to take some corpus of text You put it into this model and it spits out and embedding and it means that what it spits out is a representation Where each word? It's described by a set of coordinates so in this case, it's something like 300 coordinates 300 dimensions, but you know, we can think of it as like a three-dimensional space and
09:47
each of your words has a position and if you imagine a word men and awards King there is a vector that takes you from one to the other The cool thing about world 2vec is that these vectors the relationship between words are
10:03
Meaningful in a way that you can take this vector and instead of the word of Applying it to the word man You can apply it to the word woman and instead of getting to King you get to Queen and this is awesome That allows you to do this kind of vector arithmetic. There's a question
10:20
Axes are arbitrary. There are these vectors in high-dimensional space. So this is just a conceptual Representation in two dimensions of this 300 dimensional thing just to get the idea across that there are relationships between words I don't know. It's
10:43
so word2vec is taking a word and Putting it in some kind of space that Axes don't necessarily have Interpretable meaning it's just modeling relationships So things close to each other, for example might be related things far to each other far from each other might not be related
11:03
I don't want to make this about word2vec we can talk about it Afterwards, but it's not the topic of the talk The point is that you know, this is an awesome technique. It's very very widely used. It's extremely useful And it's been used in many many papers
11:21
Including at NIPS. So NIPS is this premier conference for For AI basically any paper with neural in the title is very welcome there And last year there was a paper that describes how word2vec is also biased. So Turns out that word2vec is trained on Google News. So it's a huge corpus of data and you know, there is no
11:48
Most people I assume writing news are not Biased intentionally, but just because of the way the you know, our society works right now there are some biases there and one of the things they discovered is that if you
12:02
Take the word man again and you You can get to a computer programmer. So there's relationship You know this vector you can think of it simplifying as saying, you know It takes you from that from went to like a profession if you take the same vector applies to a word woman It takes you to homemaker. So, you know, there's clear bias there about
12:25
Which kinds of professions people are expected to have even though it's not really fair question Absolutely, so it's the problem of the training data
12:42
sure Right, so the point is that Even though your technique might be completely fine If you're using biased training data, you will get a biased result and it's not surprising but it took three years after publishing the original word2vec paper to
13:01
For this paper detecting the bias to be found The paper also shows the ways of addressing this and the ways are basically Trying to Morph the change the sort of pre-processed training data to remove these kinds of biases if you've heard of
13:20
Metric distance learning it's kind of similar idea to work the space in a way to satisfy some constraints Some relationships should be kept some other relationships should be thrown away and there are also other things you can do Some of them Yes
13:41
Right, so, you know if you have some training data that you have gathered yourself and you know that there are some features in it that Are actually biased but there might be biased due to the way you captured it and you might You might explicitly after thinking through it. You might explicitly want to not use these features for classification
14:02
So you might just get rid of these columns and you might be fine, but that's not always the case It's possible that some other features in your training set and code these features that you just got rid of So even though you get rid of them, you know, if you think of an example of like Data about people, you know name and where they live and gender and race and so on
14:23
Let's say you want to get rid of gender and race If you look at a role describing, you know Giving someone's name and let's say interests or postcode does you have a good chance of figuring these out? And our models are Also capable of doing this, you know in fact we spend lots of time making sure that our models are capable of building up these hierarchical representations and
14:46
And running with it so they might still use that to Sort of to you know to do the classification event or you might not want them and to understand why this is problematic I just want to share a little story that Zainab Tufekci also
15:01
Mentioned so she went to this conference for HR professionals at some point and apparently there. Everyone was really excited About the system for taking a huge pile of CVS and matching the best people to the job openings, right? so we don't have to throw troll through your CVS manually just do this for you and You know, that sounds great. If you ever did any hiring, you know
15:23
It's a huge pain to read all this stuff and then try to figure out who's a good fit for what? So if we had a system that does it automatically, that would be awesome Let's put a pin for that in that for a second At the same time there I'm aware of at least two papers who did this It's possible to take data about someone from social media and predict
15:43
Sort of classify how the risk of this person getting depression at some point in the future one paper used post from Instagram the other paper used post from Twitter and they were actually able to Fairly accurately detect who's lucky to get depression sometime soon before the initial analysis. Sorry the initial diagnosis
16:03
So, you know, it's kind of cool that it's possible and I'm sure it can be good used for good But it's also a little bit scary and I'm sure you can now put the two and two together Imagine that the system that you built for You know matching people to jobs is based on training data. You have people that work for you already, you know, you know who are
16:23
High performers who are low performers and you can sort of extract the features and try to train your system on that There is no reason why you shouldn't use publicly available social media data The scary thing is there that the system might learn to discriminate on things that you really don't want to discriminate on like
16:41
you know likelihood of depression or Likelihood of being pregnant and things like this that you are explicitly trying not to discriminate against It might still be encoded into data and you might not even know about that And this is kind of a tricky problem to defend against and I think the best tool I know is I think Catherine mentioned it
17:01
In her pi date Amsterdam talk as well. Is this legal term used in the US? It's called disparate impact and it's there's a precise definition of that in the formula and it kind of treats your model as a black box and you can figure figure out whether your Black box is biased or not by you know, tweaking the parameters seeing what comes out
17:23
The thing is that it takes a bunch of effort you need to be conscious of The fact that this is possible and actively try to investigate it But I'm I'm hope I hope I can convince you that this is an effort well spent The other idea is that some errors can be very unintuitive. So like I mentioned, you know cats being
17:45
Misclassified as a penguin That's kind of you know, we can kind of understand it And there is again at nips but this year and in December there will be a competition run for the first time about adversarial examples and the aim is to you know, you have a classifier and your aim is to
18:05
construct a data sample that looks fine to humans, but will get Deliberately misclassified by by the classifier by your model So, you know We can add some specially constructed noise to your cat image to get it classified in a specific way at the same time The other teams will be try to build models that are robust to this kind of thing
18:23
I think it would be very interesting thing to watch But there's another even more scary problem perhaps That there are things that No one really intends to happen, but they happen anyway, because some models when they're not interpretable and a lot of them are not
18:41
Make mistakes in a way in ways that we can't really comprehend as humans very well now there was a talk earlier today It was really awesome about interpretable models and there is lots of research going into that area I think you know things will probably get better from this perspective, but it's still not perfect and I just wanted to sort of
19:00
Point this out because it's something to keep in mind that the errors that AI tends to make are often very different the kinds of errors that the humans can make Multiply them that's right. So the idea of stupid questions promised to come back to that Did you ever see paper where this image comes from?
19:23
Okay, so just a disclaimer this paper was not peer-reviewed It was just published on archive so you know don't take too much out of it, but it made lots of waves It was even covered in mainstream media, and I think it's important to talk about it In this paper they had a data sets of facial images and for each one They had label saying this person is a criminal this person is not
19:43
Guess what they try to do take face and predicts whether someone is a criminal or not And you know technically it's it's not really interesting They just took Alex nets if you know about CNN's and kind of retrained it a little bit and got a very good accuracy You know after the paper was published lots of people were outraged and rightly so
20:02
There are sort of many obvious problems with that, but let's just think about some like The row at the bottom is labeled as non criminals They're all the top is criminals like do non criminals smile like are people with white collared shirts never criminals
20:20
I think that's a little questionable, so you know again Silly example in some ways, but that's also something important to keep in mind make sure that the question you're asking actually make sense finally This one is a little bit more obscure That's right, so it's from a show Little Britain, and this is Carol. She's a receptionist and
20:43
She often helps people that try to get admitted to hospital, or you know get a bank loan and her trick is like Computer says no and like to everything, but I think it also illustrates the points. You know we
21:00
Started off developing these helpers including machine learning but not only to make our jobs easier to make us more efficient at making decisions and so on and But now turns out a lot of time we just defer our decisions to the computer and when the computer says no we just are not allowed to do it or unable to Do that and?
21:20
This is again something to just think about and keep in mind Whenever you are Developing something that might you know might be able to help you Just consider the implications and what happens when the system gets it wrong And I also really liked the quote and again in Katherine's keynote yesterday There was something about a guy who helped develop a system for a bank to write checks
21:44
you know he was wondering if it was an ethical thing to do because if they didn't do it, then the bank would have to innovate in a sort of Organizational way, but they had this new technical thing and they could just preserve the state status quo And I think lots of that is happening right now as well with AI there are millions of decisions that need to be taken
22:02
Every second we're incapable of doing that as humans So we just give it to the machines, but maybe that's not always the right thing to do So basically what I would like you to do is First of all read up on this topic you already seem well aware of you know that this is a problem Start thinking about it and talking about this with your co-workers and friends. There are lots of meetups and conferences
22:24
And I encourage you to really take this seriously I think it can be really awesome if we do this right, but it can be kind of catastrophic if we don't So the answer to why computers can be assholes is Because we make them
22:42
So please don't Thank you Question do we have time for questions?
23:02
Okay, so thank you very much for your talk. It's kind of like enlightening what I was I want to ask you is like maybe You should turn the question around like you're saying the data that we're putting into our models is biased And we need to kind of like prevent our models from this biased data
23:21
Maybe we should turn it around if you're thinking about ethical implications and kind of like say well We found out that our models are biased and we're actually able to quantify this and we can publish this and can tell you Well, there is this kind of bias in this data and kind of like make people aware of these issues
23:40
Because what we we do engineering models and we do classifications and at the first point We have a lot of false positives and it's the fault of the model and then afterwards we find false positives And we can go to the engineers and we can tell them Well, there is a false positive, but maybe your data is wrong and we can improve quality there. So maybe That's that's an approach we could take in the future
24:00
Yeah I think I totally agree as kind of a more more optimistic way of posing the problem and maybe that's more constructive and This talk that I mentioned from Mikhail just a couple of talks ago Don't know if you've seen it, but there was he presented a Python package that Is called Eli five explain it like I'm five it takes a model and then tells you, you know How the model makes decisions so it doesn't necessarily answer what the question whether it's biased or not
24:24
But it might help you interpret it and then be able to answer it So I think there's much more work to be done in this area, but going this direction I think is the right thing to do So what are your concrete recommendations with respect? I mean you just said be aware of this stuff
24:42
Like to pick them some of the most problematic examples predictive policing using that to figure out where to deploy police and Predicted rate of recidivism and something that started to happen recently at least in the u.s. Is using that in sentencing judgments So making people's prison sentences longer if you think they're more likely if the writer says they're more likely to
25:03
Be recidivist. Is this something that absolutely shouldn't happen or just you know, what is your specific recommendation be aware of? I mean, you know as soon as you get into if I could just point out you I mean you cited the criminal the criminal justice system of the state of Florida is like an anti pattern for him to do stuff
25:22
Right. It's one of the worst examples in the world of how to do these things And so any model working in that system is going to give you garbage results. Okay Sure, it's you know, it's still something that I believe should be pointed out So, I mean as soon as you get into ethical questions and moral questions, there are no silver bullets
25:43
So I can't give you a recommendation saying do that because even if I tell you do that and it's most Right most of the time dark there will be cases where it's wrong So, you know, I will hesitate to give you a straightforward recommendation That's why the best thing I can come up with is think about it if people building compass
26:01
Explicitly thought about racism and tested for it. This could be avoided now Of course compass is embedded in larger context of the whole justice system of the state and also the US so, you know it wouldn't prevent everything but like we have the power to change things a little bit at the time and I think we should try to do that
26:23
Okay, there was an idea that We can Find out that some model is biased and try to fix a bias by some fixes but I don't really like it because If you fix it by hand you can
26:45
Interfer another your personal bias into this data set and You and the second point is you get you are getting a fake reality Sure, so it's a good point But the way I look and I kind of also thought that for a while
27:04
The way I look at it and I'm not sure if it's right but it sounds more right to me is that it's not about taking Pristine data and messing with it. It's taking messed up data and messing with this in a different way There is no needle of them is absolutely right But you know you're messing it with it manually because you have the context you understand what it represents and you have opinions about
27:27
Okay, I think this shouldn't happen. So I'll try to make sure it doesn't happen now. There's no You know, there are very few examples where you can just take data and say it's the perfect reflection of the world so
27:41
To me that's that's kind of the answer. You just try to Make things a little bit better But none of them is like the true state the data you have after you gather it is not true in an absolute sense
28:04
That we use correlation because we are unable to get causation really and therefore actually these all these methods I mean as an ethical rule should be only applied to things which are not really Hot negative consequences for an individual because every classifier will for individuals get wrong answers. So
28:25
Okay, so I mean I'm not quite sure the question is but sure, there are I Think thinking about thinking about it from like a creation perspective
28:40
It's it's true sometimes, but there are other things that are much more difficult to put in this context like reinforcement learning, you know There are once you start getting into that You know intelligent agents they might also like the relationships between things that they do and the reward they get might be Very non-straightforward the
29:01
rewards they get might be maximized in a way that is totally not what we actually intend them to do and it's just I Think that's part of the problem But there are many different areas that can be kind of many different problems can be discovered When you think about what you do from that perspective
29:20
So I like the idea somebody mentioned over there Okay, so you've taken this model and run it against this corpus and found out Afterwards. Oh wait, there's a lot of bias here because of the corpus Turning that back around and saying okay. So what do we need to do to change things so that
29:42
in the world Say the news stories are less reflect are less Showing bias or things like that so that we can actually turn it back and fix the world. Yep Yeah, absolutely, so I think Again, no easy answers, but the reason why I think this is important
30:02
Is that when you build a solution that's biased in a way, it doesn't only reflect the world It actually reinforces the bias because of this, you know Perceived objectiveness people tend to trust things like that. So you actually are making things worse if you do this By not, you know by not releasing biased models or whatever you can do
30:24
That's one thing one way to prevent You know injustice in the world as as grand just as it sounds but obviously we should you know Go out of our bubble and also lots of other things we could do, you know get involved in your I don't know Local newspapers, you know have them out or I know join a political party or whatever
30:43
You think is the right thing to do, you know doesn't necessarily have to do with programming. We have time for three more questions Hello one of the things that seems to happen is that people publish a story saying sorry publish an article saying
31:06
this news outlet or the news outlet news on Google News is biased in this way, but there Would you suggest that companies try to take these data model these machine learning algorithms and run them internally and then analyze their purely their own data to see if they can
31:25
detect bias In-house, so they don't need to be Named and shamed they can do it internally to maintain their standards and then try to Sure Yeah, so I mean I think of that a little bit like
31:41
Like you know any kind of QA like finding bugs You can try to find them in-house or you can wait for your customers to tell you is the same thing here You can have some process for trying to discover these things before you release it Or you can wait for ProPublica to come to you and say hey, you know, you're biased. So absolutely I think it's not I don't think it's very common right now
32:01
At least I'm not aware of that, but I think this kind of testing of Models whenever you retrain your model you have to test whether it's still performing well or not We should definitely think of incorporating These kind of checks into into these processes, too You mentioned there was a study done on Instagram and Twitter users to determine
32:23
future likelihood of depression Firstly, how did they possibly find out if people were depressed in the future unless it was a controlled study and number two What are your thoughts on? If a medical study if you're doing a medical study on people you have to run it by an ethics officer Or an ethics board to make sure that it's ethical. Do you think something like that should be mandatory for models?
32:45
Which will affect people's lives in this way Okay, so to answer the first question. I'm not really sure I'll have to reread the paper or there are references here if you want to do that. It's a good question The second question was do we need ethics committee to like approve our models when we use them in production, basically
33:12
So should the software industry move to the model that medical industry is doing and you know have an extensive ethical review I don't know that you know, there are definitely different requirements
33:22
I think some software is sort of mission critical and like governs medical equipment, for example And it definitely does go through this process already If you're Instagram, you know, it might seem like oh, we're just posting filtered photos, you know, what can go wrong? So you might think that's not necessary But then again things like this happen and all of a sudden you're involved. So I don't know whether some kind of mandatory official
33:46
Requirement for doing these things is would be a good idea because it has a set of drawbacks as well You have to iterate much slower and so on But I think where we are now is at the other end of spectrum because we don't think about it enough I think we should pull towards doors that site even if we don't want to get all the way there. I
34:03
Have a last question What can we actually do to make the general public more aware that? Computers actually could be as follows great question If you're a writer write stories if you're a filmmaker, you know make animations
34:21
I think I think if we are aware of that as a community and it seems like we are getting more of that things will seep through like there are lots of Sort of knowledge or myths if you want about AI in the general public already I know everyone knows Siri or like Google Photos and stuff like this. These things slowly seep through and
34:42
like my most boring but straightforward answer is just Be aware of that and everything else will follow But if you want to specifically focus on public outreach, I think that's awesome, but I'm not an expert in that area. So
35:01
So the question about how did they find out whether the person was depressed was people who? reported being diagnosed as depressed They did a study on a lot of people and used that to predict so they had posted on Twitter For example that hey, I just got diagnosed it as being depressed. So that was where they got the
35:26
Data on. Oh, they're depressed Thank you. Thank you. And let's thank much again for his talk. Thank you