Bestand wählen
Merken

Opening Keynote: How an Algorithmic World Can Be Undermined

Zitierlink des Filmsegments
Embed Code

Automatisierte Medienanalyse

Beta
Erkannte Entitäten
Sprachtranskript
and if
if the and the energy here is absolute
spectacular on it is a total delight to be here with you today you know it wasn't just play and wanting to come here for many years and joked that they've asked me for 5 years and every time a guy I'm pregnant or having a
baby is a really awkward things I promise that as soon as I start having babies join you answer this constraint babies but now I'm here and I'm really delighted to be here so that I prepared for you today is of a strange QLogic issues all meant to come and challenges question algorithms but is done in these different pieces so work with me and tell me how this works but you have paranoia about online sexual predators had reached an all time high it was this constant refrain where people were saying that the Internet was dangerous for kids you heard it in the US you heard it in Europe and it was the concern about originally MySpace and then Facebook this was a site of serious danger the U.S. Congress has started putting together laws like the stopping online predators that could put together a task force about Internet safety as a researcher was a really frustrating moment that she knew all the data about risks young people I could tell you in boring detail what was actually going on what was actually making young people on but it matter because it was about facts or evidence that to some familiar right it was about whether or not people can get upset about the Internet over something and at that moment it was sexual crimes on the Internet was the issue this is the moment to me when I was pass to put together a collection of all of the data on online sexual relations for the regulators in America and 1 of the leading regulators came to me and then go find different data I don't like what you found OK T
is on the other hand was let on the topic were absolutely concerned that the Internet was unsafe they heard the rumors may have seen the TV show they seem to catch a predator and they were convinced so where there were kids being abducted so they needed to be safe online most people sort of looked at that someone yet another way in which adults want to ruin all the fun right and so they sat there and said OK whatever we're going north parents and try very hard not to be seen using the internet when in fact that actually we were doing the whole time but at the same
time there is a beautiful sight called 4-channel if you don't know what or channels I recommend the Wikipedia entry only up at that time it was seen as the underbelly of the internet and since then we've produced more and more underbelly of so we sort of think about those layers time it was a site that was primarily about honor made pornography that 2 major interests of 15 year old boys of and the fund producing means so they saw this conversation around on my section relation decided it was time to have a little bit of fun along comes oprah winfrey she
decides to talk about how the Internet was a really dangerous dangerous place which pose a lot of her stories from online for and indeed of our producers were looking at this and some of the fortune folks were having fun so what happened is that they managed to get Oprah to say line on national TV a wonderful statement let me read something that was was posted on message boards from someone who claims to be a member of a known pedophile network it said this he doesn't forgive you doesn't forget his group has over 9 thousand penis is everything children so I want you to know their organize and they have a systematic way of putting children and they use the internet to do it now if you know anything about 410 this is hysterical right it's not only are you making beautiful references all sorts of means over 9 thousand the references to anonymous Accetturo we to get oprah winfrey to say something ridiculous on live national television and this of course is the ability to have the attention economy so what happens with we think about trolling
shift focus for a 2nd so going to go back and forth a decade later most of the systems that we're talking about are shaped by algorithms search engines size shift cipher report contents and in order to basically use machine learning to determine what might be most relevant to any given query recommendation systems ranking algorithm they underpin huge chunks of our online experience on but meanwhile outside of the core internet practices we see on the use of data-driven technologies affecting criminal justice affecting credit-scoring affecting housing the ability to get employment I we started is see start after start think that algorithms a solution to everything I have become hot every company out there is positioning self at the Art AI solution to something so little what their action arguing but something we solve for AI on a satisfactory level even mean by i in the conversation I people had different theories on what I realize quickly is that I had just become the new big data was the mythology that if we just do something more with data we can solve all of these otherwise intractable problems but a senior executive at a tech industry company I explain even more precisely he was out of course all of these technology companies are investing in AI because what's the alternative natural stupidity we don't want invest in natural stupidity artificial intelligence some great well that's using other bases we got a problem because we
have this question about what does accountability look like how do we think about challenging the systems no of course here in Germany you are at the forefront of trying to challenge the systems questioning how these technologies can be used for building up different forms of resistance I really asking hard and important questions about where this is all going and I love artists and journalists critics and scholars who push that edges as a thank you for all of you doing that hard work on the that's really important
oxide what you were doing there are also people with dark agendas were finding other ways of making the systems accountable looking at ways of manipulating the systems at scale it's not just about manipulating oprah savings on national TV it about the use of dirty political campaigns the idea black ops advertising the ability to use any tool available to you to manipulate for particular agenda regardless of the social costs and this is where we're starting to see a whole new ecosystem unfold that has become more and more sophisticated over the last decade raises huge questions about what does it mean the whole accountable not just nation states or corporations but large networks of people who were actually working across different technologies is a lot learning to do it is to unpack and build that through what
started from examples in 2009 was a pastor in Florida in the United States he wanted to suppress the message on that Islam is of the devil and so pastor of a small church around 50 people he put up this sign in front of his church in order to invite people to share his views generally speaking not that many people right around Gainesville Florida in the scheme of things set so it reached different folks who grumbled then we got a little form he said I eventually spread this mentions louder for media and so he for used to burn the Koran it is threatening to burn the Koran also it's of at prejudicial organizations unsourced Szostak and they really press Releases telling him not to burn the Koran this year starts covering the press releases originally in Florida but it starts to scale and scale was it hits young who knows what that hits Google they become something people want to talk about it so he ends up on CNN talking about all of this people putting pressure on them to not do this its way back down cause he's got a lot of attention in this is really fundamentally what he wanted the attention 1 people stop paying attention to the issue as so he revises it in 2011 he was part of a group of people who burned occur on so what happened was that the news media said no we will not cover this discovering this is amplifier methods that's not important we will not go unwitnessed this act a little longer decided that they would do with is exactly right about it and what's that blogger wrote about it it rippled its way up the chain back to cnn he went it ends up on the front page of The New York Times the times of course is translated into many different languages around the world and so what happens is that why start breaking out in different parts of the world and notably 1 of those riots in Afghanistan results at least 12 people dead including 7 are United Nations workers why is this
important just like to imagine that they are the new for reporters of incidents they have a moral responsibility to report on anything that is newsworthy but I argue that failing to recognize that when they do they can take great heart is where we see the irresponsibility of this dynamic where journalism has paired and become part of the broader information media landscape where we need to think about the moral responsibility of being a voice of being an amplifier Micali John Donovan and I have a lot of time thinking about what strategic silence looks like and also the time when we say that like you want journalists to be silent to you the head of the New York Times actually said to me that this is a terrible idea the U.S. government asks to be silent all the time and we choose that we need to report a secure emphasizing the wrong word there the word or phrase strategic silence the idea is to be strategic and if you can't be strategic be silent you need to be strategic not just about what it means to speak but held the processes of amplification can ripple and cause harm this is something that's just no in fact 1 of the reasons why John Ivan collaborating on this is that she had gone back the history of strategic silence with regard to the 2 plants plant United States I have looked back into the history of suicide we know that when we report on suicide on related issues we amplify are the Gaussian consequences we actually journalist as as a community decided to be strategically silent World Health Organization put out a report encouraging them to do so that is all become undone because of the Internet to give you a sense of the amplitude of harm when Robin Williams died by suicide we cite 10 per cent increase in suicide over the next few months with the 32 per cent of replication of his math mechanism of suicide these are people's lives are at stake the chain of information flow is part of what
gets exploited is not no longer just about going to the gate keepers as and journalists is about finding ways of moving things across the Internet in through journalist and back around that becomes part of the gift I think that's the work of Ryan Holiday who was a basically the person manipulated the systems for as advertising purposes he figured out the key to getting large amount of attention was not to purchase advertisements that wasn't possible if you a little known and not recognize it was to create spectacle that little-known bloggers and feel the need to cover because they're paid based on the number of articles they can write and then to move up the chain to get the bloggers to be amplified back in the Twitter back into Google News up into mainstream journalism is you can create spectacle and move across the chain you can control the information infrastructure and that of course is
where the trolls show also figured out exactly these strategies they figured out how to get messages to move systematically across it whether they are paid at large scales by foreign states or the group a white nationalist looking to move a particular ideology across at what they do is pretty systematic next free also as a sockpuppet accounts on Twitter fake accounts at all they have to do is write to journalists and ask questions and what they do is they may ask the journals the questioning like what's going on with this thing and journalists under pressure to find stories report go looking around immediately search something in Google and that becomes a tool of exploitation because they come across on Wikipedia entry that is being conveniently modified to have phrases that might be of interest to them they end up getting the websites sites or blogs have been set up for them and that's how narratives are moved there moved by mainstream media because they're a tool of a variety of talk trolls we have to ask questions about
accountability who is now the blame here he is a Twitter for letting you create anonymous accounts that the journalist is looking for a scoop under pressure what other news organization whose standards of declines because they're trying to appease their private equity masters so we blame the Internet making it so easy to upload videos at Wikipedia or create your own website what reporters who feel that they have to cover something because as being covered by somebody else what you see is an accountability ecosystem not 1 actor being from place so with that environment looks like and think through the responsibility of it on November 5th
2017 a 26 year old walked into a church in Sutherland Springs Texas and began shooting most people never heard of the town it's very small town and also they got all sorts of news alerts as we get the United States and as many of you get from a overseas we're looking at our country what are you doing but we get these basically saying a lot there's an active shooting somewhere at every major news outlet in a basic relatives you get a stream on your phone active shooting of modeled Z a you know so brings but Texas so I looked a dozen of his like all I know is about to happen and I jumped online to watch it unfold in a beautiful moment of explaining exactly what I expected far
right groups 100 coordinated online they love and life shootings the United States because they like to associate the shooter was some particular agenda and at the time the goal was to associate of life years with and chief of particularly white of shooters and chief is referred to as as as I fascism and there are people who are part of this network but is actually not that large far right communities has spent a lot of time from being and TV up to look bigger than it actually is and look more violent than it actually is with the idea that if they can create as a false equivalency the news media will fill the need to report on them simultaneously so far i communities produce 175 accounts on Twitter where they take claim everything our responsibility for different actions on they look to be real news organizations constantly site vacancy counts in doing so it's not just about that because they really wanted to own was Google search you know Google search during alive shooting is extremely important to be as most people were trying to figure out what's going on I will go in search something related to you the incident into a search box that it's a little understanding Google is that Google doesn't know how to respond to you but you know basically a breaking news story prior to this moment if you looked up Sutherland you would see detailed demographic histories of the city's load pages the told you about property the reason nothing of substance about Sutherland Springs but what they did is they 1st target Twitter and Reddit why because Google pulls in Twitter and Reddit during breaking news events to try to understanding contextualize as fast as possible so by targeting Twitter and Reddit they connect to get things going they move across a ton of different fake accounts the poll different questions they throw at journalist asking questions though I'd what's going on this is just a story and they're just generally talking about it and they start asking questions is it anti-fire is it is it Antifer the question is really important because events and people start covering it in different ways so was priceless is the 1st news organization that they got was Newsweek Newsweek thing that they were being responsible wrote an article said she for responsible for Sutherland springs murders according to Far Right Media that detailed how this was completely inaccurate but the job was on Newsweek because what these groups cared about with the fact that Google News is at the top of a Google search so if there's breaking news getting top-tier news media coverage is important and thing about top-tier news coverage is that they can give you the whole headlined is is not space for in Google there's only a character limit and that character will it turns out to be the 1st 6 words of that story and anti-fire responsible for Sutherland springs murder the result was that everybody is looking at that as though that is the story see this time and time again we see this for example in our shootings related apart with the idea of moving at this is a crisis after through the system display information ecosystem like
this requires a tremendous amount of digital literacy you need to know where the weaknesses on never skills told and time to experiment so it was all about that is a variety of people been building those skills for the better part of a decade it's not just researchers or moonlighting as for-profit strategic communications companies it's also western teenagers who have been curtailed by their parents and by this to be something fun regardless of the ideology It's frustrated 20-somethings who label themselves as needs not currently in education employment and training or beta as part of a male communities visibly that they have the power of alpha males you may have heard the notion that in so recently it about and different people who are December unemployed disenfranchised knowing as though they can gain less system just like they've gained in video games the idea that they can come together and be something bigger than they were if the grace sign of what we once called call centers because in the Philippines in India folks have been trained on trying to address content moderation have figured out all the boundaries of the major technical systems and as a result of found ways to make a lot more money exploiting those systems for different characters on its activists were trying to change the world who actually playing in the but of course not all of those activists a progressive so creating seems to have it as an economy was really gone as a youth culture experiment but has become a full formal basis of what we think of as this information today so let's go back into the media to look at how
this begun it is that intended Russia today began disseminating highly controversial advertisements to ground its model question more the beautiful model fits into every media literacy narrative we've ever had of and resultant posters and it triggering all sorts of anxiety for example in London on posters were put up but around this campaign and immediately taken down as propaganda and then there was also the coverage about whether or not they were cracked propaganda because they were just asking you a question things so let's see what that actually looks like is climate change more
science fiction than science fact and a really small print how reliable is the evidence suggests human activity impacts on climate change the answer is it always clear-cut it is only possible to make a balanced judgment if you are better informed by challenging the accepted view we reveal aside the news that you wouldn't normally see because we believe that the more you question the more you know now you did already believe climate change that the architecture of a question would be really powerful to you in fact for those of you think it was in critical appressed perspectives just start to substitute the actual topic just how reliable is the evidence to suggest that the correlation between exercise and weight loss that sounds like a research agenda this is the beauty of a campaign like that's it's seating doubt in the very fabric of how we produce knowledge It's destabilizing knowledge in a systematic way is by creating false equivalency is that the media will pick up on this is a project if you're looking to actually engage in APIs logical war the news
isn't the right frame sure there are financially motivated actors producing patently inaccurate content to make a buck have you seen it happen this is a long history of things I mean I love the UFO history it's really wonderful and wanna talk about big news is actually a feeling of distrust Torres News creation that is produced outside of their world view use that's produced by people they don't trust news that seems stilted from the vantage point which they operate and why we see content that is political like artsy be perceived as of fake news from this context and why a huge chunk Americans believe at CNN and your times is actually fake news because here the epistemology kicks in at this moment the concerned how we
construct knowledge people construct knowledge through different bond ways of making sense of the world some emphasize the scientific method others focus on experience still others round what they think about religious doctrine all fine and well in those epistemologies aligned to reduce result the same narrative but if you're trying to engage in a culture war the best thing you can do is try this let those epistemologies try to make it seem as though they have nothing in common there at odds with the mother is I love this quote by Corey Doctorow draw reading you can't from afar we're not looking for a crisis of what's true were living through a crisis about how we know whether something is true were not disagreeing about facts we disagree about epistemology the establishment version of epistemology is we use evidence to arrive at the truth vetted by independent verification but last us when we tell you that it's all been independently verified by people who are properly skeptical and not the was embodies the people they were supposed to be fact-checking the alternative facts of this logical method goes against the so-called independent experts who was supposed to be verified the so-called evidence-based truth were actually invented the people there was the fact-checking in the end it's all a matter of faith you have to have faith in their experts or have face but that we are after that feels more truthful that narrative is so powerful in splitting people and it's so or confusing to people when they don't see a world view outside their own when they're not engaging with epistemologies that are different than their own because what beautiful and troubling about this dynamic is that it's a way of actually asking who gets to control knowledge something that we struggle with whenever even those were producing scientific evidence so what's this
campaign was a brilliant not because convince people actually change the world view but because it's slowly introduced or wedge into a system that way
what is introduced it still is a systematic bias big all the way through it we need to look at this question from another angle which is 1 fundamental bias is that underpin huge amount of our technical systems and how do we see that then amplified through these disinformation campaigns and where they can't come to converge in undermining all of our data infrastructure let me start with an example without
this we need is a Harvard professor and she was trying to seal of figure out which began a paper she had written before and very academic in the room you know you've written a paper in a while and search your name in the paper in a search bar bubbling you'll find it and that's what she did which is prize because up came a whole slide of ads what is the what tiniest Sweeney's arrest record and using argon I've never been arrested what's this about soldiers with the journalist the timing start asking questions trying to figure out what's going on and the French she figured that what was going on had more to do with the fact that to searching for a name that had to do with her name in particular that she gas accurately the she's rule of of variety of different names into the search engines you would get different ads from this company it seems though the company had been producing 6 different ads with its system approaches effects so she took the most popular baby names in the United States that are associated with African-American culture and the most popular baby names that are very classically in traditionally associate with Caucasian culture and she threw them on the you know at Google to see what had a pop up sure doesn't take a lot of PhD in figure out very quickly the the names that were white baby names were not likely to get criminal justice related products they tended to get ads for background checks on but the and that were associated with black beans and to be far more likely to be criminal justice related products so what's going on here that actually let you pivot offer rates and the time you knew that she was the former acts of Federal Trade Commissioners of a technology expert to she is something else was happening happens is that the way that add words works is that you can pick out of something like name and then to figure out what's names are better depends on who clicks on things so what happens people average people you searching for different names were more likely to click on a trail just related products when searching for a black name then when searching for white names and so the result is that Google learned the races dynamics of American society and prompted back out at every single person that amplification power is taking the localized cultural bias and amplify it was everyone this course is the
cornerstone of huge set of problems when we look at search engines take a searchlight baby or c I'm sure you've figured out by now that we search for baby you get this strange collides of white babies in perfectly pose of beautiful sensibilities CEO gives you a bunch of men wearing suits usually white on on the search engines trying to give you these results now here's what's going on and how this happens why people use image search they use image search to produce power point slides like this 1 right then that they search a generic term like baby not because they're looking to see what categories of babies exist but because there was a picture of a baby for their PowerPoint deck as so what you looking for stock photos the companies to figure that this is a really good way of making money put small photos in the process get people to collect though by the high high big hybrids 1 for their for the PowerPoint and so they've mastered they've gone optimized by labeling pictures with search terms the likely for PowerPoint objects and throwing that into the search process at the results for a query like baby you don't get natural babies because most people don't label photos that they've taken of their own children with a baby right they get it because stock photos now left problematic years that stock photos are extremely prejudicial the way they're constructed only a they perfectly poison like we're ways that I hope FIL fully down the right like who does that they don't this particularly constructive manner but there I the normative 2 words on the different ideas of who is you know valuable attractive Accetta within society problem because the voters coming through the stock photo companies 1st a term like baby are primarily white it's about what happens for a search company they see all of this stuff it looks like it's highly right because of the SCO has been very much are manipulated different ways and they can't actually see races of of of an image like that so they don't know what's going on but also there's a media block telling them that they doesn't bad they start to look the problem is is that they can actually see what's going on here and so then they return they rely on their users they start trying to randomize other pictures to put up and see what the users will click on or back to really racist users because the users don't click on pictures that are not photogenically perfect they don't on pictures of more diverse babies they click on pictures of white babies under the age of what I guarantee you deal with the society that's extremely prejudicial at multiple angles feeding the entire data system and here so we have a
fundamental flaw of any kind of machine learning systems Machine learning systems are set up to discriminate and I don't net in the cultural sense that you're used to when we think of discrimination we think of the ability to assert power over someone through cultural discrimination what the systems are set up to do is to say that meant our data in different ways to cluster rate data that is of fundamental active data analysis is to create clusters but where those classes are laden with cultural prejudice and bias that aims at reinforcing the discrimination are they were just discussing now grapple with cause
prejudices and biases truly important that pattern isn't as easy as 1 might think is figuring out how to deal with people who were spending tons of money money energy gaming systems coupled with prejudicial activities of search on by average people makes a really hot a hot place to try to figure out what's going on It's also a things
get gained malice talk about something the gaming problems here in 2012 the United States it was really hard to avoid the names of train on Martin and George Zimmerman Zimmerman had murdered the teenage Martin in Florida and and claims self-protection under are ridiculous laws called standard round I'm not going to defend U.S. gun laws I know the feelings here of the feelings are shared by the United States and many people believe that these laws are important so what happened was that you there's a whole set of dynamics going on around gun laws around and brutal murders of young black men in our country is this hit a level of the fur fervor in the United States every media outlet was covering it not all young people pay attention the media outlets and ends up
line of a white teenager was the name of Deligne Rove wasn't paying much attention the media but he kept hearing this in the ether somewhere and finally decided to search that the mainstream on Martin you know what a figure out what was going on and decided to read the Wikipedia entry and she was written as Wikipedia entries all are in a very neutral voice detailing the dynamics coming from a position he was coming from he concluded by the end of the Wikipedia article The George Zimmerman was clearly in the right and the train on Martin was was at fault importantly there was a phrase that was put inside that Wikipedia entry for him to stumble upon that phrase was black-on-white crime he took that phrasing through in the goal we know this because the detailed in his manifesto he took that phrase into Google and what by running a search on black-on-white crime he came to a cycle the Council for conservatives citizens which is a white supremacist white natural sites the results is that he's the next 2 years going into forums on my nationals exploring the the issues coming to his own informative ideas of of white pride in ways that are deeply disturbing and deeply deeply racist and on June 17 2015 he sat down for a with a group of African-American churchgoers during a bible study in South Carolina sat there for an hour before we opened fire on them killing 9 and injuring 1 and very clear about that is manifesto and his subsequent of trial and he wanted to start a race war so what's going on there are 2 things that you need to unpack when you understand the manipulation of the systems devices is a notion of red tell a red builds a phrase like black-on-white crime that is meant to entice you in you are learning more phrase that actually means something until it means something right which is to say that no process of Wikipedia would actually excluded the term black-on-white crime from a Wikipedia entry before this case is sort of unfolded dealt with erectile comes from the matrix as the moment where Morpheus says you take the blue pill the story ends you wake up in your about bed and believe whatever you want you take the red pill you stay in Wonderland and I show you how deep the rabbit hole goes wrong like nationals perspective the idea is to set went pills all over the place that invite people to come and explore ideas of white nationalism is a large invitation most people don't trip over them some do at the moment they trip over them they got an invitation and that's a really interesting moment for radicalizing people because the group is trying to stage radicalization through serendipitous encounters and huge amounts of the Internet are flush with all sorts of these terms of meant to invite you into these frames it's also true for about talk radio in other environments and then you back to search the power research is that there's a whole a set of places where search queries are crap the Michael Goliath he likes to talk about the the data avoids data points space where normal people don't construct websites actually give counter narratives to white nationalist ideologies people do not
began a search query with did the Holocaust happen they don't create websites with white pride when they're trying to push back against radicalization until of course this becomes visible what you actually have these terms you have an ability to roll them out and some constantly you search engine optimization to master the from this we don't fill these data was until we find out about them and then we filled with very serious content the southern poverty law center FactCheck dot org Snopes all tell you that this
is white supremacist white nationalist propaganda but the problem is is that if you're already down these past if you've already entered the final you don't care about those sites you wanna learn more to the question how you stop as follows in the 1st 1st place FIL 1 of the things we start to see in this process of data
words that he want something starts to get filled so black and white crime is started get filled by the roof cases people started flushing out all you see is a shift and so white nationalist stopped radio now talks about white
victims of black crimes which is a new data word out there I we've gone through back into darkness was come back out and to try to figure out where this fits in the broadest set the context the more powerful Texas technical system has the more that people are intent on abusing systems and the way you actually remedy this is not just by trying to patchwork across that this by trying to understand the dynamics of power in the alignment of context the lectures by moving away from the search and social media space because I'm going to see this in other environments is look at how stack up so consider it to me like precision medicine my guess is that there is no 1 in this room whose like gonna hall we should not fall of cancer on J only speaking we believe in addressing medical challenges they don't wanna pay for we may not want to deal with the consequences of trying to do the work we may not well invested in different ways but we generally believe that medicine is a good thing to pursue a scientific process that's important though we care about is how we do it ethically had a we do it responsibly we look back to the history of abuses in medicine things like test key where things like the not the experimentally same will never do that again if we can bound the process of collecting and managing data an ethical manner we generally believe it that didn't come home from
I argue that the sort of 3 different ways in which most of the data and up into the system the 1st is what you would think of as data by choice datapoint choice is the idea that you were given over that data by consenting with full knowledge of everything that we used by that data but this is only ways of a lot of the tech industry hopes is how they collect data it what a lot of scientists hope about how they collect data in an ideal world it how we would get data in the 1st place that everybody is that even those consensual moments start to get corrupted in different ways data is not you know that in terms of concise consensual foremost it is not actually done in the extreme of opposite a little to the extremes in order to understand
this at the opposite of stream is data by words and did like origin is best exemplified by a really our morbid of American policy so in the United States but there was a ruling by the Supreme Court that said that collecting DNA at the point of an arrest was equivalent to collecting a photograph or a fingerprint are justices clearly have not taken basic biology on and so they miss the idea that there may be a lot more DNA and then I had a recognizable identifiers itself and the city of Orange County here that the cat in the police department orange county in California decided to take this to a logical extreme that program called spit and equips which is the motorways somebody might be arrested they ask them to provide genetic material and if they provide a genetic material they would not be arrested they will be acquitted right if they got huge databases of genetic material on a daily others used based in is the genetic material we have situations like what we saw that last week which I choose as a voluntary process on where to use them for law enforcement only 1 of the things that happened for example from the state programs in Los Angeles is that love was an officer show up asking to meet your brother really you didn't know you had a brother right that the cost of that because it's not just cost of for you it's cost every entire social network the
laboratory of data that we deal with in this ecosystem is data by circumstances this is you posting things on Instagram because you're hanging out with your friends and you hope everything will be OK OK is that most of that mass of data is in that circumstantial for kind the precision medicine we don't necessarily think about data that's been done her consent now we look at precision medicine as an opportunity a place so we can actually pursue things because generally we believe in the pursuit there are other realms where we don't necessarily agree with the person is good their the realm of policing not everybody has the same view on police to give you 2 extreme verges on that some people believe that policing is entirely about weeding out bad actors and punishing them as a punitive model for the system to other will lead people believe that is about different kinds of creating community trust and support obviously what you want is somewhere in the middle but probably where is really complicated so the audience depending on what your goal is you will do radically different things with the data analytics process like a predictive policing if your goal is to arrest people that is your target you will send police to a very different set of our directions you will also discourage them from spending time in the community because that way they will be less biased and that means that the goal of your system and affects your data analytics process no criminal justice
is a fraught area in general but it was in a slightly different way because there's a whole ecosystem called risk assessment scores predecessors scores in the idea of taking information and determining whether or not somebody is it's difficult with was successes words is that we end up with bias data as were not surprised about but we also end up with this challenge about how people take that information to being judges in theory are supposed to receive those scores and be able to override the information to together and make a human judgment on top of whatever that SKorea's and that's part of what gets the scoring process off the hook right human judgments at this but there's something funny that happens these are elected where there are you know they're they're employed in 1 way or another and have a result what happens is that when those recessive scores are put into motion and people and judges look at them is that applies to go in alignment with the to basically new work within whatever those words are because the overriding they could be in big keeps having political he fought and that's where he was the loop on all that we think they might be what's fascinating about humans in the loop is that we often use as a solution as though human judgment is the best thing ever on the net actually be helpful to get humans there following National Alice went through the history of the Federal Aviation Administration in the United States is in the Administration that oversees things like autopilot I like the seventies or a series of hearings about whether or not we should have autopilot in place pilot with unquestionably safer for flyers to this day if any of you fly on a regular basis that person come on the loudspeaker telling you that they're your pilot and they're there to make it a safe and wonderful place and this is most likely hadn't thought of playing in a long time right because that person is there to babysit an algorithmic systems there are only responsibility is to step in with alvorlig system fails and be able to successfully absorb all the pain and responded that's most people can't do that they've been the of the job they haven't been flying planned this plane and certainly not a high situations they don't know why the failure occurred there is a as result most of those planes Coke our collapsed because of human error funny thing because what Medellin argues is that really the whole point of that pilot is it really the help out in an emergency it's to be an organizational liabilities by to absorb all the pain and responsibility for the organization to be the 1 accountable for the technology and some assets and she describes this as a moral crumple zone crumbles being the part of the car that it impact we put humans in that process if
you want him as a loop a lot it is about understanding again that moment alignment the motive power I'm on the border of our nation called crisis text Liber people can write it trained counselors when they're in a moment of crisis in order to get our support and we have to use a ton of data analytics to try to make certain their counselors are more sophisticated and everything that we do we try to figure out ways of training counselors B so the things that everybody else is learned we can do that 1st is where a nonprofit 2nd because we're trying to actually make things better for our those in crisis at a given point and because everybody agrees on that mission and that alignment is beautiful it's also very rare where is what's at stake whenever you look
at these systems it's not just about how this technology is structure but it's how we have to understand it holistically within the hour broader environment take scheduling software this is the software that determines who gets on job at any given point the reality is that you actually designed scheduling software to be about optimizing the experience for workers I can promise you that's not how it's being deployed in major retail outlets it's being employed in order break union by guaranteeing people don't actually work with 1 another in the United States is being deployed to make certain that we will work no more than 32 hours they don't get health benefits Accetturo it's not about the algorithm it's about the process
it's also where we see new forms of manipulation these we talk about all the ways in which this data can get bias things can go wrong they can also be exploited this is right now in the world of experimental research it's fascinating for all of you were following a time this vehicles make paper as some colleagues have found ways of modifying stop so that our time is vehicles believe them to be signs Imagen this is a way of intervening massively OK so we come the end here I
gave this crazy tore through issues of bias discrimination ways in which context matters the ways in which things get exploited think about accountability here how think about responsibility of course matters the most acid with power but who is setting the agenda under what terms with what level of oversight and who attacking the system achieve what agendas on both sides of the Atlantic there is growing concern about the power of tech companies in shaping information landscape pretty amazing to watch especially in here in Germany there there's reason that were watching this because there's a massive reconfiguration of the social contract and power dynamics in society and a lot of it is incorporated technology but it's not simply about the platforms is about a set of new cultural logics what I like to think about for
the frame of bureaucracy that would argue that today's hadronic systems are actually an extension of what we understand bureaucracy to be and the decision making process is about the ways in which we can actually shift responsibility moral responsibility across a very complex system it will that is the ability to understand the making of cultural infrastructure we start to see the way in which data is configured in ways of controlling a larger system it also means that the mechanisms of accountability the mechanisms of regulation came look simply at an individual technology but they need to look at it within a broader set of ecosystems we spent the last hundred years obsessing about how to create accountability procedures around bureaucracy we have figured out how to do this well within technology but
also acknowledge that the ability to regulate bureaucracy has not actually worked so well run history and progress is something that has been systematically abused at different points in 1963 higher rent controversy early publisher analysis and the Israeli trial of it off Agmon she described excruciating detail on how this made ranks and as as officer on played a significant role in facilitating the Holocaust'' logistics processes but not cuts actually was smart effect with really doubt it was the way in which he became part of a military bureaucracy and believed himself to be following orders and the way in which the distribution of responsibility can be something that would be exploited for a variety of people with negative of agendas Schiphorst famously subtitled about the banality of evil which is something I think that we should meditate on especially X 3 day we think about these technologies how is it that it's not as a sign of their intentions but the structure and configuration that causes the pain is itself from Kafka's The Trial in 1914 to Iran's ideas of not the infrastructure we've seen some of the dynamics aware bureaucracy can be mundanely to actually horribly abuse so and I would argue that the same hadronic systems are introducing that wide range of challenges for us right now so
these are the roots of today's problems are much deeper than how an algorithm prioritizes content or maximizes for the present goal just that they're all actors are learning to construct and twisted of what is the new technologies were seeing are actually becoming a logical extension of neoliberal capitalism the result is that we give people what they want with little consideration the how individual interests of financial interest may end up resulting in underwriting of social structures because what's at stake is not within the bounds of that technology with a broader social configuration so I'm a big believer in
Larry lessons argument that systems are regulated by 4 forces the market the law social norms and architecture and this is where this question of regulation comes into being because I'm confident you all will basically be pushing the directions on regulating major tech platforms on but 1 and you purely looking to regulate the architecture of the system without disturbing the financial infrastructure that makes them possible a willing to dismantle the surveillance infrastructure the props of targeted advertising even if it means that your politicians and other on Capitol structures are going to lose their ability to meaningfully appetizer people are you willing to dismantle financial ice cap was infrastructure that pushes companies too far hockey stick that profits quarter-over-quarter and what will you do to grapple with the causal factors that manipulate systems where you willing to do it makes so that prejudicial data doesn't get amplified through them because what at stake is not just what we can set up legally in terms of the company but how we can actually structure of the right kinds of norms and cultural structures around it and
security are on the rise both here and in the U. S. and technology is not the cause technologies the amplifier it's nearing magnifying the good bad and ugly of every aspect of this and just as bureaucracy was manipulated to malicious purposes were watching technology be a tool for both good and evil what we fundamentally seeing is a new form of a social security vulnerability socio-technical vulnerability fundamentally
look that acts at over and over again questions about power and this is where 1 is leave you for the next couple of days to really think about things I really the ability to really grapple with information war and histological war goes far beyond thinking about how to control or make sense of a particular technology it requires us to think about all the interfaces between technology and the newest technology and financial instruments techno technology and optimizing for social interests technology in our public good and frankly exactly sure how we meaningfully regulators systems because what I know for sure is the regulating the tech companies alone isn't gonna get us where we once thank you the
Algorithmus
Nebenbedingung
Web Site
Subtraktion
Facebook
Mereologie
Momentenproblem
Relativitätstheorie
MySpace
Gesetz <Physik>
Internetworking
Task
Hypermedia
Energiedichte
Verzeichnisdienst
Algorithmus
Forcing
Rechter Winkel
Regulator <Mathematik>
Arithmetisches Mittel
Internetworking
Web Site
Bit
Rechter Winkel
Relativitätstheorie
Garbentheorie
Quick-Sort
Internetworking
Umsetzung <Informatik>
Mereologie
Ortsoperator
Natürliche Zahl
Gruppenoperation
Gruppenkeim
Ranking
Physikalische Theorie
Internetworking
Komplexitätstheorie
Übergang
Virtuelle Maschine
Message-Passing
Algorithmus
Webforum
Stützpunkt <Mathematik>
Inhalt <Mathematik>
Gerade
Verschiebungsoperator
Algorithmus
Befehl <Informatik>
Datennetz
Abfrage
Physikalisches System
Fokalpunkt
Quick-Sort
Arithmetisches Mittel
Chiffrierung
Whiteboard
Speicherabzug
Ordnung <Mathematik>
Verkehrsinformation
Neuronales Netz
Nichtlinearer Operator
Zentrische Streckung
Subtraktion
Bildschirmmaske
Datennetz
Physikalisches System
Aggregatzustand
Resultante
Mereologie
Prozess <Physik>
Selbst organisierendes System
Mathematisierung
Formale Sprache
Gruppenkeim
Inzidenzalgebra
Login
Homepage
Internetworking
Hypermedia
Bildschirmmaske
Datenreplikation
Endogene Variable
Schreib-Lese-Kopf
Zentrische Streckung
Kraftfahrzeugmechatroniker
Sichtenkonzept
Physikalischer Effekt
Relativitätstheorie
Nummerung
Quellcode
Datenfluss
Druckverlauf
Verkettung <Informatik>
Hypermedia
Mereologie
Wort <Informatik>
Information
Ordnung <Mathematik>
Verkehrsinformation
Message-Passing
Zentrische Streckung
Web Site
Web log
Gruppenkeim
Zahlenbereich
Physikalisches System
Exploit
Hoax
Internetworking
Druckverlauf
Verkettung <Informatik>
Verknüpfungsglied
Twitter <Softwareplattform>
Mereologie
Hypermedia
Strategisches Spiel
Information
Schlüsselverwaltung
Verkehrsinformation
Message-Passing
Varietät <Mathematik>
Aggregatzustand
Schießverfahren
Momentenproblem
Selbst organisierendes System
Relativitätstheorie
Quick-Sort
Videokonferenz
Internetworking
Streaming <Kommunikationstechnik>
Druckverlauf
Twitter <Softwareplattform>
Endogene Variable
Fünf
Programmierumgebung
Verkehrsinformation
Standardabweichung
Quelle <Physik>
Resultante
Momentenproblem
Datensichtgerät
Gruppenkeim
Twitter <Softwareplattform>
Inzidenzalgebra
Raum-Zeit
Homepage
Fehlertoleranz
Prozess <Informatik>
Vorzeichen <Mathematik>
MIDI <Musikelektronik>
Message sequence chart
Call Center
Schießverfahren
Datennetz
Kategorie <Mathematik>
Ereignishorizont
Hoax
Ego-Shooter
Quelle <Physik>
Randwert
Twitter <Softwareplattform>
Rechter Winkel
Digitalisierer
Multi-Tier-Architektur
Facebook
Information
Varietät <Mathematik>
Telekommunikation
Web Site
Subtraktion
Wellenpaket
Quader
Selbst organisierendes System
Gruppenoperation
E-Mail
Äquivalenzklasse
Homepage
Hypermedia
Computerspiel
Webforum
Reelle Zahl
Endogene Variable
Inverser Limes
Jensen-Maß
Polstelle
Leistung <Physik>
Assoziativgesetz
Videospiel
Betafunktion
Physikalisches System
Softwareschwachstelle
Basisvektor
Hypermedia
Mereologie
Wort <Informatik>
Einfügungsdämpfung
Mereologie
Sichtenkonzept
Gewicht <Mathematik>
Hochdruck
Mathematisierung
Äquivalenzklasse
Mathematische Logik
Quick-Sort
Perspektive
Hypermedia
Modelltheorie
Unternehmensarchitektur
Korrelationsfunktion
Konstruktor <Informatik>
Expertensystem
Subtraktion
System Dynamics
Sichtenkonzept
Punkt
Kontrolltheorie
Momentenproblem
Rahmenproblem
Versionsverwaltung
Annulator
Kontextbezogenes System
Fokalpunkt
RFID
Heegaard-Zerlegung
Inhalt <Mathematik>
Versionsverwaltung
Stochastische Abhängigkeit
Brennen <Datenverarbeitung>
Lesen <Datenverarbeitung>
Resultante
Soundverarbeitung
Expertensystem
Subtraktion
Mereologie
System Dynamics
Sichtenkonzept
Winkel
Pivot-Operation
Schlussregel
Physikalisches System
Automatische Differentiation
Biprodukt
Bitrate
Rechenschieber
Datensatz
Weg <Topologie>
Suchmaschine
Mittelwert
Wort <Informatik>
Figurierte Zahl
Leistung <Physik>
Varietät <Mathematik>
Resultante
Subtraktion
Abstimmung <Frequenz>
Prozess <Physik>
Punkt
Datenanalyse
Stoß
Klasse <Mathematik>
Term
Virtuelle Maschine
Suchmaschine
Digitale Photographie
Cluster <Rechnernetz>
Hybridrechner
Ganze Funktion
Bildgebendes Verfahren
Basisvektor
Leistung <Physik>
Fundamentalsatz der Algebra
Suite <Programmpaket>
Physikalischer Effekt
Kategorie <Mathematik>
Winkel
Kraft
Fakultät <Mathematik>
Abfrage
Physikalisches System
p-Block
Bitrate
Gruppenoperation
Rechenschieber
Objekt <Kategorie>
Generizität
Menge
Rechter Winkel
Hypermedia
Wort <Informatik>
Mereologie
System Dynamics
Wellenpaket
Unrundheit
Physikalisches System
Gesetz <Physik>
Übergang
Energiedichte
Menge
Spieltheorie
Mittelwert
Mustersprache
Hypermedia
Standardabweichung
Resultante
Matrizenrechnung
Web Site
Multiplikation
Mereologie
Prozess <Physik>
Wellenpaket
Punkt
System Dynamics
Rahmenproblem
Ortsoperator
Momentenproblem
Minimierung
Natürliche Zahl
Gruppenkeim
HIP <Kommunikationsprotokoll>
Benutzeroberfläche
Gesetz <Physik>
Term
Raum-Zeit
Internetworking
Spezialrechner
Hypermedia
Webforum
Perspektive
Radikal <Mathematik>
Inhalt <Mathematik>
Figurierte Zahl
Gerade
URL
Leistung <Physik>
Beobachtungsstudie
Abfrage
Physikalisches System
Quick-Sort
Blu-Ray-Disc
Menge
Rechter Winkel
Hypermedia
Dreiecksfreier Graph
Energieerhaltung
Information
Programmierumgebung
Softwaretest
Web Site
Prozess <Physik>
System Dynamics
Physikalisches System
Kontextbezogenes System
Raum-Zeit
Hypermedia
Wort <Informatik>
Schlüsselverwaltung
Programmierumgebung
Verschiebungsoperator
Leistung <Physik>
Subtraktion
Prozess <Physik>
Punkt
Momentenproblem
Datenhaltung
Familie <Mathematik>
Schlussregel
Physikalisches System
Mathematische Logik
Term
Quick-Sort
Office-Paket
Streaming <Kommunikationstechnik>
Rechter Winkel
Digitale Photographie
Elektronischer Fingerabdruck
Identifizierbarkeit
Wort <Informatik>
Computerunterstützte Übersetzung
Extreme programming
Ordnung <Mathematik>
Optimierung
Auswahlaxiom
Aggregatzustand
Ebene
Resultante
Subtraktion
Prozess <Physik>
Punkt
Selbst organisierendes System
Datenanalyse
Physikalische Theorie
Richtung
Loop
Verzeichnisdienst
Prozess <Informatik>
Hook <Programmierung>
Endogene Variable
Modelltheorie
Hilfesystem
Sichtenkonzept
Systemverwaltung
Reihe
Ruhmasse
Physikalisches System
Menge
Rechter Winkel
Mereologie
Basisvektor
Wort <Informatik>
Information
Fehlermeldung
Prozess <Physik>
Punkt
Wellenpaket
Momentenproblem
Datenanalyse
Open Source
Physikalisches System
Loop
Scheduling
Message-Passing
Algorithmus
Prozess <Informatik>
Software
Kontrollstruktur
Ordnung <Mathematik>
Datenstruktur
Programmierumgebung
Leistung <Physik>
Mereologie
System Dynamics
Physikalisches System
Kontextbezogenes System
Systemplattform
Mathematische Logik
Term
Übergang
Design by Contract
Systemprogrammierung
Bildschirmmaske
Menge
Vorzeichen <Mathematik>
Endogene Variable
Information
Leistung <Physik>
Distributionstheorie
Punkt
Prozess <Physik>
System Dynamics
Rahmenproblem
Spannweite <Stochastik>
Negative Zahl
Arithmetische Folge
Rangstatistik
Vorzeichen <Mathematik>
Endogene Variable
Logistische Verteilung
Datenstruktur
Maßerweiterung
Schnitt <Graphentheorie>
Konfigurationsraum
Regulator <Mathematik>
Analysis
Verschiebungsoperator
Soundverarbeitung
Kraftfahrzeugmechatroniker
Physikalisches System
Kontextbezogenes System
Algorithmische Programmiersprache
Entscheidungstheorie
Office-Paket
Komplexes System
Menge
Mereologie
Ordnung <Mathematik>
Varietät <Mathematik>
Resultante
Parametersystem
Extrempunkt
Physikalisches System
Term
Gesetz <Physik>
Mathematische Logik
Systemplattform
Teilbarkeit
Richtung
Gebundener Zustand
Algorithmus
Forcing
Rechter Winkel
Wurzel <Mathematik>
Datenstruktur
Maßerweiterung
Normalvektor
Unternehmensarchitektur
Konfigurationsraum
Regulator <Mathematik>
Bildschirmmaske
Feinstruktur <Mengenlehre>
Physikalischer Effekt
Softwareschwachstelle
Computersicherheit
Information
Physikalisches System
Leistung <Physik>
Schnittstelle

Metadaten

Formale Metadaten

Titel Opening Keynote: How an Algorithmic World Can Be Undermined
Serientitel re:publica 2018
Autor Boyd, Danah
Lizenz CC-Namensnennung - Weitergabe unter gleichen Bedingungen 3.0 Deutschland:
Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen und das Werk bzw. diesen Inhalt auch in veränderter Form nur unter den Bedingungen dieser Lizenz weitergeben.
DOI 10.5446/36648
Herausgeber re:publica
Erscheinungsjahr 2018
Sprache Englisch

Inhaltliche Metadaten

Fachgebiet Informatik
Abstract Algorithmic technologies that rely on data don’t necessarily support a social world that many of us want to live in. We must grapple with the biases embedded in and manipulation of these systems, particularly when so many parts of society are dependent on sociotechnical systems.

Ähnliche Filme

Loading...
Feedback