Handshakes, Citizen Science and Evolution

Video in TIB AV-Portal: Handshakes, Citizen Science and Evolution

Formal Metadata

Handshakes, Citizen Science and Evolution
Title of Series
Number of Parts
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Release Date
Production Place
Erlangen, Germany

Content Metadata

Subject Area
In the 1980s Robert Axelrod invited submissions for a computer tournament in which people submitted strategies for the Iterated Prisoner's dilemma. This is a game in which individuals choose to either cooperate or defect with each other. Since then similar work has been used to understand the evolution of cooperative behaviour in an evolutionary setting. Like a large number of early (and sadly ongoing) research code, the code from Robert Axelrod's work was lost. Similar ongoing work is often done with poor sustainable practice for the software involved. In 2015, a Python library aiming to reproduce Axelrod's work was put on github (the Axelrod library). It has since accumulated more than 200 strategies with contributions from Academics and hobbyists alike. This vast OPEN treasure trove of game theoretic tools is now being used to undertake a number of research projects. Including one that looks at an evolutionary process called a Moran Process which is a model of a population in which the makeup of future generations of the population depend on how well individuals of the current generation perform. In 2012 a piece of research claimed that there was no advantage to having long memory of interactions. The work this talk will describe demonstrates how that's not true in evolutionary dynamics. Indeed: complex strategies have been trained using reinforcement learning and the huge number of strategies available through the Axelrod library to perform particularly well. Interesting behavioural aspects also emerge: without external input, the strong strategies evolve "handshakes". These handshakes allow them to recognise friend or foe in the population and act accordingly. This work not only has implications at a game theoretic and reinforcement learning level but can also help understand how and why complex behaviour can emerge in evolutionary settings.
Link time print break bit part events program number Word networks terms time evolution networks university core archive organization Games office library
Compass time model set hot instances rules Twitter Expect types Word mean job environmental rights scratch Games man
probability distributions randomness bottom study factor time tournament sources stations model indication inclination part total number web goodness strategies terms Matrix selection utilized job Cat Games dilemmas interaction fit bit lines evolution types job alternate forms environmental Case rights Games recording library
Algorithmen effective study information state factor state time dilemmas model set train job strategies mathematical object rights recognize Arrow structure
meta multi hard match Lineares zeitinvariantes System study tournament Plotly machines scenes train job strategies recognize testing Console results
point Algorithmen loop strategies state time model machines life lines recording library
laptops sources slides collaboration information encoding software developers part part photo core Games library
sources Mathematical strategies open source projection computer science organization part students Games Computational
sources encoding part
again you are welcome back everyone are going to be starting in a minute or so as people are trickling back in from the coffee break so this time he agreed to swap slots with by lady also but otherwise the program will see it is announced and is going to be talking about the game theory about the p.. they are a prize at the next summer to nickel it as a yet he had to close my eyes and we work yet right so welcome wince and thanks for it like i hope that the organization also no no problem and thanks for paying for having me and i know i know how hard organizing new things can be and how much work it is so i thank you for. for putting on this great event my name's vents. i've already treated out links to this talk as well as links to the pre print that i'm actually going to be talking about so this talk is going to be kind of following this this archive prix print here in explaining some of the ideas and how use pipe and to to do that. i'm not a magician at cardiff university in wales in the uk still in europe. i'm also a softer sustainable institute follow us been mentioned a little bit so best practice in terms of software as use in research is important to me and finally i'm a game theorists i'm one of the core developer. years for the axelrod library which is nicole at those poster is about it and that's what this this logo is about and the work i'm going to talk to about today is not just mine there's five authors on on the paper for office on the paper part me. and also a huge number of contributors to the actual axelrod library itself and so that's why the the title of this talk if you've got the program in front of you has the word citizen science in it because i really feel so it's a nice scientific for.
so this is a tweet i don't know how well you can see it at at the back is a tweet eyesore a little while ago by a post-op from penn state called kirsty mccloy and she treats sets up flawless he competition trial loses will fight over a hot pot podium there can own. only be one winner so the the idea is i understand by my naive understanding is there is the span there's to lizards in the pan and there's one podium that's warm in the pan and lizards a cold blooded so they like the warm podium and so. she's going to be able to look at how the fight over this podium right. perhaps an sharon extrapolating from a tweet perhaps to study over a longer too little time if it's always the same blizzard that wins the podium perhaps. that's interesting certainly nothing i know anything about and this is the picture.
that came up again i don't know how you can see the back but what the pictures showing his the to these are just lying on top of each other on top of the podium so everything was put in place for these two losers to fight as a to death but on the astrue to fight for the podium and and the to this is just. said hey let's just health let's just let's just cooperate right let's just work together we can both enjoy the podium and that's just all get along and so they didn't to facts and i'm going to use the word defect and cooperate in this talk to fact meaning not cooperate so. in this instance fight scratch whatever it is losers do instead they cooperated they just said let's just work it out and this tweet kind of in compass is everything that game theory tries to do apologies to the camera man i realize i walk around so i just give up. thank you. so this tweet is really nice kind of income says everything about game theory in that a lot of modeling we do is we set out rules from above and we let that dictate the environment to be the final rules and we measure the environment but game theory does the opposite it creates the environment. and then he finds out what the behavior is from an ok so in this particular incense kirsty created the environment had an expectation of what the behavior would be right that this is would fight and was wrong ok at the iss cooperated as is a really nice kind of example of game do you can trying to understand. and what what behaviors going to be given an environment and in particular lot of cambridge interested in is why we nice to each other right why do we cooperate ok why do people work really really hard to put on. these types of conferences right why do people do that and one tool game theory has to study this is called a moron process so more on process is a very simple birth death process that puts behavior in an evolutionary setting and the keynote yesterday julie.
actually spoke about a really cool paper where essentially they were putting scientific behavior in this kind of set i haven't read the paper just yet i'm looking forward to it and said here is we have a population and so perhaps blue of the lizards that a co-operative and read other users that are not co-operative. and we select one of the lizards so one of the behaviors and that's selection can be based on fitness of how well they do in their environment we reproduce that's where this birth steps we have another one and then we choose another individual completely random not necessary depending on. the fitness and that individual is removed ok so the more and process is a model of an evolution process in which the population numbers stay the same there's no no no and has only. you can see you can write a moron process basically evolution in thirty lines of piping if you include a dock strength that i don't expect you read that i'm going as i'm going to zoom in on the lines fourteen to twenty six and this is just meet me repeating what i just said so the idea is we go through every. player and a population and we start building up a school and that school all comes out of playing a game every player plays against every other player in the population and we add up that that score the game is up to one little bit just an umpire a cat and then this is the selection part where. our fitness just informs and over all probability distribution based on how good we were that doesn't allow week people to continue and to have birth but also allows a but is more favorable toward strong in terms of fitness and finally we simply get rid of a random and. this is the game we're going to talk about so i have mentioned that we're talking about cooperation and one of the famous games for the study of cooperation is the prisoner's dilemma so that's a two player game. some of you might already be aware of the prisoners on a from you know your own knowledge but a couple of months ago something went viral nicky case put a wonderful web game together that went around where you could look at cooperation soft that's basically what i'm talking about and it's a wonderful wonderful tool. i recommend playing around it. and so this is the game and we have two players a role player let's call them lizards a role izzard and a column izzard and the role is or decides which column which row iran and the columns or decides which column were in and the first matrix is the utilities to the role izzard and the second matrix is the utilities to the columns are they have to. strategies to cooperate or the fact so to just lie there at the top of the podium or to be ready to fight a cat. will call lying near the top of the podium their first strategy so cooperation is the first tragedy if they both cooperate they both get a score of three which is pretty good. but if i see you lying at the top of the podium basically ready to share the spoken with me and i just come in and start fighting you you're not ready for the fight so while immediately get a utility of five love the whole podium myself feel real good about pushing you off the podium and you'll get a utility of zero because you'll be sad. at that so if you put us in the first row i should the fact i should put us in the second column and also holds if you're standing on top of the podium in a very aggressive way ready for a fight then that means that if i just walk up you're going to throw me off so. i'm going to come prepared for a fight will both did very damage may be no even make it to the top of the podium who knows front and so when we just look at the prisoner's dilemma by itself we have what's called the national calabria which is that we both and up in his bottom left the bottom right corner of just defecting that so many come back to the question. one of why we nice to each other right and because game theory says we shouldn't because something like this is more or less valid of any interaction. and robert axelrod is a game there is too in the nineteen eighty's really started a bunch of research about the prisoners that an eye he invited individuals to submit strategies for the prisoner's dilemma but the strategy is being what happens if we play it multiple times so we don't just played with.
once because if we just played once we should effect there is nothing subtle about that however if you start playing in multiple times in our reputation becomes important and it's all about how to invest around reputation that becomes interesting to rob axelrod created some invited academics the ninety's to some a computer code would play this game against each other. and and would see what happens and this leads me to a python live recall the axelrod python library is almost three years old and it allows you to do this type of research so for example we can create a tit for tat as a strategy and we can play cooperate or and we can get them to play against each other for. i've turned and we see cooperated persisted for tat they both cooperated for the five tench so they just got along from the start i to lizards from the start jumped on top of a podium and got along and tit for tat versus the factor factor just blindly looks for a fight takes advantage of a. station so what happens is we start cooperated starts up by cooperate excited for tat starts by cooperating the factor defect and then tit for tat realize what's happening and starts to acting against a factor and finally got to for tatters alternator of their stars by cooperating and then just ultimate could name. and and what we see here is in kind she were to retire is doing to for tat is in fact just doing whatever alternator did in the previous turn and axelrod's torments kind of spend a whole bunch of research because the fact that all that tit for tat one both these tournaments. and one east germans against much more complicated strategies people like well that's kind of an inclination to begin to understanding why we cooperate is rather because it's good for tat is fundamentally a strategy that will cooperate. and.
we can look at that in an evolutionary setting so putting those strategies in a moron process and so what we're seeing here is someone processes created with the x. ray light reaches lots of propping capabilities build an and we see if we start off with a population of the factors and we throw one cooperated in air over time. cooperated gets pushed back out so the defectors have resisted the invasion of the cooperate but the opposite is also possible we can throw one cooperate iran and after time the whole population could could change so that's kind of what we're interested in is why over time to evolve this cooperative behavior but at the strategy. a very simple cooperate and the fact are not subtle there in fact rather done right it just always do the same thing they don't gain any information from who they're playing they don't build up an understanding of what's going on. so this wartime took to about today made use of a structure of the strategy this is called a finite state machine which is a while study mathematical object but in in a prisoner's dilemma setting basically we have a history of both place and we have the states that change over time and they represent.
how we feel cool. we can trainees was cold the dojo so actually trainee strategies using genetic algorithms. and they look like this so all the junk algorithm is doing is basically moving arrows around changing actions and building up a strategy and so what we're doing is we're using a genetic algorithm to change the train strategies to perform in a genetic process we put all the strategies that.
we had some of them train some of them pre-programmed in a big tournament but this torn it wasn't just a match it was an evolutionary process and here are some of the results so we have the top invaders we have some strategies are able to invade twenty percent of the time and at the top resisters i will talk about the top resisters here.
one tear through until three the scene bowl bear of these trains find same machines and they are in fact basically an invasion of all i was really cool about than looking at this plot this shows what these tier one strategy does over multiple turns against multiple studies.
every row is another strategy and in this are always turns and what we're seeing is t f one always starts with two corporations and of the faction within talent to do that the algorithm trained to do that and what that means is that he had wanted to hear these stories that if one in fact has a secret.
and check. where if we follow this line against a player that's playing the same find it's a machine they'll both cooperate twice then to fact and then as long as they have that final the faction they'll end up in this state here where they'll just be in a loop of corporations so that's why they're so good at resisting invasion because they have a secret handshake. walk around like oh you're me cool let's just get along and do that very very quickly and so that is a really need finding because it came out of the algorithm to go back to those litter that we didn't say it we got out of it and so it kind of points towards why and how evolutionary processes can involve south africa. mission mechanisms. the reason this is possible is because we had a huge diversity of strategy which we can train at the time reading the paper we are hundred sixty four strategies when in fact have more than two hundred eleven now and that's all possible because writing a strategy for libraries a simple p.r. on get have and we've had contributors from all walks of life.
this is my last slide kind of is lots of information if you want photo if any of this interesting this is the slides a photo of. nickel that you've already seen his went to college on this and own mark on the other two core developers of the library and. the actual information from libraries their pension game theory my courses there and that's actually built up on a bunch of jupiter notebooks where i teach him to traverse years the final thing i'll finish on his journey is one his collaborators who is part of course had texas had to.
and. continue to open source project and we had contributes from game theorists computer scientists but actually a lot of students that just common summit the strategy of julian fact it's way more than seven strategy she made fundamental build changes to the deliberate self the great improved it but that that's been really great contribution were when i say couldn't science. we don't just have people landing of some course on a computer the not using people are actually writing strategies that we're using that's faster than thing that i finish on an organizer pike on uk if your engine pipeline uk come see me but accommodations are the super expensive so good luck.
and that's all i have to say.