Optimizing the Driving Behavior of Self-Driving Cars Using Genetic Algorithms

my name is maximum and then I'm I'm currently a student at the forest University and also working at the giant Cologne and the I'm going to talk about a little bit the boat to optimize the driving behavior of self-driving cars using genetic I would consider no thanks for showing up at this early time I didn't actually expect that many people

to come and the last day of frost on this little so 1st of all why what am I talking about them usually when people hear about machine learning they think about the universe into granting because this is what's in the media all the time but there's also another thing another legislation in which is less known it's genetic algorithms which also has a very nice and so few applications to optimize the characters in the main switch on as known and people the don't have that much not domain knowledge and control for example if we want to use have tried and casts as you would for

example in approximate the optimizing carcinogen you know it's speaking what we yeah sorry in the better the notion of you may have to look for them not not that they're in the and of I think is but an all the OK and uh drops is a the free and open source software used for 3 D patterns simulated present simulation and it has a very nice sophisticated physical model for the rest simulation it takes into account the material the tires are made of there are also the dynamics of the cow which you driving and also hold a lot behaves and different kinds of worlds and and it's widely used by the conferences like for example the gecko Conference USA 4 years ago to do the big challenge um did I wouldn't challenge where went to the their agents for best behavior owners to certain set of practice and he also what would you mind is currently working on agents which can control the cars which will probably come up in 1 of the next conferences their dissipated and usually when you're writing

software for self-driving cars and this office very complex and other than development and testing because the devil has to think of so many corner cases and there's so many parametres like for example the distance to then the car you can do this when you're driving a certain speed that you have to know how far you have to stay away from other cars in the traffic or how far can you go and presented with parameters are very very hard to do this by hand and development is all it would be very nice to have a algorithm had with this and

here 1 with the solution might be genetic I algorithm you me all you need to develop a

very very basic control of not taking into account many things to watch if you were to go everything everything manually and then this algorithm is used to train and evolve the behavior of the call to find the best possible the driving strategies with a given software

so i'm I've been saying this term today I wouldn't a few times no and what are these I wouldn't actually genetic

algorithm is I wouldn't inspired by nature by the natural evolution also we humans

so much from and it's basically the method for optimizing parametres there in a domain where we have to maybe limited or no domain knowledge at all which might from physics or like Vulcan arriving I'm also note that prompt the so I don't really know that much maybe I know when to break and the very careful driver but for years we can use algorithm to learn better strategy and the only thing we need is the problem to be the solvability them death but a situation that simulation sorry and also this uh the promise of this be dispose of into a set of parameters describing what we're doing so we can have actually something that really evolves and these parrots would be called you know or individual from now on and ch usually either of them starts by initializing the set of parents that in several 7 that's ever genome's called population we all the power redesign 1 individual of of individuals and each of those has to go to this relation then we see how would it completed the tech correct or maybe the incomplete and only got a certain amount of meters on the track and then the algorithm will study evolving in approving those parameter sets of solutions and maybe find the best driver yet the so um with the

evaluation on the top left it would be what i was

already talking about when you're putting the parameters set into into trucks it was stature so the cars will complete the track and we get a fitness value back which were which tells us how with the count found in total mn then the the 1st step of the algorithm itself is a selection part on the top right

and this basically right humans going to the discovery of selecting then the parents for the next iteration of children only the in 2 and the 7 number of nodes gets selected like so and there and usually we assign a circulated to each individual but if said it is a better fitness but we might get had to have a better chance of being selected and if it has the red maybe didn't communicate at all or maybe was very slow it has a lower fitness and has a lower chance of getting selected as a parent but still Allerton individuals might be selected for the next iteration because a very good performing the individual make new with a high target speech can be learned from the system behavior for weaker individually and in relation of those um it might include sexual and so the combination is done with the cross over on the bottom right and for cross over there and take it was the parents which just selected and greater offspring from those them this is usually done by just taking baby random genes from 1 of the parents and combining them with ranging from other known but keeping the order of the same or take taken the 1st half of the genome from 1 of the parents and In the 2nd half from the other parent and thereby committed attacked like it would be with human in the the question of children and then as we know sometimes the document people get a red-headed rated child and this is obviously maybe OK those parents maybe someone had a different girlfriend little by phone at some point or and there was a limitation and this is also introduced with mutation process where change gene very low number of that you know of the genes that enters the introduced some new information but by maybe increases the target speed or other distance to another car on the track and yet so how do we put this together with

talks In the 1st of all or

concept through the cracks themselves which will tell us some with how far the colors on the left on the right of the track and most suited to the correct it will tell us the current speed maybe recurrently driving at the rostrum the motor as early and we have 19 which find is distributed over the front half of the car and depending on the suspension this then visualize and how far the car is with the pitch of the sentence and the TREC water and he uh sense for example the the center to the right in the front here we know OK there's a very steep curve to the right so we have to start driving right and then this helps us determining the temperature so what the readers are richly learning you preferably very simple agent it will just try to accelerate the to a maximum speed then he calls we know if we're talking to fast you might not be able to break before curve for before wall and just write it or the mode for IBM for consisting of them especially driving beginners let me know there's a lot of if you don't have enough runs amended the color just turn off or if you're trying to show you shift and the current I in front of the traffic lights and in and also if you remove all received at the optimal position it's much better for all efficient and fast acceleration and then for groups of course we know that we need to decelerated that before a curve and than their reading so we need to do afarensis between and so that you know we should consider sustain occur and also needs to know how strong breaking before and we need to determine in the same framework independence to other curvature and also if the context in which we might still want to follow a to complete the text so we might just limited maximum speed of a car but learning the permit also so let's see how well this works in training and this was done with 30 randomly

generated individuals that 50 generations and as you can see in the very 1st generation it took about 190 5 seconds for the cost to compute the correct and by running through this algorithm over and over again and evolving the after about 7 to 10 generations we don't 155 seconds for the fact that and after this the slope of the learning curve slow down a bit and after about 50 generated through don't 142 seconds of training images what word but we can see that most of the learning is done in the very 1st generations and I'd left him about the just compute this 1st and others have also used it I wouldn't already in

industry for example this was done by NASA and the when NASA had to develop an antenna for the S 5 missions uh they had engineers working on this for years and years and they didn't come

up with a viable solution fulfilling all the damages for the mission so at some point they just decided to watch while the constraints into stimulation and used it in a divided in 2 and 4 of this uh quite a counter intuitive antenna design which was then the part of the spacecraft it's still in orbit and and it turned out to work better than anything the engineers state that we so in a way yeah yeah maybe this fortified which they did with that that actually move yeah yeah that's just about could they can gradually solution in the beginning and and also at automated university they

use intuitive arguments to optimize the shapes of the Moabites which are basically bicycles you can later on in and about efficiency so you have less resistance left right from the error it when you when you writing yeah this is just the number of shapes generated with evolutionary methods that was certain parametres like for example good pressure from the road and the greatest possible depending on the size of the set so let's see how well In end only and

we will it's the the to all

1 of those ages would behave striving

uh this one's on my machine yeah I laugh at 1st the models this nation's we then we can see it's some midsummer amounts of cells not the end it knows OK just a very good goes down to about 70 kilometres problem curve and we can see that the super-secret it's quite aggressive but soon working also there's already is a little bit of energy is so it's quite aggressive much seems to be performing quite well and with only about data like that a lot of time in of the time we work with the however and the parameters for this were 20 and as was trained with also agree in the similar and

and what is the simulation of what the actual OK I thought it was threatened but what's will accept that 1 within yeah all the all they yes you can compute the companies 1 have his mode and then choose as many processes you want and 1 and 2 simulations at the same time which is a nice when you have a population of 100 to individual to training contain all of them at once and would have to wait for the others to complete this the story so I think of it was used on all of you in the and you can also include several into training and here it is a potent census true if you want to know about its center it to the more than it can move and find quite sophisticated riven behavior we've seen this that if there's a car chasing it it might just try completely to the 1 of the common sense block to so that knows OK there's 1 behind me and it cannot possibly and the 1 behind might be trained in a way it knows OK if there's a strong curve maybe I can try to push it against the wall and it will get damaged do they get any more questions the

just jumped over the question of in

June and uh this was object and I think the shock

fastest driver when the composites were championship was 1 minute and 5 seconds of an advantage in this kind as about 1 minute 20 seconds of electric generators of training of you know what you and we achieved about the same spirit is speed that time with new networks we also train you navigations with its which doesn't have any of all was already known at work having the senses I just described as input and then had some during the the biggest settings steering and braking and acceleration uh as all ports and this got after a lot longer training to about the same speeds so yes yes yes frame this year and then the question was that of the of comparison to other machine learning methods and so and so you can Kennedy moving obstacles in between dynamic uh yeah well it has this opponents and so so I think of the opponent car driving on the kinetic would be the opponents it's trying to dodge but what we get in by the and so he was in here and it was the in the time it was it was and and that it's only used for raising the world OK and so on and with it has this them a diminishing you have the and the other person is efficient islands can be combined he applied to other to permit optimization sure uh so if you have some simulation on some efficient way to test it you can always use it a guy with them to try to approximate the almost optimal set of parameters in the end you so what do we see yes yeah by the and because this different in a guy with a mandate permit itself is downside that this because I think like if given the audience you're not really wrong about this is that too much to go into detail about the parameters but for example they 1 achieve a lot to say 4 of them you can always keep the bed eventually and copied to the next generation so you don't lose progress and you don't have the population at some point where no 1 is performing as well as in the provision for and you will you can have and of across rate so only a certain number of individuals to actually perform cross over you after the mutation rate of how strong the and redistributed and how many that it indicated and hold large parts of the genomic data in this huge amount of parameters which you can set an because I the this totally depends on the problem this talk depends on the problem it's the problem to men and and you when you use gender violence you need to develop a feeling for this what's good and what's bad for your problem that the same you know you yes my and its way to the structure of new In the last so yes this was also tried regression loss if this was also tried in different rates yeah this was the display age was mainly trained floor as literals but we can also perform 21 7 what's at once and then we can yeah well ask this is what I need to know about that yeah yeah I can do this if you want right now yeah but it is rather rocks on almost every aspect of it is also all of this I repeat this on to my force gone BOF profile might be doubling it's a little of this but all of this will be used in the year of the whole list of all your you the question is I think if is also worth if the training has to be done differently with opponents and if you would want into this this guy doesn't use opponents themselves so it was completely ignore opponents and we just crash into them was its behavior and have to try to the a track again there you have to have enable 1 sensors and to perform training completely differently if you at this enabled you know there was disappointment for this agent you want to do I think that's it room and if the