[Music] the next talk will be presented by break the system and the talk is entitled the Mars rover onboard computer the stage is yours

[Applause]

this took entirely too long here we go

over the last 50 years a lot of missions have tried to reach Mars

unfortunately Mars is heart and landing a mass is even harder so a lot of these failed especially the Landers funniest failure by the way is mass climate orbiter which launched in 1999 and had one component that used the imperial system and one component that used the metric system for measurements and when they try to exchange numbers the thing came way too close to mass and burned up in the atmosphere sadhana was the first

cute little rover about this big about the size of a skateboard and deployed in 1997 to the Martian surface Spirit and

Opportunity the Mars exploration Rovers or mer landed in 2004 they're about this large the size of a golf cart I want to say and they're wildly successful spirit was active from 2004 to 2010 whereas opportunity was active for a full 14 years until just this summer when her solar panels got covered by a dust storm

and she went to sleep she traveled for a distance of 45 point 16 kilometers completing a full marathon before the end of her mission also funniest landing

method ever with a series of airbags it's like bouncing across the Martian landscape

today I wanna introduce you to mass science laboratory also known as curiosity curiosity is a completely

autonomous vehicle and she has to be because radio signals from Earth to Mars travel about between 10 and say 40 minutes depending on the positions of Earth and Mars

she has an atomic power source the MM RTG so she's completely independent of solar power and curiosity's top speed is

100 meters per hour that's not a typo

one of her coolest instruments is the freakin laser which she uses to zap stones from up to seven meters away that's the other end of the stage here and uses a spectrometer on the resulting gases to analyze them the rover has

spent over 2,200 souls on Mars a soul by the way as a Martian day it takes about 24 hours and 40 minutes during that time

she drove almost 20 kilometres and she's still going she exceeded her original mission requirement by a huge margin so

say hi curiosity

curiosity has been in different

configurations the crew stage in interplanetary stage navigating with star maps and making course correction maneuvers to plan the perfect arrival on Mars the entry descent landing

configuration with its combination of guidance thrusters heat shields supersonic parachutes retro rockets freaking sky crane that's a that's a whole other talk right here like that's not this talk sadly and finally the

rover configuration we see now all of these configurations were controlled by a single onboard computer using sophisticated and very modular software so let's have a closer look at that today quick introduction hi I'm Daniel

I'm a software engineer I'm an amateur space nerd and I learned a lot about space hardware and software when trying to build satellites for my startup I work at keep safe creating privacy

protecting apps thanks for paying my hotel room by the way and I want to look at the curiosity mission today through the lens of a person who writes software and uses hardware a quick disclaimer by

the rim I've used word of these people here Emily Lackawanna dr. Katharine Weiss and dr. mark mamon extensively in this talk and I admire them greatly but I didn't have the time to contact them so if you find any and all mistakes in my in my presentation they're mine and mine alone and not these people's mistakes let's talk about curiosity's

hardware

this is the bie red 750 a radiation hardened version of the IBM PowerPC 750 created by bas bae systems it can be clocked up to 200 megahertz and it's used in space a lot on curiosity this chip is clocked at 133 megahertz this is the chip and this is the chip within its

port you see two different boards here

one is or both sizes here as far as I know curiosity uses the both sides on the right on the board we have 256

kilobytes of EEPROM we have 256

megabytes of RAM and we have a whole lot of two gigs of flash storage and we have all of that twice in case something

fails the rover can switch from the a side computer to the B side computer they are both completely independent they also have a few components that belong only to one side of the computer for example and if this is going to be important later on the navigation cameras of course all of this is

radiation hardened and there are a lot of techniques to radiation harden a chip and board and this is another talk that I would like to hold someday but today is also not this talk so I'm gonna leave you this slide and tell you that the last space shuttle used a 386 equivalent processor Hubble has been upgraded to a 486 and the International Space Station uses other command computers are using 386 s this is because radiation hardening is hard and takes a lot of time until the chips come out so space stuff is always very very slow [Applause]

what kind of software can we run on this hardware turns out we're using vxworks

six-point-seven a simple real-time operating system and this screenshot is 7 but it looks cool VxWorks is also used in other NASA spacecraft various space X spacecraft the Boeing 787 the cuca industrial robots of my hometown Augsburg and also the the Toshiba eBridge range of photo

copiers and before you ask yes it can run Java but in this case it doesn't

the software that runs on curiosity has a long lineage there has been a continuous codebase that started in the 90s when JPL rode very code for the prototype prototype Rovers and then part of that was used in sadhana and then that got rewritten and whatever and was used in the Mars exploration Rovers and then it was ported from C to C plus plus and refactored into a component based architecture for MSL aka curiosity and

yes we are using C++ not for C++ as you'd use on a desktop phone one thing that's missing is processes for example the X words gives you this concept of tasks but they're not as independent as processes they all use one big chunk of memory and it's up to you to make sure that they don't like stomped all over each other we also don't have exceptions

templates iostream multiple inheritance and the only operator we're overloading are the new and delete operators that I used to allocate new memory for objects or for things and to deallocate that that memory and they are overwritten by the custom memory allocator that JPL wrote for VxWorks this is a pretty cool piece of software because it's guarantees graceful access to these defined memory pools and it has their well-defined behavior for out of memory conditions which is very important as you can imagine it supports multiple pools in different areas of the RAM so that's that's how you can separate your motor processes it provides Diagnostics there's a display that shows you each new and delete operation and it can also give you a map of the RAM at on on request the rover can even downlink a map of its RAM if needed the same allocators also used for development on JPL's UNIX machines which is of course very helpful and there's no garbage collection here which were being too expensive instead you have to be very very careful not to have any memory leaks and these are usually and forced by unit tests [Applause]

if I if I add a title of something by the way you should be you should be able to google it and watch it on YouTube those talks are super interesting one of the philosophies behind the flight software is its component-based architecture functions are grouped in two components and a component exports an interface and identifies the interfaces that it needs to function

that means that you can replace any and all components of the system as long as they export the same interface it also means you can work on your own component without having to touch all the other parts of the system this is very helpful for development and also for testing and extensibility you'll also also notice that the components here are organized in layers the lower level components deal with the

Harvard directly they turn switches they move actuators they prepare data for being set for being sent back to earth they also abstract away redundancy components there's a lot of redundant components where just a thing is just twice in the rover in case one of them fails and the lower levels will automatically fail over to a backup component without the upper levels having to know the intricate details of what exactly went wrong so they will get a notification that says something went wrong and I'm using the backup but they don't don't have to know how the backup is activated or how it's being used further up we have components that coordinate these very atomic functions into actions and finally we have

components that group these actions into whole activities like land on the surface or take and analyze a surface sample the land on the surface module by the way is not on here because it was deleted right after it was like I right after the spacecraft actually landed on the surface

the most complex module in the whole codebase is the surface navigation module which was in some form also used for the Mars exploration Rovers already on mer it was about 21 percent of the full code base for MSL or curiosity it's about 10% of the whole code base is just the surface navigation module pathfinding moving that's the kind of stuff there's about 100 in the individual or actually over a hundred individual source code modules each with its own owner another philosophy is

fault protection mobility and science modules should not have to worry about the spacecraft safety instead the

infrastructure in addition to providing the platform to do mobility and science must keep the system safe and away from the dreaded safe mode unsafe actions should not be allowed and should be blocked the pattern here is hole should be detected and corrected as low in the layers as possible finally we have just loads and loads of testing we have unit

tests we have static analysis and we

have validation and verification where we run code on the various test beds like this complete duplicates of curiosity down on the ground and then

there's tests as you fly once the software is uploaded behaviors will be tried out and tested on Mars during the mission [Applause]

so this is really cool curiosity's flight software can be remotely updated from Earth so engineers on the ground can change the software and introduce new functions and behaviors or tweak the landing code the mobility module by the way the thing that allows the rover to move her arm and drive around was actually developed while the spacecraft was already in cruising stage towards Mars it was not part of the launch load at all it was uploaded once the rover was safely on the ground it's kind of like a like the console you got for Christmas you unpack its you plug it in and then it just has to update for a week in this case

how does the software update mechanism work first the software update is uploaded into the active computers RAM and run there and if the system behaves erroneously it can reboot and load the old version from its flash storage then the rover is going through an extensive series of tests to see if everything works as expected if that passes the software gets uploaded into the flash storage and booted from after another series of tests that half of the computer is updated and the process then gets repeated the same way for the other side because we want to have both computers ready for anything in case we need to fail over to the backup memory rather

days start are around 9:00 or 10:00 in the morning local martian time when curiosity boots up from her dream mode while in dream mode various FPGAs already flip on the heating system and prepare everything so that the rover is warm once the main computer is booting up and then she gets directions from the ground and sets off for her day she

drives she collects data she runs experiments she drills she zaps it's pretty fun but very exhausting the

day ends shortly before sundown when the last satellite passed across the and that is used to upload the last bits of data for the day before the rover goes back to sleep curiosity does have various antennas and some of them could talk to us directly but that would be at

the speed of about a 14 point 4k modem using the relay satellites the data rate goes up to about 500 megabits per day that's about 62 megabytes per day so the data with the highest priority gets sent pretty quickly and the rest and just follow at some point and then she needs to go to sleep the mmm RTG the atomic power source

produces about 110 watts of energy the rover needs 45 to 70 watts even while sleeping a hundred and fifty watts while

she's awake and up to 500 watts while driving so she spends most of her life asleep and recharging her batteries being awake for about six hours each day

lucky her

next I want to talk about how actually how the driving part actually works

Rover drivers use a system called RSVP

the rover sequencing and visualization program to plan out drives this has been used an update updated over the years and has been used for previous Rover missions as well in in in RSVP the

terrain data that curiosity sends back is being reconstructed in 3d and the human robot Rovers can look at it like fly through it do local simulations of activities some sanity checking does this really make sense in this context and then send orders to the rover these orders could for example be like driving

the easiest driving mode is blank driving just point the rover in to any

directions and go for ten meters just blindly this is dangerous and cumbersome however because you can only drive as far as you can see and even then like the further away things you can see like maybe there's things hidden by perspective and then you have to wait for more orders from the ground even then you might into you might run into obstacles the thing though is that this is way easier computationally and way more predictable so it was used a lot especially in the beginning of the mission

and we have visual odometry odometry is finding out how far you've come by counting real revolutions or other things thing is audiometers in wheels don't work here because the curiosity can't detect slippered in real time and the metal wheels they slip a lot on rock and sand instead what she does is she takes before-and-after pictures with her stereoscopic cameras and compares them she will auto select various features in the terrain that she finds interesting and follow them along as she drives so she will she will go and then see like a super interesting rock just I look at the rock for a while and see how far she's come this is all done on the rover this is not being sent back to earth or something so this is a completely separate system and it works very very well and then there's Auto nav

where the Rover drivers set a target or a destination location and the robot tries to find her way towards the target completely autonomously while avoiding the red no-go areas that are designated by ground control curiosity has stereo

cameras and takes pictures with them

features in the image pair are correlated and triangulated to get the distance of these features the range data must satisfy a series of tests before being seen as correct like unclear and misleading parts of the image like pixels that are only in one of the images or parts of the robot of the rover are being ignored

using these stereoscopic images curiosity will then create a geometric model of the surrounding terrain and

subdivided into grid cells of the resolution of a robot wheel she will

then rate each of these cells according to its Traverse ability how much she'd

like to drive they're taking in app into account slopes steps roughness excessive

tilt and other environmental dangers like things that just stick out too much

from the ground and would scratch her belly pan these obstacles are expanded

by the radius of the rover so she will for that

her sensor will not touch these things and she will stay far enough away from them this gridded reversibility map is

used for each step of the navigation so she will then project a number of paths onto the map and a valued evaluate them is this safe is this will this get me closer to my destination she then chooses the safest path that gets her closer to the goal and drives a short distance on their path and after each step the navigation process starts again until the goal is reached or no safe path is found or the rover is commanded to stop to find her way

curiosity uses a version of a-star search optimized for driving kindly increments at a time between map updates because of this optimization the the algorithm is much faster and more efficient than regular a-star search and therefore warm but better to use on the rover Auto knife is still too slow to

use all the time but as the mission went on the robot did more and more

autonomous driving because people learned how to trust another Rover and give it the correct directions and also

the software was improved over and over

this is curiosity looking back at her very first Drive she surprised the engineers here because she avoided a very very tiny rock you can see it at the top where this little corner so so so so she went straight and then there was this rock and she was the girl this is very dangerous this and we've seen

this before this is basically exactly than thrive being replayed in higher speed in RSVP and look at the head moving around and also the other cameras you can see them with them they also take pictures and this is visual odometry in Auto nav working all right

what other features do we have we have

the Molly camera the rover has an arm and at the end of that arm there's a camera turns out it connects you take pictures of itself for example of the belly pan or of the wheels directly

after landing to see if they survived the landing incorrect

using that arm she can also take larger panel panoramas by taking multiple pictures and therefore cover her entire body this is what the result looks like

this is curiosity's very first selfie if you stitch them together they look

pretty cool like they are amazing pictures and enough to look at them the

arm is usually photoshopped out because you will only see half the arm all the time because it doesn't fit into the panorama let me tell you about what

happened on Sol 200 suddenly the rover couldn't safe data anymore transmitting images live back to the ground worked fine but saving them for later upload would fail their overall so rebooted multiple times without apparent reasoning [Applause] in the end the engineering team decided to switch to the b-side computer from there they could assess the damage and it turns out that the a-site computer has a fault a fault in its flash you can't really tell what it is but they have to now disabled half of the flash in the a science computer and this seems to work fine curiosity has been was on the B side computer from then on them at all and this is more work than you can imagine because as I

told you earlier there are various

cameras especially the navigation cameras that are directly connected to each of the boards so if you switch switch from the a side to the B side turns out that the cameras they are slightly offset and this gives you different angles and these have to be programmed into the software of course or it will give you 40 data and then it also turned out that during the heat and the Martian summer these cameras warp slightly and this warp like whoops the images a little bit and that makes that that makes the the triangulation of the stereoscopic images very hard so they also had to write code while on earth just by looking at the pictures how to how to calculate out this warping the whole thing

exactly one year after landing on Mars curiosity celebrated her first birthday this is Sam an instrument that includes a very exactly controlled sieve that it that conceive with like these perfectly programmable vibrations add the first birthday after landing was programmed to play the following vibrations

[Applause] [Applause]

imagine the tiny rover Curiosity sitting in the Martian desert just like singing a birthday song for herself this only happened once by the way you hear sometimes on the Internet the rumor that this happens every year it happens exactly once and the first birthday it has been six birthdays in her five birth birthday since so and those were all just regular working days after some

time on Mars engineers noted that the wheels were getting torn up way quicker than expected the wheels they are only

about a millimeter thick and they're made from machined aluminium what

happened was that curiosity ended up in terrain that wasn't experienced by up here so Donna normally these small rocks as you see on this picture for example when the rover drives over them they get

just pushed to the side or pushed into the ground but this clearly is not

happening here the terrain is ripping deep gashes and holes into the wheels so

JPL engineers on the ground like Amanda Stephie here constructed various ground test beds that simulated different new ground conditions and tested the wheels

to failure as they say so that they know exactly which types of terrain are the most dangerous to drive on and it turns

out that if you have rocks that are cemented into the ground and they have very sharp edges at the top and they don't really move aside because they're completely fixed in place then the back wheels will force the front wheels that can't really move because of the sharp rocks onto those rocks with a huge torque that the rover has and this leads to these punctures so the driving

software was changed to allow the wheels to move at different turn rates so to avoid this this pushing onto the sharp rocks thing also the rover has been very carefully steered out of the rocky areas and is now back on the more sandy areas that she prefers

unsold 21 72 that is September 15 of this year another compute computer problem was detected curiosity was completely healthy but could not access parts of her memory where her where she stalls data for later uploads this is clearly a serious serious problem but it doesn't endanger the rover safety you'll notice no reboots this time for now the rover was switched back to the the a side and is merrily doing science again while engineers in the background are trying to diagnose the problem and you remember the a side is safe until as long as it doesn't address the faulty part of the flash what does the future

hold for curiosity so curiosity's

primary mission was from 2012 to 2014 and she's still going strong in 2018 so that's pretty cool turns out a few months ago at a conference in Toulouse NASA and JPL engineers said that there were about 10 kilometers left in the wheels if conditions are about the same we can also expect the mm RTG power source to last for another 10 years or so so if nothing happens she will live on for a while she has more or less already reached her goal this is Mount sharp

aka Aeolus Mons the central peak in Gale

Crater and Gale Crater was once filled with water and because there's lots of lots of different sediments at the bottom of Gale Crater there's a boatload of interesting science to be done here and after driving for 20 kilometres curiosity is now in the foothills or as JPA calls them the butts of sharp Mount sharp so there's lots of cool stuff to look at in about two years we should see the

arrival of mass 2020 hurry Aziz sister / it has the same chassis and the same body same software Manas slightly updated wheels and a completely new set of science instruments it will also be the very first Rover to include a microphone so that we can hear what the surface of Mars sounds like and maybe just maybe if we listen close enough we can hear a tiny faraway birthday song

[Applause]

in here in the AIA exactly and we start with a microphone - two small questions felt first why you call her over she and

the second one can you say something more about era Pangolin does the software have has have different kinds of errors which are treated differently right good question so the first question was why is it a she and it turns out that jpn at NASA call all their Rovers by female pronouns so I just did the same because just seems nice and the second question was about error handling and there are there's a whole pattern that is being discussed by Katharine Weiss for example at one of the software engineers and architects of their software where there's like a whole error detection patterns and they do all the all in the same way they have like three status codes green yellow and red where green is just everything is nominal yellow is something happened but the lower-level component was able to fix it by I don't

know like moving like routing something around or switching to a backup component or whatever and then there's red where the where the component just says like okay something something way like higher up in the chain has to deal with this so this way they can like bubble up these error status codes thanks okay we have already more questions than we can take I am afraid but we continue with number five please okay thanks for your nice talk and I wants to know how does the image recognition for the past finding that's the rover compares the images or does it have a kind of lighter sensor or how does it work and it is all just regular images so there's no lidar I'm looking at my backup slides just now because I have I have a lot on this so basically I'm fine I'll find it later basically it's just regular image recognition there's an algorithm in there called roxtor which can recognize rocks in the sand like the ones you see here actually by their contours and also because if because you have stereoscopic images you can

calculate like a 3d map of the terrain number four please oh I've heard that

some components echo and controlled by Ross robot operating system is right Roboto something rating system us robot operating system it might be true I have to be honest I haven't heard of that term before so that's the edge of my knowledge apparently numbers eight I guess so when the autonomous driving softer was designed would be probably around 2005 today we have autonomous cars being designed for pure cameras is NASA on a dead end a different path will they integrate what is being designed today for cars here or is this going to be completely in a different frame which is not compatible are they going to augment or rewrite from scratch so I don't I don't have any affiliation with NASA but if you want to hire me I'm here I don't have any affiliation with NASA so I can't speak for them to me it seems like it's very too different use scenarios where the one where like autonomous cars are like they use all kinds of like detection specific to roads whereas the rover has to navigate a rocky terrain but I can imagine like because there's a lot of research going on that and this research is public that they might take the best bits of that research and integrate them like these people are incredibly smart okay we take one question from the number six Pacific reason why there's no radar or little sensor included because I think that would make getting 3d images more easy I do not know of a

reason it might be because it's just like the whole design is basically from the early 2000s maybe light I wasn't wasn't good enough back then what I can recommend to you by the way is this incredibly good book that how curiosity does instace job by Emily Lackawanna from the Planetary Society like this will definitely answer this question I imagine but I don't know number one please our update signed is the communication encrypted I know this parity bits and hashes going on that's your answer many hours and Mars birthday but how its organized herself to work in different times per day or it's in the same time every day the robot always works in its own or in her own time zone so if I say ten o'clock I mean ten o'clock in the morning or in on Mars so about a few hours after the Sun has risen a Martian day a soul has about 24 hours and 39 something minutes so it's very close to Earth Day but there's some some movement between Earth days and Mars days so they they're there's some offset that's like but worse and worse so for the first year or so or maybe even two years the JPL team that controlled the Mars rovers actually worked in shifts that corresponded to Martian days and that led of course to night shifts weekend shifts all kinds of stuff and they have since created this whole system where they oh they can actually work just during the Earth Day and it results in about these 30 days stretches where the rover still gets new updates and orders every day and then in between there's always like a few days where just the rubber just sits there doesn't have anything to do because the people need to sleep and during that time it will just upload all the data that it has collected because of the slow uplink speed okay two more number five and then number two and then we close the session I great talk on the two incidents when they had to switch the computer sites do you know if they found out what the cause of that was was it like cosmic rays frying something or the thermal expansion working they did not like this is the thing like and there's also other incidents that I didn't talk about like for example that drill broke at some point and in all of these cases you can't really find out what's going on because to do that you would have to go there open the open the lid and just look in and see the things so like all they know is what their software can tell them so their software can tell them it's crashing when it accesses this part of the memory so don't hold it that way well thanks okay and finally number two I those two computers named a and B are they of the same hardware design or are they different they are they're both both the chip that I that I told you about earlier the BAE 750 I'll be I'll be in the in the bar just outside the exit the overflow bar if you have any more questions find me there okay with that let's thank the speaker again [Applause] [Music] [Music]