The next talk will be presented by Break the System and the talk is entitled The Mars

Rover Onboard Computer. The stage is yours.

This took entirely too long, here we go.

Over the last 50 years a lot of missions have tried to reach Mars. Unfortunately Mars is hard and landing on Mars is even harder. So a lot of these failed, especially the landers. Funniest failure by the way is Mars Climate Orbiter which launched in 1999 and had one

component that used the imperial system and one component that used the metric system for measurements and when they tried to exchange numbers the thing came way too close to Mars and burned up in the atmosphere. Sojourner was the first cute little rover about this big, about the size of a skateboard

and deployed in 1997 to the Martian surface. Spirit and Opportunity, the Mars Exploration Rovers or MER, landed in 2004. They're about this large, the size of a golf cart I want to say and they're wildly

successful. Spirit was active from 2004 to 2010 whereas Opportunity was active for a full 14 years until just this summer when her solar panels got covered by a dust storm and she went

to sleep. She travelled for a distance of 45.16 kilometres, completing a full marathon before the end of her mission. Also funniest landing method ever with a series of airbags that's like bouncing across the Martian landscape.

Today I want to introduce you to Mars Science Laboratory, also known as Curiosity. Curiosity is a completely autonomous vehicle and she has to be because radio signals from Earth to Mars travel about between 10 and say 40 minutes depending on the positions

of Earth and Mars. She has an atomic power source, the MMRTG, so she's completely independent of solar power and Curiosity's top speed is 100 metres per hour. That's not a typo.

One of her coolest instruments is the freaking laser which she uses to zap stones from up to seven metres away, that's the other end of the stage here, and uses a spectrometer on the resulting gases to analyse them.

The rover has spent over 2,200 soles on Mars. A sol, by the way, is a Martian day. It takes about 24 hours and 40 minutes. During that time she drove almost 20 kilometres and she's still going. She exceeded her original mission requirement by a huge margin.

So say hi, Curiosity. Curiosity has been in different configurations. The cruise stage in interplanetary stage, navigating with star maps and making course

correction manoeuvres to plan the perfect arrival on Mars. The entry descent landing configuration with its combination of guidance thrusters, heat shields, supersonic parachutes, retro rockets, freaking sky crane, that's a whole other talk

right here, like that's not this talk, sadly. And finally, the rover configuration we see now. All of these configurations were controlled by a single onboard computer using sophisticated and very modular software.

So let's have a closer look at that today. Quick introduction. Hi, I'm Daniel. I'm a software engineer. I'm an amateur space nerd and I learnt a lot about space hardware and software when trying to build satellites for my startup. I work at Keepsafe, creating privacy-protecting apps.

Thanks for paying my hotel room, by the way. And I want to look at the Curiosity mission today through the lens of a person who writes software and uses hardware. A quick disclaimer, by the way. I've used work of these people here, Emily Lactawala, Dr. Catherine Weiss, and Dr. Mark

Mamone extensively in this talk, and I admire them greatly, but I didn't have the time to contact them. So if you find any and all mistakes in my presentation, they're mine and mine alone and not these people's mistakes. Let's talk about Curiosity's hardware.

This is the BAE RAD 750, a radiation-hardened version of the IBM PowerPC 750. Created by BAE Systems, it can be clocked up to 200 MHz and it's used in space a lot.

On Curiosity, this chip is clocked at 133 MHz. This is the chip, and this is the chip within its board. You see two different boards here, or board sizes here. As far as I know, Curiosity uses the board size on the right. On the board, we have 256 kilobytes of EEPROM, we have 256 megabytes of RAM, and

we have a whole lot of 2 gigs of flash storage. And we have all of that twice. In case something fails, the rover can switch from the A-side computer to the B-side computer.

They are both completely independent. They also have a few components that belong only to one side of the computer. For example, and this is going to be important later on, the navigation cameras. Of course, all of this is radiation-hardened, and there are a lot of techniques to radiation-harden

a chip and board, and this is another talk that I would like to hold some day, but today is also not this talk. So I'm going to leave you this slide and tell you that the last space shuttle used a 386 equivalent processor, Hubble has been upgraded to a 486, and the International

Space Station uses, or the command computers are using 386s. This is because radiation-hardening is hard and takes a lot of time until the chips come out, so space stuff is always very, very slow.

What kind of software can we run on this hardware? Turns out we're using VxWorks 6.7, a simple real-time operating system.

This screenshot is seven, but it looks cool. VxWorks is also used in other NASA spacecraft, various SpaceX spacecraft, the Boeing 787, the KUKA industrial robots of my hometown Augsburg, and also the Toshiba e-bridge

range of photocopiers. And before you ask, yes, it can run Java, but in this case, it doesn't.

The software that runs on Curiosity has a long lineage. There has been a continuous code base that started in the 90s when JPL wrote code for their prototype rovers, and then part of that was used in Sojourner, and then that

got rewritten and whatever, and was used in the Mars exploration rovers, and then it was ported from C to C++ and refactored into a component-based architecture for MSL, aka Curiosity. And yes, we are using C++.

Not full C++ as you'd use on a desktop phone. One thing that's missing is processes, for example. VxWorks gives you this concept of tasks, but they're not as independent as processes. They all use one big chunk of memory, and it's up to you to make sure that they don't

stomp all over each other. We also don't have exceptions, templates, Iostream, multiple inheritance, and the only operator we're overloading are the new and delete operators that are used to allocate new memory for objects or for things, and to deallocate that memory.

And they are overwritten by the custom memory allocator that JPL wrote for VxWorks. This is a pretty cool piece of software because it guarantees graceful access to these defined memory pools, and it has a well-defined behaviour for out-of-memory conditions, which

is very important, as you can imagine. It supports multiple pools in different areas of the RAM, so that's how you can separate your mod processes. It provides diagnostics. There's a display that shows you each new and delete operation, and it can also give

you a map of the RAM on request. The rover can even downlink a map of its RAM, if needed. The same allocator is also used for development on JPL's Unix machines, which is, of course, very helpful, and there's no garbage collection here, which would be too expensive.

Instead, you have to be very careful not to have any memory leaks, and these are usually enforced by unit tests. If I add a title of something, by the way, you should be able to Google it and watch

it on YouTube. Those talks are super interesting. One of the philosophies behind the flight software is its component-based architecture. Functions are grouped into components, and a component exports an interface and identifies

the interfaces that it needs to function. That means that you can replace any and all components of the system, as long as they export the same interface. It also means you can work on your own component without having to touch all the other parts of the system.

This is very helpful for development and also for testing and extensibility. You'll also notice that the components here are organised in layers. The lower-level components deal with the hardware directly. They turn switches, they move actuators, they prepare data for being sent back to

earth. They also abstract away redundancy components. There's a lot of redundant components where a thing is just twice in the rover in case one of them fails, and the lower levels will automatically fail over to a back-up

component without the upper levels having to know the intricate details of what exactly went wrong. So they will get a notification that says something went wrong, and I'm using the back-up, but they don't have to know how the back-up is activated or how it's being used. Further up, we have components that coordinate these very atomic functions into actions.

And finally, we have components that group these actions into whole activities like land on the surface, or take and analyse a surface sample. The land on the surface module, by the way, is not on here because it was deleted

right after the spacecraft actually landed on the surface. The most complex module in the whole codebase is the surface navigation module, which was in some form also used for the Mars exploration rovers already.

On Mer, it was about 21 per cent of the full codebase. For MSL or curiosity, it's about ten per cent of the whole codebase is just the surface navigation module, pathfinding, moving, that kind of stuff. There's about 100 individual or actually over 100 individual source code modules,

each with its own owner. Another philosophy is fault protection. Mobility and science modules should not have to worry about the spacecraft's safety.

Instead, the infrastructure in addition to providing the platform to do mobility and science must keep the system safe and away from the dreaded safe mode. Unsafe actions should not be allowed and should be blocked. The pattern here is fault should be detected and corrected as low in the layers as possible.

Finally, we have just loads and loads of testing. We have unit tests, we have static analysis, and we have validation and verification where we run code on the various test beds like this complete duplicate of curiosity

down on the ground. And then there's test as you fly. Once the software is uploaded, behaviours will be tried out and tested on Mars during the mission.

So this is really cool. Curiosity's flight software can be remotely updated from Earth. So engineers on the ground can change the software and introduce new functions and behaviours or tweak the landing code. The mobility module, by the way, the thing that allows the river to move her arm and

drive around was actually developed while the spacecraft was already in cruising stage towards Mars. It was not part of the launch load at all. It was uploaded once the rover was safely on the ground. It's kind of like the console you got for Christmas. You unpack it, you plug it in, and then it just has to update for a week in this

case. How does the software update mechanism work? First, the software update is uploaded into the active computer's RAM and run there.

And if the system behaves erroneously, it can reboot and load the old version from its flash storage. And the rover is going through an extensive series of tests to see if everything works as expected. If that passes, the software gets uploaded into the flash storage and booted from.

After another series of tests, that half of the computer is updated, and the process then gets repeated the same way for the other side, because we want to have both computers ready for anything in case we need to fail over to the backup computer.

Rover days start around 9 or 10 in the morning, local Martian time, when Curiosity boots up from her dream mode. While in dream mode, various FPGAs already flip on the heating system and prepare everything so that the rover is warm once the main computer is booting up.

And then she gets directions from the ground and sets off for her day. She drives, she collects data, she runs experiments, she drills, she zaps. It's pretty fun, but it's very exhausting.

The day ends shortly before sundown, when the last satellite pass occurs. And that is used to upload the last bits of data for the day before the rover goes back to sleep. Curiosity does have various antennas, and some of them could talk to Earth directly, but that would be at the speed of about a 14.4K modem.

Using the relay satellites, the data rate goes up to about 500 megabits per day. That's about 62 megabytes per day. So the data with the highest priority gets sent pretty quickly, and the rest can just follow at some point.

And then she needs to go to sleep. The MMRTG, the atomic power source, produces about 110 watts of energy. The rover needs 45 to 70 watts even while sleeping. 150 watts while she's awake, and up to 500 watts while driving.

So she spends most of her life asleep and recharging her batteries, being awake for about six hours each day. Lucky her. Next I want to talk about how the driving part actually works.

Rover drivers use a system called RSVP, the Rover Sequencing and Visualization Program to plan out drives. This has been used and updated over the years and has been used for previous rover missions as well.

In RSVP, the terrain data that Curiosity sends back is being reconstructed in 3D, and the human rover drivers can look at it, like, fly through it, do local simulations of activities, some sanity checking, does this really make sense in this context, and then send

orders to the rover. These orders could, for example, be blind driving. The easiest driving mode is blind driving. Just point the rover into any directions and go for 10 meters just blindly.

This is dangerous and cumbersome, however, because you can only drive as far as you can see, and even then, like, the further away things you can see, like, maybe there's things hidden by perspective, and then you have to wait for more orders from the ground. Even then, you might run into obstacles.

The thing, though, is that this is way easier computationally and way more predictable, so it was used a lot, especially in the beginning of the mission. Then we have visual odometry. Odometry is finding out how far you've come by counting wheel revolutions or other

things. The thing is, odometers and wheels don't work here because Curiosity can't detect slippage in real time, and the metal wheels, they slip a lot on rock and sand. Instead, what she does is she takes before and after pictures with her stereoscopic cameras and compares them.

She will auto-select various features in the terrain that she finds interesting and follow them along as she drives, so she will go and then see, like, a super interesting rock and just, like, look at the rock for a while and see how far she's come. This is all done on the rover. This is not being sent back to Earth or something, so this is a completely separate

system and it works very, very well. And then there's AutoNerf, where the rover drivers set a target or a destination location and the rover tries to find her way towards the target completely autonomously while avoiding the red no-go areas that are designated by ground control.

Curiosity has stereo cameras and takes pictures with them. Features in the image pair are correlated and triangulated to get the distance of these features. The range data must satisfy a series of tests before being seen as correct, like unclear

and misleading parts of the image, like pixels that are only in one of the images or parts of the rover are being ignored. Using these stereoscopic images, Curiosity will then create a geometric model of the surrounding terrain and subdivide it into grid cells of the resolution of the rover wheel. She will then rate each of these cells according to its traversability, how much she'd

like to drive there, taking into account slopes, steps, roughness, excessive tilt, and other environmental dangers like things that just stick out too much from the ground and would scratch her belly pan. These obstacles are expanded by the radius of the rover so that her center will not

touch these things and she will stay far enough away from them. This gridded traversability map is used for each step of the navigation. So she will then project a number of paths onto the map and evaluate them.

Is this safe? Will this get me closer to my destination? She then chooses the safest path that gets her closer to the goal and drives a short distance on that path. And after each step, the navigation process starts again until the goal is reached or no safe path is found or the rover is commanded to stop.

To find her way, Curiosity uses a version of A-Star Search optimized for driving tiny increments at a time between map updates. Because of this optimization, the algorithm is much faster and more efficient than regular A-Star Search and therefore better to use on the rover.

AutoNav is still too slow to use all the time, but as the mission went on, the rover did more and more autonomous driving because people learned how to trust the rover and give it the correct directions and also the software was improved over and over.

This is Curiosity looking back at her very first drive. She surprised the engineers here because she avoided a very, very tiny rock. You can see it at the top where this little corner is. So she went straight and then there was this rock and she was like, oh, this is very dangerous, I'm going to walk around. And we've seen this before. This is basically exactly that drive being replayed in higher speed in RSVP.

And look at the head moving around and also the other cameras. You can see them, but they also take pictures. And this is visual odometry in AutoNav working. All right, what other features do we have? We have the Ma Li camera.

The rover has an arm and at the end of that arm, there's a camera. Turns out it can actually take pictures of itself. For example, of the belly pan or of the wheels directly after landing to see if they survived the landing correctly. Using that arm, she can also take larger panoramas by taking multiple pictures and

therefore cover her entire body. This is what the result looks like. This is Curiosity's very first selfie. If you stitch them together, they look pretty cool. Like they are amazing pictures and I love to look at them.

The arm is usually photoshopped out because you will only see half the arm all the time because it doesn't fit into the panorama. Let me tell you about what happened on Sol 200. Suddenly, the rover couldn't save data anymore. Transmitting images live back to the ground worked fine, but saving them for later

upload would fail. The rover also rebooted multiple times without apparent reasoning. In the end, the engineering team decided to switch to the B-side computer. From there, they could assess the damage and it turns out that the A-side

computer has a fault in its flash. You can't really tell what it is, but they have now disabled half of the flash in the A-side's computer and this seems to work fine. Curiosity was on the B-side computer from then on.

This is more work than you can imagine because as I told you earlier, there are various cameras, especially the navigation cameras, that are directly connected to each of the boards. So if you switch from the A-side to the B-side, it turns out that the cameras are slightly offset.

This gives you different angles and these have to be programmed into the software, of course, or it will give you faulty data. Then it also turned out that during the heat in the Martian summer, these cameras warp slightly and this warp warps the images a little bit and that makes the triangulation of the stereoscopic images very hard.

So they also had to write code while on Earth just by looking at the pictures, how to calculate out this warping. It's a whole thing. Exactly one year after landing on Mars, Curiosity celebrated her first birthday.

This is SAM, an instrument that includes a very exactly controlled sieve that can sieve with these perfectly programmable vibrations. The first birthday after landing was programmed to play the following vibrations.

Imagine the tiny rover Curiosity sitting in the Martian desert just like singing a birthday song for herself.

This only happened once, by the way. You hear sometimes on the internet the rumor that this happens every year. It happens exactly once and the first birthday. It has been six birthdays, five birthdays since. So, and those were all just regular working days.

After some time on Mars, engineers noted that the wheels were getting torn up way quicker than expected. The wheels, they are only about a millimeter thick and they are made from machined aluminium. What happened was that Curiosity ended up in terrain that wasn't experienced by Oppie or Sojourner.

Normally, these small rocks, as you see on this picture, for example, when the rover drives over them, they get just pushed to the side or pushed into the ground. But this clearly is not happening here. The terrain is ripping deep gashes and holes into the wheels.

So, JPL engineers on the ground, like Amanda Steffi here, constructed various ground test beds that simulated different new ground conditions and tested the wheels to failure, as they say, so that they know exactly which types of terrain are the most dangerous to drive on. And it turns out that if you have rocks

that are cemented into the ground and they have very sharp edges at the top and they don't really move aside because they're completely fixed in place, then the back wheels will force the front wheels that can't really move because of the sharp rocks onto those rocks with the huge torque that the rover has

and this leads to these punctures. So, the driving software was changed to allow the wheels to move at different turn rates. So, to avoid this pushing onto the sharp rocks thing, also the rover has been very carefully steered out of the rocky areas and is now back on the more sandy areas that she prefers.

On sol 2172, that is September 15 of this year, another computer problem was detected. Curiosity was completely healthy but could not access parts of her memory

where she stores data for later uploads. This is clearly a serious problem but it doesn't endanger the rover's safety. You'll notice no reboots this time. For now, the rover was switched back to the A side and is merely doing science again,

while engineers in the background are trying to diagnose the problem. And you remember, the A side is safe as long as it doesn't address the faulty part of the flash. What does the future hold for Curiosity? So, Curiosity's primary mission was from 2012 to 2014

and she's still going strong in 2018, so that's pretty cool. Turns out, a few months ago at a conference in Toulouse, NASA and JPL engineers said that there were about 10 kilometers left in the wheels if conditions are about the same.

We can also expect the MMRTG power source to last for another 10 years or so, so if nothing happens, she will live on for a while. She has more or less already reached her goal. This is Mount Sharp, aka Aeolus Mons,

the central peak in Gale Crater. And Gale Crater was once filled with water and because there's lots of different sediments at the bottom of Gale Crater, there's a boatload of interesting science to be done here. And after driving for 20 kilometers,

Curiosity is now in the foothills, or as JPL calls them, the butts of Mount Sharp. So there's lots of cool stuff to look at. In about two years, we should see the arrival of Mars 2020.

Curiosity's sister rover. It has the same chassis and the same body, same software, more or less, slightly updated wheels, and a completely new set of science instruments. It will also be the very first rover to include a microphone so that we can hear

what the surface of Mars sounds like. And maybe, just maybe, if we listen close enough, we can hear a tiny, faraway birthday song.

Tears in my eyes, actually. What's left for today?

So I guess we have lots of questions. There are microphones here in the aisle, exactly, and we start with a microphone, too. Hi, thanks for the talk. Two small questions. First, why you call the rover a she?

And the second one, can you say something more about error handling? Does the software have different kinds of errors which are treated differently? Right, good question. So the first question was, why is it a she?

And it turns out that JP and NASA call all their rovers by female pronouns, so I just did the same, because it just seems nice. And the second question was about error handling. There is a whole pattern that is being discussed by Katherine Weiss, for example,

one of the software engineers and architects of the software, where there's a whole error detection pattern, and they do it all in the same way. They have three status codes, green, yellow, and red, where green is just everything is nominal. Yellow is something happened, but the lower level

component was able to fix it by, I don't know, like rooting something around, or switching to a backup component, or whatever. And then there's red, where the component just says, okay, something way higher up in the chain has to deal with this.

So this way, they can like bubble up these error status codes. Thanks. Okay, we have already more questions than we can take, I am afraid, but we continue with number five, please. Okay, thanks for your nice talk, and I want to know how does the image recognition works for the path finding?

Does the rover compare the images, or does it have a kind of lighter sensor, or how does it work? It is all just regular images, so there's no LIDAR. I'm looking at my backup slides just now, because I have a lot on this. So basically, I'll find it later. Basically, it's just regular image recognition.

There's an algorithm in there called Rockster, which can recognize rocks in the sand, like the ones you see here, actually, by their contours. And also, because you have stereoscopic images, you can calculate like a 3D map of the terrain.

Number four, please. Hello, I've heard that some components, or this echo, are controlled by Ross Robot Operating System, is that right? Robot something rating system? Ross Robot Operating System.

It might be true, I have to be honest, I haven't heard of that term before. So, that's the edge of my knowledge, apparently. Number eight, I guess. So, when the autonomous driving software was designed,

would be probably around 2005. Today we have autonomous cars being designed for pure cameras. Is NASA on a dead end, a different path?

Will they integrate what is being designed today for cars here? Or is this going to be completely in a different frame which is not compatible? Are they going to augment or rewrite from scratch?

So, I don't have any affiliation with NASA, but if you want to hire me, I'm here. I don't have any affiliation with NASA, so I can't speak for them. To me, it seems like it's two very different use scenarios where autonomous cars, like they use all kinds of detection specific to roads,

whereas the rover has to navigate a rocky terrain. But I can imagine, because there's a lot of research going on, and this research is public, that they might take the best bits of that research and integrate them. These people are incredibly smart.

Okay, we take one question from the number six. Is there a specific reason why there's no radar or LIDAR sensor included? Because I think that would make getting 3D images more easy. I do not know of a reason. It might be because it's just,

like the whole design is basically from the early 2000s. Maybe LIDAR wasn't good enough back then. What I can recommend to you, by the way, is this incredibly good book, How Curiosity Does Its Job, by Emily Lactawala from the Planetary Society. This will definitely answer this question, I imagine.

But I don't know. Number one, please. What about security? Are updates signed? Is communication encrypted? I know there's parity bits and hashes going on.

That's your answer? Number four, please. For six hours per day, now I'm not sure how many hours in Mars per day,

but how it's organized herself to work in different times per day, or it's in the same time every day? The rover always works in its own or in her own time zone. So if I say 10 o'clock, I mean 10 o'clock in the morning on Mars,

so about a few hours after the sun has risen. A Martian day, a sol, has about 24 hours and 39 something minutes. So it's very close to Earth Day, but there's some movement between Earth Days and Mars Days. So there's some offset that's worse and worse.

So for the first year or so, or maybe even two years, the JPL team that controlled the Mars rovers actually worked in shifts that corresponded to Martian days, and that led, of course, to night shifts, weekend shifts, all kinds of stuff. And they have since created this whole system where they can actually work just during the Earth Day,

and it results in about these 30 day stretches where the rover still gets new updates and orders every day. And then in between, there's always like a few days where the rover just sits there, doesn't have anything to do because the people need to sleep. And during that time,

it will just upload all the data that it has collected because of the slow uplink speed. Okay, two more, number five, and then number two, and then we close the session. Hi, great talk. On the two incidents when they had to switch the computer sides, do you know if they found out what the cause of that was?

Was it like cosmic rays frying something or the thermal expansion working? They did not. Like this is the thing, like there's also other incidents that I didn't talk about, like for example, the drill broke at some point. And in all of these cases, you can't really find out what's going on because to do that, you would have to go there, open the lid and just look in and see the things.

So like all they know is what their software can tell them. So their software can tell them it's crashing when it accesses this part of the memory. So don't hold it that way. Thanks.

Okay, and finally, number two. Hi, those two computers named A and B, are they of the same hardware design or are they different? They are, they're both the chip that I told you about earlier, the BAE 750. I'll be in the bar just outside the exit, the overflow bar. If you have any more questions, find me there.

Okay, with that, let's thank the speaker again. Thank you.