Dear viewer, these subtitles were generated by a machine via the service Trint and therefore are (very) buggy. If you are capable, please help us to create good quality subtitles: https://c3subtitles.de/talk/1058 Thanks!
[Music]
the next talk will be presented by break
the system and the talk is entitled the
Mars rover onboard computer the stage is
yours
[Applause]
this took entirely too
long here we
go
over the last 50 years a lot of missions
have tried to reach
Mars unfortunately Mars is hard and
landing on Mars is even harder so a lot
of these failed especially the
Landers funniest failure by the way is
mass climate Orbiter which uh launched
in 1999 and had one component that used
the imperial system and one component
that used the metric system for
measurements and when they tried to
exchange numbers the thing came way too
close to mass and burned up in the
atmosphere sjena was the first cute
little Rover about uh this big about the
size of a
skateboard and deployed in 1997 to the
Martian
surface spirit and opportunity the MS
exploration Rovers or mer landed in
2004 they're about this large the size
of a golf cart I want to
say and they're wildly
successful Spirit was active from 2004
to
2010 whereas opportunity was active for
a full 14 years until just this summer
when her solar panels got covered by a
dust storm and she went to
sleep she traveled for a distance of
45.6 KM completing a full marathon
before the end of her mission
also funniest Landing method ever with a
series of airbags just like bouncing
across the Martian
landscape today I want to introduce you
to mass science laboratory also known as
curiosity curiosity is a completely
autonomous vehicle and she has to be
because radio signals from Earth to Mars
travel about between 10 and say 40
minutes depending on the positions of
Earth and
Mars she has an atomic power source the
mmrtg so she's completely independent of
uh solar
power and curiosity's top speed is 100 m
per hour that's not a
typo one of her coolest instruments is
the freaking
laser which she uses to zap stones from
up to 7 m away that's the other end of
the stage here and uses a spectrometer
on the resulting gases to analyze
them the Rover has spent over 2,200
Souls on Mars a saw by the way is a
Martian day it takes about 24 hours and
40
minutes during that time she drove
almost 20 km and she's still going she
exceeded original Mission requirement by
a huge
margin so say hi
curiosity Curiosity has been in
different
configurations the cruise stage uh in
interplanetary stage navigating with
star maps and making course correction
Maneuvers to plan the perfect arrival on
Mars the the entry descent Landing
configuration with its combination of
guidance thrusters heat shields
supersonic parachutes retro Rockets uh
freaking sky crane uh that's a that's a
whole other talk right here like that's
not this talk
sadly and finally the Rover
configuration we see
now all of these configurations were
controlled by a single onboard computer
using sophisticated and very modular
software
so let's have a closer look at that
today quick introduction hi I'm Daniel
I'm a software engineer I'm an amateur
space
nerd and I learned a lot about space
hardware and software when trying to
build satellites for my
startup I work at KeepSafe creating
privacy protecting apps thanks for
paying my hotel room by the way and I
want to look at the Curiosity Mission
today through the lens of a person who
writes software and uses
Hardware a quick disclaimer by the way
I've used work of these people here uh
Emily lakawana Dr Katherine Weiss and Dr
Mark Mayon extensively in this talk and
I admire them greatly but I didn't have
the time to contact them so if you find
any in all mistakes in my in my
presentation they are mine and mine
alone and not these people's
mistakes let's talk about curiosity's
Hardware
this is the Bae red 750 a radiation
hardened version of the IBM power PC
750 created by ba BAE Systems it can be
clocked up to 200 MHz and it's used in
space a lot on curiosity this chip is
clocked at33
MHz this is the Chip And this is the
chip Within its board you see two
different boards here one is uh or board
sizes here uh as far as I know curiosity
uses the board size on the right on the
board we have 256 kiloby of
eprom we have 256 megabytes of RAM and
we have a whole lot of 2 gigs of flash
storage and we we have all of that
twice in case something fails the Rover
can switch from the the aside computer
to the bside computer they are both
completely
independent they also have a few
components that belong only to one side
of the computer for example and this is
going to be important later on the
navigation
cameras of course all of this is
radiation hardened and there are a lot
of techniques to radiation harden a chip
and board and this is another talk that
I would like to hold someday but today
is also not this talk so I'm going to
leave you this slide and tell you that
the last space shuttle used a 386
equivalent processor Hubble has been
upgraded to a
486 and the International Space Station
uses or the command computers are using
386 this is because radiation hardening
is hard and takes a lot of time until
the chips come out so space stuff is
always very very
slow
what kind of software can we run on this
Hardware turns out we're using VX Works
6.7 a simple realtime operating
system um this screenshot is seven but
it looks cool uh VX works is also used
in other NASA spacecraft various SpaceX
spacecraft the Boeing
787 the ca industrial robots of my
hometown
Augsburg and also the um the Toshiba
ebridge range of photo
copers and before you ask yes it can run
Java but in this case it
doesn't
the software that runs on Curiosity has
a long lineage there has been a
continuous code base that started in the
'90s when JPL wrote very code for the
pro their prototype Rovers and then part
of that was used in
sjena and then that got Rewritten and
whatever and was used in the mass
exploration Rovers and then it was
ported from C to C++ and refactored into
a component based architecture for MSL
AKA
curiosity and yes we are using
C++ not full C++ as you'd use on a
desktop
though uh one thing that's missing is
processes for example VX Works gives you
this concept of tasks but they're not as
independent as
processes they all use one big chunk of
memory
and it's up to you to make sure that
they don't like Stomp all over each
other we also don't have exceptions
templates IO stream multiple
inheritance and the only operator we're
overloading are the new and delete
operators that are used to allocate new
memory for objects or for things and to
deallocate that uh that memory and they
are overwritten by the custom memory
allocator that JPL wrote for
VX
works this is a pretty cool piece of
software because it guarantees graceful
access to these defined memory pools and
it has a well defined behavior for uh
out of memory conditions which is very
important as you can imagine it supports
multiple pools in different areas of the
ram so that's that's how you can
separate your notot processes it
provides Diagnostics there's a display
that shows you each new and deletable
operation and it can also give you a a
map of the Ram at on on request the
Rover can even downlink a map of its Ram
if
needed the same allocator is also used
for development on jpl's Unix machines
uh which is of course very helpful and
there's no garbage collection here which
would be too expensive instead you have
to be very very careful not to have any
memory leaks and these are usually
enforced by unit
[Applause]
tests if I if I add a title of something
by the way you should be you should be
able to Google it and watch it on
YouTube those talks are super
interesting one of the philosophies
behind the flight software is its
component-based
architecture functions are grouped into
components and a component exports an
interface and identifies the interfaces
that it needs to
function that means that you can replace
any and all components of the system as
long as they export the same interface
it also means you can work on your own
component without having to touch all
the other parts of the
system this is very helpful for
development and also for testing and and
extens
ability you'll also also notice that
that the components here are organized
in
layers the lower level components deal
with the hardware directly they turn
switches they move actuators they
prepare data for being uh s for being
sent back to Earth they also abstract
away redundancy components there's a lot
of redundant components where just a
thing is just twice in the Rover in case
one of them fails and the lower levels
will automatically fail over to a backup
component without the upper levels
having to know the intricate details of
what exactly went wrong so they will get
a notification that says something went
wrong and I'm I'm using the backup but
they don't don't have to know how the
backup is activated or how it's being
used further up we have components that
coordinate these very Atomic functions
into
actions and finally we have
components that group these actions into
whole activities like land on the
surface or take and analyze a surface
sample the land on the surface module by
the way is not on here because it was
deleted right after it was like right
after the spacecraft actually landed on
the
surface the most complex module in the
whole code base is the surface
navigation module which was in some form
also used for the mass exploration
Rovers already on my uh it was about 21%
of the full code base for MSL or
curiosity it's about 10% of the whole
code base is just the surface navigation
module pathf
finding moving that's that kind of stuff
there's about 100 individual or actually
over 100 individual source code modules
each with its own
owner
another philosophy is fault protection
mobility and science modules should not
have to worry about the spacecraft
safety instead the infrastructure in
addition to providing the platform to do
mobility and science must keep the
system safe and away from the dreaded
safe mode unsafe actions should not be
allowed and should be blocked the
pattern here is fult should be detected
and corrected as low in the layers as
possible finally we have just loads and
loads of testing we have unit tests we
have static
analysis and we have validation and
verification where we run code on the
various test beds like this complete
duplicate of curiosity down on the
ground and then there's test as you
lie once the software is
uploaded behaviors will be tried out and
tested on masss during the
[Applause]
mission so this is really cool
curiosity's flight software can be
remotely updated from Earth so Engineers
on the ground can change the software
and introduce new functions and
behaviors or tweak The Landing
code the mobility module by the way the
thing that allows the river to move her
arm and drive around was actually
developed while the spacecraft was
already in cruising stage towards Mars
it was not part of the launch load at
all it was uploaded once the Rover was
safely on the ground it's kind of like a
like the console you got for Christmas
you unpack it you plug it in and then it
just has to update for a week in this
case
how does the software update mechanism
work first the software update is
uploaded into the active computer's RAM
and run there and if the system behaves
erroneously it can reboot and load the
old version from its flash
storage then the Rover is going through
an extensive series of tests to see if
everything works as
expected if that passes the software
it's uploaded into the flash storage and
booted
from after another series of tests that
half of the computer is updated and the
process then gets repeated the same way
for the other side because we want to
have both computers ready for anything
in case we need to fail over to the
backup
computer R days start around 9 or 10 in
the morning local Martian time when
curiosity boots up from her dream mode
while in dream mode various fpgas
already flip on the heating system and
prepare everything so that the Rover is
warm once the main computer is booting
up and then she gets directions from the
ground and sets off for her
day she drives she collects data she
runs experiments she drills she zaps
it's pretty fun but it's very exhausting
the day ends shortly before Sundown when
the last satellite pass
occurs and that is used to upload the
last bits of data for the day before the
robber goes back to
sleep curiosity does have various
antennas and some of them could talk to
Earth directly but that would be at the
speed of about a
14.4k
modem using the relay satellites the
data rate goes up to about 500 abits per
day that's about 62 megabytes per
day so the data with the highest
priority gets sent pretty quickly and
the rest can just follow at some
point and then she needs to go to
sleep the
mmrtg uh the atomic power source
produces about 110 watts of energy the
Rover needs 45 to 70 Watts even while
sleeping 15 50 Watts while she's awake
and up to 500 watts while
driving so she spends most of her life
asleep and recharging her batteries
being awake for about 6 hours each
day lucky
her next I want to talk uh about how
actually how how the driving part
actually
works Rover drivers you use a system
called RSVP the Rover sequencing and
visualization program to plan out drives
this has been used and update updated
over the years and has been used for
previous Rover missions as
[Applause]
well in in in RSVP the terrain data that
Curiosity sends back is being
reconstructed in
3D and the human robot drivers can look
at it like fly through it um do local
simulations of activity
some sanity checking does this really
make sense in this context and then send
orders to the Rover these orders could
for example be blind
driving
the the easiest driving mode is blind
driving just point the Rover into any
directions and go for 10 m just
blindly this is dangerous and cumbersome
however because you can only drive as
far as you can see and even then like
the further away things you can see like
maybe there's things hidden by
perspective and then you have to wait
for more orders from the
ground uh even then you might into you
might run into
obstacles uh the thing though is that
this is way easier
computationally and way more predictable
so it was used a lot especially in the
beginning of the
mission then we have visual
odometry Odom omry is finding out how
far you've come by counting wheel
revolutions or other things thing is
odometers in wheels don't work here
because curiosity can't detect slippage
in real time and the metal wheels they
slip a lot on rock and
sand instead what she does is she takes
before and after pictures with her
stereoscopic cameras and Compares them
she will Auto Select various features in
the terrain that she finds interesting
and follow them along as she drives so
she will she will go and then see like a
super interesting Rock just like look at
the Rock for a while and see how far
she's
come this is all done on the Rover this
is not being sent back to Earth or
something so this is a completely
separate system and it works very very
well and then there's autonav where the
Rover drivers set a Target or a
destination location and the Rover tries
to find her way towards the target
completely autonomously while avoiding
the red nogo areas that are designated
by Ground
Control Curiosity has stereo cameras and
takes pictures with them features in the
image pair are correlated and
triangulated to get the distance of
these features the range data must
satisfy a series of tests before being
seen as correct like unclear and
misleading parts of the image like
pixels that are only in one of the
images or parts of the Rob of the Rover
are being ignored
using these stereoscopic images
curiosity will then create a geometric
model of the surrounding terrain and
subdivide it into grid cells of the
resolution of a r wheel she will then
rate each of these cells according to
its traversability how much she'd like
to drive there taking into account
slopes steps roughness excessive tilt
and other environmental dangers like
things that just stick out too much from
the ground or would scratch her her
belly pan these obstacles are expanded
by the radius of the Rover so she will
so that her Center will not touch uh
these things and she will stay far
enough away from
them this grided traversability map uh
is used for each step of the
navigation so she will then project a
number of paths onto the map and
evaluate evaluate them is this safe is
this will this get me closer to my
destination she then chooses the safest
path that gets her closer to the goal
and drives a short distance on that path
and after each step the navigation
process starts again until the goal is
reached or no safe path is found or the
Rover is commanded to
stop to find her way C curiosity uses a
version of AAR search optimized for
driving tiny increments at a time
between map updates because of this
optimization the the algorithm is much
faster and uh more efficient than than
regular a star search and therefore more
better to use on the
Rover autona is still too slow to use
all the time but as the mission went on
the Rover did more and more autonomous
driving because people learned how to
trust the Rover and give it the correct
um directions and also the software was
improved over and
over this is curiosity looking back at
her very first drive she surprised the
engineers here because she avoided a
very very tiny Rock you can see it at
the top where this little corner so so
so she went straight and then there was
this rock and she was like oh this is
very dangerous I'm going to
around
um this and you've seen this before this
is basically exactly that drive being
replayed in higher speed in RSVP and
look at the the head moving around and
also the other cameras you can see them
but they also take pictures and this is
Visual odometry and autona
working all right what other features do
we have
have we have the Mali camera the Rover
has an arm and at the end of that arm
there's a camera turns out it can
actually take pictures of itself for
example of the belly pan or of the
wheels directly after landing to see if
they survived the landing
correctly using that arm she can also
take larger pan
panoramas by taking multiple pictures
and therefore cover her entire
body this is what the result looks like
this is curiosity's very first selfie if
you stitch them together they look
pretty cool like they are amazing
pictures and I love to look at them the
arm is usually photoshopped out because
you will only see half the arm all the
time because it doesn't fit into the
Panorama let me tell you about what
happened on Soul
200 suddenly the Rover couldn't save
data anymore transmitting images live
back to the ground worked fine but
saving them for later upload would
fail there were also rebooted multiple
times without apparent
[Applause]
reasoning in the end the engineering
team decided to switch to the B-side
computer from there they could assess
the damage and it turns out that the
aside computer has a fault A Fault in
its flash you can't really tell what it
is but they have now disabled half of
the flash in the acience computer and
this seems to work fine Curiosity has
been uh was on the B side computer from
then on that though and this is more
work than you can imagine because as I
told you earlier there are various
cameras especially the navigation
cameras that are directly connected to
each of the boards so if you switch from
the a side to the B side turns out that
the cameras they're slightly offset and
this gives you different
angles and these have to be programmed
into the software of course or it will
give you faulty data and then it also
turned out that during the heat in the
marsh and summer these cameras warp
slightly and this warp like warps the
images a little bit and that makes that
that makes uh the the triangulation of
the stereoscopic images very hard so
they also had to write code while on
Earth just by looking at the pictures
how to how to calculate out this warping
it's a whole
thing exactly one year after landing on
Mars Curiosity celebrated her first
birthday this is Sam an instrument that
includes a very exactly controlled seeve
that that can SE with like these
perfectly programmable
vibrations at the first birthday after
landing it was programmed to play the
following
[Applause]
vibrations
imagine the tiny rover Curiosity sitting
in the Martian desert just like singing
a birthday song for
herself this only happened once by the
way uh you hear sometimes on the
internet the rumor that this happens
every year this happens exactly once and
the first birthday it has been six
birthdays since uh five birth birthdays
since so and and those were all just
regular working
[Applause]
days after some time on Mars Engineers
noted that the wheels were getting torn
up way quicker than
expected the wheels they are only about
a millimeter thick and they're made from
machined
aluminium what happened was that
Curiosity ended up in terrain that
wasn't experienced by opio or
soda um normally these small rocks as
you see on this picture for example
um when the Rover drives over them they
get just pushed to the side or pushed
into the
ground but this clearly is not happening
here the terrain is ripping deep gashes
and holes into the
wheels so JPL Engineers on the ground
like Amanda Stephy here constructed
various ground test beds that simulated
different new ground conditions and
tested the wheels to failure as they say
so that they know exactly which types of
terrain are the most dangerous to drive
on
and it turns out that if you have rocks
that are cemented into the ground and
they have very sharp edges at the top
and they don't really move aside because
they're completely fixed in place then
the back wheels will force the front
wheels that can't really move because of
the sharp rocks onto those rocks with
the huge torque that the Rover has and
this leads to these
punctures so the driving software was
changed to allow the wheels to move at
different turn rates so to Avo avoid
this this pushing onto the sharp rocks
thing also the r has been very carefully
steered out of their Rocky areas and is
now back on the more Sandy areas that
she
prefers on so
2172 that is September 15 of this
year another comp computer problem was
detected curiosity was completely
healthy but could not access parts of
her memory where her where she stores
data for later
uploads this is clearly a Ser serious
problem but it doesn't endanger the
Rover safety you'll notice no reboots
this
time for now the Rob was switched back
to the the a side and is merily doing
science again while engineers in the
background are trying to diagnose the
problem and you remember the aite is
safe until and as long as it doesn't
address the faulty part of the
flash what does the future hold for
curiosity so C curiosity's primary
mission was from 2012 to
2014 and she's still going strong in
2018 so that's pretty cool turns out um
a few months ago at a conference in too
NASA and JPL engineers said that there
were about 10 km left in the wheels if
conditions are about the
same we can also expect the mmrtg power
source to last for another 10 years or
so so if nothing happens she will live
on for a
while she has more or less already
reached her goal this is Mount sharp AKA
aolis mons the central peak in Gale
crater and gaale crater uh was once
filled with water and because there's
lots of lots of different sediments at
the bottom of Gale crater uh there's a
boatload of interesting science to be
done here and after driving for 20
km curiosity is now in the Foothills or
as jpa calls them the butts of sharp
Mount sharp so there's lots of cool
stuff to look
at in about 2 years we should see the
arrival of Mars
2020 sorry curiosity's sister rovera it
has the same chassis and the same body
uh same software morus uh slightly
updated wheels and a completely new set
of science
instruments it will also be the very
first Rover to include a microphone so
that we can hear what the surface of
mass sounds
like and maybe just maybe
if we listen close enough we can hear a
tiny far away Birthday
[Applause]
song
okay I have tears in my eyes actually
uh we have some times left for uh for
the Q&A so I I guess we have lots of
questions there are microphones in the
uh uh in here in the aisle exactly and
we start with a microphone 2 I thanks
for talk uh two small questions F first
uh why you call uh the Rover she and the
second one um can you say something more
about error handling
uh does the software have has have
different kinds of Errors which are
treated
differently right good question so the
first question was why is it a she and
it turns out that jpn at Nasa call all
their Rovers by female pronouns so I
just did the same because just seems
nice and the second question was uh
about error handling and there are there
is a whole pattern that is being
discussed by Katherine Weiss for example
one of the software engineers and
architects of the software um where
there's like a whole error detection
patterns and they do it all all in the
same way they have like three status
codes green yellow and red uh where
green is just everything is nominal
yellow is something happened but the
lower level component was able to fix it
by I don't know like moving like rooting
something around or
switching to a backup uh component or
whatever and then there's red where the
where the component just says like okay
something something way like higher up
in the chain has to deal with this so
this way they can like bubble up these
error status
codes thanks okay we have already more
questions than we can take I'm afraid
but we continue with number five please
okay thanks for your nice talk and I
want to know how does the image
recognition works um for the past
finding does the Rover uh compare the
images or does it have a kind of lighter
sensor or how does it work um it is all
just regular images so there's no liar
uh I'm looking at my backup slides just
now because I have I have a lot on this
um so basically I'm I'm find I'll find
it later uh basically it's just regular
image recognition there's an algorithm
in there called rockster which can
recognize Rock and the s like the ones
you see here actually uh by their
Contours and also because of if because
you have stereoscopic images you can
calculate like a 3D map of the
terrain number four please hello I've
heard that some components or this Echo
um are controlled by Ross Robert
operating system is that right um robot
something rating system Ross Robot
Operating
System it might be true I have to be
honest I haven't heard of that term
before uh
so that's the edge of my knowledge
apparently number eight I guess um so
when the autonomous driving software was
designed would be probably around 2005
live
um today we have autonomous cars being
designed for Pure uh
cameras is Naza on a dead end a
different path will they integrate what
is being designed today for cars here or
is this uh going to be completely in a
different frame which is not not
compatible are they going to augment or
rewrite from scratch so I don't I don't
have any affiliation with NASA but if
you want to hire me I'm here uh I don't
have any affiliation with NASA so I
can't speak for them to me it seems like
it's very two very different use
scenarios where the one where like
autonomous cars are like they use all
kinds of like detection specific to
roads whereas the Rover has to navigate
a rocky terrain um but I imagine like
because there's a lot of uh research
going on that and this research is
public that they might take the best
bits of that research and integrate them
like these people are incredibly
smart okay we take one question from the
number
six um is there a specific reason why
there's no uh radar or lead sensor
included because I think that would uh
make getting 3D images more easy
I do not know of a reason it might be
because it's just uh like the the whole
design is basically from the early 2000s
maybe Linda wasn't wasn't good enough
back then um what I can recommend to you
by the way is this incredibly good book
The how curiosity does in its job by
Emily lakawana from the planetary
Society like this will definitely answer
this question I I I imagine but I don't
know number one please what about
security are updates signed is the
communication uh
encrypted I know there's parity bits and
hashes going on
well that's your
answer number four
please uh hi you said that it worked uh
for six hours per day now I'm not sure
how many hours in Mars per day but uh
how it's organized herself to work in
different times per day or it's in the
same time every day the Rover all always
works in its own or in her own time zone
so if I say 10:00 I mean 10:00 in the
morning in on Mars so about a few hours
after the sun has risen uh a Martian day
a soul has about 24 hours and 39
something minutes so it's very close to
a Earth Day um but there is some some
movement between Earth days and Mars
days so they they there's some offset
that's like worse and worse so for the
first year or so or maybe even two years
the JPL team that controlled the Mars
rovers actually worked in shifts that
corresponded to Martian days and that
led of course to night shifts weekend
shifts all kinds of stuff um and they
have since created this whole system
where they they can actually work just
during the Earth day and it results in
about these 30day stretches where the
rover still gets new updates and orders
every day and then in between there's
always like a few days where just the
Rover just sits there doesn't have
anything to do because the people need
to sleep and during that time it will
just upload all the data that it has
collected because of the slow Uplink
speed
okay two more number five and then
number two and then we close the session
hi uh great talk um on the two incidents
when they had to switch the computer
sides uh do you know if they found out
what the cause of that was is it like
cosmic rays frying something or the
thermal expansion working they did not
like this is the thing like um there's
also other incidents that I didn't talk
about like for example the drill broke
at some point and in all of these cases
you can't really find out what's going
on because to do that you you would have
to go there open the open the lid and
just look in and see the things so like
all they know is what their software can
tell them so their software can tell
them it's crashing when it accesses this
part of the memory so don't hold it that
way okay thanks okay and finally number
two yeah hi hi um those two computers
named A and B on they of the same
Hardware design or are they different
okay they are they are both both the the
chip that I that I told you about
earlier the Bae
750 I'll be I'll be in the in the bar
just outside the exit the Overflow bar
if you have any more questions find me
there okay with that let's thank the
speaker
[Applause]
again
[Music]