*preroll music*
Camille Gay: Hello, everyone. My name is Camille. I am a researcher at Toyota Motor Corporation,
and this is a presentation about RAMN, a platform that we developed to make
education and research in the automotive systems more accessible. The automotive
industry can be inaccessible to many people because automotive projects involve
prohibitive costs and be tied to NDAs that everybody is going to sign. What we want
to propose with this project is an inexpensive test bed to study and research
automotive systems, which is both open source and developed with open source
tools so that at least anyone can get access to a good alternative for education
and research. The main focus of this test bed is security, but you will see that the
usage of the test bed is not limited to security, and I will keep the security
talk mostly for the end of the presentation. I will start by giving a
short introduction about automotive systems. Then I will present the design
details of the test bed besides demonstrations and concrete details about
its hardware and software. As an example of how the test bed can be used, I will
spend some time experimenting with cruise control and by that I mean I will go
through the whole development process, starting from evaluating the differential
equations of a simple mechanical model. I will experiment with various control
strategies, implement them in C and make measurements in a driving simulator using
only data from the CAN bus. And I will do all that using only open source tools.
This is to demonstrate how the test bed can be used, but also to have a concrete
project that I can use as a reference to explain after what concretely would have
been different if we were experimenting with a real electronic control unit. I
will also explain how we can get close to automotive hardware and software without
signing NDAs. So the second part of the talk is mainly here to give you more
information about the automotive industry in case you are not familiar with it.
Before I start, let me just clarify that this is not an advertisement. We are not
selling anything we present here and do not profit financially from it. We're
simply showing design files whithout permising licenses and without
royalties. OK, so first, let me give you a very quick introduction about automotive
systems. You can see your car as a collection of systems divided in four
different domains, the powertrain domain, which includes the engine and the
transmission. The chassis domain, which includes the steering column and
suspensions. The body domain, which includes the lights, the doors and
heating, and the infotainment domain, which includes navigation and
connectivity. Many of the different systems that can be found in the car are
controlled by electronic control units or ECUs for short. There are many kinds of
ECUs in the car. Sometimes hundreds of them. And usually it's a lot hard to
understand. They have a limited number of inputs, generally data from sensors and
actuators. And they have a limited number of outputs generally to control actuators.
So, for example, an airbag ECU may use an accelerometer as its input and an airbag
trigger as its output. The role of the ECU would be to use data from the
accelerometer to detect a shock and output a signal as actuator to detonate an airbag
when the shock is detected. It is very common for ECUs to read data from other
ECUs. Most of the time ECUs will need to share data with other ECUs on the same
domain. In the case of [*unclear*], for example the transmission control unit
gets input from the engine ECU to determine the correct gear. If the data is
critical that connection may even be redundant. ECUs may also need to
communicate with ECUs on a different domain. For example the brake system,
usually in the chassis domain, willl need to communicate its data to store them, usually
in the body domain. Most of the time the technology that is used for communication
is CAN. And CAN technology uses a bus topology, which means a CAN message will
be received by all ECUs on the same CAN bus, there is no authentication or
encryption at the link layer. So any message can be sent by any ECU. And
security features need to be implemented at higher layers. A standard CAN frame
consist mainly of an arbitration ID of 11 bits and a payload of 8 bytes. CAN-FD is a
recent evolution of CAN where the payload size may be extended up to 64 bytes. For
an ECU network manufacturers will assign a meaning to each arbitration ID and each
bit in the payload, so the file that determines the traffic on the CAN bus is
often referred to as a DBC file. For example, assuming a lamp controller and
two lamps on the CAN bus. The manufacturer may decide that ID 123 is used by the lamp
controller to communicate the command of both lamps, that ID 124 is used by the
left lamp to give feedback about the status and that ID 125 is used by the
right lamp to give feedback about its status. Each of those messages will be
broadcasted periodically by the assigned ECU on the CAN bus and will serve as a
basis for most data exchange between ECUs. So that's it for the introduction, there
are many reasons why people would be interested in automotive systems and ECU
networks. The opportunity that gets by far most attention from the media is
vulnerability research. But there are also other reasons. For example, owners: They
want to check their car's compliance with regulations such as emissions regulations
and privacy regulations. For example, GDPR. Other owners may want to exercise
their rights to repair as they are guaranteed by the country they live in.
And finally, some owners may want to experiment and innovative DIY features or
simply satisfy their curiosity and educate others. And while those may be valid
reasons to experiment with the car, manufacturers are typically against people
tinkering with a car because they may be worried about their intellectual property
being stolen, about vulnerabilities being exploited or people hurting themselves and
others while tinkering. And what probably suffers the most from this delicate
situation is education and research in automotive security, because people can
not easily get access to safe equipment or access the information that they would
need. In the long term, this may mean that manufacturers will have less security
technologies to choose from to secure their cars and that less talents would be
available to develop and evaluate them. As the development, of course, involves many
people from many companies, so it is important to make sure that everyone
involved is competent in automotive security. And some people are pushing for
more open sourcing cars and who knows, maybe one day the car will be 100%
open source, but if it happens, it is going to take a long time. And
manufacturers themselves do not have access to a hundred percent of the source
code of the cars they make because ECUs contain intellectual property from other
companies. So this is mostly a political topic. So there is not much we can
contribute to as researchers. However what we can contribute technically right now is
to try the other way around and use what is publicly available to make it
accessible, to learn and research automotive systems. And this is what we
try to do with RAMN, which is the topic of this presentation. The objective is to
provide a platform for research that is: Open, and by that we mean it should be
easy to modify the source code, and reprogram the ECUs; Accessible, and by
that we mean inexpensive, small and requiring no prior skills in automotive
systems; Safe - with no risk of accidents or legal repercussions; and motivating,
something that you can interact with so that you get the same kind of experience
as you do when you experiment with a real car. So already some solutions are
available if you want to experiment with an ECU network. Besides, of course, a real
car, the first one is making your own test bed from real ECUs so we can see many
hackers sharing the test bed at security conferences. And usually if you see
something like this, you stop and you immediately get interested. So it is a
nice way to motivate people to learn. Unfortunately, those are not easy to
reprogram because manufacturers do not share information about the ECUs and they
require a lot of skills to build. So it's not accessible to everyone. Another option
is to use development boards such as Arduino, and that is what you can see
mostly on academic papers. They have the advantage of being reproducible and you
can modify the source code as you want. So they can be used in many cases for
research. But they lack many safety features offered on an actual ECU hardware
and software. Even if you are able to simulate a CAN bus, you don't get the same
level of interaction as you do with a real car. So it's not something that motivates
people and makes them want to learn more. And the third option is to use a
professional test bed such as Pasta - another work from our team. This is a good
option because you get access to the source code and you can reprogram the ECUs
and the CAN bus network is already simulating a full car. So the groundwork
is already done. So major drawback are that it is very expensive. So it is not
accessible to everyone. So there are some options to study and research automotive
systems, but none of them seem to be both accessible and motivating at the same
time. So many people don't even think of running automotive systems because it
never occurred to them that they could like it. And in comparison with other
industries, you have so many ways to get started, if you want to learn about Linux,
you can start with a Raspberry Pi. If you want to learn about electronics you can
start with an Arduino and so on. So we wanted something that would give a similar
experience, but for automotive systems. So we noticed that most of the test beds that
people are using to experiment with ECUs are made of 4 ECUs. So the ECUs are often
communicating with each other using a CAN bus. So we tried to fit all that in a PCB
the size of a credit card. And we named that PCB RAMN. It features four ECUs
connected over a common CAN bus or CAN-FD bus, which is accessible from outside by a
terminal block. One of the ECUs is also connected to USB, which is also the power
supply. PIN-sockets can be used to connect sensors, actuators and additional
hardware and the board features many probes to easily access important electric
signals. The 4 ECUs simulate a CAN network with messages identical to Pasta. The name
RAMN is obviously a reference to Pasta as this is a cheap alternative, mostly aimed
at students. In real cars CAN messages typically have different payload sizes,
but by default we operate with maximum payload size, to demonstrate heavy
traffic, so the basic format is like this: Arbitration ID, 2 bytes for the data,
2 bytes for a counter and 4 bytes of additional data, random data used as a
placeholder for additional data such as checksum or MAC. You can easily modify the
CAN bus's arbitration ID end format. And here we are assuming a full by-wire
vehicle, which means all physical functions of a car are accessible to the
CAN bus, which is usually not the case on current cars. The block diagram of RAMN
looks like this. And as I explained earlier, all ECUs are periodically
exchanging messages on the CAN bus. If you connect a CAN adapter and have a look at
the traffic, it will typically look like this. So the board itself is enough to
simulate an ECU network, but it does not look very motivating. What we wanted on
top of this was some sensors and actuators to make it more interactive. So we created
4 expansion boards for sensors and actuators. To simulate the infotainment
domain, we simply use a screen. For the body domain, we use an engine key and some
LEDs. For the chassis domain we mainly use a slide switch to simulate a side brake
and a rotating potentiometer to simulate the steering wheel. And for the power
train domain we use slide potentiometers for the brake and accelerator and a
joystick for the shift lever. The EC connected to USB implements a standard CAN
or CAN-FD interface, either of a standard serial port using Slcan or over a native
interface on Linux thanks to the counterlight firmware projects. If you
connect the board to a USB port on the computer, it should be recognized as a USB
to CAN adapter. So it is not necessary to own an external CAN adapter to get
started. This is a demo of what it looks like to use RAMN. Just connect it over
USB, if you use Linux, you can get it to be recognized as a standard CAN network
interface. So it will show up in ifconfig. Then you can use various tools available
on Linux to observe the traffic, for example CAN sniffer. Here you can see the
traffic explained earlier, the CAN IDs on the left and the payload on the right. So
with this, we can simulate a ECU network with sensors and actuators, which is
enough for basic interactions. But it still doesn't feel like you are actually
experimenting with a car. Ideally the ECUs should be performing realistic ECU
functions, not just lighting up LEDs based on some switches and potentiometers. And
for this we thought of a connection in a closed loop with an open source driving
simulator which is an affordable and safe solution. To feel like you are driving the ECU network.
Fortunately, there is a great open source driving simulator for autonomous driving
research. It is called CARLA. It features a Python API. So it is very easy to
interact with it and it also comes with an example self-driving algorithm. So you can
immediately start experimenting with your virtual self-driving car. So we wrote some
scripts so that most sensors values for example speed and altitude would be
simulated on the computer running CARLA. Then broadcasted on the CAN bus. On the
other side we made it so that all controls of CARLA such as throttle, steering and
brakes will be decided by the ECU network. And this is what you could refer to as
HILs or Hardware-In-The-Loop simulation in the automotive industry, RAMN is not as
advanced as professional tools, but at least it is accessible. So to achieve
manual control, it is not complicated with CARLA. On the manual control example
provided with the API there is a while true game loop that with events of the
keyboard, applies controls, then updates the physics simulation. So CARLA does not
simulate the CAN bus by default so we created a Python class to easily interact
with the CAN bus using the CAN message specifications of RAMN. To integrate it
with CARLA we just need to replace keyboard controls with actuator control
data read from the CAN bus. To close the loop we broadcast sensor data using CAN
messages based on data retrieved from the Python API of the physics simulator. And
with this we are able to control the car manually, the potentiometers on the board.
Here I gently press accelerator, release the handbrake, and I can control the car
using the steering wheel on the expansion board.
*Video stops*
So manual control is nice, but
automatic control is better because ultimately we want to focus on working on
the CAN bus. So on the original CARLA project, there was also an example script
for automatic control. Again there is a while true loop where the code basically
simulates the physics, lets the self- driving AI make a decision, then applies
the controls to the physics simulation. To integrate RAMN in this certain algorithm,
again, we need to replace the apply control part with the controls from the
ECU network. We also need to send CAN messages with sensor data retrieved from
the Python API of the physics simulation. That is identical to having manual
control. What we need to do more is also send a message to broadcast the status of
the AI to the ECU Network, so that the ECU network knows what controls the AI
algorithm is requesting. So periodically the AI would broadcast its status by
sending messages over USB, which are converted into CAN messages by the ECU
input to USB. All ECUs on the network will receive those messages and will decide the
actual controls of the car based on their own algorithm. For example, the power
train ECU may decide to apply brakes depending on the highest value between the
AI brakes and the brake potentiometer. All ECUs will receive the CAN message and
apply the control, some ECUs may filter that message if they do not need it. Some
ECUs like the body ECU may process it and take action. For example, if the brakes
are engaged, the body ECU will light up the stop lamp on the expansion board.
Finally, the ECU connected to USB will forward the brake controls to the
simulator that will apply the brakes in the physics simulation. So this is what it
actually looks like, all I need to do again is release the handbrake and the car
will drive itself. The car is controlled by the ECU network, so when the car stops
in the simulation, it is because the controls were applied by the power train
ECU. You can see that the stop lamp lights up because the body ECU also received and
processed the break CAN message. Since the ECU is in charge of the controls, I can
force the car to stop any time by forcing brakes on the power train ECU potentiometer.
So if you connect an external CAN adapter to the CAN bus,
you'll be able to send and receive messages. So using RAMN you can experiment
with the CAN bus of the self-driving virtual car. When you want to reprogram an
ECU in the real world, you have two options. Either you interact with the
hardware bootloader of the ECUs microcontroller, which depends on the
microcontrollers model and manufacturer, or you interact with diagnostics and
calibration software, which depends on the car model and manufacturer. Diagnostic and
calibration are often done on the CAN bus, but other options may be available.
Protocols are defined by standard documents. You often hear about UDS and
OBD-II which both rely on the same transport layer ISO-TP. But there are also
other protocols, such as Keyword Protocol 2000 or XCP. All those protocols can often
also be implemented over other layers. For the hardware bootloaders it depends on the
microcontroller manufacturer. For example for an STM 32 microcontroller you can
reprogram the firmware by interacting over CAN according to the application note
3154, over USB according to the application note 3156. Or using other
protocols defined in other application notes. In the case of RAMN, we use the
hardware bootloader to reprogram the ECUs and you can also use a UDS protocol over
CAN. And in the future we may consider adding XCP and KWD 2000. How to reprogram
an ECU using calibration and diagnostic software is a topic that is already
heavily discussed at automotive security talks, I will not spend time on this
topic. You can find the definition of the standards online. Usually you need to pay
for them. But you can also find summaries for free on some websites. For example,
Wikipedia. What is more interesting to discuss here is the hardware bootloader.
The various ways to force a microcontroller into bootloader mode are
described in the application note 2606. The format of CAN messages we need to send
to interact with the bootloader is defined in the application note 3154. And it's not
complicated. The format is simply common byte plus argument. All that within the
same CAN message payload. So we wrote some script to make it easier to interact with
the bootloader. So here I am showing the script that we use to interact with the
bootloader. First thing I do is retrieve all information from the bootloader,
including the version, the supported commands and the Chip ID. I then use a
read memory command to dump all known sections of memories. This includes the
ECU firmware. So it can be a good start to start experimenting with reverse
engineering. We can also activate memory without protection and see what happens
when you try to dump the memory again. And in this case is not allowing memory dumps
anymore. You can remove the memory protection, which will wipe out the
memory, and after you do that, you can use the bootloader to reprogram the ECUs. For
the hardware we also designed additional expansion boards with various features.
First one is an expansion to connect a JTAG debugger. Second one is about to add
extra QuadSPI Memories, which is why you see packages such as EEPROM, FRAM, NAND,
SRAM and so on. The third one is a board to add a trusted platform module. And the
last one is an interface to easily connect a chip whisperer. All those expansions are
compatible with each other, so you can stack them and create a very advanced ECU
network with all ECUs between a TPM and one gigabit of memory. And it looks better
when the expansions are stacked on top of each other. Since we would like to use
RAMN for education, we try to vary the type of electrical signals using the
expansions. And we added many external probes to easily connect external tools. You
can use one of the many excellent probes to connect an oscilloscope and have a look
at analog signals. For example, here's the brake potentiometer. Or connect a logic
analyzer and have a look at digital signals, for example, to analyze SPI
communication between the ECU and the screen. For the hardware design we kept it
simple using only two layers and keeping all components on the same side. We
designed the hardware with flash tolerances which should be easy to produce
with most PCB fabrication services. We use components with large size. So if you have
some skills in soldering, you should be able to solder one yourself. We designed
the hardware using KiCAD, which is an open source tool for designing hardware. You
can easily modify the schematics, layout and even generate nice 3D views. So
firmware is designed using STM32CubeIDE, it is accessible to beginners, and you can
easily reconfigure peripherals and clocks from the graphical interface. You will
even get some statistics, such as the estimated power consumption. But of
course, you do not need to use the graphical interface and libraries if you
would rather program everything yourself by interacting directly with registers. We
use free RTOS as a Real-Time operating system. The basic software of ECUs has
several tasks running in parallel, one for receiving CAN messages, one for sending
CAN messages and several periodic tasks to take care of the ECU control loops. Those
tasks receive data from different interrupt services within, using queues and
DMA memory overwrites. The tasks can also exchange data between each other using
queues or memory overwrites. Again, to keep the project accessible to beginners,
we did the OS configuration using the graphical interface where you can see all
tasks and their configuration, for example the priority. You can add or remove
interrupts. And you can also configure the various queues in memory. There is still a
lot of memory left even though the memory controller is less performant. So you can
easily add your own application on top of this. That's all for the hardware and
software section, you can find more details on GitHub. I would like to move to
the next section and show you a usage example. There are usually two budgets for
people working in automotive security Either they are automotive engineers who
learn about security or they are security engineers who wrote about automotive
systems. Since this is a security conference and I assume most people do not
need me to explain how the platform can be used to study and research basic security
topics, I will focus on the automotive side. So diagnostics and calibration topic
is already covered by many security talks. I will not spend time on this. So
I will spend time on something that is not often mentioned at security
conferences: Control algorithms and safety critical hardware and software. And for
this, I would like to demonstrate the design of a PID controller for cruise
control as an example. I will show you how being knowledgeable in control systems may
be relevant to security engineers, and how many of the activities that are done by
engineers in the automotive industry can also be simulated using open source tools,
including RAMN. Once we have an implementation in C that works, I would
then use it as a reference to talk about safety critical systems and the
differences with real ECU hardware and software. So let's get started. Cruise
control is very simple: When a human is controlling the throttle with the
accelerator pedal depending on the skill the car may have an uneven speed. What we
want is an ECU that optimizes the control of the throttle so that we can maintain a
steady speed. An automatic car can be very easy to model. If you press the
accelerator pedal, which opens the throttle, you will get speed out of your
car. But the relationship between speed and throttle is not as simple as a
multiplication. No, we have to follow the laws of dynamics. In this case that's the
sum of forces on the car is equal to its mass times its acceleration. We can
consider that there is a force pushing the car that is proportional to the throttle,
and that there is an opposing force proportional to the speed due to friction.
When we solve this diversal equation, what we expect to see is that the speed follows an
exponential curve. And a simple way to control the speed that may come to your
mind is open loop control. You make some measurements on the flat road of the
relationship between throttle and maximum speed, and you keep in a lookup table.
When the user asks the ECU to reach a certain speed, the ECU can use the lookup
table to find what throttle controls should be applied based on past
measurements. And this may work in some situations, but according to the laws of
dynamics, as soon as we reach an upward slope, the car will lose speed because of
gravity. So at least that is what we expect, but we should verify that on the
CAN bus. This is something we can use RAMN for. Here again, using an external CAN
adapter connected to a second PC. On that PC I simply receive data from the physical
CAN bus. For the rest of the section I will only be modifying the firmware on the
power train ECU. I will not change the simulator, I will not even reboot it.
So in the simulator, I drove around the city to try to find a nice place to experiment.
More precisely, I looked for a place with flat road followed by an upward slope.
Then I programmed the power train ECU to a apply a constant throttle, which is only
one line of code. And after reprogramming the ECU I let the car drive straight and I
recorded data from the CAN bus. I used an open source tool for CAN bus analysis
called Bus Master. Bus Master allows you to load DBC files or to input manually the
format of your CAN frames. Here I simply told Bus Master what were the CAN
arbitration IDs of the throttle control message and the speed sensor message. And
what was the format of the payload. Once I do that, I can plot the evolution of the
throttle and speed over time. And the results we get are exactly what we
expected from our differential equations. The speed of the vehicle is following an
exponential curve and as soon as we reach the upward slope, we start loosing speed
because the throttle is constant. There are some noise and non-linearities at low
speed but overall it seems our model of the car is correct. We are approaching the
problem correctly. What we can see here is that it takes about 40 seconds for one
test-drive. And 40 seconds is already too long. So before testing the ECU in the
driving simulator, we want to use the software that can simulate differential
equations so that we can see the impact of the cruise control algorithm directly,
without having to wait for 40 seconds. Most of the time this is done using a
professional tool such as Matlab and Simulink. But here we will use open source
tool Scilab. It has a graphical interface where we can connect inputs and outputs
from a block diagram. Differential equations are a bit hard to deal with
because the relationship between inputs and outputs is complicated. What you
typically do in control theory is use a Laplace transform, which will change the
variable called t to a complex number called s. This may be complicated with a
contradictory background, but you just need to know that a differentiation is
equivalent to a multiplication by s and that integration is equivalent to division
by s. In our system it is easier to model the Laplace transform because we now have
a simple relationship between throttle and speed. Speed equals transfer function of
car times throttle. And based on the measurements from the CAN bus, we can
evaluate the transfer function of the car to be equal to approximately 1 / (1 + 4s).
We can simulate the car in Scilab by using a block function which has the exact
same transfer function. Using Scilab, we can now test various scenarios and get the
results immediately. Here I am testing the scenario in which we start from zero
speed, apply a constant throttle and after 20 seconds, we add a new force - gravity -
which corresponds to the slope. And this is what we call here the disturbance. And
with Scilab simulation we can verify we get waveforms similar to our measurements
on the CAN bus. With a constant throttle, the speed follows an exponential curve
that is close to maximum speed after around 14 seconds. And as soon as there is
a disturbance: the gravity here, showing green, we can check that the car looses
speed because there is no reaction from the throttle. So to fix that the solution
is obvious: The ECUs need to have feedback. We need the ECU to make use of
the speed sensor data that it can find on the CAN bus so that it can adapt the
throttle to the actual speed of the vehicle. So first solution that may come
to mind to software engineers is Bang-Bang Control. Bang-Bang Control is quite
simple. You measure the speed and if it is above a certain threshold, you stop
applying the throttle. If it goes below a certain threshold, you apply the throttle
again. This is extremely easy to implement in C on the ECU. And once we reprogram the
ECU on RAMN, we can make measurements on the CAN bus and verify that this time we
are not loosing speed anymore when we reach a slope. But as you can see, there
are some oscillations. And as you can imagine, the oscillations are not
something very nice for passengers. Apparently people driving like this is the
reason cruise control was invented. I do not know if that story is true, but I can
believe it. So Bang-Bang Control made a good start, but it is not good enough for
cruise control. And the most famous algorithm used in control theory is the
PID controller. You can find a lot of resources online. The PID controller is
one of the control mechanism used in software of the self-driving AI of CARLA.
In the PID controller you measure the difference between the target speed and
the current speed. You call that different error and you can control the throttle
using the sum of 3 control blocks. The error multiplied by Kₚ. The integral of
the error multiplied by Kᵢ and the derivative of the error multiplied by Kd.
Kₚ, Kᵢ and Kd are constant called gains. And you need to have a very fine tuning of
those gains to achieve optimal control. Here I can simulate the PID controller
using Scilab with different blocks. Remember that the division by s is an
integration, and the multiplication by s is a derivation. Thanks to the simulation
I am able to try many values without having to actually drive the car. And when
we are able to find correct values for the PID controller, we get this. The ECU is
able to reach a target quite quickly without oscillations and without
overshooting the maximum speed. And when there is a disturbance so gravity, it will
dynamically adapt the controls of the throttle so that the target speed is
maintained. So this is good, because this is what we want for cruise control. But is
a gain of only one control block is correct, the speed of the vehicle may look
like something totally different, potentially dangerous. And this is why the
integrity of calibration data is important not only from a safety point of view, but
also from a security point of view. Because an attacker should not be able to
make an ECU have a dangerous behavior with only one small change. The last thing I
need to explain is how to implement these algorithms in C, and this is not obvious
because we're dealing with integration and derivations, which are not possible for
digital functions. So there are many ways to implement this in a digital PID
controller in C. I will just explain two approaches. The first approach is to stay
in the time domain and approximate the derivation by the difference between two
successive errors divided by the sampling time. And we can approximate the integral
operation with a Riemann sum which is a running sum of errors so far multiplied by
the sampling time. This may look a bit intimidating, but when you look at it
closely, you can see it is just a combination of constants and values that
can be computed from current and past sensor values from the CAN bus. The actual
implementation in C looks like this. We need to define 2 variables. One to store
the running sum of errors and one to store the error of the previous control loop
execution. In the control loop, we define constant gains for each stage. We compute
the current error. We add the error to the sum of all errors. We compute the
difference between current errors and previous error. We then add all those
values with the respective gain to the output variable and then we clamp that
output in case it goes out of bound. We then apply the throttle control and save
the current error in the previous error variable for use in the next iteration.
The second approach is to use a Laplace transform of the PID controller. We first
need to convert it to a Z transform, the equivalent of Laplace transform but for
digital signals. It looks a bit complicated, but there are many tools to
do the conversion for you. If you want to do the conversion by hand, one way is to
use the bilinear transformation in which you replace s by this, to approximate the Z
transform of your ECU. And again, this may all look a bit intimidating, but you can
actually compute the throttle output by using the throttle output from 2
iterations ago, the current error, the previous error and the error before that
which can all be computed from sensor values on the CAN bus. And while this
control algorithm is equivalent to our previous implementation, it looks totally
different. And what I would like to stress here that identical control algorithms may
have very different software implementations, which may be relevant for
reverse engineers. I only show the first implementation not waste time, but you can
see now that the ECU in RAMN is able to make the car maintain a constant speed and
for dynamic control of the throttle. So that's it for the example. I just wanted
to show that RAMN can be used for realistic activities of the CAN bus and
that the ECUs are not just doing some random easy work. The control theory part
may be complicated and if that was going too fast for you at least I hope it proves
there is a lot of things to discover and experiment with. And that all that
learning can be done with open source tools. Now I would like to discuss what
would have been different with real ECUs because, as you can imagine, actual ECU
software is not as simple as this. I will also show you what alternatives we have.
Technologies hidden behind NDAs, so that we can get as close as we can to real
ECUs. We showed in RAMN that this cruise control ECU worked, but we only tried it
on a single scenario, namely a flat road followed by an upward slope. But what
about this scenario in which a car in front is driving slowly? Or what about a
scenario in which we are in a downward slope? In this case the ECU will not be
able to prevent a car from going too fast because it does not have access to the
brakes, which could lead to an accident. And the difficult problem here is not
whether you can think of one more scenario. The difficult problem is can you
think of them all are you sure? And thinking of all potential scenarios,
quantifying the risk and implementing countermeasures is what makes automotive
systems really different. And to prove that an ECU is reasonably safe, you
actually need to follow ISO 26262 standard, which is a standard for
functional safety. The standard defines different requirements at many levels. Not
all ECUs are equally critical, so safety relevant issues are assigned and
automotive safety integrity level or ASIL for short. And ASIL A is a less critical
level and ASIL D is the most critical level. So if you were to design a real
cruise control ECU for use in a real car, you could not just connect some random ECU
to the CAN bus and try it on the highway. You will need to go through a lot of
analysis, such as HAZOP, HARA, FMEA, STPA and so on. Usually there are so many
requirements that they cannot be tracked with only a human and a sheet of paper.
They are managed using dedicated tools such as Rational Doors. Now let's discuss
how the software would be different. The main contribution of automotive software
is to realize control algorithms safely. But without countermeasures, many things
could go wrong. For example there could be bugs in the ECU application code. And then
without any bug the software will be robust enough when there are transient
errors in the hardware. There could also be problems with the toolchains that
compile the firmware and problems with the libraries and RTOS. And when we have a
close look at the PID controller implementation which seemed good enough in
our testing, you can see it is actually a terrible implementation. We are mixing
integers and unsigned integers and floats without proper checks and type casting. We
are not checking for overflows and also random software issues. And this is not
acceptable, both for safety and security. And in this case, the problems were
obvious on purpose, but sometimes it can be very hard to spot because they stem
from very subtle computing issues. And those issues may lead to system failures.
So to avoid such scenarios, the automotive industry usually mandates the use of a
language subset which restricts what developers can do, but makes sure that
numerical errors are less likely to happen. So usually in the automotive
industry, the standard that is used is MISRA-C and it is very similar to CERT-C,
which is popular in the security industry. Using a language subset is only one of the
requirements that are dictated by ISO 26262. There are many other requirements.
At a high level they try to enforce a low complexity of the software. For example,
by restricting the size of components, restricting the copying between software
components and making sure the scheduling is appropriate. And that there are not too
many interrupts. But we also have more concrete requirements, such as restricting
functions to 1 entry and 1 exit point, forbid dynamic memory, avoid global
variables, limit the use of pointers and so on. The other issues we have to deal
with, which is riddled with bugs is transient errors. In the harsh
environment, data is not always reliable. There may be a bit flip occurring outside
of memory, for example, because of noise and communication lines, but also may be
bit flips occurring inside a microcontrollers memory. For example, because of cosmic
rays. Those issues do not originate from software, they originate from hardware,
but they do need to be addressed by software because remember, in the case of
the example cruise control ECU, just one bit flip could lead to unwanted behavior of
the ECU and the car. So to address those issues, automotive controllers need
special countermeasures. For example having redundant memory or having
redundant CPUs. In ECUs, you will typically find microcontrollers that have
been designed specially for automotive use. All those microcontrollers require
you to sign an NDA so you can not just buy them and start programming them. So that
makes it a bit hard to study an actual ECU microcontroller and real automotive
software. ISO 26262 is not the only standard for safety critical systems. ISO
26262 is actually derived from IEC 61508, so they are both similar in their
concepts. And IEC61508 microcontrollers they do not require NDAs for most other
activities you may be interested in. And more completely RAMN can be used with
STM32L4 or STM32L5 microcontrollers. And for those microcontrollers, you do not
need an NDA to download guidelines on how to implement safe software. For example,
you can find a list of features that are required for safety applications, and you
can request more data that you would need to actually achieve compliance with
IEC 61508, such as the FMEA and FMEDA. But to obtain those data, you would need to sign an
NDA. No, I personally do not think that those data are essential for education and
research. So using such microcontrollers is a good alternative. But again, let me
stress that this is an alternative for learning and researching, not for actual
use in a car. I don't have time to detail all the safety features, let me just talk
about memory redundancy, since this one impacts the code of the application of the
example cruise control ECU. So in the example, we wrote the gain of each stage
in code memory defining them as constants. For safer applications, this would not be
here. They belong to another section, the data flash where ECC protection can be
activated. If possible, calibration data should be stored twice with checksums and
preferably MACs. If you're not familiar with ECC memory, it is a kind of memory
that can detect bit flips and sometimes automatically corrects them. Another
memory is also available for the RAM, but not at all addresses. So in the
application, we have to ensure that safety critical variables are placed in a section
of RAM in which bit flips can be detected, in this case in section SRAM2. For data in
RAM that are actually constants such the gains you may also want to activate write
protection, a feature that is only available in SRAM2. Last slide about
software. In the example cruise control, we are using GCC toolchain in ISO 26262 it
is a requirement to have what is called a tool confidence lever to ensure that
poor chains will not introduce errors as it could be the case with some optimizations.
So normally you cannot use GCC. Realtime operating systems and libraries may also
have problems. That is why they need to be certified. Both STM32-HAL and FreeRTOS are
compliant with MISRA-C which is nice, but they are not compliant with ISO 26262.
However, it looks like ST is bringing Azure RTOS into the ecosystem and that one
is actually precertified ISO 26262. Maybe in the future it would be an option to
experiment with with an actual ISO 26262 operating system. So now let's talk a bit
about hardware. In case it was not clear to you, you cannot use commercial
electronics to implement an ECU. Smartphone vendors will often warn you not
to let a device in your car because a parked car can reach extreme temperatures
that commercial electronics are not designed to resist. And if you think that
the life inside the cabin is hard, you should think about an ECU which has to
stay in the engine compartment and operate without failures. And you would not think
of putting a smartphone or an Arduino here and trust your life with it. And extreme
temperatures are just one of the many environmental factors that make it
difficult for an ECU to stay reliable. The ECU also need to resist high humidity,
corrosive gases, vibrations, micro cuts, load dumps, electrostatic discharges,
electromagnetic noise and so on. And when subjected to such a harsh environment,
many things could go wrong with electronics. You probably know about
corrosion, but many other physical phenomena are at risk of happening to the
components. Solder cracs, intermetallic growth, whiskers, dendrites,
electromigration, etc. For example, whiskers are metal growing out of
electrical components and dendrites are metals leaving the plus side towards the
minus side. And many other phenomena may result in a dangerous failure. So
obviously, ECUs need to be designed to resist harsh environments and have
countermeasures against all those potential failures. ECUs need to pass
various tests that simulate harsh environments. Those tests are usually
defined by manufacturers and the test specifications are not made public. What
is made public, however, is the test specifications for individual electronic
components. And those tests are usually defined by AEC. So the Automotive
Electronic Council, and you can have a look at them online. For RAMN, we tried to
follow design guidelines similar to those of real ECUs. But of course, we cannot
follow actual rules as it to be much less accessible. Completely, we selected
AEC-Q100 grade zero components for everything except connectors and
microcontrollers, because those may require NDAs or not be easily accessible.
Depending on the part reference, ECU microcontrollers may be usable from -40 to
125°C. RAMN tried to stay close to automotive grade, but it is still not
automotive grade, especially in the reliability department, so it can not be
used as a real ECU. As a result, we try to stay close to automotive hardware is to
help researchers evaluate the impact of manufacturing tolerances and environments
because remember, manufacturers are making millions of cars that need to operate on a
large temperature range. So if you are developing, for example, a security
technology that relies on hardware characteristics such as the clocks of the
ECUs, you will need to prove that the technology works despite manufacturing
tolerances and harsh environments. And with RAMN it is easy to have a large
sample of ECU networks and since they are small they can easily fit in various
testing equipment. And now let's move on to the last section, security. In the
automotive industry, we just can not apply the same reasoning as you do in many other
industries. For example, a credit card, if you detect the temperature is too cold, it
may think that it is a tampering attack and decides to shut down because it is not
safety critical. On the other hand, the car needs to start quickly because the
user should not be left out in the cold and also credit cards have an expiration
date. So they do not need to guarantee security for more than a few years. But
cars do not have an expiration date. If they are well maintained they may be used
for several decades and the security technologies should keep on working. So in
the end, automotive security technologies have different requirements.
Unfortunately, according to past research, a security technology is often less
reliable when you extend its operating temperature range and its lifetime. For
example, at low temperatures, they may be liable to cold boot attacks. At high
temperatures, it has also been shown that electronics tend to be less reliable
concerning glitching attacks and in those papers high-temperature means something
like 60 or 100°C, far from the maximum temperature required for some ECUs. Also,
it has been shown that the higher age for electronics usually results in different
security properties. And you may think that the safety features of automotive
microcontrollers would prevent some attacks such as glitching attacks, but it
has been shown that ECC memories are also susceptible to glitching attacks. And that
even ISO 26262 ASIL-D microcontrollers, which is the highest level of safety may
be susceptible to glitching. So safety features often help, but there aren't
really enough to ensure security. What is also different with automotive is that you
need to rethink the strategy in case of security problems, for example, with
credit cards, it is not uncommon for authentication to fail randomly. When the
credit card fails to work usually you just need to try it once more and it will
probably work and everything stays again. No life is at risk. But the car cannot
have the same strategy. If you add authentication to the brake CAN message
and you start receiving break request that fail authentication what should the car
really do. Because it may be a cyber attacks, which you want to avoid, but you
should not relay out the possibility of a random malfunction or a false positive for
an intrusion detection system. And by adding complexity into the system, you
always increase the odds of a problem. And which one would be worse between breaking
because of a cyber attack or not breaking because of a malfunction? And there is no
easy way to answer that question. But what I want to stress here is that many
security countermeasures that people suggest for cars such as encrypting the
CAN bus, permanently disabling debug ports or obfuscating the firmware, they may not
necessarily be the best ideas, because if you suspect a malfunction with an ECU, you
need to investigate the problem seriously because it may harm people. You cannot
just replace a car as you would with a credit card or a smartphone.
So technologies that can truly take into account both automotive requirements and
security requirements are better. And we should make sure that education and
research in these areas are accessible to many researchers without NDAs or
prohibitive costs. Now, of course, you can use RAMN to try out different attacks. The
first obvious one is to inject CAN messages to alter the behavior of the car.
Here for example the breaks. Another kind of security that I did not mention in this
presentation is physical security for sensors and actuators. Here I am
demonstrating what happens when it overtake the control of the steering wheel
actuator. A human will probably break in this situation. The self driving algorithm
in CARLA here does not realize it has lost control of the steering wheel and is still
trying to correct the trajectory when a better decision would be to stop the car.
So this is the end of the presentation. We developed an inexpensive, safe and
interactive platform to study and research automotive systems. The platform is
accessible to beginners. It is not automotive grade, but is close enough for
research and educational purposes. The project is open source and with permissive
licenses. If you have questions or ideas, do not hesitate to contact us, especially
if you are involved with education, research, training, CTF, etc. And thank
you for watching.
Herald: Camille, thanks for this comprehensive talk, this was amazing. We
unfortunately don't have much time for the questions or answers, but there is
one question that popped up, which is about the hardware and your PCB. How did
you design it? How much does it cost actually, how can you get hold of that
thing? Camille: So I designed everything with
KiCAD. And I mean, I think a few years ago it was very hard to design hardware, but
now you have footprint libraries available online. It has become very easy. The board
was between 50 to 100€ for a quantity of one. So microcontrollers are obviously the
expensive parts and the PCB assembling parts - it is up to the PCB Fabrication
Service. If you have questions just ask me on GitHub, I would be happy to answer.
*rC3 postroll music*
Subtitles created by c3subtitles.de in the year 2021. Join, and help us!