*36C3 Intro musik*
Herald: The next talk will be titled 'How to Design Highly Reliable Digital
Electronics', and it will be delivered to you by Szymon and Stefan. Warm Applause
for them.
*applause*
Stefan: All right. Good morning, Congress. So perhaps every one of you in the room
here has at one point or another in their lives witnessed their computer behaving
weirdly and doing things that it was not supposed to do or what you didn't
anticipate it to do. And well, typically that would have probably been the result
of a software bug of some sort somewhere inside the huge software stack your PC is
running on. Have you ever considered what the probability of this weird behavior
being caused by a bit flipped somewhere in your memory of your computer might have
been? So what you can see in this video on the screen now is a physics experiment
called a cloud chamber. It's a very simple experiment that is actually able to
visualize and make apparent all the constant stream of background radiation we
all are constantly exposed to. So what's happening here is that highly energetic
particles, for example, from space they trace through gaseous alcohol and they
collide with alcohol molecules and they form in this process a trail of
condensation while they do that. And if you think about your computer, a typical
cell of RAM, of which you might have, I don't know, 4, 8, 10 gigabytes in your
machine is as big as only 80 nanometers wide. So it's very, very tiny. And you
probably are able to appreciate the small amount of energy that is needed or that is
used to store the information inside each of those bits. And the sheer amount of of
those bits you have in your RAM and your computer. So a couple of years ago, there
was a study that concluded that in a computer with about four gigabytes of RAM,
a bit flip, um, caused by such an event by cosmic background radiation can occur
about once every 33 hours. So a bit less than than one per day. In an
incident in 2008, a Quantas Airlines flight actually nearly crashed, and the
reason for this crash was traced back to be very likely caused by a bit flipped
somewhere in one of the CPUs of the avionics system and nearly caused the
death of a lot of passengers on this plane. In 2003, in Belgium, a small
municipal vote actually had a weird hiccup in which one of the candidates in this
election actually got 4096 more votes added in a single instance.
And that was traced back to be very likely caused by cosmic background radiation,
flipping a memory cell somewhere that stored the vote count. And it was only
discovered that this happened because this number of votes for this particular
candidate was considered unreasonable, but otherwise would have gotten away probably
without being detected. So a few words about us: Szymon and I, we both work at
CERN in the microelectronics section and we both develop electronics that need to
be tolerant to these sorts of effects. So we develop radiation tolerant electronics
for the experiments at CERN, at the LHC. Among a lot of other applications, you can
meet the two of us at the Lötlabor Jena assembly if you are interested in what we
are talking about today. And we will also give a small talk or a small workshop
about radiation detection tomorrow, in one of the seminar rooms. So feel free to pass
by there, it will be a quick introduction. To give you a small idea of what kind of
environment we are working for: So if you would use one of your default intel i7
CPUs from your notebook and would put it anywhere where we operate our electronics,
it would very shortly die in a matter of probably one or two minutes and it would
die for more than just one reason, which is rather interesting and compelling. So
the idea for today's talk is to give you all an insight into all the things that
need to be taken into account when you design electronics for radiation
environments. What kinds of different challenges come when you try to do that.
We classify and explain the different types of radiation effects that exist. And
then we also present what you can do to mitigate these effects and also validate
that what you did to care for them or protect your circuits actually worked. And
of course, as we do that, we'll try to give our view on how we develop radiation
tolerant electronics at CERN and how our workflow looks like to make sure this
works. So let's first maybe take a step back and have a look at what we mean when
we say radiation environments. The first one that you probably have in mind right
now when you think about radiation is space. So, this interstellar space is
basically filled with, very high speed, highly energetic electrons and protons and
all sorts of high energy particles. And while they, for example, traverse close to
planets as our Earth - these planets sometimes do have a magnetic field and the
highly energetic particles are actually deflected by these magnetic fields and
they can protect the planets as our planet, for example, from this highly
energetic radiation. But in the process, there around these planets sometimes they
form these radiation belts - known as the Van Allen belts after the guy who
discovered this effect a long time ago. And a satellite in space as it orbits
around the Earth might, depending on what orbit is chosen, sometimes go through
these belts of highly intense radiation. That, of course, then needs to be taken
into account when designing electronics for such a satellite. And if Earth itself
is not able to give you enough radiation, you may think of the very famous Juno
Jupiter mission that has become famous about a year ago. They actually in the
environment of Jupiter they anticipated so much radiation that they actually decided
to put all the electronics of the satellite inside a one centimeter thick
cube of titanium, which is famously known as the Juno Radiation Vault. But not only
space offers radiation environments. Another form of radiation you probably all
recognize this when I show you this picture, which is an X-ray image of a
hand. And X-ray is also considered a form of radiation. And while, of course, the
doses or amounts of radiation any patient is exposed to while doing diagnosis or
treatment of some disease, that might not be the full story when it comes to medical
applications. So this is a medical particle accelerator which is used for
cancer treatment. And in these sorts of accelerators, typically carbon ions or
protons are accelerated and then focused and used to treat and selectively destroy
cancer cells in the body. And this comes already relatively close to the
environment we are working in and working for. So Szymon and I are working, for
example, on electronics, for the CMS detector inside the LHC or which we build
dedicated, radiation tolerant, integrated circuits which have to withstand very,
very large amounts and doses of short lived radiation in order to function
correctly. And if we didn't specifically design electronics for that, basically the
whole system would never be able to work. To illustrate a bit how you can imagine
the scale of this environment: This is a single plot of a collision event that was
recorded in the ATLAS experiment. And each of those tiny little traces you can make
out in this diagram is actually either one or multiple secondary particles that were
created in the initial collision of two proton bunches inside the experiment. And
in each of those, of course, races around the detector electronics, which make these
traces visible. Itself, then decaying into multiple other secondary particles which
all go through our electronics. And if that doesn't sound, let's say, bad enough
for digital electronics, these collisions happen about 40 million times a second. Of
course, multiplying the number of events or problems they can cause in our
circuits. So we now want to introduce all the things that can happen, the different
radiation effects. But first, probably we take a step back and look at what we mean
when we say digital electronics or digital logic, which we want to focus on today. So
from your university lectures or your reading, you probably know the first class
of digital logic, which is the combinatorial logic. So this is typically
logic that just does a simple linear relation of the inputs of a circuit and
produces an output as exemplified with these AND and OR, NAND, XOR gates that you
see here. But if you want to build - I mean even though we use those everywhere
in our circuits - you probably also want to store state in a more complex circuit,
for example, in the registers of your CPU they store some sort of internal
information. And for that we use the other class of logic, which is called the
sequential logic. So this is typically clocked with some system clock frequency
and it changes its output with relation to the inputs whenever this clock signal changes.
And now if we look at how all these different logic functionalities are
implemented. So typically nowadays for that you may know that we use CMOS
technologies and basically represent all this logic functionality as digital gates
using small P-MOS and N-MOS MOSFET transistors in CMOS technologies. And if
we kind of try to build a model for more complex digital circuits, we typically use
something we call the finite state machine model, in which we use a model that
consists of a combinatorial and a sequential part. And you can see that the
output of the circuit depends both on the internal state inside the register as well
as also the input to the combinatorial logic. And accordingly, also the state
that is internal is always changed by the inputs as well as the current state. So
this is kind of the simple model for more complex systems that can be used to model
different effects. Um, now let's try to actually look at what the radiation can do
to transistors. And for that we are going to have a quick recap at what the
transistor actually is and how it looks like. As you may perhaps know is that in
CMOS technologies, transistors are built on wafers of high purity silicon. So this
is a crystalline, very regularly organized lattice of silicon atoms. And what we do
to form a transistor on such a wafer is that we add dopants. So in order to form
diffusion regions, which later will become the source and drain of our transistors.
And then on top of that we grow a layer of insulating oxide. And on top of that we
put polysilicon, which forms the gate terminal of the transistor. And in the end
we end up with an equivalent circuit a bit like that. And now to put things back into
perspective - you may also note that the dimension of these structures are very
tiny. So we talk about tens of nanometers for some of the dimensions I've outlined
here. And as the technologies shrink, these become smaller and smaller and
therefore you'll probably also realize or are able to appreciate the small amount of
energy that are used to store information inside these digital circuits, which makes
them perhaps more sensitive to radiation. So let's take a look. What different types
of radiation effects exist? We typically in this case, differentiate them into two
main classes of events. The first one would be the cumulative effects, which are
effects that, as the name implies, accumulate over time. So as the circuit is
placed inside some radiation environment, over time it accumulates more and more
dose and therefore worsens its performance or changes how it operates. And on the
other side, we have the Single Event Effects, which are always events that
happen at some instantaneous point in time, and then suddenly, without being
predictable, change how the circuit operates or how it functions or if it
works in the first place or not. So I'm going to first go into the class of
cumulative effects and then later on, Szymon will go into the other class of the
Single Event Effects. So in terms of these accumulating effects, we basically have
two main subclasses: The first one being ionization or TID effects, for Total
Ionizing Dose - and the second one being displacement damages. So displacement
damages do exactly what they sound like. It is all the effects that happen when an
atom in the silicon lattice is actually displaced, so removed from its lattice
position and actually changes the structure of the semiconductor. But
luckily, these effects don't have a big impact in the CMOS digital circuits that
we are looking at today. So we will disregard them for the moment and we'll be
looking more at the ionization damage, or TID. So ionization - as a quick recap - is
whenever electrons are removed or added to an atom, effectively transforming it into
an ion. And these effects are especially critical for the circuits we are building
because of what they do is that they change the behavior of the transistors.
And without looking too much into the semiconductor details, I just want to show
their typical effect that we are concerned about in this very simple circuit here. So
this is just an inverter circuit consisting of two transistors here and
there. And what the circuit does in normal operation is it just takes an input signal
and inverts and basically gives the inverted signal at the output. And as the
transistors are irradiated and accumulate dose, you can see that the edges of the
output signal get slower. So the transistor takes longer to turn on and off.
And what that does in turn is that it limits the maximum operation frequency of
your circuit. And of course, that is not something you want to do. You want your
circuit to operate at some frequency in your final system. And if the maximum
frequency it can work at degrades over time, at some point it will fail as the
maximum frequency is just too low. So let's have a look at what we can do to
mitigate these effects. The first one and I already mentioned it when talking about
the Juno mission, is shielding. So if you can actually put a box around your
electronics and shield any radiation from actually hitting your transistors, it is
obvious that they will last longer and will suffer less from the radiation damage
that it would otherwise do. So this approach is very often used in space
applications like on satellites, but it's not very useful if you are actually trying
to measure the radiation with your circuits as we do, for example, in the
particle accelerators we build integrated circuits for. So there first of all, we
want to measure the radiation so we cannot shield our detectors from the radiation.
And also, we don't want to influence the tracks of these secondary collision
products with any shielding material that would be in the way. So this is not very
useful in a particle accelerator environment, let's say. So we have to
resort to different methods. So as I said, we do have to design our own integrated
circuits in the first place. So we have some freedom in what we call transistor
level design. So we can actually alter the dimensions of the transistors. We can make
them larger to withstand larger doses of radiation and we can use special
techniques in terms of layout that we can experimentally verifiy to be more
resistant to radiation effects. And as a third measure, which is probably the most
important one for us, is what we call modeling. So we actually are able to
characterize all the effects that radiation will have on a transistor. And
if we can do that, if we will know: 'If I put it into a radiation environment for a
year, how much slower will it become?' Then it is of course easy to say: 'OK, I
can just over-design my circuit and make it a bit more simple, maybe have less functionality,
but be able to operate at a higher frequency and therefore withstand
the radiation effects for a longer time while still working sufficiently well at
the end of its expected lifetime.' So that's more or less what we can do about
these effects. And I'll hand over to Szymon for the second class.
Szymon: Contrary to the cumulative effects presented by Stefan, the other group are
Single Event Effects which are caused by high energy deposits, which are caused by
a single particle or shower of particles. And they can happen at any time, even
seconds after irradiation is started. It means that if your circuit is vulnerable
to this class of effects, it can fail immediately after radiation is present.
And here we also classify these effects into several groups. The first are hard,
or permanent, errors, which as the name indicates can permanently destroy your
circuit. And this type of errors are typically critical for power devices where
you have large power densities and they are not so much of a problem for digital
circuits. In the other class of effects are soft errors. And here we distinguish
transient, or Single Event Transient errors, which are spurious signals
propagating in your circuit as a result of a gate being hit by a particle and they
are especially problematic for analog circuits or asynchronous digital circuits,
but under some circumstances they can be also problematic for synchronous systems.
And the other class of problems are static, or Single Event Upset problems,
which basically means that your memory element like a register gets flipped. And
then of course, if your system is not designed to handle this type of errors
properly, it can lead to a failure. So in the following part of the presentation
we'll focus mostly on soft errors. So let's try to understand what is the origin
of this type of problem. So as Stefan mentioned, the typical transistor is built
out of diffusions, gate and channel. So here you can see one diffusion. Let's
assume that it is a drain diffusion. And then when a particle goes through and
deposits charge, it creates free electron and hole pairs, which then in the presence of
electric fields, they get collected by means of drift, which results in a large
current spike, which is very short. And then the rest of the charge could be
collected by diffusion which is a much slower process and therefore also the
amplitude of the event is much, much smaller. So let's try to understand what
could happen in a typical memory cell. So on this schematic, you can see the
simplest memory cell, which is composed of two back-to-back inverters. And let's
assume that node A is at high and node B is at low potential initially. And then we
have a particle hitting the drain of transistor M1 which creates a short
circuit current between drain and ground, bringing the drain of transistor M1 to low
potential, which also acts on the gates of second inverter, temporarily changing its
state from low to high, which reinforces the wrong state in the first inverter. And
at this time the error is locked in your memory cell and you basically lost your
information. So you may be asking yourself: 'How much charge is needed
really to flip a state of a memory cell?'. And you can get this number from either
simulations or from measurements. So let's assume that what we could do, we could try
to inject some current into the sensitive node, for example, drain of transistor M1.
And here what I will show is that on the top plot you will have current as a function
of time. On the second plot you will have output voltage. So voltage at node B as a
function of time and at the lowest plot you will see a probability of having a bit
flip. So if you inject very little current, of course nothing changes at the
output, but once you start increasing the amount of current you are injecting, you
see that something appears at the output and at some point the output will toggle,
so it will switch to the other state. And at this point, if you really calculate
what is the area under the current curve you can find what is the critical charge
needed to flip the memory cell. And if you go further, if you start injecting even
more current, you will not see that much difference in the output voltage waveform.
It could become only slightly faster. And at this point, you also can notice that
the probability now jumped to one, which means that any time you inject so much
current there is a fault in your circuit. So for now, we just found what is the
probability of having a bit-flip from 0 to 1 in node B. Of course we should also
calculate the same for the other direction, so from 1 to zero. And usually
it is slightly different. And then of course we should inject in all the other
nodes, for example node B and also should study all possible transitions. And then
at the end, if you calculate the superposition of these effects and you
multiply them by the active area of each node, you will end up with what we call
the cross section, which has a dimension of centimeters squared, which will tell
you how sensitive your circuit is to this type of effects. And then knowing the
radiation profile of your environment, you can calculate the expected upset rate in
the final application. So now, having covered the basic of the single event
effects, let's try to check how we can mitigate them. And here also technology
plays a significant role. So of course, newer technologies offer us much smaller
devices. And together with that, what follows is that usually supply voltages
are getting smaller and smaller as well as the node capacitance, which means that for
our Single Event Upsets it is very bad because the critical charge which is
required to flip our bit is getting less and less. But at the end, at the same
time, physical dimensions of our transistors are getting smaller, which
means that the cross section for them being hit is also getting smaller. So
overall, the effects really depend on the circuit topology and the radiation
environment. So another protection method could be introduced on the cell level. And
here we could imagine increasing the critical charge. And that could be done in
the easiest way by just increasing the node capacitance by, for example, putting
larger transistors. But of course, this also increases the collection electrode,
which is not nice. And another way could be just increase the capacitance by adding
some extra metal capacitance, but it, of course, slows down the circuit. Another
approach could be to try to store the information on more than two nodes. So I
showed you that on a simple SRAM cell we store information only on two nodes, so
you could try to come up with some other cells, for example, like that one in which
the information you stored on four nodes. So you can see that the architecture is
very similar to the basic SRAM cell. But you should be careful always to very
carefully simulate your design, because if we analyze this circuit, you will quickly
realize that this circuit, even though the information is stored in four different
nodes, the same type of loop exists as in the basic circuit. Meaning that at the end
the circuit offers basically no hardening with respect to the previous cell. So
actually we can do it better. So here you can see a typical dual interlocked cell.
So the amount of transistors is exactly the same as in the previous example, but
now they are interconnected slightly differently. And here you can see that
this cell has also two stable configurations. But this time data can propagate, the low
level from a given node can propagate only to the left hand side, while high
level can propagate to the right hand side. And each stage being inverting means
that the fault can not propagate for more than one node. Of course, this cell has
some drawbacks: It consumes more area than a simple SRAM cell and also write access
requires accessing at least two nodes at the same time to really change the state
of the cell. And so you may ask yourself, how effective is this cell? So here I will
show you a cross section plot. So it is the probability of having an error as a
function of injected energy. And as a reference, you can see a pink curve on the
top, which is for a normal, not protected cell. And on the green you can see the
cross section for the error in the DICE cell. So as you can see, it is one order
of magnitude better than the normal cell. But still, the cross section is far from
being negligible, So, the problem was identified: So it was identified that the
problem was caused by the fact that some sensitive nodes were very close together
on the layout and therefore they could be upset by the same particle. Because as we
mentioned, that single devices, they are very small. We are talking about dimensions
below a micron. So after realizing that, we designed another cell in which we
separated more sensitive nodes and we ended up with the blue curve, and as you
can see the cross section was reduced by two more orders of magnitude and the
threshold was increased significantly. So if you don't want to redesign your
standard cells, you could also apply some mitigation techniques on block level. So
here we can use some encoding to encode our state better. And as an example, I
will show you a typical Hamming code. So to protect four bits, we have to add three
additional party bits which are calculated according to this formula. And then once
you calculate the parity bits, you can use those to check the state integrity of your
internal state. And if any of their parity bits is not equal to zero, then the bits
instantaneously become syndromes, indicating where the error happened. And
you can use these information to correct the error. Of course, in this case, the
efficiency is not really nice because we need three additional bits to protect only
four bits of information. But as the state length increases the protection also is
more efficient. Another approach would be to do even less. Meaning that instead of
changing anything you need in your design, you can just triplicate your design or
multiply it many times and just vote, which state is correct? So this concept is
called tripple modular redudancy and it is based around a voter cell. So it is a
cell which has odd number of inputs and output is always equal to
majority of its input. And as I mentioned that the idea is that you have, for
example, three circuits: A, B and C, and during normal operation, when they are
identical, the output is also the same. However, when there is a problem, for
example, in logic, part B, the output is affected. So this problem is
effectively masked by the voter cell and it is not visible from outside of the
circuit. But you have to be careful not to take this picture as a as a design
template. So let's try to analyze what would happen with a state machine
similar to what Stephan introduced. If you were to just use this concept. So here you
can see three state machines and a voter on the output. And as we can see,
if you have an upside in, for example, the state register A, then the state is
broken. But still the output of the circuit, which is indicated by letter s is
correct because the B and C registers are still fine. But what happens if some time
later we have an upset in memory element B or C? Then of course the state
of our system is broken and we can not recover it. So you can ask yourself what
can we do better in order to avoid this situation? So that just to be sure. Please
do not use this technique to protect your circuits. So the easiest mitigation could
be to use as an input to your logic to use the output of the voter cell itself.
What it offers us is that now whenever you have an upset in one of the memory
elements for the next computation, for the next stage, we always use the voter
output, which ensures that the signal will be removed one clock cycle later. So
you will have another hit sometime later, basically, it will not affect our state.
Until now we consider only upsets in our registers but what happens if we have
charge in our voter? So you see that if there is no state change, basically the
transient in the voter doesn't impact our system. But if you are really unlucky
and the transient happens when the clock transition happens, so when whenever we
enlarge the data, we can corrupt the state in three registers at the same time, which
is less than ideal. So to overcome this limitation, you can consider skewing our
clocks by some time, which is larger than the maximum charge in time. And now,
because with each register samples the output of the voter a slightly different
time, we can corrupt only one flip-flop at the time. So of course, if you are
unlucky, we can have problematic situations in which one register is
already in your state. The other register is still in the old state. And then it
can lead to undetermenistic result. So it is better, but still not ideal. So as a
general theme, you have seen that we were adding and adding more resources so you
can ask yourself what would happen if we tripplicate everything. So in this case,
we tripplicated registers, we tripplicate our logic and our voters. And
now you can see that whenever we have an upset in our register, it can only affect
one register at the time and the error will be removed from the system one clock
cycle later. Also, if we have an upset in the voter or in their logic it can be
larged only to one register, which means that in principle we create that system
which is really robust. Unfortunately, nothing is for free. So here I compare a
different tripplication environments and as you can see that the more protection
you want to have, the more you have to pay in terms of resources being power in the
area. And also usual, you pay small penalty in terms of maximum operational
speed. So which flavor of protection you use depends really on
application. So for most sensitive circuits, you probably you want to use
full TMR and you may leave some other bits of logic unprotected. So another, if
your system is not mission critical and you can tolerate some downtime, you can
consider scrubbing, which means periodically checking the state of your system and refreshing it
if necessary if an error is detected using some parity bits or copy of the data in
a safe space. Or you can have a watchdog which will find out that
something went wrong and it will just reinitialize the whole system. So now,
having covered the basics of all the effects we will have to face, we would like
to show you the basic flow which we follow during designing our radiation hardened
circuits. So of course we always start with specifications. So we try to
understand our radiation environment in which the circuit is meant to operate. So
we come up with some specifications for total dose which could be accumulated and
for the rate of single event upsets. And at this moment, it is also not very rare
that we have to decide to move some functionality out of our detector volume,
outside, where we can use of the sort of commercial equipment to do number
crunching. But let's assume that we would go with our ASIC. So having the
specifications, of course we proceed with functional implementation. This we
typically do with hardware describtion languages, so verilog or VHDL which you may
know from typical FPGA flow. And of course we write a lot of simulations to
understand whether we are meeting our functional goals or whether our circuit
behaves as expected. And then we selectively select some parts of the
circuits which we want to protect from radiation effects. So, for example, we can
decide to use triplication or some other methods. So these days we typically use
triplication as the most straightforward and very effective method. So you can ask
yourself how do we triplicate the logic? So the simplest could be: Just copy
and paste the code three times at some postfixes like A, B and C and you are
done. But of course this solution has some drawbacks. So it is time consuming and it
is very error prone. So maybe you have noticed that I had a typo there. So of
course we don't want to do that. So we developed our own tool, which we called
TMRG, which automatizes the process of triplication and eliminates the two main
drawbacks, which I just described. So after we have our code triplicated and of
course, not before rerunning all the simulations to make sure that everything
went as expected. We then proceed to the synthesis process in which we convert our
high level hardware description languages to gate level netlists, in which all the functions
are mapped to gates, which were introduced by Stefan, so both combinatorial and
sequential. And here we also have to be careful because modern CAD tools have a
tendency, of course, to optimise the logic as much as possible. And our logic in most
of the cases is really redundant. So it is very easy; So, it should be removed. So we
really have to make sure that it is not removed. That's why our tool also provides
some constraints for the synthesizer to make sure that our design intent is
clearly and well understood by the tool. And once we have the output netlist, we
proceed to place and route process where this kind of netlist representation is
mapped to a layout of what will become soon our digital chip where we placed all
the cells and we route connections between them and here there is
another danger which I mentioned already, it's that in modern technologies the cells
are so small that they could be easily affected by a single particle at the same
time. So we have to really space out the big cells which are responsible for
keeping the information about the state to make sure that a single particle cannot
upset A and B, for example, registered from the same register. And then in the
last step, of course, we'll have to verify that everything, what we have done, is
correct. And at this level, we also try to introduce some single event effects in our
simulations. So we could randomly flip bits in our system. We can also inject
transients. And typically we used to do that on the netlist level, which works
very fine. And it is very nice. But the problem with this approach is that we can
perform these actions very late in the design cycle, which is less than ideal.
And also that if we find that there is problem in our simulation, typical netlist
at this level has probably few orders of magnitude more lines than our initial RTL
code. So to trace back what is the problematic line of code is not so
straightforward. At this time. So you can ask yourself why not to try to inject
errors in the RTL design? And the answer was, the answer is that it is not so
trivially to map the hardware description language's high level constructs to
what will become combinatorial or sequential logic. So in order to eliminate
this problem, we also develop another open source tool, which allows us to...
So we decided to use Yosys open source synthesis tool from clifford, which
was presented in the Congress several years ago. So we use this tool to make a
first pass through our RTL code to understand which elements will be mapped
to sequential and combinatorial. And then having this information, we will use
cocotb, another python verification framework, which allows us programmatic
access to these nodes and we can effectively inject the errors in our
simulations. And I forgot to mention that the TMRG tool is also open source. So if
you are interested in one of the tools, please feel free to contact us. And of
course, after our simulation is done, then in the next step we would really tape out. And
so we submit our chip to manufacturing and hopefully a few months later we receive
our chip back. Stefan: All right. So after patiently
waiting then for a couple of months while your chip is in manufacturing and you're
spending time on preparing a test set up and preparing yourself to actually test if
your chip works as you expected to. Now, it's probably also a good time to think
about how to actually validate or test if all the measures that you've taken to
protect your circuit from radiation effects actually are effective or if they
are not. And so again, we will split this in two parts. So you will probably want to
start with testing for the total ionizing dose effects. So for the cumulative effect
and for that, you typically use x ray radiation relatively similar to the one
used in medical treatment. So this radiation is relatively low, energetic,
which has the upside of not producing any single event effects, but you can really
only accumulate radiation dose and focus on the accumulating effects. And typically
you would use a machine that looks somewhat like this, a relatively compact
thing. You can have in your laboratory and you can use that to really accumulate
large amounts of radiation dose on your circuit. And then you need some sort of
mechanism to verify or to quantify how much your circuit slows down due to this
radiation dose. And if you do that, you typically end up with a graphic such as
this one, where in the x axis you have the radiation dose your circuit was exposed
to. And on the y axis, you see that the frequency has gone down over time and you
can use this information to say: "OK, my final application, I expect this
level of radiation dose. I mean, I can still see that my circuit will work fine
under some given environmental condition or some operation condition." So this is
the test for the first class of effects. And the test for the second class of
effects for the single event effect is a bit more involved. So there what you would
typically start to do is go for a heavy ion test campaign. So you would go to a
specialized, relatively rare facility. We have a couple of those in Europe and would
look perhaps somewhat like this. So it's a small particle accelerator somewhere.
They typically have different types of heavy ions at their
disposal that they can accelerate and then shoot at your chip that you can place in a
vacuum chamber and these ions can deposit very well known amounts of energy in your
circuit and you can use that information to characterize your circuit. The downside
is a bit that these facilities tend to be relatively expensive to access and also a
bit hard to access. So typically you need to book them a lot of time in advance and
that's sometimes not very easy. But what it offers you, you can use different types
of ions with different energies. You can really make a very well-defined
sensitivity curve similar to the one that Szymon has described. You can get from
simulations and really characterize your circuit for how often, any single event
effects will appear in the final application if there is any remaining
effects left. If you have left something unprotected. The problem here is that
these particle accelerators typically just bombard your circuit with like thousands
of particles per second and they hit basically the whole area in a random
fashion. So you don't really have a way of steering those or measuring the position
of these particles. So typically you are a bit in the dark and really have to really
carefully know the behavior of your circuit and all the quirks it has even
without the radiation to instantly notice when something has gone wrong. And
this is typically not very easy and you can kind of compare it with having
some weird crash somewhere in your software stack and then having to have
first take a look and see what actually has happened. Typically
you find something that has not been properly protected and you see some weird
effect on your circuit and then you try to get a better idea of where that problem
actually is located. And the answer for these types of problems involving position
is, of course, always lasers. So we have two types of laser experiments available
that can be used to more selectively probe your circuit for these problems. The first
one being the single photon absorption laser. And it sounds this relatively
simple in terms of setup. You just use a single laser beam that shoots straight up
at your circuit from the back. And while it does that, it deposits energy all along
the silicon and also in the diffusions of your transistors and is therefore also
able to inject energy there, potentially upsetting a bit of memory or exposing
whatever other single event effects you have. And of course, you can steer this
beam across the surface of your chip or whatever circuit you are testing and then
find the sensitive location. The problem here is that the amount of energy that is
deposited is really large due to the fact that it has to go through the whole
silicon until it reaches the transistor. And therefore it's mostly used to find
these destructive effects that really break something in your circuit. The more
clever and somehow beautiful experiment is the two photon absorption laser experiment
in which you use two laser beams of a different wavelength. And these actually
do not have enough energy to cause any effect in your silicon. If only one of the
laser beams is present, but only in the small location where the two beams
intersect, the energy is actually large enough to produce the effect. And this
allows you to very selectively and only on a very small volume induce charge and
cause an effect in your circuit. And when you do that now, you can systematically
scan both the X and Y directions across your chip and also the Z direction and can
really measure the volume of sensitive area. And this is what you would typically
get of such an experiment. So in black and white in the back, you'll see an infrared
image of your chip where you can really make out the individual, say structural
components. And then overlaid in blue, you can basically highlight all the sensitive
points that made you measure something you didn't expect, some weird bit flip in a
register or something. And you can really then go to your layout software and find
what is the the register or the gate in your netlist that is responsible for
this. And then it's more like operating a debugger in a software environment.
Tracing back from there what the line of code responsible for this bug is. And
to close out, it is always best to learn from mistakes. And we offer our mistakes
as a guideline for if you ever feel yourself the need to design radiation
tolerant circuits. So we want to present two or three small issues we had and
circuits where we were convinced it should have been working fine. So the first one
this you will probably recognize is this full triple modular redundancy scheme that
Szymon has presented. So we made sure to triplicate everything and we were relatively
sure that everything should be fine. The only modification we did is that to all
those registers in our design, we've added a reset, because we wanted to initialize
the system to some known state when we started up, which is a very obvious thing
to do. Every CPU has a reset. But of course, what we didn't think about here
was that at some point there's a buffer driving this reset line somewhere. And if
there's only a single buffer. What happens if this buffer experiences a small
transient event? Of course, the obvious thing that happened is that as soon as
that happened, all the registers were upset at the same time and were basically
cleared and all our fancy protection was invalidated. So next time we decided,
let's be smarter this time. And of course, we triplicate all the logic and all the
voters and all the registers. So let's also triplicate the reset lines. And while
the designer of that block probably had very good intentions, it turned out
that later than when we manufactured the chip, it still sometimes showed a complete
reset without any good explanation for that. And what was left out of the the
scope of thinking here was that this reset actually was connected to the system reset
of the chip that we had. And typically pins are on the chip or something that is
not available in huge quantities. So you typically don't want to spend three pins
of your chip just for a stupid reset that you don't use ninety nine percent of the
time. So what we did at some point we just connected again the reset lines to a
single input buffer. That was then connected to a pin of the chip. And of
course, this also represented a small sensitive area in the chip. And again,
a single upset here was able to destroy all three of our flip flops. All right.
And the last lesson I'm bringing or the last thing that goes back to the
implementation details that Szymon has mentioned. So this time, really simple
circuit. We were absolutely convinced it must work because it was basically the
textbook example that Szymon was presenting. And the code was so
small we were able to inspect everything and were very much sure that nothing
should have happened. And what we saw when we went for this laser testing experiment,
in simplified form is basically that only this first voter. And when this was
hit, always all our register was upset while the other ones were
never manifested to show anything strange. And it took us quite a while to actually
look at the layout later on and figure out that what was in the chip was rather this.
So two of the voters were actually not there. And Szymon mentioned the reason for
that. So synthesis tool these days are really clever at identifying redundant
logic and because we forgot to tell it to not optimize these redundant pieces of
logic, which the voters really are. It just merged them into one. And that
explains why we only saw this one voter being the sensitive one. And of course, if
you have a transient event there, then you suddenly upset all your registers and that
without even knowing it and with being sure, having looked at every single line
of verilog code and being very sure, everything should have been fine. But that
seems to be how this business goes. So we hope we had been we had the chance and you
were able to get some insight in in what we do to make sure the experiments at the
LHC work fine. What you can do to make sure the satellite you are working on
might be working OK. Even before launching it into space, if you're interested into
some more information on this topic, feel free to pass by at the assembly I
mentioned at the beginning or just meet us after the talk and otherwise thank you
very much. *Applause*
Herald: Thank you very much indeed. There's about 10 minutes left for Q and A,
so if you have any questions go to a microphone. And as a cautious reminder,
questions are short sentences with. That starts with a question. Well, ends with a
question mark and the first question goes to the Internet.
Internet: Well, hello. Um, do you also incorporate radiation as the source for
randomness when that's needed? Stefan: So we personally don't. So in our
designs we don't. But it is done indeed for a random number generator. This is
sometimes done that they use radioactive decay as a source for randomness. So this
is done, but we don't do it in our experiments.
We rather want deterministic data out of the things we built.
Herald: Okay. Next question goes to microphone number four.
Mic 4: Do you do your tripplication before or after elaboration?
Szymon: So currently we do it before elaboration. So we decided that our tool
works on verilog input and it produces verilog output because it offers much more
flexibility in the way how you can incorporate different tripplication
schemes. If you were to apply to only after elaboration, then of course doing a
full tripplication might be easy. But then you - to having a really precise control
or on types of tripplication on different levels is much more difficult.
Herald: Next question from microphone number two.
Mic 2: Is it possible to use DCDC converters or switch mode power supplies
within the radiation environment to power your logic? Or you use only linear power?
Szymon: Yes, alternatively we also have a dedicated program which develops radiation
hardened DCDC converters who operate in our environments. So they are available
also for space applications, as far as I'm aware. And they are hardened against total
ionizing dose as well as single event upsets.
Herald: Okay next question goes to microphone number one.
Mic 1: Thank you very much for the great talk. I'm just wondering, would it be
possible to hook up every logic gate in every water in a way of mesh network? And
what are the pitfalls and limitations for that?
Stefan: So that is not something I'm aware of, of being done. So typically: No. I
wouldn't say that that's something we would do.
Szymon: I'm not really sure if I understood the question.
Stefan: So maybe you can rephrase what your idea is?
Mic 1: On the last slide, there were a lesson learned.
Stefan: Yes. One of those? Mic 1: In here. Yeah. Would you be able to
connect everything interchangeably in a mesh network?
Szymon: So what you are probably asking about is whether we can build our own
FPGA, like programable logic device. Mic 1: Probably.
Szymon: Yeah. And so this we typically don't do, because in our experiments, our
power budget is also very limited, so we cannot really afford this level of
complexity. So of course you can make your FPGA design radiation hard, but this is
not what we will typically do in our experiments.
Herald: Next question goes to microphone number two.
Mic 2: Hi, I would like to ask if the orientation of your transistors and your
chip is part of your design. So mostly you have something like a bounding box around
your design and with an attack surface in different sizes. So do you use this
orientation to minimize the attack surface of the radiation on chips, if you know
the source of the radiation? Szymon: No. So I don't think we'd do that.
So, of course, we control our orientation of transistors during the design phase.
But usually in our experiment, the radiation is really perpendicular to the
chip area, which means that if you rotate it by 90 degrees, you don't really gain
that much. And moreover, our chips, usually they are mounted in a bigger
system where we don't control how they are oriented.
Herald: Again, microphone number two. Mic 2: Do you take meta stability into
account when designing voters? Szymon: The voter itself is combinatorial.
So ... - Mic 2: Yeah, but if the state of the rest
can change in any time that then the voters can have like glitches, yeah?
Szymon: Correct. So that's why - so to avoid this, we don't take it into account
during the design phase. But if we use that scheme which is just displayed here,
we avoid this problem altogether, right? Because even if you have meta stability in
one of the blocks like A, B or C, then it will be fixed in the next clock cycle.
Because usually our systems operate on clocks with low frequencies, hundreds of
megahertz, which means that any meta stability should be resolved by the next
clock cycle. Mic 2: Thank you.
Herald: Next question microphone number one.
Mic 1: How do you handle the register duplication that can be performed by a
synthesis and pleasant route? So the tools will try to optimize timing sometimes by
adding registers. And these registers are not trippled.
Stefan: Yes. So what we do is that I mean, in a typical, let's say, standard ASIC
design flaw, this is not what happens. So you have to actually instruct a tool to do
that, to do re timing and add additional registers. But for what we are doing, we
have to - let's say not do this optimization and instruct a tool to keep
all the registers we described in our RTL code to keep them until the very end. And
we realy also constrain them to always keep their associated logic tripplicated.
Herald: The next question is from the internet.
Internet: Do you have some simple tips for improving radiation tolerance?
Stefan: Simple tips? Ahhhm... Szymon: Put your electronics inside a
box. Stefan: Yes.
*some laughter* There's there's just no
single one size fits all textbook recipe for this as it really always comes down to
analyzing your environment, really getting an awareness first of what rate and what
number of events you are looking at, what type of particles cause them, and then
take the appropriate measures to mitigate them. So there is no one size fits all
thing I say. Herald: Next question goes from mycrophone
number two. Mic 2: Hi. Thanks for the talk. How much
of your software used to design is actually open source? I only know a super
expensive chip design software. Stefan: You write the core of all the
implementation tools like the synthesis and place and route stage for the ASICS,
that we design is actually a commercial closed source tools. And if
you're asking for the fraction, that's a bit hard to answer. I cannot give a
statement about the size of the commercial closed tools. But we tried to do
everything we develop, tried to make it available to the widest possible audience
and therefore decided to make the extensions to this design flaw available
in public form. And that's why these tools that we develop and share among the
community of ASIC designers and this environment are open source.
Herald: Microphone number four. Mic 4: Have you ever tried using steered
iron beams for more localized, radiation ingress testing?
Stefan: Yes, indeed! And the picture I showed actually, uh, didn't disclaimer
that, but the facility you saw here is actually a facility in Darmstadt in
Germany and is actually a micro beam facility. So it's a facility that allows
steering a heavy ion beam really on a single position with less than a
micrometer accuracy. So it provides probably exactly what you were asking for.
But that's not the typical case. That is really a special thing. And it's probably
also the only facility in Europe that can do that.
Herald: Microphone number one. Mic 1: Was very good very good talk. Thank
you very much. My question is, did you compare what you did to what is done for
securing secret chips? You know, when you have credit card chips, you can make fault
attacks into them so you can make them malfunction and extract the cryptographic
key for example from the banking card. There are techniques here to harden these
chips against fault attacks. So which are like voluntary faults while you have like
random less faults due to like involatility attacks. You know what? Can you explain if
you compared in a way what you did to this?
Stefan: Um, so no, we didn't explicitly compared it, but it is right that the
techniques we present can also be used in a variety of different contexts. So one
thing that's not exactly what you are referring to, but relatively on a similar
scale is that currently in very small technologies you get two problems with the
reliability and yield of the manufacturing process itself, meaning that sometimes
just the metal interconnection between two gates and your circuit might be broken
after manufacturing and then adding the sort of redundancy with the same kinds of
techniques can be used to make, to produce more working chips out of a
manufacturing run. So in this sort of context, these sorts of techniques are
used very often these days. But, um, I'm and I'm pretty sure they can be applied to
these sorts of, uh, security fault attack scenarios as well.
Herald: Next question from microphone number two.
Mic 2: Hi, you briefly also mentioned the mitigation techniques on the cell level
and yesterday there was a very nice talk from the Libre Silicon people and they
are trying to build a standard cell library, uh, open source standard cell
library. So are you in contact with them or maybe you could help them to improve
their design and then the radiation hardness?
Stefan: No. We also saw the talk yesterday, but we are not yet in
contact with them. No. Herald: Does the Internet have questions?
Internet: Yes, I do. Um, two in fact. First one would be would TTL or other BJT
based logic be more resistant? Szymon: Uh, yeah. So depending on which
type of errors we are considering. So BJT transistors, they have ...
Stefan in his part mentioned that displacement damage is not a problem for
seamless devices, but it is not the case for BJT devices. So when they are exposed
to high energy hadrons or protons, they degrade a lot. So that's why we don't
use them in really our environment. They could be probably much more robust to
single event effects because their resistance everywhere is much lower. But
they would have other problems. And also another problem which is worth
mentioning is that for those devices, they consume much, much, much more power, which
we cannot afford in our applications. Internet: And the last one would be how do
I use the output of the full TMR setup? Is it still three signals? How do I know
which one to use and to trust? Stefan: Um, yes. So with this, um,
architecture, what you could either do is really do the full triplication scheme
to your whole logic tree basically and really triplicate everything or, and
that's going in the direction of one of the lessons learned I had, at some point
of course you have an interface to your chip, so you have pins left and right that
are inputs and outputs. And then you have to decide either you want to spend the
effort and also have three dedicated input pins for each of the signals, or you at
some point have the voter and say, okay. At this point, all these signals are
combined. But I was able to reduce the amount of sensitive area in my chip
significantly and can live with the very small remaining sensitive area that just
the input and output pins provide. Szymon: So maybe I will add one more thing
is that typically in our systems, of course we triplicate our logic internally,
but when we interface with external world, we can apply another protection
mechanism. So for example, for our high speed serialisers, we will use different types
of encoding to add protect..., to add like forward error correction
codes which would allow us to recover these type of faults in the backend later on.
Herald: Okay. If ...if we keep this very, very short. Last question goes to
microphone number two. Mic 2: I don't know much about physics. So
just the question, how important is the physical testing after the chip is
manufactured? Isn't the simulation, the computer simulation enough if you just
shoot particles at it? Stefan: Yes and no. So in principle, of
course, you are right that you should be able to simulate all the effects we look
at. The problem is that as the designs grow big and they do grow bigger as the
technologies shrink, so this final net list that you end up with
can have millions or billions of nodes and it just is not feasible anymore to
simulate it exhaustively because you have to have so many dimensions. You have to
change when you inject. For example, bit flips or transients in your design in any
of those nodes for varying time offsets. And it's just the state space the circuit
can be in is just too huge to capture in a in a full simulation. So it's not possible
to exhaustively test it in simulation. And so typically you end up with having missed
something that you discover only in the physical testing afterwards, which you
always want to do before you put your, uh, your chip into final experiment or on your
satellite and then realise it's it's not working as intended. So it has a big
importance as well. Herald: Okay. Thank you. Time is up. All
right. Thank you all very much.
*applause*
*36c3 postroll music*
Subtitles created by c3subtitles.de in the year 2021. Join, and help us!