Dear viewer, these subtitles were generated by a machine via the service Trint and therefore are (very) buggy. If you are capable, please help us to create good quality subtitles: https://c3subtitles.de/talk/1083 Thanks!
[Music]
driver
development has been dominated by sea
for many many years and it sadly still
is sea is inherently flawed and there
are better Alternatives but those are
always fighting against the stigma of
being slow and cumbersome to use so
today here our speakers are going to
tell you that this is in fact not true
you can write highlevel drivers with
high level languages and they perform
very very well here to talk about this
are Paul Simon and Sebastian warm Round
of Applause
please
thank you um yeah I'm Paul with me are
salon and Sebastian and just a few quick
notes before we get started I speak
quite fast usually so sorry to the
translators and sorry to everyone
listening um if you are watching this on
a stream recording on media. cccd there
is a button to your right where you can
reduce the playback speed if you are
watching on YouTube you can also um
reduce the play spee somewhere also
please like And subscribe on YouTube
um you might have already seen that we
have quite a lot of names up on these
slides today and these are all the
people who somehow contributed to this
talk and these are all my students with
me today are Simon and Sebastian who
both did our bachor teasers with me I'm
a PhD student at Technical University of
Munich and I'm researching performance
of software packet processing
system and today we are going
specifically to talk about network
drivers while while look at network
drivers as a case study well it's
obviously our research area so it's the
next best thing to do and also user
space network drivers are all the r now
and user space drivers is where you can
use all the fancy languages I have
already talked about user space network
drivers here last year so a quick recap
of what I presented here last year is
that was the iie project and the I
project is a I think I started where I
wanted to show off how you can write a
user space driver that is actually
readable and understandable and fast at
the same time so the goal is this to be
used for educational purposes it's
around a thousand lines of C code that's
full of references to data sheets specs
and so on and um well if you want to
know more about that watch my talk last
year just a quick diff since then we
have added support for we Ionix and we
now a regant setup so you no longer need
Hardware to play around with that you
can check it out on GitHub um yeah then
I've wrot it in C back then well why
would you write a driver and C um seems
like a kind of obvious question why
wouldn't you because most drivers are
written in C and if you're going for
educational use with the driver then
might as well use the language that all
the other drivers are in it's also the
lowest common denominator of all the
systems programming languages meaning
everyone should be able to read C and I
also think that C cord can be quite
beautiful in some cases
um uh can we can we get a quick show of
hands who here thinks they can read C
code or so this is way more than
expected it was like basically everyone
raising their hands so let's have a look
at some C code um this is
some this is some actual code from our
driver that was added by a student and
when I initially got a pull request for
this from my student I was like well no
we can't this m it's like the goal was
to have readable code you can't add this
and then we discussed this a little bit
in the end we ended up adding it uh to
the code base because it's really
necessary there's no better way to do
this than this marro andc uh who here
knows who who can immediately recognize
the Marco or knows what it
does I see I see one
hand two a few hands okay not bad not
bad does it help if I show you the
actual name
anyone who who can who recognize this
Marco some more hands yeah okay but um
still this is kind of um little bit fake
inheritance or a little bit to abstract
drivers it's commonly used in drivers um
to to abstract the different drivers
I've uh MOS copied from the Linux kernel
and I've searched through the Linux
kernel sources and found 15,000 users of
this maros so it's not at all unusual to
have C code like this in your driver uh
so I then agreed to add it to the code
even though probably almost no one can
read it the actual code also has a
comment to a blog post that explains
what this Marco does um but it it also a
CH the problem that maybe a lot of
people think they can re but then they
encounter something like that and
suddenly yeah point is it can be ugly
and it can not only be ugly in terms of
how it looks or Fields while programming
it can also be ugly when it comes to
security Properties or security issues
this are screenshot from cve details.com
no I don't expect you to be able to read
all these figures but basically it shows
there are security bus in the Linux
kernel this is specifically all the
Linux kernel bus found in the last 20
years or so
and now you could say well why is the
language to blame you can write bad code
in any language and what we now would
have to do is basically we would have to
go through all these buck and check
whether they uh could be prevented by
using a memory save or better language
and it seems like a lot of work but
luckily someone else already did that um
last year there was a paper by Cutler at
all who developed a um operating system
in go and they looked at all code
execution bus in uh think 2017 in the
Linux colel and looked through all of
them manually and tried to figure out
whether they could have been prevented
by using a different programming
language for 17% of them they weren't
sure um because it's a kind of weird bu
and they couldn't tell whether that
would could have been prevented for 22%
that were clear logic bu that's clearly
you would have made that bu probably in
any language other language wouldn't
have helped but for over 50% of the bus
were related to memory both use after
free outof bound accesses and so on and
these can be prevented by using a better
programming language now these are 40
preventable bucks uh in this case study
and we looked at them to figure out how
many of these bugs were in drivers
versus bucks in other parts of the
kernel because we are specifically
looking at drivers well 39 of them were
on drivers um the other one was in the
Bluetooth deck and of the drivers the
qualcom Wi-Fi driver had 13 bucks um yes
I know I was shocked bux in the qualcom
Wi-Fi driver who would have
thought now based on these results uh
should you be writing new code in C in
2019 well probably not if you have a
choice but you just often don't have a
choice if you're writing some colel code
for some reason sure you can write a
module in Rust but good luck getting it
uped and other languages like good luck
writing a colel module in JavaScript
that's probably not going to work and if
it's going to work it's probably a bad
idea now this is why we looking at user
space drivers because they can be
written in virtually any language so we
are not constrained by any environment
here and the the question would now be
are all the languages equally good
choice can I do it in any language
should I do it in any language which
language should I use is a jit comp or
garbage collector problem in the driver
now I initially wanted to like write one
driver in one high level language to
have a case study and then extensively
evaluated but then I thought maybe it
would be a better idea to write drivers
in all the languages but it turns out I
don't speak all the languages but
luckily I could recruit the help of a
few students and this is a screenshot
from um my website at the University
where I do like announcement for tees
and I just added um announcements for
writing Network in Rust in go in Java in
C in husk and so on
and then at first my my colleagues at
the University looked at me in weird
ways and they're like are you serious
you realize this the same announcement
all over again some of them still can't
tell if I'm serious about this but um
yes I am and I got a lot of response
from students I think I talked to a
total of 30 students or so who wanted to
do one of these te and then um these two
were one of the first ones to talk to me
and I tried to scare them all away
initially I told them all it's going to
be really hard you can get an easier
teas it's probably not a good idea you
need to know a lot of lowlevel stuff and
so I scared away 20 of the 30 I talked
to in the end we did um 10 te and yeah
quite nice results so far a few are
still ongoing but I hope we will have 10
different languages for drivers um
soonish I think finished uh depending on
how your count finished is six or seven
and yeah also turns out giving a talk
here is a really nice way to recruit
students because uh a lot of my students
uh mentioned that they saw my talk and
um contacted me afterwards for teasers
um now what did I tell my students how
to how to go ahead to to write a driver
um well I explain to them basically the
very basics of how to write a user space
driver how to talk to a modern PCI
Express device and what you need to do
basically there are three different ways
to talk to Modern PCI Express devices we
are ignoring a few Legacy things here if
you like we old IO instructions not on
here sorry um first way to talk to the
device directly simplest way is memory
mapped IO memory mapped IO is just a
magic memory area that is mapped to
that's mapped into your process and
directly goes through to the device and
if you read write that memory the device
gets the request and can reply to it
that's usually used to expose device
registers on Linux you can just mmap a
magic file via the uio framework and
then you have access to that from your
uh user Space Program second way is kind
of how the device talks to you or how
the device talks to the rest of the
system with direct memory access that is
just a way how the device can read and
write arbitrary memory locations and for
user space drivers you just have to
figure out where our memory add space is
mapped physically then we can tell the
device to write something there and then
it will just show up in our process
without us having to do anything in the
in the kernel and the Third Way are
interrupts um we will not be using
interrupts here because we don't need
them for a high-speed Network driver but
um let's I it on here because sometimes
people say well you can't use interrupts
from a user space driver that is
incorrect you can use the vfi sub system
that has full support for interrupts but
we won't be using them here now what did
I tell my students about how they should
go to write their drivers well basically
they should just remove the current
kernel driver do the magic m call on the
on the right magic file then figure out
the physical addresses and then just
write the driver it's it's really easy
so
um then um we have a lot of hardware at
the University and I gave them all
access to servers with 10 GB network
cards of the Intel I GBE family this is
a really common network card that you
will find on a lot of servers like the
default goto 10 GB network card on any
servers you will often find it on board
or embedded in some of the CPUs even the
nice part about this it has a very very
nice data sheet publicly available that
documents basically everything um fun
fact we found it easier to program
against this Hardware blackbox uh with a
good public data sheet than we found it
to implement the vid iio spec which is
the open specification has several open
source implementations but yeah
um then the this network C is a little
bit older it's 10 years old or and it
has a nice property that it's still very
low compared to newer Network CS if you
implement this on newer Network CS you
are usually just exchanging messages
with some fmw and that's just boring
because the firmware implements
everything sure the older network cards
also have some firmware but you get a
lots of more lowlevel access to the card
and you you don't feel like you are just
talking to a FM where you feel like you
are implementing an actual driver
yourself so now um these were the basic
things I I told my students I will now
give over to Sebastian um who will show
a little bit of C code
about um well how to write a code how to
write a driver in and whether that could
be done in a high L language thank you
very much Paul um so uh he just showed
you um what we have to do in general I'm
going to take a bit more detailed look
at this so uh first of all of course we
have to figure out our uh PCI uh
addresses um you can do this VI
lspci um and then you get a list of all
the PCI devices
and you just look for something like
Intel Corporation 82599 uh something
that that looks like this and uh then
there at the front of the line um
there's an address looks kind of similar
to a MAC address and well that's the
address we need um so now we got the
address with this address we can uh just
go there and unload our driver that's
basically it you just ride there um Next
Step up we have to map our PCI register
from uh the address
space and for that we yeah we open our
magic file and execute an map um The
Challenge here is mainly that uh for
every high level language you want to do
this in you have to have some kind of
way to actually use an map either via
library or in the worst case you just
have to use some C code for
that um next thing
uh this is an uh extract of the data
sheet uh in the data sheet yeah it's
basically like this uh you have the
register names register offsets and then
you can just uh go through the whole
document and you find all the registers
you need to uh read and set things uh
for the driver and yeah a quick example
um Network CS have LEDs often and we can
make them blink turn on and off and how
do we do this well we have to to get the
uh Base address of our registers at the
offset that we found in the data sheet
and then just switch the bit and this
turns the light off and on and yeah
write it back of of course um to the
register uh also this is one of the very
few uh valid uses of volatile and C um
because we have to really prevent
compiler optimization here as uh we have
um memory access uh from multiple
sources so yeah
um next up how do we handle packets via
dma um packets are transferred via uh Q
interfaces uh receive cues transmit cues
and these are often called Rings because
well they are organized in a ring like
structure um and uh these rings are
configured via uh mmio and uh when this
is done we can access the device via dma
uh these Rings uh then usually contain
pointers to package packets um and these
packets are then also accessed bya
dma um the details vary a bit between
cards uh and devices but um that is not
unique to NX this uh process this is
pretty much the same thing for all pcie
devices they all kind of work
similar so what are the challenges for
highle languages we have seen some for
example the map call um we somehow have
to get access to this with with the
proper Flags um another thing we have to
ex handle externally allocated memory um
and the layout this memory comes in um
also we've seen we need volatile to
prevent compile optimization so we have
to uh have some kind of semantics uh so
we can enforce this in other
languages and of course um often
especially for lowlevel stuff
uh you often have to use some kind of
unsafe code for uh highle languages
because well many operations are
inherently unsafe when you uh operate
here um but we try to contain this
unsafe code to as few places as possible
um uh to uh contain this to a small area
um okay now uh some basic challenges um
I'm going to hand back over to Paul who
says something about the goals for our
implementations okay um basically this
is what I've told my students about what
I'm expecting for these implementations
I wanted to have the same feature set as
my C driver which was kind of used as
the reference implementation and it also
was supposed to have the similar
structure but at the same time we wanted
to have code that looks like it was
meant to be written in that language
it's always a difficult trade off to um
well use safety features wherever
possible but if they cost too much
performance do we really need them in
all the places where can we use the
safety features where do they make sense
and then we wanted to quantify that the
idea is to really have in the end like
10 driver implementations that we can
quantitatively evaluate and look at the
performance of all of them look at the
safety properties of all of them look at
properties like safety for memory
accesses is guaranteed for packet
buffers yes no for other stuff yes no
and so on and I'm now going to look at a
few of these languages basically I'm
only picking a basically I'm only
talking one or two minutes about each of
the languages for students who are not
here but have already finished their
teers and then we are going to have a
deeper look at go and rust um but I will
now start with the the other languages
where it's only a very very short high
level overview well first one is C which
seems a little bit unusual but we found
a student for that so why not
um and no we didn't develop a driver for
Windows um Microsoft Corr is available
on Linux on works really well for those
who don't know c c is a Jus in time
compiled garbage collected memory save
language and it has a relatively obscure
or rarely used unsafe mode and unsafe
mode features full support for pointers
so um you can basically write code that
looks like C you just have to compile
give the compiler a special flag to tell
that hey I'm going to use unsa stuff
um then how can we access external
memory or F memory there's for example
there are a few nice reppers in NC Shar
this P invoke or the unmanaged memory
stream and so on but that turned out to
be slow too slow for our implementation
so we use that unsafe mode which
basically looks like this you see the
unsafe keyword here and the other stuff
it just looks like C and it also feels
like C when you are writing it so um
it's a really nice language to the
drivers and again the unsafe code is
contained to a few well-known places
that can then be audited compared to a c
driver where the unsafe code is all over
your code base and you don't know where
the bug is here we know if there's a
buckets probably in there that we have
while not checking the buff size
properly or something and okay that's
all it for C another unusual language
for drivers is uh Swift um also found a
student who mentioned well I can of do
it in Swift I was like oh I didn't even
think of Swift um sounds like a good
idea yeah um so no we didn't develop a
mechos or iOS driver Swift is also
available on Linux Swift is a compiled
language that's compiled via lbm memory
management is done via reference
counting there is no garbage collector
and it's mostly memory safe now again
have to use some kind of pointers um
there's unsafe buffer pointer there's
also unsa Ro pointer and more classes um
and we are using these things to make
packets that are stored in the dma
buffers available to the um application
using the driver and for example here's
a property that reps some memory in a uh
unsafe buffer pointer wrapper and the
unsafe buffer Point weer forces you to
specify uh how big the buffer is and
then it does the bounce check for you
when running in debug mode um then
pointers are kind of a little bit more
verose compared to C um but basically
you can use them like like pointers and
operator overloading here can help for
some of the other managed um buffer
pointers uh War buffer pointers there is
already an overloaded operator so it
looks like an array access but yeah um
totally possible to do that in Swift
even if you're uh writing a driver
instead of a uh UI application
now for the fans of functional
programming we have a um fully working
implementation in okl okl is a compiled
language that has garbage collection for
memory management it's also memory save
and it's the first functional language
we are going to look look at and one
nice quick feature of okl is the seuck
library the seu library allows you to
specify memory layout uh like this kind
of looks like a well C struct and then
it generates from that code for accessor
and there is really nice compared to the
Swift example where you would have to
hard code the offset somewhere and it's
much nicer to have this kind of code
generator here which uh does the right
thing for you and um also can do
automatic engine swapping and so on and
then the code and okl now looks quite
different compared to what you might
expect from a driver code and this is
for example just a function that counts
how many packets have been received in
the receive ring by checking the flex
and we can see here that we are using
the uh get ATX WB status which is the
geta function from the previous stock
declaration we're checking a flag in
that and then we are just counting the
packets then we know okay we have
received 10 packets or whatever since
the last call to that function and we
can now pass them on to the user of our
driver um more functional programming we
also have an implementation in hcal hcal
again compiled language memory
management via garbage collection it's
memory safe and also
functional few nice or few nice features
of Hill that you might not know about is
the um system Pro memory package has a
lot of lot of really helpful uh
functions this is um compared to to emel
where we had to write some C code to get
map and mlock with the right Flex
working and um here that was everything
available and the foreign package has uh
nice functions like Peak bite poke bite
and so on where you can just do a memory
access and another thing we are using in
hcll quite a lot are the some types
because uh a lot of things in the
drivers are where you have in C a union
there's basically this this part where
you write some data in a dma buffer in
some format the device reads this data
and the um like for the transmit packet
descriptor we have the transmit read
format which looks One Way once the
device has transmitted the packet it uh
goes back to the same memory location
and overwrites with a different thing so
we then need to read the same data
something else and this basically a
fancy C Union and this is um little bit
nicer to work with than the D unions
okay these were the languages um of
students who are not here this year um
and you can check out all the
implementations on on GitHub there will
be a QR code to scan on the last slide
and I'm now going to hand back to
Sebastian who is going to do a deep dive
into go yeah thank you very much uh so
now you've seen few languages next up is
go um what is go go is a compile
programming language it's been developed
by Google it's uh generally uh it's
generally purpose language but as it's
been developed by Google it's mainly
designed for distributed systems because
that's what Google does distributed
systems a driver is not a distributed
system so why should we even use go well
uh go does offer a few things that are
quite nice um it has a runtime that uh
has a garbage collection and also
enforces memory and type safety uh also
uh it has very large standard Library so
we don't need to use any other code
except uh standard Library code um so
how do we program drivers and go
actually in most cases it's just like C
um there are a few main uh differences
though um uh on one hand uh we don't
have pointer arithmetic we have pointers
but we don't have arithmetic um and this
is what we need for managing our dma
memory um and the second point is we
don't have volatile uh for the memory
barriers uh for register accesses
mainly um so what do we do instead to
compensate for that uh first of all we
can manage the uh dma memory via
slices uh that's pretty easy and the
second thing is we can use unsafe
pointers for pretty much all the rest um
unsafe pointers are arbitrary pointers
um so that's good but they do circumvent
around time so we have to be careful
with them um and this is what we use for
uh mainly physical address calculation
and register access uh and there is also
a rule set for um uh unsafe pointers so
they are still valid and of course we uh
we follow this rule set um so two quick
examples first of all mols uh uh here
you can kind of see how we manage the
dma
memory um first of all we allocate the
dma memory initialize the mempool and
then um this mol. buff this is actually
the the whole um memory mapped area and
we can just uh Subs slice this into
packet buffers so you see that's pretty
easy um next thing is for physical
address calculation this is where we uh
need our unsafe pointers uh for the
first time um because we have to
translate our virtual addresses into
physical addresses that the network card
can then use to actually send and
receive packets um and for this uh
because the run time uh checks many
things and uh it uh and you have to
explicitly convert types uh you first
have to convert your um uh pointer uh to
an unsafe pointer and then you can uh
convert it to an integer type the UN
pointer and uh yeah it's it's uh uh how
you do this in go um also no volatile no
problem um I said we need a volatile um
as we share registers with the network
card and we need uh some kind of
compiler memory barrier to prevent
reordering um we don't have volatile and
go but uh syn Atomic functions do
prevent reordering among them amongst
other things they provide stricter
guarantees than uh volatile um but yeah
uh it doesn't really cost us any
performance here so we're just going to
use that and um yeah so we can use
Atomic store and load for integer types
um to uh then get our memory barriers so
as a
conclusion um I thought go was actually
quite nice to work with um safety
properties have improvenet adal as Paul
said before uh wrote a kernel Ino um and
uh yeah so it is safe uh it it gets you
some safety guarantees um and the other
thing it's kind of a personal opinion
but I think it looks like C code in
beautiful um but it also has downsides
in the best case it's approxim
approximately 10% slower than C and um
if you get to more suboptimal cases uh
it's even worse compared to C um but
yeah we have to live with that also the
scripto access can be a bit ugly but
well as long as it works so um next I'm
going to hand over to Simon who did this
in Rust thank you Sebastian so let's
talk about rust what is rust well uh the
rust website says it is a safe
concurrent and practical systems
language sounds great that's exactly
what we need to write a users based
Network driver so uh is there anything
else we should know well yes uh rust has
no garbage collector so we have less
overhead for memory handling um it has a
unique ownership system and some rules
for moving and borrowing values um so
with these uh rules we can accomplish R
goal of memory safety and we have unsafe
like in other languages presented before
um what is the ownership system well it
is the core feature of rust um actually
it's it's basically just a set of three
Simple Rules uh rule number one each
value has a variable that is its owner
rule number two there can only be one
owner at a time and rule number three
when this owner goes out of scope the
value is freed um these three rules uh
combined um with the rules for borrowing
values keep our safe from memory Buck
like double freeze and um as they are
inforced at compile time uh we don't
have any performance penalties at
runtime so our programs are similar to C
programs but uh we have the great
advantage of memory
safety um what does it look like in our
implementation well um we have packet
structs for our Network packets um that
own dma memory and these packets are
passed between uh the users of our
driver and our driver and with them
along ownership is passed as well and uh
that's pretty cool because uh when the
packet is passed to the user only the
user can modify the packet and the
packet content and when it's passed back
to the driver only the driver can modify
it so we have uh basically um safe uh
packet handling um unlike in other
languages and um at the bottom of the
slide you can see um how how you would
use the driver based on our driver's
interfaces you can see um how to receive
modify and send packets and um yeah
there's no way to screw up so uh for
example you cannot forget to free
packets because uh packets are freed
automatically when they go out of scope
and uh are return to the memory pool of
our driver so um yeah this is safe code
and and there's nothing you can do wrong
here um but unfortunately we also have
unsafe code in our in our driver um what
is unsafe code well not everything can
be done in safe rust for example calling
forign functions and D referencing RW
pointers is unsafe but uh this is
nothing unusual um the idea is to reduce
unsafe code to a few places and do some
checks to make unsafe code
safe what does it look like in our
driver well um for example we have this
set register meod a method to um Set uh
the registers of our device and um we
use pointer right volatile to write to
some uh register of our device and
before we do that we have some assertion
in our code um to assert that the uh
address we are going to write to is
indeed inside of the map memory
region so we have we have some great
code but is it fast well uh to find uh
that out we set up a test bed to
Benchmark our drivers um we have two
servers uh package generator and the
device under test they are connected bir
directionally with two 10 gbit per
second connections um we use the moonen
moonen package generator written by Paul
because obviously it's the best package
generator
um
um yeah and uh on the device under test
uh we have a simple bir directional
packet forwarder that we Implement that
we implemented on top of our drivers in
all languages and yeah so let's look at
the results of our measurements um so
this is a graph showing the throughput
of our forward on the x-axis you can see
the CPU speed on the y- AIS the packets
per second um we look at uh packets per
second because uh the main overhead is
per packet not per bite and um the top
of the y-axis is uh 30 million packets
per second because that's about uh 20
gigabit per second at minimum size
packets um you can see the different
plots for the different languages um
it's linear scaling with the CPU speed
uh rust is the fastest Swift is yeah
well it performs incredibly poor and um
so the thing is uh usually you you don't
manually change your CPU speed so we ask
ourselves is is there anything else we
can modify
um yes there is you can uh change the
the batch size so um how many packets
you send uh at once to the PCI device um
because that uh avoids some overhead and
um Kel drivers usually use a batch size
of one on
transmission um and higher batch sizes
are one of the main reasons why user
space drivers are faster than kernel
drivers uh batch size 32 to
64 packets is a very good batch size
because higher batch sizes um are bad
because we have more L1 cach
misses and well we asked ourselves why
does uh Swift perform uh that bad and
Paul is going to tell you why deser that
yeah so if you have some program that is
performing not as expected well what do
you do you run some profiling on it and
then you get a lot of data and then you
need some way to visualize this a common
way to visualize profiling results is
this thing called a flame graph
basically the x-axis is time spent in a
function and the y- axis is the depth of
the call stack and no I don't expect you
to be able to read all of this these are
just the function names that are in here
and if you look at the topmost functions
here these are the leaf functions where
the time is then actually used and we
can characterize all these functions
what it's doing in there and um well we
found out it's due to Swift's memory
management
Swift adds calls to the magic internal
release retain functions for each object
used in each function um just to keep
track of the reference counting and
that's well basically no problem if
you're just writing a UI up with some
buttons or something but if you are
writing a driver that has to pass
through millions of millions of packets
to a lot of functions all the time then
it turns out it spends uh well 76% of
its time in these release retain calls
um so it could be four times faster if
it had other way to manage memory and um
for comparison in go we spend less than
half a percent in the garbage collector
because it's well a quite simple
application so now the big advantage of
having the these uh semantics with
reference counting in Swift is of course
that there are no unpredictable pause
times and the garbage collector and go
might just stop your driver for all the
time or for some time um the question is
now is it a good idea to have a garbage
collector in something like a driver
now the good thing is we can measure
that because we have this forwarding
application and we did that we um
measured the latency of all the packets
that we forwarded this latency was
measured at around 16 million packets
per second and what you see here is the
cumulative distribution function of the
latency of rust and this is basically a
almost perfect normal distribution
centered at around 8 microc which is a
very nice result uh for comparison a
hardware search takes around 1 microc to
forward a packet um but 8 microc is for
a software forwarding thing is really
nice and fast and now to the same graph
let's add the other languages and now we
see well goes kind of similar but C is a
little bit slower at the top um but I do
realize that this graph might look a
little bit confusing so let's really
quickly explain it how to how to read it
if you're not familiar with these cdfs
now we um look at a value at the y- axis
for example5 so 50% then we go over to
the language go down to uh the xais and
that just means that 50% of the packets
take less than 8.9 microc to be
processed with c and the other 50% take
more so when looking at any latency
where a garbage collector or
unpredictable spikes are involved then
it's always a good idea to not look at
the median but at something like the
99th percentile
and for C 1% of the packets take longer
than 30 microc um and 1% of the packets
is a lot if you're processing a lot of
packets it's like one in a 100 packets
and you're doing millions of them per
second so you are going to get these
worst case latencies quite often what we
really want to know is not the 99th
percentile but the
99.9999 percentile or something like
that so we will need to zoom into that
graph here if you're are zooming into a
graph you usually change the exis to be
logarithmic to zoom in well in this case
it would zoom into the wrong part of the
graph so we also have to subtract the
exas from one yielding the complimentary
cumulative distribution function on a
logarithmic axis now this is inverted
and a little bit a confusing graph but
um this is what you would see in a
academic publication talking something
about latency or anything like that but
I think it's a it's a really confusing
graph but you can quickly see the the
percentiles um and the the top top uh no
the the bottom line would be the latency
of one packet in a million packets um
what we can do with this graph to make
it a little bit more approachable is we
can basically just rotate it and change
the axis description of the x-axis which
was the y- axis before and the x-axis is
now the percentile and the y- AIS is the
latency and now it's easier to read we
can for example look at
99.99 and um check out where it uh which
Laten at this percenti is for the
different languages and this is a kind
of graph that you will see for a lot of
latency evaluations of anything that has
latency spikes if done properly um you
unfortunately often see people doing
latency evaluations and then providing
like the average or median latency which
is from many cases a completely used as
value um most of them probably just
don't know better but if you want to
evaluate latency please have a graph
like this in the end um there is if you
want something to Google there is the
library called HDR histogram which which
can generate these graphs um from
latency measurement data and that's just
a really nice way to characterize
garbage collection or just in time
compilation latency or anything like
that now we got a driver that is nice
and fast and has a relatively low
latency for for most languages but we
have not yet really looked at Safety and
Security beyond what was offered by the
language um because our driver still
runs as rout like all or virtually all
user space drivers run as root by
default and well why is this the case
well I've shown the this code before um
there are a few operations in the
initialization that just require you to
be root like mapping the PC Express
resource requires you to beot for
implementation details in the Linux coin
we need nontransparent hugee Paces for
the dma buffer they require you root to
allocate them um locking memory requires
so these are clearly all functions
related to initialization or setup so
the obvious idea is well WR a small
program that does that for us keep that
simple and out of that and check that
it's good and then drop all the
Privileges can we do that um yeah sure
we can do that uh it's relatively easy
just drop privileges after uh setting up
memory but that's still not really um
secure in any way uh even though you're
now running um as an unprivileged user
and to understand why this does not work
as you might want it to work we have to
take a high Lev look at how memory
access works on a modern system and this
is what a modern system looks like at a
well very high level view we have the
CPU here at the top with our application
running on it and in the bottom left we
have a PC Express device the btom right
we have some memory and if we now want
to do some memory access from our
application it goes like this it goes
through a thing called the mmu the mmu
is the memory management unit and the
memory management unit translates your
virtual address that you have in your
program to a physical address that can
be used by the memory
controller now the security or the
isolation between processes is
controlled by this mmu and only the
kernel can reprogram to the mmu to
guarantee the
isolation now if we want to access our
device via um PCI Express for example
memory M.O it also goes through the mmu
the mmu then knows this is not going to
memory but to PCR express it talks to
the device and um that's also all fine
now you could argue clearly these are
the two kinds of memory accesses we do
both are checked via the mmu so what's
the problem well the problem is that
when we tell the device to do something
with memory I've previously mentioned we
tell the device uh physical addresses to
use them and well of course the device
does not go through the mmu or uh
because it doesn't know it doesn't have
a concept of it's used by a process or
it's being used by this kind of process
it just has full access to all of your
memory and so if you somehow own the
program um that is running as an
unprivileged User it's quite a trivial
exercise to get any data from the
important area that you shouldn't be
allowed to access or write any data
there by just telling the device to do
it for you meaning any application that
has direct access to a PCI Express
device is effectively rote even if you
drop privileges so the obvious solution
is to somehow make this access pass go
through the mmu and there is this fancy
Hardware component called the io mmu
which is exactly for this use case this
can be found on any modern CPU that has
hardware virtualization features because
it's mainly being used to do a pass
through of PCI Express devices to
Virtual machines in a safe and secure
manner but you can also use it for user
space drivers you just need a way to
tell the kernel to configure the io mmu
with the proper restrictions for us and
then the access looks like this and we
just need to configure it in a way that
the io mmu uh has the same permissions
for that device like the mmu has for our
user space program and then it's
perfectly um secure if your program gets
owned attacker doesn't have any
privileges um beyond what your process
has and uh this is also useful for
safety because when I initially wrote a
driver I kind of killed server when I
apparently misconfigured the dma and it
overr something that was important for
the file system apparently and then it
was dead and I had to reinstall it so
that wouldn't have happened if I had
started with Theo mmu in the first place
um how do you do that um to to use theu
specifically on Linux which has a night
nice subsystem called the wef iio
subsystem and um this what we can use we
just need to prepare the system as a
root this is a one-time step as root we
can can bind the device to the vfio
driver instead of to no driver at all we
can change the owner of the resulting
magic device file and pass it to an
unprivileged user then we have to give
the unprivileged user permissions to
lock memory to allocate dma memory um
but we can also restrict them can like
you are allowed to allocate 512
megabytes of locked dma memory and then
all the remaining steps can be done as
an unprivileged user the unprivileged
user can now call an map on the um New
device and then the unprivileged user
can communicate with the kernel VI IO
control commands on the magic device and
it can also tell the colel to please
allocate dma memory um is also actually
a better way to allocate dma memory for
technical reasons um and then you can
just use the device as you had before so
you just need to change the setup steps
and then you have a user space driver
that can run with well no special
privileges at all and the IU will check
all the accesses and the colonel will
make sure that you can't configure the
IU in a wrong way you can just tell it
to please configure the i omu in such a
way that the device can access my
current address space but not anything
else and we have implemented this in our
C driver um the my student who
implemented this is actually here today
but he was afraid to come up on stage
but if you have any questions you can
talk to him afterwards there
um now we have this awesome driver which
is say secure and everything and yet
some people still argue user space
drivers are useless I already have a
driver in the kernel why do I need
another driver why would you want to
Rite a driver and obvious answer to the
question is why wouldn't you want to
write a driver it can be fun I had a lot
of fun when I WR the the first driver so
um yeah then maybe you just need a quick
and dirty driver for a weird device that
you found somewhere maybe you just need
quick development Cycles where you want
to um don't want to shoot down your
colel all the time or reboot all the
time if you have a weir device that you
are developing this maybe you're
developing a custom device maybe you
have some fpga BO that you want to talk
to real quick without being involved in
some Kel stuff or maybe you just need a
feature that's found on the hardware but
not yet on the uh device driver there
are some features there is like a like a
lot of stuff in the past you have infent
IP offloading which wasn't in the open
source driver uh and so on but something
we have done recently is Hardware time
stamping the latency measurements I
showed before required us to take time
stamps of 15 million packets per second
with nanc level position and that is
quite hard and people usually use
special hardware for this and we
ourselves have used net fpgas in the
past SS which um well can be a lot of
fun to use these uh nice devices but at
the same time it's prohibitive from a
cur perspective or from a user
experience perspective for some people
who just needed to take some uh time
stamps so we want to do it on some cheap
cheap off the shelf commodity network
cards and turns out a lot of these or
some of the newer cards have a hardware
feature that can just add a time stamp
to the incoming buffer of all packets
that they receive but sadly none of the
existing drivers uh supported this
feature but I've shown you how you can
access registers you can just skip the
step where you unload the original
driver you can just poke the just map
the thing while the original driver is
running poke the right register in the
right way tell it to do this please and
now you got a time stamp at the end of
every packet sure it probably breaks
your uh TCP stack or whatever if there's
extra data it but like pcap doesn't care
or in this case you just did it with
dpdk and got a ra packet buffers and it
also doesn't care if the device thinks
there's four extra bytes or eight extra
bytes um for this setup that we had here
we used the embedded Nick on our xon um
and we had a fiber optic splitter to
sample time stamps of all the packets
before and after device under test and
that's dealed us with this measurement
of all the things and um
yeah so this is just a simple use case
why you would want a driver to conclude
I can only really say that I think
drivers should be written in better
languages now I think you shouldn't be
start writing a new user space driver
and C noway sadly if you look at the
world of user space drivers there is
mainly dpdk which is a network user
space driver is all C there's well it's
all C because it's mainly copy pasted
from um col drivers um then there's spdk
which has nvme drivers also all C and
the big exception to this is snap which
has drivers in Lua which is quite nice
um and well our implementations which
have lot of different things and we
really want to compare these other uh
these languages you can scan this QR
code or just Google for XY languages on
GitHub or or whatever and then there's
like this meta repository that has links
to all the implementations has will have
a link to the talk um and yeah yeah
basically check it out write your own
driver no colel code needed thanks for
your
attention thank you very much Paul Simon
and Sebastian we do have time for
questions please line up at the
microphones and to get it started it
please a question from our signal angel
from the
internet so the IRC first of all was
wondering why was The Bash proposal only
offered as a bachelor
thesis uh yeah the The Bash question
kind of expected as like I was adding
random language and thought well why
can't you do it in bash um the the
problem is I tried to implemented in
bash it doesn't work the way I wanted to
to work I would need to write more C
code the way I tried it it does not work
is I um wrote a short C program that um
called the map thing and then just slept
forever and then the idea was to get the
to access the address space of that
program that just called the map via the
procfs system and then there's like this
magic thing which has all the things and
then with DD you can write and read from
there but it breaks at some point
because you can it goes through when
reading but if you write something at
some point via Pro that doesn't go
through the PC Express so didn't work
and um I only wanted as a bachelor
teasers because kind of a master should
maybe be a little bit more serious than
a joke
driver another question from internet
yeah maybe a more serious one um Gordon
is asking how do you use the chest to
handle interrupt requests or a code with
strict timing
requirements um if I have strict timing
requirements I'm not going to use
interrupts interrupts are horrible um if
I have strict timing requirements I just
pull the device all the time this is how
basically all user space drivers work
they just ask the device is there a new
packet one million times a second um
interrupts are one of the slowest way to
communicate between CPU and Device just
receiving the interrupt requires you to
do a contact switch on the CPU then you
have to do a contact switch back because
you don't want to do too much in the
interrupt Handler and then you have to
pull the device anyways because the
interrupt just tells you that something
has changed so if you really care about
the latency then you just P it device
all the time and for for user space
interrupt check out um vfio the vfio
framework you can do like an EOL on
something and then you get notified if
there's an interrupt if you really need
that okay let's go to the hall questions
uh please keep your questions to one
sentence only and only ask questions
because there's many of them microphone
number two please so when you compare
different user space um drivers in
different languages why was uh rust
slower than C given that that it the
memory safety is compared
to yeah well we have a few more memory
operations because of the safety so um
we we have to um move the the packet
structures from inside the driver uh to
to outside to the user and uh you don't
have that in see so rust was a bit
slower because of that but I think you
could optimize it maybe a bit better
because yeah well it was just a thesis
so I I I didn't have that much time but
yeah I I guess it could be a bit faster
but um I think it would still be a bit
slower than C uh also my C driver
doesn't do any bounce checks at all it
just assumes you're doing the right
thing microphone number three
please um you mentioned Hasler in the
beginning but it wasn't in the
comparison can you talk about that um
hll is not yet optimized for performance
it would have been unfair to add a
unfinished version for it it's currently
still quite slow and I I didn't want to
add it there microphone number four
please awesome T awesome T thank you
very much uh have you considered using
total programming languages like Idis or
cck where your compiler can check the
logic of your
driver um I don't have a student for
that to implement it
but yeah microphone number one please
okay I have SE several language with the
garbage collector included including GH
so my question is
how often do the GC stop the war happens
and what is the general Heap
sizes um how often I don't know how
often um the latency showed you how long
it takes uh which was like up to 40
micros seconds which depending on your
application might or might not be a
problem there's a few other data from we
we referen this paper of the the guys
who implemented the whole operating
system in go they mention they see up to
200 or 300 microc of of pause times and
Heap size uh to be honest I didn't think
I really measured Heap size uh maybe
something about um uh the garbage
collector um I'm not even sure if how
how much it actually works because the
go profiling just uh dropped this note
as it was so little time compared to the
others so it's basically uh irrelevant
except for probably some
latency microphone number three please
yeah I missed a couple of languages um
uh opencl and Cuda not not because
they're particularly interesting
languages but it would be interesting to
have the GP direct to the network there
is a paper out there that's called
packet Shader um
think yes they do exactly
what there's also another paper out
there called raising the bar for GPU
packet processing or something like this
they basically argue against that the
the main problem is if you are
transferring the packets between the
network card and the GPU this is kind of
slow and you need to use gigantic batch
sizes the the package Shader guys used I
think batch size of 4,000 or 8,000 which
affect latency and they don't have a
proper latency evaluation wonder why um
but if you're interested in GPU packet
processing read that packet Shader paper
it's a few years old thank you
microphone number one please so uh how
do you deal with iio ordering uh the x86
will guarantee you that uh this order in
which the CPU posts IO accesses is the
same as uh the device receives them but
on other platforms this is not the case
um yes and if you do it with uio the uio
doesn't give you any such
guarantee um well for for memory
ordering it's highly specific to the
device you're using which um memory
ordering semantics you're using the the
weird thing is in the in the device we
are using here the the Intel device in
the there is one one location where I'm
so so really sure that I do need a uh
release memory order semantics because
there's something which clearly sets
some flag and then the device with some
other memory based on this and I'm like
99% sure that there should be released
memory order there but none of the
driver implementations has any release
memory order barrier there um so we
don't we don't need it here and um other
than that for a go and rust we of course
have lowlevel Primitives to to Ure or to
enforce um Hardware memory barriers for
other languages um if you for example
check out the Snap driver which is
written in Lua they have a little C step
which calls to WR ense instruction at
the right uh place thanks number four
please uh thank you um the abstract of
this talk mentioned that um the user
space c implementation uh was uh 6 to 10
times faster than the inernal
implementation was this just because of
the batch sizes or were there other
reasons for that
so this is um mainly due to the to the
batch size the Cur implementation when
using xtp is um also faster so was
compared to Kernel without xtp kernel
with xtp I think we are 30% faster or so
if you have a konel that can XT do xtp
between different Nicks because most
Nicks can just send back out different
different topic but yeah next question
again from the internet so R is asking
have you considered writing user space
drivers for the more inherently in
secure due to specification complexity
things like the Bluetooth stack or
things like
this um yeah would be would be
interesting
um yeah I guess that's a good topic for
for the research we just went for
Network because it's really common to
have um user space Network stuff all the
dire now like look at your iPhone or
whatever there's a user space TCP St
running on it so yeah number three
please hello thanks for the talk just a
little question does the IU affect
performance um we are not yet sure um
you can ask him afterwards as far as we
have evaluated it it does not yet affect
performance there's also a paper called
um what's it called pcie bench um pcie
bench something um sicom last year I
think and um you can read this paper
they have a performance evaluation of
the IU so yes there is some effects
because the uh TB size is smaller on the
IU but we couldn't measure it in our toy
setup yet microphone number one please
hi um when you access ring buffers you
usually have to have like a an EXs On's
macro that you use to access the memory
in C because C is actually allowed to
change the semantics of the
implementation in a way where instead of
say reading memory and storing in a
variable and then using that variable
would translate into something that
would read the memory
twice and um this is this was a problem
was xsa 155 I believe um where in
netback net front the communication
actually broke in in that way because
you had to talk tub um how would you
enforce like a single memory access and
all of those programming languages
enforce is always a hard word um the
thing is most of these programming
languages do need to copy the descriptor
when using it so the the ring stores the
descriptors the descriptors are
basically pointers to some other buffers
and the critical part is when reading
the descriptor and I think all of the
language implementation copy the whole
descriptor which is 16 bytes okay so the
memory is always copied
guaranteed I hope so um I mean please do
check the in the in the go code we had
the um the the atomic read thing and I
hope that only do one access and
otherwise it would consider a bu and go
if my Atomic read is does it twice and
see I also copy it I think we have time
for one last question from microphone
number two please um how can we convince
other people especially other developers
and the business people of the necessity
of moving away from C learning a new
language and investing the time of
developing stuff in
it yeah I honestly don't know um it's
like a mystery to me why people keep
writing stuff in C um yeah
no idea
sorry thank you very much to Paul Simon
and
[Applause]
[Music]
Sebastian
yeah