Dear viewer, these subtitles were generated by a machine via the service YouTube and therefore are (very) buggy. If you are capable, please help us to create good quality subtitles: https://c3subtitles.de/talk/2734 Thanks!
[Music]
the next talk will be by Thomas Freer
he's an expert on containers and network
security it will be about energy
consumption of data centers especially
following the ai ai hype thank
you yeah thank you to this slate hour so
I I assume you are really interested in
the
topic um who are we or who am I so I'm
part of this u
in this is some group of people
interested more as in in
medical uh it and AI is part of it we do
open source projects with privacy um in
mind use social innovation we have some
funding from from um beond ster and the
initial funding was done by the holistic
Foundation personally I do normally
kubernetes Cloud security I have
background but this is decades ago in
statistical
physics and I've worked uh for certain
public projects poono for for the German
open code portal which is kind of gitlab
for the German Administration and do
Consulting on the it plan which is the
planning Council and now I'm doing paid
uh Consulting on open desk and for the
federal it
commission so let's talk about this and
uh I think I don't know who of you had
seen something or had this kind of
device in his or her
hands this is an Nvidia h00 this is the
most hype thing in the moment you see it
fits in your hand and it's a more or
less a modern version of a big graphics
card and if you see the power
consumption if it would be working you
would not have it in your hand because
it is producing 700
watts of uh Heat this means if you stack
these things together into a rack you
immediately have 30 kilowatt per rack
instead compared to the normal data
center at the moment which has 3 to 6
kilow per
rack and let's compare it with the old
model everybody has uh more or less a
brain in his
head uh until you drink too much
alcohol and uh here we have um yeah the
leg Legacy model let's say that and it
took several hundred years to
develop and what you actually also have
is an energy consumption of 20
watt so there's a big difference 700
watt 20 wat why is there a big
difference let's look into it if we look
into the modeling of a neuron in a
in the typical AI everything is wrong
with it nearly everything so the hype is
50 years old and they simply said yes
this percep one which is a summing
function and more or less a threshold
function that's a neuron and if you'll
go to somebody a friend of mine is
Professor
for um Neurology in in Cologne he is
laughing at me immediately because this
on the right side this is a neuron you
have um a lot of energy
producing stuff here and then you have
this synoptic terminals so you have this
synopsis and this means this is real
neuron this is not a neuron um what
comes closest to this perceptron is kind
of synoptic terminal and if you look
into the comparison everything with yeah
the computer is an electronic brain now
we have artificial intelligence nearly
everything is wrong and you should
simply subtract the mar marketing a
computer is not a brain and never was
not in the 50s Not in the 60s and
probably not in the near F
future uh if we Compare the numbers this
is from the Nvidia tensor cor data sheet
you see typical CPU which is the most
advanced CPU at the moment has 80
billion transistors the transistor is
not a neuron it's just a switch and if
you compare to the basic neural units of
the brain the neuron synopsis and the
action potential somebody has counted it
it has been counted several time you get
into it that you
have around 100 billion so this is the
American
billion um this 10 to the 9 so in in
German Mard U neurons and they have some
10,000 connections to other neurons so
the
complexity is much higher than the
number of transistors on a
single um Nvidia
h100 and you get this for 20
wats and if you compare it further you
see uh it U the the the brain has a
completely different way of of uh
processing signals it is really firing
it is just um by the number of of fry
rings and upis per per second um this is
the stuff how it shows how much
information or how much how close it is
to the threshold so effectively this
synoptical thing here which is a closer
thing to a percept one is um
um identified by exactly this function
um they um interpret it as a neuron
which is not the new this is a neuron
and effectively if you look into it this
typical thing which is simulated but
they have simulated it in the same way
that it would work in a a brain it is a
two layer uh neural network so the it
neural network so we have have to have
this in mind and this means uh we are
still much much further away from the
actual complexity of the
brain and now we stack this entire thing
into RS and Rec typical W is whatever if
you've gone to if you ever visited a
data center normally you have this
standardized recs and with this new type
of Nvidia gpus we have have this kind of
Nvidia rack and this means we have here
700 wat time 8 and we have typically
three of these units so you can
immediately calculate you have you need
6 time 3,000 watts
to um produce the energy which is
consumed by the Nvidia graphic cards so
effectively you need here more or less
30
Kow and this is not the thing you get in
the typical data center if you have in a
data center if you go into the data
center you have typically uh 30 * 30 so
around a thousand of these RS each of
the normal RS is about 900,000 so this
is number is a few years old so this
means in a typical room of a data center
where you have uh thousand of these RS
you have a billion of Euro of money uh
together and this
means um this is a big investment if you
if
you um want to uh install a data center
with this kind of thing you have the
additional complication that you need a
different cooling so this is it's not
possible to have a passive cooling
anymore you need an active Cooling and
this means you need a liquid cooling and
this means you need completely different
concept of a data
center if you look into the Google
version Google has their own version of
tzor processing units these are seven
recs and this means they are all
interconnected network connections come
additional to the power typical uh GPU
consumes you have this seven
RS and uh you have
to invest a lot of um thermodynamics and
and uh heat technology to get rid of the
heat in these data centers so this is uh
what makes the um AI at the moment so
expensive if you go to an University or
to a library and they use AI they are
always complaining about the cost of the
of
the AI um processing and this comes from
yes we need this this kind of data
center for this Ai and this means we
have to invest
it if we go back to the classic data
centers without AI we have compared to
other Industries more or less Aviation
and uh the data centers on the same size
if we compare this now with artificial
intelligence it comes on the top and
this means we have here
an unknown number I would say at the
moment at least two times uh as much
energy which is necessary for um
creating AI in a data
center uh another thing is the Privacy
uh and and digital sovereignity problem
um normally you have your own data
centers
internally um or you can use collocation
or you can use a
hyperscaler and the hyperscalers
obviously because they are scaling have
more efficient data center than you can
produce in your building and it's
significantly so you have here five
times the workload um the intensity so
the tons of CO2 compared to a
hyperscaler data
center what you also see is it it's
nearly impossible to get real numbers
but if you compare the typical data
centers 15
megawatt um which is a small one a small
one consumes as much power as three
Hospital
buildings or uh as an almond Orchid or
as a 2 * 18 hole
um golf course but this is not energy
anymore this is water so it needs also
additionally water to
cool um the data center so this is
something you also should have in mind
if you compare numbers of a Data Center
and it's so strong uh you see the
industry is green rushing things so
Google is very proud of yes we have this
P value this means it's a total data
center energy consumption to the
consumption which go really is used from
the equipment so this is more or less a
value here this is the um the payload
and here is a total energy you need to
use and Google is very proud of uh
telling you yes we are below
1.01 which is um simply green washing
because they only can get this with a
number of water which is substantial in
some areas so Google in in in the area
of Frankfurt could not build a data
center directly in Frankfurt they had to
go to Han because Frankfurt is running
out of En
and running out of water for the data
centers and there are several movement
around the world if you see Ireland for
example has 18% of the energy
production um going into Data Centers
and the people are protesting if you
have the I don't know if you have heard
it Ireland has uh to reduce the carbon
Dio seed H
has passed a law that they should go
away with a cow that because cows are
another in in the terms of CO2
production competitor to Data Centers
and I don't know if you can imagine
Ireland without cows same in the
Netherlands same in Brandenburg because
we don't have enough water we don't have
enough energy at
these um places and then um yeah how
much energy does artificial intelligence
use and uh it's hard to get the numbers
but if you compare if you collect the
graphs then you see Okay C GPT used
1.2 something Gatt hours
roughly uh 120 us homes for a year but
they don't tell which version and they
don't tell
you um for which kind of training what
you also see is this P Pue value of this
um uh Nvidia graphic cards is much worse
than this 1.1 of Google it's uh 1.58 so
nearly 60% of the energy is
wasted and you have normally anecdotical
num so how much uh does it cost you have
air travel of one person to New York San
Francisco or human life average one year
and you can compare it to training
models this is interesting because you
see here the the real huge numbers um
but finally the really interesting
number I know don't know if you can use
this and this happened in
somehow after 2010 is that um the AI
has um its own version of Mo's law so
normally Mo's law this is these are Lo
this is a logarithmic scale says you
have uh every 2
years a doubling of um computing power
and here in 2010 the training of the big
models turned to another Mo's law which
said okay we are
doubling the energy or those so CPU
consumption every 3. four months so you
have nearly four doubling periods a year
and not one doubling period every 2
years and if you compare it for this
Alpha go zero model and would use it or
would use in age 100 for it but at that
time h100 did not exist you see it is
taking one around 67 megawatt hours to
train this model so you should have a
real reason for these big models playing
go okay is probably not the best idea
but it it was a proof of concept and it
showed that it is possible to have some
real
useful outcome of AI but then uh if you
another thing is if if you Compares this
with with
Bitcoin um it is more or less replacing
Bitcoin in that leak uh that you have uh
the energy consumption of entire
countries um which are used for mining
bitcoins and the interesting part is
both
Bitcoins as
meta uh so this is um
Facebook
metaverse and
the um artificial in intelligence are
based on gpus from one and the same
vendor Nvidia and they um got a value at
the stock of 1 trillion so in German I
billion dollar b v and this means um is
it is driven by The Same Spirit of yes
we want to sell
gpus
if you look into typical applications
you
see uh I did this personally in critical
infrastructure for trading software uh
we had problems of
measuring uh the load of the application
compared to the load of a typical open
shift cluster and then we recalculated
the needs for the software and then we
have seen okay it would fit on rasbery
but we are using a kubernetes cluster
for that same thing I once uh dissolved
the team of Oracle
um admins and then they moved simply to
adop and this is another Trend that the
the people in a company who are using
the most resources the most horsepower
or most computing power um have the
highest rank in a in in the
hierarchy and this means that that a lot
of these paradigms are not technolog
logical but they are more or less
psychological I call it more or less
server matchism who is owning the most
resources is the most important one in
this
uh in this uh
companies also have the rebound effect
so this means if we would assume AI is
now getting
cheaper let's say 20% the usage would
increase more than double that amount
this means this is known as Yan's
Paradox it has been first described for
steam engines and it is an example for
traveling costs rebound effects in cloud
computing typically include
self-provisioning if you make a team
enabling in self-provisioning it is
using more resources than before and
it's getting expensive even you think
okay now the team can do everything on
its own no it just uh getting more
expensive another example of data
centers this is a plan data center in
should go live in Berlin in 2026 I
believe um this means we have uh a 100
megawatt data
center uh and it's one of the first one
uh integrated into the energy
management there is a cooporation with
local Berlin power provider and uh it's
integrated into District heating this
means we reuse 50 to 60% of the heating
energy for um for warming
houses and um you might have seen anus
talk about uh the public data centers
this it's the same in Industry only 6%
of the data centers are
integrated and that means uh we have to
look into how we can uh integrate it
better the cooling if you don't want to
waste the entire water um has to change
you need ad diabetic cooling which means
we have two streams one uh stream of
incoming
air uh and then you have an heat
exchange and then uh the outcoming the
heat here is uh going into the outcoming
uh
heat flow and this means you don't have
the if you don't do it this way you get
all the
um the liquidity or not the liquidity
the humidity of the um of the air in the
data center and this is something you
don't want so you have to do this
exchange and this means with this kind
of um closed heating systems or cooling
systems you have um a pu value of 1.3
but you don't waste water so if water is
your problem uh ad diabetic cooling is
your
solution another thing is if you look
into um smaller uh units you have data
centers in
containers and you can run an entire
open um stack version so the sing Cloud
stack on it this is has to be done by by
h Brook and you have a data center in a
real
container and you can use
photovoltaic for the energy production
and you can even store energy in uh
hydrogen tanks if if you really want it
but this means you can have an isolated
Data Centers of this
size uh in your backyard more or
less uh I asked them um what what they
are really doing they are up cycling
heat to
85 celsus and it's scalable between 50
kwatt and 15 megawatt it is makes
economic sense uh from 250 kwatt so you
have on this scale you have two gaw
hours of heat
production per year and you can Heat 400
houses but customers normally want at
least 5 megawatt data centers this means
you have 40 Gatt and 8,000 houses so it
can be also used for
industrial usage this means uh yeah the
first real estate companies are
interested because this can
significantly lower your heating
costs now thing is if if you can really
do the exchange in another way you can
uh have a sand
battery that you store the heat because
if you you will create this heat also in
summer and this is not what you really
want but what we really want with with
um uh artificial intelligent is
something similar to our brain this is
around since 20 years it's called
neuromorphic com Computing and um it's
compared to a CPU it mimics the way our
neurons work in our brain so you have
Advantage is you have massively parallel
processing you have spiking ual networks
and it is completely asynchronous so it
is also this means it's also completely
non-deterministic and this means we have
a problem that we at least in in this
model um it's even harder to find out
what really happens in this
models uh there's some an example from
Intel they say they could save um 10
fold the energy which is good there are
memoro so there you have units which are
also memory and compute something that's
completely different from the Forman
model and here in uh Australia they are
something creating something which is
named deep
house uh and they want to build the
first big neurom mfia supercomputer in
the next year hope they uh are
sucessful okay um this was all more or
less negative what is what do we really
want we want artificial intelligence
definitely for image processing for
protein folding or real applications
like that what we don't want is that our
AI consumes all our energy and the
answer is a little bit stupid again what
do we need to produce energy yes let's
try nuclear reactors and um if you have
followed the discussions in Germany in
the last year it was more or less
irrational we have this kind of yes can
we turn it off um and if in in the US
they had this mini reactors from from
new scale um but they have been killed
by Green energy because even the small
reactors are much more expensive than
the real reactors
uh so the remote Fusion reactor on our
in our sky or wind energy is much more
effective than nuclear energy this is
just a marketing hype to get this uh
kind of Technology back into
business um what we see if we really
look into the
details I think we will see a 5 to 10
fold increase in energy consumption ion
if you do it this way this is a danger
to the local climate and it cannot be
satisfied globally it is competing with
other
Industries and uh nuclear power plants
by the way we are running also out of
uranium and most of the uranium is not
under Democratic control and we should
go away from this GPU naive way of
implementing
percepton uh unfortunately the the other
stuff is not really uh ready
for um production I hope that the
Australians have success next year but
we will see what happens we also see
massive chatry and uh massive financial
interest on this
topic so I skipped a lot of things for
example if you want to get rid of the
bias of the typical large language
models you have to train it more um we
have racist bias we have we have misen
bias we have all kinds of biases means
uh we have to train more carefully and
uh yeah this is another aspect and would
be a completely new talk so I hope uh
you enjoyed a little bit and I'm done
with my slides uh and now we
can go to
celebrate the end of the day thank
you
well thank you there was some food for
thought is there any question from the
audience yeah maybe one
ah
uh
yeah ah
hello
[Applause]
yeah hi sorry for the delay uh could you
please elaborate on how because you
mentioned the problem of uh water
consumption for data centers how how is
the water consumed the water simply
vaporized so the easiest way of cooling
a system is just vaporize it and then um
get rid of the Steam and if you do this
you lose a lot of more water than you uh
have in a close circuit this ad diabetic
cooling is a closed circuit but it is
not as effective as just um spreading um
water onto the system and then if you
don't if you have enough water okay but
at the moment we have we don't have
enough water in in certain regions and
you said the open circuit is more
efficient than the closed circuit for
cooling yes but then the water is lost
you have to yeah I understand I was just
making Shi thank
you and and this is complicated so so
you um I think it's hard to get below
1.3 uh of this Poe value if you use a
closed uh water
circuit yeah hello yeah no one question
um the usage the the high power usage
for AI models is when they are trained
um is there separation of the
calculation for the usage of existing
trained models and how much is that
versus the training itself that's that's
completely true normally you calculate
um I think 100 or a thousand times more
GPU power for training but if you have a
100 or thousand times more users it's
more or less on par so if you make
something like large language models
like CH GPT very popular it will also
consume a lot of power um during the
time it is simply used so it's um it's
hard to distinguish normally for a
single training run it's it's really a
factor of 100 or thousand but for many
users of the same model it might be uh
as power consuming as a as a training so
you also can find the numberers
somewhere how much is used per query on
a large language model like you can
calculate it down to the query but
actually this caring curve is this
exponential uh Trend the doubling of of
uh CPU con consumption every two do four
every 3. four months um cannot be
um cannot be caught by more efficiency
in the in the GP
in this model so Nvidia is I think
everybody is working on that but you
have a factor of uh doubling um amur's
law every two years and then you have uh
more or less in two years you have seven
uh doubling periods of the of the
consumption so this does not fit
together this unfortunately something
you cannot um you cannot discuss away
and anything else is more or less
anecdotic so you see yeah compared with
a power consumption of a hospital or
something like this yes this this gives
you an impression but it is
not telling you the real um Trend behind
this uh entire AI power
consumption yeah thank
you all right oh let's go you um is it
possible to um like throttle the data
centers to like um uh um like undo
spikes in the energy consumption and to
stabilize the networks of like energy
energy networks yeah every everybody is
is thinking about uh saving energy and
and uh you can have batteries on this
the data centers itself have a lot of
um of of
um uh of throttling not of throttling of
of
um um
they're trying to equalize the
consumption but effectively this is part
of your of your power grid so the power
grid providers should learn somehow to
train um um to to to to save energy in
batteries or in hydrogen hydrogen is is
the the worst idea or the the the power
grid should be as big that you can
simply um flatten the entire differences
because course you have a energy Park in
the northe and and solar energy locally
so this is this is something they try to
do anyway but therefore they need um
yeah Power transport and and this is
it's more another bottleneck in in in
our transmission
grids yes
uh yeah one more question um you
mentioned power grids and battery power
and now I'm wondering have you looked up
what uh what amount of power is lost
just in a conversion between AC and DC
is that marginal or is that actually a
size that's what uh actually uh DC on 1
million volt is very effective and
the uh the the turning it into to AC is
is quite effective so
uh I think you can neglect it compared
to the consumption of of let's say
electric cars they would be another
thing or data centers
than all right cool with that let's call
it a night thank you thank you for
coming
[Applause]
[Music]