*36C3 preroll music*
Herald-Angel: This Talk will be about… I have to read this. Mathematical diseases
in climate models and how to cure them. And I don't have the slightest idea what
these two guys are talking about now. And when I asked them, they said, just tell
the people it's about next generation climate models and how to build them.
Which is cool. Throw that on Twitter. Please welcome Ali Ramadhan and Valentin
Churavy.
*applause*
Ali Ramadhan: Can you guys hear us? Is this OK?
Valentin Churavy: I'll stand back... Ramadhan: I'll stand back a little bit.
OK, cool. Thank you. So if you guys saw the last talk by karlab... karlabyrinth or
something. So we're kind of expanding on her talk a little bit. So she talked a lot
about kind of uncertainties… *audio feedback from microphone*
uncertainties in climate models. And one point that she did make was that most of
the uncertainty actually comes from humans. But there's a really huge
uncertainty that also comes from… comes from the models. So we're talking more
about the model uncertainties, which is kind of uncertainties because of unknown
or missing physics, kind of how to cure them. So it'll be kind of a weird talk. So
I'll talk a little bit more about the climate modeling part and then kind of how
to cure them involves using new programing languages. And that's where Valentin will
talk about Julia. So we'll kind of just start with maybe just giving kind of an
idea of why it's so hard to model the climate. So if you've… maybe you've seen
images like this a lot where it's like a… it's a satellite image basically of
clouds. It's used for like weather forecasting. But you can immediately see
there's lots of, you know, lots of really small clouds. So basically, if you want to
build a climate model, you've got to be able to resolve all the physics in these
clouds. So you can actually zoom in a lot. And see the clouds look pretty big over
here. But if you zoom in on kind of Central America, then you see even smaller
clouds. And if you zoom in even more so, so you zoom in on the Yucatan Peninsula,
then you can see the clouds are really, really small. So you're… there are maybe
five smaller clouds, some some of the clouds are, you know, a hundred meters or
something. And as the last talk kind of suggests that most climate models are…
they resolve things of, you know, up to 50 kilometers. So anything smaller than 50
kilometers, the climate model can't really see. So you have to kind of take that… it
kind of has to account for that because clouds are important, and if you have more
clouds, then that reflects some of the heat out. So maybe you cool. But it also
traps more of the heat in so maybe you warm. And if you have more clouds, maybe
you warm more. But if you have less clouds, maybe you warm even more. So it's
kind of unsure. We actually don't know if clouds will make the climate warmer or if
they'll make the climate cooler. So it's important for your climate models to kind
of resolve or see these little clouds. So kind of where the mathematical disease
comes in, is that you don't… we don't know what equation to solve. We don't know
exactly what physics to solve, to see, to kind of resolve the effect of these little
clouds. So it's kind of the the mathematical disease. We don't know how to
do it. So you instead use a… well, it's called a parametrization, which is the
mathematical disease. So in the atmosphere, the big mathematical disease
is clouds. But if you look at the ocean, you kind of get a similar… You have also
similar mathematical diseases. So if you for example, this is model output. We
don't have good satellite imagery of the oceans. So if you if you look at, for
example, model output from an ocean model, high resolution ocean model here, it's
kind of centered on the Pacific. So you can kind of see Japan and China and the
white kind of lines. Those are streamlines or that the lines tell you where the water
is going. So you could see a lot of kind of straight lines. You see this curve here
current off of Japan, but you see lots of circles. So the circles are these eddies
and they're kind of the turbulence of the ocean. They move, they kind of stir and
mix and transport a lot of salt or heat or carbon or nutrients or… you know, marine
life or anything. It's the main way the ocean kind of moves heat from the equator
to the pole. It kind of stirs things around. So they're really important for
kind of how carbon moves in the ocean, for how the ocean heats up. And here they look
pretty big. But again, you can zoom in and you'll see lots of small scale structures.
So we're going to switch to a different model output and different colors. So here
here's kind of the same area. So you see Japan in the top left. But what's being
plotted is vorticity. So you have to know what that is. It's kind of a measure of
how much the fluid or the water is spinning. But the point is that you have
lots of structure. So there's lots of, you know, big circles, but there also lots of
really little circles. And again, your climate model can only see something like
50 kilometers or 100 kilometers. But as you can see here, there's lots of stuff
that's much smaller than a hundred kilometers. So if you superimpose kind of
this this grid, maybe that's your climate model grid. And, you know, basically for
the climate model, every one of these boxes is like one number. So you can't
really see anything smaller than that. But there's important dynamics and physics
that happens in like 10 kilometers, which is a lot smaller than what the climate
model can see. And there's even important physics that happens that like 100 meters
or 200 meters. So if you want if you want to, you know, what the climate will look
like, we need to… we need to know about the physics that happens at 200 meters. So
to give an example of some of the physics that happens at 10 kilometers, here's kind
of a little animation where this kind of explains why you get all these eddies or
all the circles in the ocean. So a lot of times you have, say, hot water, say, in
the north. So the hot water here is all in orange or yellow and you have a lot of
cold water. So the cold water is in the south and it's purple. And then once this…
once you add rotation, you end up with these eddies because what the hot water
wants to do, the hot water is lighter, it's less dense. So it actually wants to
go on top of the cold water. So usually have cold at the bottom, hot at the top.
So you have heavy at the bottom and light at the top. So when you add… without
rotation, the hot water will just go on top of the cold water. But when you have
rotation, you end up… it kind of wants to tip over. But it's also rotating. So you
kind of get this beautiful swirling patterns and these are kind of the same
circular eddies that you see in the real ocean. But this model here is like two
hundred and fifty kilometers by five hundred kilometers and it's like one
kilometer deep. So you need a lot of resolution to be able to resolve this
stuff, but not… your climate model doesn't have that much resolution. So some of the
features here, like the sharp prints between the cold and the hot water, your
climate model might not see that. So maybe if you if you don't resolve this properly,
you get the mixing rate wrong or maybe that the ocean is the wrong temperature or
something. So it's kind of important to resolve this stuff. Another one, the color
scheme here is really bad. *laughs* I'm sorry, but another one, for example, is
here. Everything's under 100 meter, so it's a cube of 100 meters on each side and
you're starting with 20 degrees Celsius water at the top. You have 19 degrees
Celsius water at the bottom initially. So it's kind of you're… as you go deeper in
the ocean, the water gets colder. And then if you can imagine, the ocean kind of at
night, it's kind of cold. So the top is being cooled and you end up with cold
water on the top. The cold water wants to be at the bottom. So it ends up sinking and you
get all this convection going on. So this is happening at a lot of places in the
ocean. You get a lot of mixing at the top. You get this kind of layer at the top of
the ocean. It's kind of constant color, constant temperature. So this mix layer is
important for the ocean. So knowing how deep that mix layer is and knowing how
much of the water is being mixed is also important for for climate. But as you can
imagine, you know, if this happens on very small scales. So you're climate model has
to know something about what's happening at this scale. So this isn't I guess the
mathematical diseases in the ocean is, the climate model cannot see this, so it has
to do something else that's maybe unphysical to resolve this stuff. And
that's a mathematical disease, I guess. Aside from the ocean and the atmosphere.
You also have the same problem with sea ice. So this is kind of just a satellite
picture of where sea ice is forming off the coast of Antarctica. So you get winds
that kind of come off the continent and they're
kind of blowing all the ice that's beeing formed
away. So you get all these little lines and streaks and they kind of
merge into sea ice. But in this whole picture is like 20 kilometers. So the
climate model doesn't see this, but somehow it has to represent all the
physics. And you have kind of similar things happening with soil moisture, land
and dynamic vegetation, aerosols. So, you know, these are kind of three places with
pretty pictures. But see, if you look at the atmosphere, so it's not just clouds.
You also have aerosols, which are like little particles, or sulfates that are
important for kind of cloud formation and maybe atmospheric chemistry. But again, we
don't fully understand the physics of these aerosols. So again, you have to kind
of parametrize them. Same thing with kind of convictions. You maybe your climate
model doesn't resolve all the very deep convection in the atmosphere so as to get
all sides to parametrize it. So I guess you have many kind of mathematical
diseases in the atmosphere. So I'm not expecting you to understand everything in
this in this picture. But the idea is: The atmosphere is complicated. There's no way
a climate model is going to kind of, you know, figure all this out by itself. And
again, you could you could do something similar for the ocean. So we can just show
an image for like two little parts of these. But the point is, you know, the
ocean is not kind of just a bucket of water standing there. So there's lots of
stuff happening deep inside the ocean. And some of it, we think is important for
climate. Some of it we don't know. Some might not be important. But again, a lot
of this happens on very small spatial scales. So we don't know or the climate
model can't always resolve all this stuff. And again, same thing with kind of sea
ice. Lots of small scale stuff is important for sea ice. And I think one
person asked about kind of tipping points and there's kind of two with like sea ice
that are pretty important. One of them is this CSL biofeedback. So if you have sea
ice that melts. Now you have more ocean and the ocean can absorb more heat. But
now the earth is warmer, so it melts more sea ice. So as soon as you kind of start
melting sea ice, maybe you melt even more sea ice and eventually you reach an earth
with no sea ice. So there's kind of research into that stuff going on, but
it's a possible tipping point. Another one is this kind of marine ice sheet,
stability, instability at the bottom of the ice shelf. So if you start melting
water, if you start melting ice from the bottom of the ice shelf, then we create
kind of a larger area for more ice to melt. So maybe once you start melting and
increasing sea level, you just keep melting more and more and increasing sea
level even more. But again, it's kind of hard to quantify these things on like 50
or 100 year timescales because it all happens on very small scales. So yeah, the
point is there's lots of these kind of parametrizations or mathematical diseases.
And once you start adding them all up, you end up with lots and lots of kind of
parameters. So this is a really boring table. But the point is, so this is like
one parametrization for like vertical mixing in the ocean. It's basically the
process that I showed the rainbow color movie about to see a climate model for
that. I'm trying to kind of parametrize that, physics might have like 20
parameters. And, you know, some of them are crazy like a surface layer fractional
like zero point one or something. And usually they keep the same constants for
all these values. Usually it's like someone in like 1994 came up with these 20
numbers and now we all use the same 20 numbers. But you know, maybe they're
different. And like the Pacific or the Atlantic or like maybe they're different
when it's summer and winter and the problem is, there's many of these
parametrizations. So you know here's like 20 parameters, but then you have a lot
more for clouds. You have a lot more sea ice. We add them all up. Suddenly you have
like 100, maybe up to a thousand kind of tunable parameters. Kind of going back to
this plot that was shown at the last talk. You can see kind of the all the models
kind of agree really well from like 1850 to 2000, because they're all kind of being
they all have different kind of parameters, but they all get kind of tuned
or optimized. So they get the 20th century. Correct. So they get the black
line. Correct. But then when you run them forward, you run them to like 2300. They
all are slightly different. So they all start producing different physics and
suddenly you get a huge like red band. So that's saying you have lots of model
uncertainty. So it's kind of on some people might say like oh this like tuning
process is like optimization. It's like not very scientific to be kind of right.
It's kind of like in the past. It's kind of like the best live we've had. But I
think, you know, we should be able to do a little bit better. Better than that. So
just to give you the idea, you know, some people would say, you know, why don't you
just you know, most of the physics, which you just, you know, resolve all the
physics, you know, but see if you want to do like a direct numerical simulation, so
it's basically saying you want to resolve all the motions in the ocean, in the
atmosphere. You basically need to resolve things down to like one millimeter. So if
you have like a grid spacing of one millimeter and you consider the volume of
the ocean and the atmosphere, you basically say you need like 10 to the 28
grid points. You know, that's like imagine putting cubes of like one millimeter
everywhere in the ocean and atmosphere.
That's how many great points you would need.
So unfortunately, you could do that. But there's not enough computer power or
storage space in the world to do that. So you're kind of stuck doing something a bit
coarser. Usually most climate models, he's like 10 to the 8 great points so that you
10 to the 20 to little words. You don't want to just run a big climate model once
you know you need to run them for very long times, usually like you run them for
a thousand years or ten thousand years when you want to run many of them because
you want to collect statistics. So generally you don't run at the highest
resolution possible. You run kind of at a lower resolution so you can run many, many
models. So because you can only use so much resolution, it seems that power
transitions or these kind of mathematical things, you have to live with them, you've
got to use them. But at least one idea is, you know, instead of using numbers that
sum. But he came up with in 1994. You might as well try to figure you know,
better numbers or maybe you know if the numbers
are kind of different in different places,
you should find that out. So one thing you could do, one thing we are
trying to do is get the pressurization is to kind of agree with like basic physics
or agree with observations. So we have lots of observations. How many we can run
kind of high resolution simulations to resolve
a lot of the physics and then make sure
when you put the prioritization in the climate model, it actually gives you
the right numbers according to basic physics or observations. But sometimes
that might mean, you know, different numbers in the Atlantic, in the Pacific or
different numbers for the winter and the summer. And you have to run many high
resolution simulations to get enough data to do this. But indeed, you know, these
days I think we have enough computing power to do that. So it's kind of do all
these high resolution simulations. We ended up building a new kind of ocean
model that we run on GPUs because these are all faster for giving us these
results. So we ended up usually most climate modeling is done in Fortran. We
decided to go with with Julia for a number of reasons, which I'll talk about. But the
left figure is kind of that mixed layer or boundary layer turbulence kind of movie.
But instead of the rainbow color map, now it's using a more reasonable color maps.
It looks like the ocean, the right is that old movie. So we're generating tons and
tons of data from using simulations like this and then hopefully we can get enough
data and like figure out a way to explain the prior transitions. But it's kind of a
work in progress. So a different idea that might be more popular here, I don't know.
Is instead of kind of using the existing permanent positions, you could say, OK,
well, now you have tons and tons of data. Maybe you just throw in like a neural
network into the differential equations. Basically, you put in the physics, you
know, and then the neural network is responsible for the physics you don't
know. So, for example, you know, most people here might not. I also don't want
to talk about differential equations because I would take a long time. So just
imagine that the equation in the middle is kind of
what a climate model needs to solve.
And the question marks are kind of physics we don't know. So we don't know
what to put there. But maybe you could put out a neural network. So number one is
kind of a possible characterisation or a possible way you could try to franchise
the missing physics where the neural networks kind of responsible for
everything. We find that doesn't work as well. So instead, maybe you tell it some
of the physics, maybe tell it about cue, which is like the heating or cooling at
the surface. And then it's kind of responsible for resolving the other stuff.
But it's still a work in progress because the blue is kind explosive your data. The
orange is supposed to be the narwhal and they don't agree. So it's still a work in
progress, but hopefully we'll be able to do that better. So this is kind of stuff
that's like a week or two old. But kind of reach a conclusion, at least from my half
of the talk. So the reason I personally like Julia as a climate modeler is we were
able to kind of build an ocean model from scratch basically in less than a year. And
one of the nice things is that the user interface
or the scripting and the model backend is all in one language,
whereas in the past used to usually write the high
level and like Python and maybe the back end is like Fortran or C. And we find, you
know, when we Julia, it's just as fast as our legacy model, which was written in
Fortran. And one of the nicest things was that basically able to write code once and
using there's a need of GPU compiler. So basically you write your code one single
code base and you go to CPUs and GPUs. So you'd want to write two different code
bases. And yeah, we find generally because it's high level language, we're all kind
of more productive. We can give a more powerful user API and Julia kind of has a
nice multiple dispatch backend so that we find that makes it easy for the users to
kind of extend the model or hack the model. And there's. Some people would say
the Julia community is pretty small. But we find there's a pretty big Julia
community interest in scientific computing. So we fund kind of all the
packages we need are pretty much available. So with our client conclude my
half by saying there is most of the uncertainty in climate modeling basically
comes from humans because they don't know what humans will do. But there's a huge
model uncertainty basically because of physics we don't understand or physics,
the kind of model cannot see. You can't resolve every cloud and you know, every
wave in the oceans you've got you've got to figure out a way to account for them.
So that's what our prioritization does. And we're trying to kind of use a lot of
computing power to kind of make sure we train or come up with good privatizations
instead of kind of tuning the model at the end. And we're hoping this will lead to
better climate predictions. Maybe you will. Maybe you won't. But at least, you
know, even if it doesn't. Hopefully we can say we got rid of the model tuning problem
and hopefully we can make. We find it that software development for climate
modeling is easier than if we did it in Fortran. I will say this kind of an
advertisement, but I'm looking to bike her on Germany for a week and apparently can't
take the next bike out of Leipzig. So if anyone is looking to sell their bicycle or
wants to make some cash, I'm looking to rent a bicycle. So yeah, if you have one,
come talk to me, please. Thank you. Danke.
*applause*
Churavy: So one big question for me always is how can we ask technologists hub that
think most of us in this room are fairly decent with computers? The internet is not
necessarily an new island for us. But how do we use that knowledge to actually
impact real change? And if you haven't there's some fantastic article:
worrydreams.com/ClimateChange. Which lists all the possible or not all the possible
but a lot of good ideas to think about and go like, okay, do my skills apply in that
area? Well, I'm a computer scientist. I do programing language research. So how do my
skills really apply to climate change? How can I help? And one of the things that
took me in this article was how, and one of the realization, and why I do my work
is that the tools that we have built for scientists and engineers, they are that
poor. Computer scientists like myself have focused a lot on making programing easier,
more accessible. What we don't necessarily have kept the scientific community as a
target audience. And then you get into this position where models are written in
a language. Fortran 74 and isn't that a nice language, but it's still not one that
is easily picked up and where you find enthusiasm in younger students for using
it. So I work on Julia and my goal is basically to make a scientific computing
easier, more accessible and make it easier to access the huge computing power we have
available to do climate modeling. Ideas, if you are interesting in this space is,
you don't need to work on Julia necessarily, but you can think about maybe
I'm to look at modeling for physical systems, modeling like one of the
questions is can be model air conditioning units more precisely, get them more
efficient? Or any other technical system. How do we get that efficiency? But we need
better tools to do that. So the language down here as an example is modelicar.
There is a project right now, modelicar is trying to see how we can push the
boundary there. The language up here is Fortran. You might have seen a little bit of that
in the talk beforehand and it's most often used to do climate science. So why
programing languages? Why do I think that my time is best spent to actually work on
programing languages and do that in order to help people? Well, Wittgenstein says:
"The limits of my language are the limits of my world." What I can express is what I
think about. And I think people are multilingual, know that that sometimes
it's easier to think for them. But certain things in one language say it isn't the
other one. But language is about communication. It's about communication
with scientists, but it's also about communication with the computer. And too
often programing language fall into that trap where it's about, oh, I want to
express my one particular problem or I wanna express my problem very well for the
compiler, for the computer. I won't talk to the machine. What if I found that
programming languages are very good to talk to other scientists, to talk in a
community and to actually collaborate? And so the project that Ali and I are both
part of has, I think, 30 ish. I don't know. The numbers are as big as the big
coupe of climate scientists modelers. And we have a couple of numerical scientists,
computer scientists and engineers and we all working the same language, being able
to collaborate and actually work on the same code instead of me working on some
low level implementation and Ali telling me what to write. That wouldn't be really
efficient. So, yes, my goal is to make this search easier. Do we really need yet
another high level language? That is a question I often get. It's like why Julia?
And not why are you not spending your time and effort doing this for Python? Well, so
this is as a small example, this is Julia code. It looks rather readable. I find it
doesn't use a semantic whitespace. You may like that or not. It has all the typical
features that you would expect from a high level dynamic language. It is using the
M.I.T. license that has a built in package manager. It's very good for interactive
development, but it has a couple of unusual wants and those matter. You need
if you want to simulate a climate model, you need to get top performance on a
supercomputer. Otherwise you won't get an answer in the time that it matters. Julia
uses just in time ahead of time compilation, the other great feature is
actually a spitting in Julia. So I can just look at implementations. I can dive
and dive and dive deeper into somebodies code and don't have a comprehension
barrier. If I if you ever have spent some time and tried to figure out how Python
sums numbers under the hood to make it reasonably fast. Good luck. It's hard.
It's written in C. See, and there is a lot of barriers in order to understand what's
actually going on. Then reflection and meta programing. You can do a lot of fun
stuff which we're going to talk about. And then the big coin for me is that you have
native keep you code generation support so you can actually take Julia code and run
it on the GPU. You you're not relying on libraries because libraries
only are can express the things. That was where writtenin there. So early on last
December, I think we met up for the climate science project and after deciding
on using Julia for the entire project. They were like, we we're happy with the
performance, but we have a problem. We have to duplicate our code for GPUs
and CPUs. What really? It can't be! I mean, I designed the damn thing, it should be working.
Well, what they had at that point was basically always a copy of two functions
where one side of it was writing the CPU code and the other side was implementing a
GPU code. And really, there were only a couple of GPU specific parts in there. And
if anybody has ever written GPU Code, it's this pesky which index am I calculation.
Worthy for loop on the CPU to would just looks quite natural. And I was like, what?
Sit. Come on. What we can do is we can just wait a kernel so he takes a body of
the for loop, extracts it in a new function. Add a little bit of sugar and
magic to court GPU kernels and CPU functions and then we're done. Problem
solved. What the code roughly would look look like isn't actually this. You can
copy and paste this and it should work. And so you have two functions. One of them
launches where you extract your kernel. Then you write a function that takes
another function and runs it function in a for loop or it launches that function on
the GPU. And then you have this little GPU snippet is the only bit of us actually
GPU, which calculates the index and then calls the function F with an index
argument. I'm done here. My, my contribution to this project was done,
Well, they came back to me and we're like, now it's not good enough. And I was like,
why? Well, the issue is they needed kernel fusion. So that's the process of taking
two functions and merging them together. I'm like, okay, fine. Why do they need
that? Because if you want to be white(?) average efficient GPO code, you need to be
really concerned about the numbers of global memory loads and stores. If you
have too many of them or if they are irregular, you lose a lot of performance
and you need good performance. Otherwise, we can't simulate the solution once. They
also actually wanted to take use GPU functionality and low level controlled.
They wanted to look at their kernels and use shared memory constructs. They wanted
to do precise risk working, minimizing the number of registers used and they really cared
about low level performance. They were like, well, we can't do this with the
abstraction you gave us because it builds up too many barriers. And I could have
given you a few more typical computer science answer, which would have been OK.
Give me two years and I'll come back to you and there is a perfect solution which
is like a cloud cover in the sky. And I write your speech spoke language that does
exactly what you need to do. And at the end, we have a domain specific language
for climate simulation that will do final volume and discontinuous cloaking in
everything you want. And I will have a PhD. Kit. Fantastic. Well, we don't have
the time. The whole climate science project that we are on has accelerated
timeline because the philanthropist that the funding that research are. Well, if
you can't give us better answer anytime soon, it won't matter anymore. So I sat
down and was like, okay, I need a box. I need something. It has minimal effort.
Quick delivery. I need to be able to fix it. If I do get it wrong the first time
around and I did, it needs to be hackable. My collaborator needs to understand it and
actually be able to change it. And it needs to be happened yesterday. Well,
Julia is good at these kinds of hacks. And as I've learned, you can actually let them
go into bespoke solutions and have better abstractions after the fact. So that
you're that you can actually do the fancy computer science that I really wanted to
do. The product is called GPUify Loops because I couldn't come up with a worse
name, nobody else could. So we stick with it. It's a Macro based. And so, Julia, you
can write syntax macros that transform the transform the written statements into
similar statements so you can insert code or remove code if you want to. At, right
now target CPUs and GPUs and we are talking about how do we get multi threaded
into the story, how do we target more on different GPUs? There are other projects
that are very similar. So there's OCCA, which is where a lot of these ideas are
coming from and Open ACC in C++ does something really similar. But basically
you write a for loop, you write an at loop in front of it, which is the magic
macro that takes a transformation. And you have two indexed statements and now you
just say I want to launch it on the GPU and it magically does a job. Get,
fantastic. So let's pick up the entire implementation of the macro at loop
without the error checking that didn't fit on the screen a couple of nights. So
everything is here and basically I'm just manipulating the for loop so that on the
GPU it only iterates one iteration per index and on CPU it iterates all of the
indices because CPU is single threaded and a GPU is many, many
multithreaded. Of course there's a little bit of magic hidden in the device function
because how do I know where I'm running? And if you're curious how to do that and
then we can talk after afterwards. But otherwise, it's a very simple,
straightforward transformation. It's written in Julia. It's a Julia function.
And. Yeah. So you don't need to understand the code here. I just want to show how quick it
can be to write something like this. If you know anything about GPU Programming at
all, there should be a little voice in the head, of the back of your head is like,
wait a second. How can you run a dynamic programming on a GPU? That shouldn't be
possible. Well, Julia can run on the GPU because it has a lot of meta programing
facilities for the port for stage programing. So I can generate code based
on a specific call signature. It has introspection, reflection mechanisms that
allow me to do some interesting stuff in the background. It is built upon LVM,
which is a common compiler infrastructure. And so I can actually write staged
function that would generate an LVM specific code for my one function and do
so do that during compile time and is a dynamic language that tries really hard to
avoid runtime uncertainties. And this is one of the challenges if you're getting
into Julia is to understand that when you're writing code that has a lot of
runtime uncertainties, you get relative slow performance, or as fast as Python.
But if you work with the compiler and you write runtime uncertainties you can get
very fast code and you can run your code on the GPU, you basically that's the
limites test. If you can run your code on the GPU, that you did your job well and it
provides tools to understand the behavior of your code. So a warning runtime
uncertainty. It does that and I don't have the time to go too deep into the answers.
There is actually a paper about this. It has a type system that allows you to do
some sophisticated reasoning type influence to figure out what your code is
doing. Mutable dispatchers actually helping us quite a lot in making it easier
to do virtualized codes. It was a case of specialization and just in time
compilation. And so just looking a little bit closer at some of these topics, if you
want to look at the entire pipeline that flow when you start while you're
functioning, call it what happens through the Julia compiler. You have tools to
introspect and all of these on the right hand side here and then you have tools to
interact on the left hand side. You can inject code back into the compiler. The
other thing is Julia has dynamic semantics. So when you difficult, you can
at runtime, redefine your function and recall it new function and it uses
multiple dispatch. So if you look at the absolute value call here, which of the 13
possible methods will it call? In C++ or in other programing languages this called
a virtual function call. So isn't Julia everything a virtual functional call? No.
This is one of the important points is when we call a function, let's say because
sign of X, we look at the type of the input arguments and then we first of all
look at which function is applicable to our input argument. So in this case, it
would be the real down here because float 64 is a subtype of real. So we choose the
right method using dispatch and then we specialize that method for the signature.
So the rule in multiople dispact is to remember is we calling the most specific
method, whatever specific might mean. So if you have this bit of example, where we
have a function F, which has three different methods and we have an integer
argument that can be matched on X, or on Y, and then we have a floating point argument
on Y and we call this with a "1,Hello". Well, we will select the methods that is
most specific for this argument, which would be the number 1 here. On the other
hand, if when we have a float 64 and the second position, then we will call the
second method. Now what happens if I pass in an integer and the first position and a
floating point in the second position? Well, you would get a run time error
because we can't make this decision. What is the most specific method? That's just
something to keep in mind. Method specialization works really similarly when
you call a method for the first time. This method sign right now has no
specializations. And then I look back, call it once and Julia will insert a
speciallisation just for Float64. Before that it could have been a Float32. The
Float64 is for this method. So Julia specializes in compilers methods on
concrete called signatures instead of keeping everything dynamic or everything
ambiguous. You can introspect this process and there are several macros that are code
lowered or code type that will help you understand that process. I think I don't
have enough time to go into detail here, but just as a note, if you have a look at
this, the percentage for means it's an assignment. So if you reference it later,
so in line 5, we will iterate on the 4 value. And then we can look at the type
information that Julia infers out of that call. We're calling the function mandel
with the U in 32 and you can see how that information propagates through the
function itself. And then if you actually do agressive inlining .., we do aggressive
inlining and optimizations and devirtualization. And so in the end, we
don't have calls anymore. We only have the intrinsics that Julia provides on which
programs are actually implemented. So this is a unsigned less than integer function.
So we are using time and find as an optimization to find static or near static
site programs. It allows us to do us agressive virtualization, inlining and
constant propagation. But it raises problems of cash and validation. So in
bygone days, this used to be the case. I could define a new function G after
calling G want a function, a new function, f after calling G once and I would get the
old restore back. That's bad. That's counter-intuitive. That's not dynamic. So
in Julia 1.0 and I think 0.5 and 0.6 already. We fix that. So we invalidating
the functions that have dependencies on the function. We just changed. But can we
see latency of your program? If you change a lot of the functions and you recall them
well hm we need to do a lot of work every time. We do constant propagation, so it
isn't very simple example. We try to reduce. We try to exploit as much
information as possible. And so if you call if you want a function F and you call
the function sign with a constant value, we actually build just turning you the
constant avoiding the calculation is the sine entirely. And that can be very
important during hot calls and in a cycle. This can sometimes go wrong or Julia can
has heuristics in order to decide when or whether or not these optimizations are
valuable. And so when you introspect your code, you might see the results that are
not that are not quite, what you want. So we don't know what the return value here
is. It's just a tuple. We know it's a tuple, nothing else. Holistic to say, not
specialize. But the nice thing about Julia and where we get performance voice that we
can actually do for specialisation and hopefully at some point view makes a
compiler smart enough that these edge cases disappear. So I can use some secrets
and foresee specialization to happen and then I can actually infer the precise of
return type of my function. Another thing to know when you're coming for more
traditional object oriented programing language is that types are not extensible,
extendable. So you can't inherit from something like Int64. You can only subtype
abstract types. We do that because otherwise we couldn't do a lot of
optimizations. When we, when we look at programms, we can't never assume that you
won't add code. We had a dinamic programming language at any time in the runtime of your
program you can't add code. And so we don't have close word semantics, which doesn't
doesn't allow us to say, hey, by the way, we know all possible subtypes here. You
might add a new type. Later on by saying a common types are not extendable.
We get a lot of the performance back. So personally, for me, why do I like Julia?
Or why do I work on Julia? It works like Pyphon, it talks like Lisp and runs like
Fortran. That's my five sales pitch. It's very hackable and extendable.
I can poke at the internals and I can bend them if I need to. It's a
bit of upon LVM. So in reality, for me as a compiler writer, it's my favorite LVM
front end. I can get the LVM code that I need to actually run. But for users,
that's hopefully not a concern. If you do our job right and it has users in
scientific computing and I'm in a prior life whilst doing a lot of scientific
computing in cognitive science wanting models. And I care about these users
because I've seen how hard it can be to actually make progress when the tools you
have are bad. And my personal goal is to enable scientists and engineers to
collaborate efficiently and actually make change. Julia is a big project and Climate
is a big project and many people to thank. And with that, I would like to extend you
an invitation if you're interested. There is juliacon every year. Where you have a
develop meet up. Last year we were about 60 people are much smaller than CCC. But
next year it will be in Lisbon. So come join us if you're interested and if you
want to meet scientists who have interesting problems and are looking for
solutions. Thank you.
*Applaus*
Herald A: Time for questions and answers, are there any questions?
Herald H: Yeah, we've got microphones over there. So just jump to the microphone and
ask your questions so that everybody could hear.
Question: What do you mean when you say dead? Julia talks like Lisp and how is
that a good thing *Lachen* Churavy: Well, it talks like Lisp, but it
doesn't look like Lisp. I assume that's what you mean. It doesn't have that many
braces. But no, Lisp has another powerful meta programming capabilities and macros. And
so we have a lot of that. If you read a little bit about the history of Lisp. The
original intention was to write NLisp, which would be Lisp with a nice syntax. And
I think Julia is my personal is NLisp. It has all these nice features, but it
doesn't have the packet syntax. Herald A: OK. Thank you.
Question: Thanks for the talk. My question is regarding the first part of the talk.
You, if I understand correctly, you simulating a deterministic system there.
So there's no additional noise term or anything, right?
Ramadhan: Well, if you had infinite precision, I think it would be
deterministic. But I think by kind design turbulence itself is not deterministic.
Well, it's a chaotic system, Question: But the district size version
itself is deterministic. You don't have the monte carlo part where you have
some noise that you would add to which might actually be justified
from the physics side. Right? Ramadhan: Well, I mean, we, if you think if
you ran the same simulation again, you would not get that. Well, I think if you
ran on the exact same machine, you would get the
same answer. So in that sense, it is deterministic.
But if you ran on a slightly different machine like truncation
error, I'd like the 16th decimal place could give you a completely different
answer. Question: Sure. So the point I'm trying. Am I allowed to continue?
Herald H: Yes, of course. There's no one else. Well, there is one person else. So you
can continue a few minutes if you want to. Thanks. *Laughter*
Question: So the point I was trying to make is,
if you add noise in the sense that it's a physical
system, you have noise in there, it might actually allow you to
solve a PDI or discretize a PD, but get a stochastic simulation itself, which might
be interesting because it often can make things easier. And also, you mentioned
neural differential equations, right? And in particular, with physical systems, if
you have an discontinuities, for example, the DT integral can actually be quite the
problem. And there is work on to just plug my colleagues work, control neutral
differential equations where you can actually also built in these
discontinuities, which might also be interesting for you guys.
Ali: That's why maybe we should talk because I don't know much about that stuff
where we're kind of just starting up. I think that so we've been doing this maybe
hopefully continuous, but maybe we'll hit discontinuities. I don't know. We should
talk, though. Q: And also the math is beautiful and has no sickness. It's the
physics that mightn't change. I'm a mathematician. I have to say that. Ali: I know
that the physics is ugly, trust me. Churavy: Just as quickly, we do have
stickers and I sell cookies, too. They are in the cookie box and on. I think they for
somebody from our community is giving a juliaworkshop and we're trying to find a
set up an assembly space and hopefully that goes out as well.
Herald H: Go on please. Question: Also, one question for the first
part of the talk I want. I wanted to ask if it's possible or if you are using
dynamic resolution in your climate models. Well, you will maybe have a smaller grid
size near the (???) and larger in the areas that are not that
interesting. Ramadhan: Like adaptive grids? So I
think we mostly do that in the vertical. So usually in the ocean, the thinking
things are interesting in the, you know,
close to the surface. We have more resolution
there. But as you go deeper, things get less interesting. So you put
less resolution there. Generally, I think in general, the idea people have asked
that before, you know, why do you always use constant grids? Why don't you use
these adaptive grids on your global, you know, models? And you the answer I've
heard I don't know if it's very convincing. I think generally there hasn't
been that much research or people who do research into adaptive grids for kind of
models. Their funding gets cut. But I like the answer I've heard is a lot of the
time, a lot of the atmosphere and ocean is turbulent. So if you especially you do
kind of adaptive refinement, then you just kind of adapt everywhere because there's
kind of turbulence everywhere. But yeah, I don't I'm not. I guess first for our
simulations we're kind of just some of the numerical methods are only fast if you
run it on a regular grid. So that's the reason we don't use adaptive
grids for our simulations. But in general, adaptive grids for climate models is
interesting beyond like it seems like there needs to be more research in that
area. So I don't know if I answered your question, but I kind of just ranted it.
Question: You did, thanks. Herald H: Go go ahead, please.
Question: Yeah, it's just a few guesses about us. I think I have. I wept quite a
bit of legacy fortune code in Python. And my question is, would there be a simple
pass converting Fortran code to Julia, preferably automatically. Do you have any
ideas about this one? Churavy: You can do it. Your Julia code
will look like Fortran code. So you haven't won anything. So, yes. As a good
starting point, you can do that. Absolutely. But you can also just call
Fortran from Julia and then totally move over. I generally don't want people to
rework their code, except if there's a good reason. Like starting from scratch
sometimes helps. It can be a good reason. Or if you say the solutions, we don't have
the necessary experts to to work with the old solution anymore. But generally, if
you have Fortran code, I would just say, well, call Julia from Fortran or from
Julia, get it up to speed and then start transitioning. Piece by piece. That makes
sense? Herald H: So any more questions? No more
questions. That's an early read. Ali Ramadhan, and Valentin Churavy, thank you very much.
*Applaus*
*36C3 postroll music*
Subtitles created by c3subtitles.de in the year 2020. Join, and help us!