*35c3 preroll music*
Herald: So our first talk is going to be presented by two speakers: Michael Sperber
and Nicole Rauch. Michael is the CEO of Active Group. He has been developing
software and teaching programming for over 30 years, and Nicole is an independent
software developer, focusing on domain driven design and event streaming and
she's been organizing Software Craftsmanship conferences. So please give
them a big round of applause for this talk.
Michael: Good morning everyone. Ah, this thing is on.
Nicole: Hey, it works. Excellent. Cool. So… soul-crushing code: I think before we
say farewell, we should think about what it actually is. And so I thought… *looks
at projection surface* I'd maybe get a presentation first. *projection of slides
starts* Excellent. So. OK. So first maybe we should talk a little bit about what's
soul-crushing code is and so I brought something for you for your entertainment.
And here we go. This is some code I found in an eclipse tutorial. If you know a
little bit about eclipse you can see this by looking at this "I…" thing here. Yeah.
And this actually… I mean I have no idea what that this does. You know, you have
this AdapterFactory which is really like… okay… and then we get an adapter and
there's lots of stuff going on. But what actually is this doing… so there is
"Todo". OK. This gives a hunch that this might be this famous Todo example, but
then I have no idea what's happening here M: but maybe that's just Eclipse, right,
and in Eclipse everything is an adapter for something else.
N: OK. Maybe. So, OK, I had expected you to say something like this so I thought
I'd bring another example. And this actually is from an open source project.
And this actually is a NativeQueryInterpreterInitiator. I also
want an interpreter initiator – who doesn't? And also it's a
SessionFactoryServiceInitiator. Wow, I'm getting impressed. So. And then it also
does lots of stuff and then. OK. We have initiateService() and then we have another
initiateService(). And then at the end… wow… we have this: getServiceInitiated.
Well OK so fine. M: But I mean that's just a bunch of kids
doing open source software. N: Right. Yeah, this is open source
software. M: So these are all hobbyists. So I mean
what would you expect from something like that, right?
N: OK fair enough. M: So, I brought a professional example.
N: Oh I see. *laughter in the audience*
N: Everybody stand back. M: So here's a professional example. So
this is from a system that does queries of some sort. I think there's a mishap in
translation from German or French to English in me or somewhere. So we can see
that it does either full text search or it does non full text search or something
like that. And so obviously in the initialization of some C++ class it checks
that flag and it instantiates some object either to the full text variant or the non
full text variant. Right. And you would think that this object oriented design,
you just go and call methods. N: Yeah…
M: You get one instance variable that you have there.
N: if and else is sooo object oriented. M: Yeah, well, that's just the
constructor. Right. This is not the bad part. The bad part is that every single
method that actually does something – like this one – looks like this, right, it
checks that flag again
*laughter in the audience*
M: and it either calls "VolltextRecherche" which is… I guess it also has something to
do with full text search… you can see that there's lines commented out and nobody
remembered to leave a comment as to why it's commented out. And you can see that
this code is really fragile, right. If you want to maintain that kind of code… every
method looks like this. You always need to remember to check that flag before you do
anything to be sure that… "Oh I'm here" or "I'm there".
N: OK yeah. So I mean… OK… So the fact that somebody uses an object-oriented
programming language does not necessarily imply they actually understand what object
orientation is all about. Right. So maybe they are just crappy programmers.
M: OK. N: So they happen to use this and then
they messed it up on the way you know… M: You mean non-crappy programmers can
produce crappy code N: And now you're getting me confused…
*both laugh* N: … probably.
M: I think I have… you have another example.
N: Yes I have a much better example here. A real object-oriented example. So this is
finance, right. So here we have options, in this case just call options, and so the
the financial guys want to talk about those options. But there are so many
different kinds of options so they cannot talk about each of them individually and
so they want to group them together in something they call a basket. So up here
you can see this basket, and it contains Google and Facebook, because they are sort
of like the same, you know. So they just group them together in this
basket and then they want to talk about properties of this basket of all those
options that are contained in it. And for example here they implement this market
data. And so this is standard object- oriented business like everything is a
class, right? And there we have two methods in there because that's also
standard object oriented business: We have a class and then we have methods in there.
And one of the first methods, GetSpot(), just gives us the spot price. So all of
these options have a number in the real world. And that's why they number this
here and now for fun they call this sicovam. I have no idea why, this seems to
be a French word for… I don't know what it means, but yeah, just go with us.
M: So. So this maturity thing that's the date at which point the option expires.
N: Yeah. So that's for the volatility. So you get the volatility for an option. An
option as defined by the underlying asset. And by the date it's due and also by the
price it's supposed to have then, and that's what is encoded here. So we have
the option with this long and then we have the maturity which is a point in time, so
we can properly use a double for this, I think. And then we have the strike which
is the price and probably some of you know that it's not a very good idea to deal
with money in terms of doubles. But on the other hand it's fairly common in banking.
So let's let's do this here as well. Okay fine. So yeah that's it. Yeah. And then
what they actually want to do so they don't only want to look at the world as it
is but you know those guys with the suspenders you know they want to analyze
and see what might happen if something would happen you know. So they want to
play what-if games and for this what-if game here they take the spot price and
they actually want to shift it. So they want to say what if the spot price were
different and what would happen then with my options. And here they are just… you
know, so if you want to modify something in object orientation you write a derived
class and this is the derived class SpotShiftedMarketData, and this derived
class overrides the GetSpot method and it takes the value of the parent class and
multiplies it with the factor. Okay. Sounds straightforward so far. And then of
course they don't only want to multiply this with a factor, but they also want to
do other modifications and they want to be able to do all these modifications without
re-compiling. So they want to play around with these and they don't want to
recompile their code every time they change something. So they want to have a
dynamically configurable code base so to speak, or dynamically configurable market
data. And if you look this up in the object-oriented literature you come up
with the decorator pattern. And so the Decorator pattern is something that allows
you to dynamically modify your code. So you have a component and then you derive a
decorator and the decorator is a delegate to something else. You want to plug-in
basically. And this is how they implemented the delegate so they wrote the
decorator as a derived class. And then they have there that they have the real
object in there and for the GetSpot method here they just ask the GetSpot method of
the underlying object. Fair enough. And now going back to the baskets, where I
started out initially. So what they did in this one derived modification here they
multiplied with a factor. And now in the basket they ... they get here. Here they
get a bunch of options, say, or a bunch of assets and now they want to calculate this
for all of them and they go into this recursively and – it's very hard to see
here – but what actually happens is they go into this recursively and do this
modified operation for all of them and because of late binding it happens that
they keep getting through this upper method again and again so adding the
factor again and again. And of course they did not figure this out by looking at the
code, because it's – like I said – really hard to understand. And of course this
is all just small snippets of the code but this is the essence. But they found this
because the values were incorrect, right, and so what they did is they compensated
for this. And for example here is the most interesting part I think: Does the first
market data derive from the second one? So they check whether they have this chain
going on and then they do something differently. Wow. I mean this is
production code, right. And this is still in operation and probably this method by
now is like hundreds of lines long because they have so many edge cases they need to
compensate for. Well. M: Yeah. So can imagine that. Yeah but I
mean it's obvious they just shouldn't have done this in C++ or using object-oriented
programming, right. N: Phew… so, do you have a better solution?
M: Oh, absolutely. Absolutely. We have slides on this.
N: You have slides! M: People should have used functional
programming, right. Which is great, because…
N: Oh, how could I forget this? M: Yeah.
N: Of course you would say that. M: Yeah. You know, I'm the functional
programming person here on stage.
N: *chuckles* M: So functional programming has all these
advantages, right. You can have immutable data. Immutable data means you have less
coupling. You don't have these complicated effects going on that you saw earlier
maybe. You can use… functional programming is very close to mathematics, so you can
apply formal verification… N: Isn't that scary?
M: Yeah. You can use all these great mathematical things in there. You can have
catamorphisms, you can have bifunctors. That's my new favorite thing. Monadic
profunctors is also something that we've been using lately. That's great. Kleisli
arrows. There's all these wonderful things in functional programming and they solve
all of these problems, right? N: Yes I could…
M: (towards the audience) Why are you laughing? Who's laughing?
N: I should have imagined you saying something like this. So… but… have you
ever considered that you're tackling the problem from the wrong angle. I mean you
and your horde of IT tech nerds, you're sitting in your cave, hacking away, having
your hoodie over your head and just sitting there and hacking and hacking.
M: (takes off his hoodie) *N chuckles, audience laughs*
M: So that problem solved. Okay anyway. N: Yeah. And I mean really just throwing
tech at the problem is not a solution, it's actually part of the problem…
M: But Nicole, we're producing software, we're producing tech, right? Have you gone
through this thing here *points outside the hall*? There's tech outside everywhere
that… How do you make that? N: Probably not by sitting in the corner
and just writing… hacking away. M: OK.
N: So have you considered that there is more to it, that it's not only just tech,
that you need to talk to each other? M: Mm hmm…
N: Yesterday I saw this robot walking around in a circle and it said "I need new
code, please talk to me", right. This is something like this.
M: OK. N: So we need to talk to each other. It's
not only tech. Tech is nice and everything.
M: So where is… Do you have like a pattern manual for that?
N: A pattern manual? Yes, of course. There are talking patterns actually for people
like you. *chuckles, audience laughs* M: *clears throat* OK. So go. Go ahead.
N: So… M: Educate me.
N: OK. So I mean there were many approaches over the years, right. Some
approaches were for example… looking at agile software development you know and it
says "individuals and interactions over processes and tools". Tools… tech…
M: OK N: So let's talk together and let's figure
out things. M: OK well but it does say working
software also, right? N: One of four, you know, says "working
software" because of course we also want working software, right.
M: Yeah. N: I mean, just talking and no coding is
also not the answer. So… M: Yeah, anyway, so last year we were at
this conference, right, and you were probably talking to somebody but I
actually attended the keynote and it was by somebody who talked about what worked
and what were the ongoing problems in software engineering. And this was an
agile company, right? They do everything in an agile manner. So supposedly they
communicate all the time, but still when they look at where they spend all of their
time and all of their work and effort they say that they still spend 53 percent on
maintenance and complexity and not on new features or… I have no idea what that
professionalization thing is actually. But it takes up 18 percent…
N: nice M: So let's get back to the technical
problem, shall we. OK N: Yeah. So I mean we already saw this
right in the market data example. I think much of this maintenance and complexity
problems is caused that we're in a world consisting of objects and so everybody
jumps on this object-oriented bandwagon, right? So what they actually end up with
is something like this *laughter in the audience*
M: OK N: And this is sort of… yeah… not helpful.
M: So maybe but I would like to explain this maybe in a different way. So modern
object oriented programming – while you can all laugh at that slide before – but
we really need to understand the problem, right. The technical problem is this,
right? So at the at the heart of modern object-oriented programming something
called imperative programming where all these objects that you showed on the
previous slides they have what's called encapsulated state. Okay? There's some
state in there. And what happens is the world progresses by all these objects
sending messages to each other. And as a result of a message, some code gets
executed that modifies that encapsulated state. Now the thing was… I mean object-
oriented programming was originally developed to support simulations of the
real world. And the problem is that the real world just does not work like that.
The real world is not a bunch of objects sending messages to each other. So one
simple example that maybe can help explain that is there is an el… Speaking of
elephants, right? N: Yes
M: Great elephant slide. So. So there is an elephant and the elephant comes in from
the jungle and walks into some kind of room, right? And the object-oriented model
for this is… well… all the entities get models, get objects, so that the elephant
has an object, the jungle has an object, and the room that the elephant enters has
an object that represents them. And you have the sequence of method calls or
message sends that tries to reproduce that sequence as well that you know the jung…
that the elephant exits the jungle and the elephant enters the room, right? The
problem is that the exiting of the jungle and the entering of the room are one and
the same act. So the real world has dependencies, it is not just a bunch of
isolated entities that are sending messages to each other. Things hang
together in the real world. So if… so I think a more useful model for thinking
about programs and how programs should model the real world has to do with…
should really go through the way that we perceive things. So, if you watch a soccer
game these days there's lots of objects that you see, right. So you see 22
players, maybe you see a ball, you see the referees and you see lots of people in the
audience and they all move. Now. So, they all change their internal state if that's
the model that you use. Now, if you want to know what's going on in the football
field, you need to observe all of these changes. But, you know what the object
orient model is for observing changes? Something called the observer pattern,
which means that you register which each of these objects and tell them: Well, if
anything changes with you, send me a message. You want your ball, send me a
message if you move, you know, you do that with all the players, supposedly, you do
that with all the 20000 audience members. And of course, when you leave the stadium
you all send the messages: Oh no, I'm no longer interested in what you're doing,
right. The world doesn't work like that. It has another problem, that all these
messages will arrive in some sort of sequential order, and that also would mean
that with all of these objects moving around, we would observe inconsistencies
all the time, the same way that you observed it with the elephants going into
the room, right. If you remember, there was an inconsistent state in the middle
here. Which was that after the first step the elephant is, for a brief amount of
time, is nowhere. It has exited the jungle, but it has not entered the room
yet. And the same is true, of course, if we have many moving objects. But yet, we
never observe, like one person getting up and suddenly appearing in another place,
or two people appearing in the same place at the same time. And that's because our
perceptive apparatus creates consistent snapshots of what we observe, right? We
look at something, and it gets stored in memory and we can think about it for a
little amount of time and analyze what's going on. And so, of course, we remember
things that were in the past, which is also something that an object oriented
model cannot do. So, I think there's fundamental problems with this object
oriented model of programming, and that's one of the fundamental things that leads
to that soul crushing code that we've been talking about.
N: Now, that actually reminds me of a very interesting thing that lots of people are
using, and that's in the UI and it's called the MVC pattern, or Model View
Controller. And if you have ever worked with us, you can see here that this all
goes in circles and you can go in any direction and then you end up here again,
or you go this way around, or anything, or follow this dashed line. And so you can go
from anywhere to everywhere. And this leads to the obvious problem that, if you
have changes in the model, and changes in view, they need to, hopefully, be
corresponding. Or, maybe not. And what do you do about this? So, what you end up
with, if you do this enough, and if you aren't very, very careful, you end up with
this. M: Night MVC fraught programs have always
looked like that, right. N: Yes. Feels really familiar doesn't it.
M: Yeah. Yeah, it does. N: And I think that's also what Alan Kay
was talking about, back in the day. So, this is a quote from 1996. But I think the
paper just appeared then. But the original quote is much older. And so he said, okay,
OOP had those motivations that we described here. But eventually, we need to
overcome this model and we need to find something better, and eliminate it
altogether. That's what he says here. M: But that never happened, right?
N: No, I don't think so. M: Never happened. So if these days, if
you look for examples of how object oriented programming works, it always is
about encapsulated state that Alan Kay wanted to get rid of. So, I think that's
the first thing when I searched for Object Oriented examples, was something with
students, and you can immediately see that a lot of attributes that you have there go
through some kind of, some method that modifies them. My favorite one is that you
have the student and you can set the grade point average of the student. So you can
just,... yeah. So, if you're not happy with the grades that your kid is getting
you just sent the object a message. Get it get a perfect GPA there. So, and let me
reiterate, Nicole, of course people should have used functional programming. Which
has all these simple languages, less complexity, higher productivity, less
bugs. You know, we have all these powerful type systems, we can do property based
testing. We can do, oh it goes on and on, and we get more predictable behavior.
Generally testing is easier, because we don't need, you know, set up and tear down
methods. You get lower couplings, you have fewer dependency cycles. So, I didn't even
say monad, right, here in this list. So you get all these concrete...
N (at the same time): Lucky me. M: You get all these concrete advantages
from doing functional programming, right? And so that's what people should do to
solve those problems. N: So do you remember
M (at the same time): You, too. N: Fred Brooks, he said there is no silver
bullet. M: OK.
N: So, what do you say about that? M: Fred Brooks is an old guy, right?
N: Yes.
*laughter*
N: And what about you? M (laughing): Yeah, I guess I am, too.
M: I'm getting there. Yeah. Well, so Fred Brooks said that, but he's an old guy. So
let me get you one example of why that maybe is not true. So in the early 90s
actually, there was a big study conducted by the U.S. Navy, on the effectiveness of
different programming languages. And they had one set problem that was about
determining the regions of influence of warships and they had different teams
write solutions for their problem in different languages. And what they ended
up doing is also, they gave that also to people who were using functional
programming, specifically in the Haskell programming language, which was still
pretty young back then. And you can see, well, the solution in Haskell is much
shorter than the solution... you know C++, definitely. You know, less than 10 times
as short than the C++ solution. I think Java wasn't as big then. But I think the
factor would also be around between 3 and 10 somewhere. Also, what's maybe
interesting, is that there are two Haskell solutions. One at the top and one of the
bottom, and... N: Probably they just split the code so
the numbers look nice.
*laughter*
M: Yeah, yeah. You would need to add them up, now that would be interesting. But
what they did is, they had a Haskell expert write a solution, and then they
also gave the solution to a student, I think, who learned Haskell for two or
three weeks and who also wrote a solution. And if you look at development time, so
somebody took 10, hours and somebody took 8 hours, and the 8 hours as a student.
That's because the Haskell expert tried to put many interesting
flourishes and super cool programming techniques into the program but the
student was actually doing pretty well. So if that is not a silver if that's
not what a silver bullet looks like I don't know what would.
N: So the Yale study. They got a fixed set of instructions and they just had to code
those instructions. Do I remember that correctly?
M: Yeah. Yeah. Yeah. N: So do you ever experience that in the
real world? Getting a fixed set of instructions from your client and then
just implementing this and they never change their mind? They never come up with
new ideas? They never say "Oh I forgot something". I actually don't experience
this. M: Let me think about it.
N: That's what we need to consider as well. Coming back to the elephant. Talking
to each other. Everybody has different ideas ...
M: Are you gonna start with that agile stuff again?
N: Everybody has different ideas when they look at something and everybody describes
things differently and so what we need to do is to figure this out together. You
know coding a bit, talking a bit, talking a bit, coding a bit. And so this needs to
go hand-in-hand. And this is sort of agile, you know. So it doesn't mean Scrum
and you know standing in a circle every day or something. Relax, okay?
M: So I personally I like to communicate with code.
N: You can do that too, yes. M: That makes me think of an example. We
were working on a project. That was in a semiconductor factory.
N: Wow. So, real stuff! M: So when we started this I thought well
semiconductor the it way gets made: there is a big machine you put in a piece of
silicon and you go like this, and chunk, and outcomes of microprocessor. And it
doesn't doesn't really work that way. One of the reasons is just that the modern
chip consists of many layers. Another one is that there's just many different
production steps that are necessary for making even a single layer. And some of
the machines that make a layer are so expensive that you can't just make an
assembly line. And also a lot of things break all the time in a semiconductor
factory. So it makes no sense to just have an assembly line and push things through
that. But things just move around among the different machines in the
semiconductor factory. Right. And so what's important is that each chip undergoes
or each wafer undergoes a sequence of steps in the factory and that needs to be
managed. And those steps it's typically for big microprocessors that might be
like a thousand steps. So you need to manage something that's that's called a
route which is just a sequence of operations. And now here's a bunch of
Haskell code. Haskell is great because the programs are so short that they fit on
slides but if there's something unclear about that code then I invite you to
interrupt me and ask. So first of all you can read that declaration at the beginning
says "data Operation" and that's just a simplified data type that describes what
an operation would be. You can read that vertical bar as "OR". So it says an
operation is either TrackIn or Process or TrackOut. TrackIn it just means putting a
wafer into a machine and Process is you do something inside the machine and TrackOut
is you take it out of the machine. And then the next thing is that the route is
just a sequence of operations and these brackets that you see there they mean list
of. So what it says is a route is a list of operations. And down here you have an
example for a very simple route that says well route number one might be a list of
the following operations. You put a wafer into a machine, you process it, you
process it some more, and then you take it out again. Clear so far? Everybody nod!
Everybody who's still awake nod! N: *laughter*
M: Don't be afraid to ask. So one thing that you do is is when you have data
types, as you just saw, is you define functions on them that describe some
aspect of what happens in a fab, in a semiconductor factory. So in fact what
happens of course is you need to execute the next step, the next operation. That
happens as part of making a semiconductor and for that we're making a function
called routeHead - the head of the route. And what you do is you write a type
signatures and type signatures are very good for communication actually. So you
put in a route and you get out a single operation and then you write equations
that describe what that function would do on different aspects or different classes
of input. So in this case you remember a route was a list of operations. There are
two different kinds of lists. One type of list is the empty list and the other kind
of list is a list that has what's called a head or first element and a rest. And
because there's two kinds of lists you write two equations. And so that's why you
see two things where it says routeHead something equals to something else. The
first equation is for the empty list. This is why you have these two empty brackets.
And the second one that says well it's a non empty list and the first thing in that
list is some operation ops or there's something called pattern matching you
match this onto the actual list that you see and often gets bound to that first
operation. We're not we don't really care about what comes after that first
operation. And so the second equation is pretty clear if you want the head of a
route that is not empty you just take the first element of that list. OK so far? The
other equation says: what do we do with an empty list? An empty list does not have an
operation. N: So you're saying you're into the second
slide of your beautiful Haskell code and you already don't know what to write
there. M: So we're re communicating right.
N: OK. OK. So M: So you're talking to some expert and
you say: You have got an empty route. What's the first operation of an empty
route? And he says: Well, empty routes they don't really have a first operation. They
only maybe have an operation, and sometimes they don't. So for that we can
create a data type that says that something might be there and sometimes
it's not. And we'll just call it "Option". Are there any Haskell programmers in this
room? This is built in of course as the "maybe" type but I'm just making a
separate type called the "Option" type and that says: The a says this can be
anything. Anything in option a means it can either be there or not. And for that
it has two constructors or two different classes of Maybe objects and one are
called the Some objects and the other ones are called the None objects. Maybe we'll
start with None. So Anything can be a None. It just says that anything is not
there. So anything can be of type option of a. And it means that something is not
there and that is the type of that particular constructor. The other
constructor says while that thing actually is there and so the constructor has to
accept something of type A and then give us something of type Option of a.
N: So you're saying it wraps this object? M: Yes it wraps the object. Okay. And so
if you're not a Hasekell programmer but maybe maybe an F sharp programmer or an ML
programmer then that is what it looks like there. And I believe it's even built into
Java these days something called Optional. So now that means we can change our
routeHead functional a little bit because our first attempt didn't work out and
instead of saying Route --> Operation we write Route --> Option Operation. This may
seem trivial to you but it already communicates a tiny little bit of
something. Then we can write routeHead of the empty list is None. So there's no
routeHead of the empty route. or if we have an Operation coming out we just write
Some in front. And so if we use that example the route that you saw earlier
where our route was "TrackIn, Process, Process, TrackOut" what we get is "Some
TrackIn" so that type communicates a little bit of what we do. Because always
are you calling the shots. Very good. The next thing that we might want to do is we
don't want to always know only the first operation. We also want to know what
happens after that. So we can use that Option and we already know all that. Of
course. An empty route will not have something coming after that first
operation. So we could write another function called routeAdvance and it takes
a route as input. That's to the left of the arrow. It gives us both an operation
and a route. That's why there's these two things in parentheses with a comma in
between. So that's a tuple. So it gives us an operation and a route but only
sometimes when that actually exists which is why there's an option wrapped around
that. So what we want to do is if we take our Route Number one we want to split it
into that first operation and a list of the rest so it should split out the
TrackIn and then give us a list of the remaining or a route of the remaining
operations that are in there. OK so far? It gets technical. Does anybody have a
question? OK. Don't hesitate to ask. I'm looking at the clock. So then this is
actually pretty easy to write now. Again we need to make two equations because a
Route is a list and lists always need two equations. So we can say routeAdvance of
the empty list is None. And routeAdvance of op and the rest is we just return a Sum
of the rest because the list already splits exactly along the line that we
wanted to between the first and the remaining elements. Right. So this is just
very simple code or at least it's short code that communicates what routes are.
N: right, I see. If I remember correctly. I mean you said you put the wafer into the
machine and then it processes it. If I remember correctly some of these
processing steps can be chemical reactions. And so it might be the case
that they must happen in a certain amount of time or something?
M: Yeah. N: Okay. So what we could actually do is
we could model something like this. Yeah. I sneaked at your part of the slide so I
checked this in. Oh yeah. So what we actually could do is we could model
something like those three steps here need to happen together in a set period of
time. M: Yeah. So yeah. So the chemical
reactions mean that your wafer might go bad. If you start the sequence of steps
and you don't finish on time then your wafer goes bad.
N: like etching and washing, for example if you etch too much, then...
M: That's always a problem in existing systems, to model that.
N: Okay. Right. Oh so cool. So maybe I can try and see how I can model this into your
existing code. M: Okay
N: cool. So let's have a look. So what we first need as we we have this Route
element here with this Operation right. And now we need another representation for
a Route element and let's call this RouteQTZone for queue time zone. And there
of course we have the duration. That means the amount of time this step needs to be
finished in, or this sequence of steps needs to be finished in. And then we have
this list of Operations here. We saw this before. Right. The list of the Operations.
And then what this gives us is of course again a routeElement. So we can combine
like ordinary steps like putting the wafer into the machine, doesn't matter how long
it takes, maybe it blocks the machine but other than that no. No issues come from
that. But if we do with the etching and the washing for example we would need to
have a queue time zone because we need to restrict the time that this takes in
conjunction. Right. Is that correct so far?
M: Yeah. N: Good. And then if we look at our
example this is the previous example we had here. We could create another example
r2. Here we have a routeQTZone. Say it may only take five whatever seconds say and
then we have two processes here that need to be finished within five seconds and
then we do the TrackOut again. OK. So far so clear? Good. If we look at this and we
discover that here we have this routeElement list. And here we have this
Operation list. This is actually sort of similar isn't it. So maybe we can make
something out of this and maybe we can actually turn this down here into
routeElements as well. M: Oh so that's what you were doing
before, right. Before up there it also said list of Operation.
N: Up there? M: When you started.... My code had Route
= [Operation]. Right. And now you did the same thing.
N: Oh yes. Yeah. Right. So now it's now it's the same. Again. So it's both [RouteElements ].
Yeah. Good point. So now if we look at this again so maybe this actually means that this
here is not just a list of routeElements but maybe this is actually a route down
there. So I actually derive some information here from the code so we
discover that our routeQTZone actually contains a Route.
M: Oh all right. N: So if we look at this here. What we
then can do here with this RouteElement. So here we could plug in any RouteElement
right? For example we could plug in a RouteQTZone which is also a RouteElement.
So coming from this example here where we have just a flat list that does contain
some RouteOperations and a RouteQTZone and then a RouteOperation we could also stack
them into each other. So a RouteQTZone could again contain a route that contains
a RouteQTZone. M: That's pretty cool.
N: Do you think that would happen. M: Yeah. That's pretty cool because now we
can nest RouteQTZones and that's something that occurs in reality.
N: Oh really does it? Cool. M: And so your model has suggested that
but it is something that our old IT systems based on Object Oriented
Programming couldn't model. That's pretty neat.
N: So now I understand why you're aiming at functional programing and saying that
you can model things better with it. M: Exactly.
N: Yeah. This is pretty cool. So we discovered this in the code and then we
were able to get back to the business people and check back with them whether
they would actually see this in practice and whether it's a valuable addition here
and we discovered that just from looking at the type information we had. Yeah and
then we can continue this and because now we have this list of RouteElements down
there. We could actually say Oh yeah list of RouteElements, we know what that is:
This is actually a route. And now we go one level up. So whenever we learn more
about Routes or whenever we change this here this Route thing it will
automatically be reflected in here because we abstracted it. Right. And that's what
functional programming is all about abstracting things and figuring out what
the common parts are. Right? M: I notice you're agreeing with me. Very
good. N: Excellent. So and now we can also of
course check how our functions will be modified because now we need to go to
all of our functions and see whether they still work. And here we have this
routeHead and also routeElementHead. And now we need to extend this because the
routeElementHead works on the routeElement. And now we have a new
routeElement which is the RouteQTZone. And of course if we want the routeElementHead
of the RouteQTZone this is of course the head of the containing route. And we have
this neat function up here already, routeHead, which gives us the head of the
route. And so we can just reuse this here. So even this has become simpler for us to
implement, because we have discovered that there is a route in there instead of just
a list. So no special handling we can just revert to the standard function we already
have. And if we look at the routeAdvance so if we want to proceed our route to the
next operation and you know proceed to the next step then of course we also need to
add this here the RouteQTZone and now the question is what happens if we advance
into a RouteQTZone process because somehow we need to keep track of when does it need
to be finished. So we initially we said OK it may only take like x time or d time and
now we need to keep track of whether this time is already taken up or not. And so in
this case what we actually do is we need to add another RouteElement and that's the
RouteQTLimit which defines when a started operation needs to be finished. So it has
an actual time which is the point in time like no duration or something just the
finishing point in time and then it's just the same as the RouteQTZone. So this
limits our process up to this end point of time. And now we can implement our
routeAdvance because whenever we have a QTZone that's where we didn't know what to
write before. So whenever we advance this route then we know that we need to come up
with a RouteQTLimit here and whenever we have this RouteQTLimit and we advance over
that one then we know that we just need to basically behave like before, we need to
split this up and also keep our time limit of course. And then just proceed into this
limited while while keeping the time limit. And here if we start this out then
we need to to determine the end point, so we take the current time which we now need
to pass in here at the top. So we take the current time add the duration and then we
know when we need to be finished with this process. And then in here yeah we just we
just work on every step while keeping the time limit.
M: So your code suggested that there was a gap in your understanding of what needs to
be represented right. Yeah. N: So I didn't know what to put to the -
what to implement for the routeAdvance. Yeah. And now I know that. Yeah. I can
handle this. M: So watching this I'm thinking that
there's still a little, if I look at it with the eye of a domain person, I think
there's still a little problem in there. N: Oh. Is it?
M: Yeah. If you look at this right remember a RouteElement is something that
can occur anywhere in the middle beginning at the end of a route so it's these three
things. Obviously we can have RouteQTZones anywhere we can have operations anywhere
but the thing is we can only enter a RouteQTZone when it's at the beginning.
Right? N: Oh I see.
M: So now. N: You mean here we could have a random
list in a random order. M: Yeah. That's right.
N: That's a good point. M: So you can't... So this suggests that
the RouteQTZone has been entered because there is a RouteQTLimit. But there is even
- there there's still an operation there in front of it.
N: Okay. Yeah. That doesn't make sense. M: Yeah. That that makes no sense. So if
we could go back from the domain knowledge to the code now a little bit we could
refine that type further and we could pull out the RouteQTLimit from that type down
there, because these are all the things that occur in the middle of a route and
pull it up to a top level type and introduce an intermediate type
that distinguishes between what's in the middle and what's at the beginning.
N: Oh I see. That's neat. M: So what have as we have this back and
forth between the code which yields insights about the domain and the domain
from the domain back to the code and we can play that game we can use the code for
communication. N: Yeah. So that's what I intended. Did
you understand this? M: Yeah so that code no longer crushes my
soul. N: Excellent.
M: So I can say Yeah right. N: Same here.
M: So. N: Cool.
M: Okay. N: So that's what the point we wanted to
make. So if you take functional programming and if you add communication
and discussions and this going back and forth and learning from both sides and
enriching both sides with the information coming from the other part then you
actually end up at a silver bullet. M: Everybody's looking confused that's a
bullet train, a silver bullet train. OK we're done.
N: Yes thank you. Herald:
So thank you for a very entertaining talk we have some time for questions, Q&A, so if
you have any questions please line up next to the microphones. We have four
microphones spread across the room and we'll start from a question from the
Internet. Signal Angel: There is a question from the
internet: As an FP beginner with a weak background in math would it be preferable
to learn functional programming in a pure functional language or in languages that
are multi paradigm? M: I'll take that one? OK. So I think the
point is that there is many different functional programming languages and
Haskell happens to be what's called a pure one but there are also languages that are
hybrid between object oriented and functional programming. Scala I think is a
prime example right now. And I mean Scala is a fine language. The problem is if you
want to combine these two paradigms you typically get something that's pretty
complicated. So Scala is a complicated language that takes more time to master.
Also, because you have both paradigms available to you at all times. It is
sometimes... You often get confused about what paradigm you should use in a given
situation. So I think both of us we haven't really seen the great advantages
you get from that hybrid model. Right? N : Yes. Also the problem is if you're a
learner in FP and you try to tackle this with something like Scala you will
inevitably fall back to what you know if you run into problems. And so maybe just
jump into the deep end and try to swim and see where you get and get help on the
Internet. Herald: Number two.
Mic2: It was a good presentation. Thank you. Every time I see some functional
programming it's refreshing and it's interesting but the description of your
presentation was talking about using functional programming in IOT. So there
was nothing specific to IOT in this presentation. There was no interaction
with the hardware. No interrupt handling. No nothing. How would you handle that for
example. M : Yeah good point. I think we had a
bunch of slides on that that we ended up dropping because of time. So my argument
would be that IOT is the same software as any other software. What's special about
IOT is the risk that emanates from IOT obviously. So if you want to do things
like interrupt handling I think my response to that would be to convert it
into functional data structures and this gives you a deterministic model for
handling that kind of stuff. So we talked about that. I think we talked a little bit
about the observer pattern which is analogous to what usually happens with
interrupt handling which is what you were talking about. And the way to do that
really is to have... of course you have a tiny bit of imperative code that hooks
your functional code to the hardware or whatever it is. But to convert your
interrupts into a list and you can... Haskell actually is pretty good at that
and that then gives you all the advantages of functional programming: the testability,
the funky abstractions. And you can use that even on the interrupts and it
turns into software just like any other functional software.
Herald: Microphone number one. Mic1: Hi. Thank you for the talk. I have
to write code for microprocessors and most of the time I only have a C compiler and
if I'm lucky I get a C++ compiler. Where can I get a silver bullet?
*laughter* N: The old Haskell compiler used to
compile to C and nowadays it doesn't do that anymore by default but probably you
can make it do that still? M: But there's a number functional
languages that compile to C so it's kind of difficult to give a one shot answer.
We've also done a project for example where we got a lot of the advantages from
functional programming by writing code in Haskell that generates the C code, right.
And so it's difficult to give you like one answer without knowing more details about
what it is that you're doing but there's certainly a spectrum of options available
in that context. Q: So functional programming is pretty
concise and pretty compact so no one really wants to use large variable names
and I saw in your examples like a D and a TTL, RTs. And I don't think that there are
so much better than the long versions you showed earlier. Like what is yout take on
that? N: So the general - I'm also struggling a
little bit with this. One thing is that this was fairly concrete code, right? But
often you go into the abstractions and in the abstraction there it's arbitrary what
you have. So you can use short names because you're not talking about concrete
things anyway. So that's why it's easier to use - also more like abstract variable
names. But what I actually do is I also use longer names so I would not always use
D and RT and something like that to get a better grip on this. But one important
thing you must not forget is that this you have a function, right, and it's two
lines, three lines. And so if you start out and you understand D and RT, for
example, from the signature and reading three lines of code with a D and RT in it
it's not that bad right? And in object oriented or Java or whatever you sometimes
have hundreds of lines and then you have a D and RT and you read this for half an
hour and then of course you forget what it means. And so, you know, it's a bit of a
balance. So I would use longer names as well probably but sometimes also shorter
because, just the code is so short. M: Like if I can add one detail as I find
myself using longer names in dynamically typed languages right there dynamically
typed functional languages. There you don't have the type that spells out what that thing is.
N: Oh right. M: And then you need to put it in the
variable name. Herald: Microphone number 1.
M1: Yeah. In one of the slides you made the claim that a functional programming
lends itself to proving. How would you actually do this?
M: Good question. So N: That's another talk right?
M: I mean in practice, so there's various ways of doing that. So first of all as you
saw, right, a Haskell program is a bunch of equations so you can use algebra as a
mathematical technique to reason about functional programs. First of all, going
to the abstract aspect of that question which is something, well, you can sort of
do it with your Java program but it is much harder to establish an algebraic way
of talking about Java programs. There's also plenty of tooling available to write
to prove aspects of your programs in functional languages, starting with a
classic tool called ACL2. We can talk on something called Idris, so there is a
newer breed of functional languages where you can put more proofs into the types of
your languages. So it just turn, it turns out it tends to be just a magnitude of an
order of magnitude easier to prove properties of functional programming
because you can use algebra and equational reasoning as you would about Java
programs. Does that answer your question a little bit?
M1: A bit. But I think Idris uses like dependent types and doesn't - isn't it
possible to use it in imperative programming as well? Maybe? I don't know.
So is that specific to functional programming dependent types? Or could it
be used... M: It is even the way that it just talks
about imperative programming is functional. Right? Using monads, by the
way. So there is that and there's this whole spiel about reasoning about effects.
But the thing is effects side effects said you have an imperative programming right?
They don't make it impossible to reason about programs but they make it much much
harder. And so in a functional program you tend, in a proper functional program that
doesn't crush your soul, you tend to have large parts that are purely functional and
that allow functional that allow equational reasoning and you have smaller
parts that maybe make - that might be a little bit more difficult that are about
the interaction with the imperative environment.
N: Also the thing that we did with the type system you know you remember? First
we have the list and we have the the RT, the queue time limit, in the middle and
that was an illegal state. And so we changed the type system to disallow this.
And so actually already the compiler tells us: "Hey this is forbidden code." Right?
And it's not syntactically wrong but it's semantically wrong. And that's what is
also like a weak kind of validation or verification in my opinion.
M: So we could talk all about all this about this all day. Sorry.
Herald: Another question from Microphone number 2.
M2: I noticed that you didn't present any mechanism to hide your implementation or
to make things private - the things that you can do in C++. And I don't miss this
mechanism but I would like to know your take on why - I think you don't miss them.
So why you don't miss them either? M: No they exist in functional languages
as well. Right? And so the mechanisms that you're familiar with private things,
modules. Functional languages tend to not use objects and classes for
modularisation. I think that's the primary difference but functional languages tend
to always have mechanisms for doing that. They're just different between languages.
So it's difficult to talk about this in a talk. We could have told you how to do
this in Haskell and write a module on hide things. But would have put more code on
the slides. N: And the other aspect is that you don't
really have to hide things because everything is pure. So if you call this
function - I don't care, you know. You can't destroy anything. Whereas in an
object oriented programming you have this - change my object to really bad
something you know and you don't want anybody else to call this because it will
really break things and then functional programming - chucking some variables get
something out - do this as much as you want. I don't care. That last one is the
take I love. And I endorse fully. Herald: Mic for number 3.
M3: You had some slides on the importance of communication. I have a hard time to
see myself communicating with business people using Haskell code. How do you what
do you *laughter* do about that at this point? How would you do the communication
with like normal people? M: I mean we clean it up somewhat. But
this actually happened. So it's sometimes a little bit of a process to get there and
sometimes... but what you could, I mean, what you noticed maybe in the code with
the routes was that playing with a Haskell code yielded domain insights that you
could have also explained to somebody in a meeting, right?
N: Without showing the code. M: Yeah. And so that's - so the
communication goes through the Haskell code, maybe not. It doesn't happen, you
know, by the Haskell code. But all I can tell you is that that actually happens is
we sometimes communicate with clients showing them code and talking to them
about, is this really what you're doing is this what your domain is like. That
happens. N: Happened to me as well. So it depends
on your clients you know. They need to have a bit of resilience so they can, you
know, and they can accept that they wouldn't understand everything. But if
you, like, talk them through and say look here's this and this goes in and that
comes out and what do you think about this. And then say - so you just not throw
the code at them but you have a informed communication to say.
M: This can also be contractually enforced. And I remember one instance
where we did that. Herald: That actually finished the
questions. Again got a big round of applause for this.
M: Thank you.
*applause*
*35c3 postroll music*
Subtitles created by c3subtitles.de in the year 2020. Join, and help us!