*34c3 intro*
Hanno Böck: Yeah, so many of you probably know me from doing things around IT
security, but I'm gonna surprise you to almost not talk about IT security today.
But I'm gonna ask the question "Can we trust the scientific method?". I want to
start this by giving you which is quite a simple example. So if we do science like
we start with the theory and then we are trying to test if it's true, right? So I
mean I said I'm not going to talk about IT security but I chose an example from IT
security or kind of from IT security. So there was a post on Reddit a while ago,
a picture from some book which claimed that if you use a Malachite crystal that can
protect you from computer viruses. Which... to me doesn't sound very
plausible, right? Like, these are crystals and if you put them on your computer, this book
claims this protects you from malware. But of course if we really want to know, we
could do a study on this. And if you say people don't do Studies on crazy things:
that's wrong. I mean people do studies on homeopathy or all kinds of crazy things
that are completely implausible. So we can do a study on this and what we will do is
we will do a randomized control trial, which is kind of the gold standard of
doing a test on these kinds of things. So this is our question: "Do Malachite
crystals prevent malware infections?" and how we would test that, our study design
is: ok, we take a group of maybe 20 computer users. And then we split them
randomly to two groups, and then one group we'll give one of these crystals and tell
them: "Put them on your desk or on your computer.". Then we need, the other group
is our control group. That's very important because if we want to know if
they help we need another group to compare it to. And to rule out that there are any
kinds of placebo effects, we give these control groups a fake Malachite crystal so
we can compare them against each other. And then we wait for maybe six months and
then we check how many malware infections they had. Now, I didn't do that study, but
I simulated it with a Python script and given that I don't believe that this
theory is true I just simulated this as random data. So I'm not going to go
through the whole script but I'm just like generating, I'm assuming there can be
between 0 and 3 malware infections and it's totally random and then I compare the
two groups. And then I calculate something which is called a p-value which is a very
common thing in science whenever you do statistics. A p-value is, it's a bit
technical, but it's the probability that if you have no effect that you would get
this result. Which kind of in another way means, if you have 20 results in an
idealized world then one of them is a false positive which means one of them
says something happens although it doesn't. And in many fields of science
this p-value of 0.05 is considered that significant which is like these twenty
studies. So one error in twenty studies but as I said under idealized conditions.
So and as it's the script and I can run it in less than a second I just did it twenty
times instead of once. So here are my 20 simulated studies and most of them look
not very interesting so of course we have a few random variations but nothing very
significant. Except if you look at this one study, it says the people with the
Malachite crystal had on average 1.8 malware infections and the people with the
fake crystal had 0.8. So it means actually the crystal made it worse. But also this
result is significant because it has a p-value of 0.03. So of course we can
publish that, assuming I really did these studies.
*applause* B.: And the other studies we just forget
about. I mean they were not interesting right and who cares? Non significant
results... Okay so you have just seen that I created a significant result out of
random data. And that's concerning because people in science - I mean you can really do
that. And this phenomena is called publication bias. So what's happening here
is that, you're doing studies and if they get a positive result - meaning you're
seeing an effect, then you publish them and if there's no effect you just forget
about them. We learned earlier that with this p-value of 0.05 means 1 in 20 studies
is a false positive, but you usually don't see the studies that are not significant,
because they don't get published. And you may wonder: "Ok, what's stopping a
scientist from doing exactly this? What's stopping a scientist from just doing so
many experiments till one of them looks like it's a real result although it's just
a random fluke?". And the disconcerning answer to that is, it's usually nothing.
And this is not just a theoretical example. I want to give you an example,
that has quite some impact and that was researched very well, and that is a
research on antidepressants so called SSRIs. And in 2008 there was a study, the
interesting situation here was, that the US Food and Drug Administration, which is
the authority that decides whether a medical drug can be put on the market,
they had knowledge about all the studies that had been done to register this
medication. And then some researchers looked at that and compared it with what
has been published. And they figured out there were 38 studies that saw that these
medications had a real effect, had real improvements for patients. And from those
38 studies 37 got published. But then there were 36 studies that said: "These
medications don't really have any effect.", "They are not really better than
a placebo effect" and out of those only 14 got published. And even from those 14
there were 11, where the researcher said, okay they have spent the result in a way
that it sounds like these medications do something. But they were also a bunch of
studies that were just not published because they had a negative result. And
it's clear that if you look at the published studies only and you ignore the
studies with a negative result that haven't been published, then these
medications look much better than they really are. And it's not like the earlier
example there is a real effect from antidepressants, but they are not as good
as people have believed in the past.
So we've learnt in theory with publication bias
you can create result out of nothing. But if you're a researcher and you have a
theory that's not true but you really want to publish something about it, that's not
really efficient, because you have to do 20 studies on average to get one of these
random results that look like real results. So there are more efficient ways
to get to a result from nothing. If you're doing a study then there are a lot of
micro decisions you have to make, for example you may have dropouts from your
study where people, I don't know they move to another place or they - you now longer
reach them, so they are no longer part of your study. And there are different things
how you can handle that. Then you may have cornercase results, where you're not
entirely sure: "Is this an effect or not and how do you decide?", "How do you
exactly measure?". And then also you may be looking for different things, maybe
there are different tests you can do on people, and you may control for certain
variables like "Do you split men and women into separate?", "Do you see them
separately?" or "Do you separate them by age?". So there are many decisions you can
make while doing a study. And of course each of these decisions has a small effect
on the result. And it may very often be, that just by trying all the combinations
you will get a p-value that looks like it's statistically significant, although
there's no real effect. So and there's this term called p-Hacking which means
you're just adjusting your methods long enough, that you get a significant result.
And I'd like to point out here, that this is usually not that a scientist says: "Ok,
today I'm going to p-hack my result, because I know my theory is wrong but I
want to show it's true.". But it's a subconscious process, because usually the
scientists believe in their theories. Honestly. They honestly think that their
theory is true and that their research will show that. So they may subconsciously
say: "Ok, if I analyze my data like this it looks a bit better so I will do this.".
So subconsciously, they may p-hack themselves into getting a result that's
not really there. And again we can ask: "What is stopping scientists from
p-hacking?". And the concerning answer is the same: usually nothing. And I came to
this conclusion that I say: "Ok, the scientific method it's a way to create
evidence for whatever theory you like. No matter if it's true or not.". And you may
say: "That's a pretty bold thing to say.". and I'm saying this even though I'm not
even a scientist. I'm just like some hacker who, whatever... But I'm not alone
in this, like there's a paper from a famous researcher John Ioannidis, who
said: "Why most published research findings are false.". He published this in
2005 and if you look at the title, he doesn't really question that most research
findings are false. He only wants to give reasons why this is the case. And he makes
some very possible assumptions if you look at that many negative results don't get
published, and that you will have some bias. And it comes to a very plausible
conclusion, that this is the case and this is not even very controversial. If you ask
people who are doing what you can call science on science or meta science, who
look at scientific methodology, they will tell you: "Yeah, of course that's the
case.". Some will even say: "Yeah, that's how science works, that's what we
expect.". But I find it concerning. And if you take this seriously, it means: if you
read about a study, like in a newspaper, the default assumption should be 'that's
not true' - while we might usually think the opposite. And if science is a method
to create evidence for whatever you like, you can think about something really
crazy, like "Can people see into the future?", "Does our mind have
some extra perception where we can sense things that happen in an hour?". And
there was a psychologist called Daryl Bem and he thought that this is the case and
he published a study on it. It was titled "feeling the future". He did a lot of
experiments where he did something, and then something later happened, and he
thought he had statistical evidence that what happened later influenced what
happened earlier. So, I don't think that's very plausible - based on what we know
about the universe, but yeah... and it was published in a real psychology journal.
And a lot of things were wrong with this study. Basically, it's a very nice example
for p-hacking and just even a book by Daryl Bem, where he describes something
which basically looks like p-hacking, where he says that's how you do
psychology. But the study was absolutely in line with the existing standards in
Experimental Psychology. And that a lot of people found concerning. So, if you can
show that precognition is real, that you can see into the future, then what else
can you show and how can we trust our results? And psychology has debated this a
lot in the past couple of years. So there's a lot of talk about the
replication crisis in psychology. And many effects that psychology just thought were
true, they figured out, okay, if they try to repeat these experiments, they couldn't
get these results even though entire subfields were built on these results.
And I want to show you an example, which is one of the ones that is not discussed so
much. So there's a theory which is called moral licensing. And the idea is that if
you do something good, or something you think is good, then later basically you
behave like an asshole. Because you think I already did something good now, I don't
have to be so nice anymore. And there were some famous studies that had the theory,
that people consume organic food, that later they become more judgmental, or less
social, less nice to their peers. But just last week someone tried to replicate this
original experiments. And they tried it three times with more subjects and better
research methodology and they totally couldn't find that effect. But like what
you've seen here is lots of media articles. I have not found a single
article reporting that this could not be replicated. Maybe they will come but yeah
there's just a very recent example. But now I want to have a small warning for you
because you may think now "yeah these psychologists, that all sounds very
fishy and they even believe in precognition and whatever", but maybe your
field is not much better maybe you just don't know about it yet because nobody
else has started replicating studies in your field. And there are other fields
that have replication problems and some much worse for example the pharma company
Amgen in 2012 they published something where they said "We have tried to
replicate cancer research and preclinical research" that is stuff in a petri dish or
animal experiments so not drugs on humans but what happens before you develop a drug
and they were only able to replicate 47 out of 53 studies. And these were they
said landmark studies, so studies that have been published in the best journals.
Now there are a few problems with this publication because they have not
published their applications they have not told us which studies these were that they
could not replicate. In the meantime I think they have published three of these
replications but most of it is a bit in the dark which points to another problem
because they say they did this because they collaborated with the original
researchers and they only did this by agreeing that they would not publish the
results. But it still sounds very concerning so but some fields don't have a
replication problem because just nobody is trying to replicate previous results I
mean then you will never know if your results hold up. So what can be done about
all this and fundamentally I think the core issue here is that the scientific
process is tied together with results, so we do a study and only after that we
decide whether it's going to be published. Or we do a study and only after we have
the data we're trying to analyze it. So essentially we need to decouple the
scientific process from its results and one way of doing that is pre-registration
so what you're doing there is that before you start doing a study you will register
it in a public register and say "I'm gonna do a study like on this medication or
whatever on this psychological effect" and that's how I'm gonna do it and then later
on people can check if you really did that. And yeah that's what I said. And this
is more or less standard practice in medical drug trials the summary about it
is it does not work very well but it's better than nothing. So, and the problem
is mostly enforcement so people register study and then don't publish it and
nothing happens to them even though they are legally required to publish it. And
there are two campaigns I'd like to point out, there's the all trials campaign which
has been started by Ben Goldacre he's a doctor from the UK and they like demand
that like every trial it's done on medication should be published. And
there's also a project by the same guy the compare project and they are trying to see
if a medical trial has been registered and later published did they do the same or
did they change something in their protocol and was there a reason for it or
did they just change it to get a result, which they otherwise wouldn't get.But then
again like these issues in medicine they offer get a lot of attention and for good
reasons because if we have bad science in medicine then people die, that's pretty
immediate and pretty massive. But if you read about this you always have to think
that these issues in drug trials at least they have pre-registration, most
scientific fields don't bother doing anything like that. So whenever you hear
something about maybe about publication bias in medicine you should always think
the same thing happens in many fields of science and usually nobody is doing
anything about it. And particularly to this audience I'd like to say there's
currently a big trend that people from computer science want to revolutionize
medicine: big data and machine learning, these things, which in principle is ok but
I know a lot of people in medicine are very worried about this and the reason is,
that these computer science people don't have the same scientific standards as
people in medicine expect them and might say "Yeah we don't need really need to do
a study on this it's obvious that this helps" and that is worrying and I come
from computer science and I very well understand that people from medicine are
worried about this. So there's an idea that goes even further as pre-registration
and it's called registered reports. There is a couple of years ago some scientists
wrote an open letter to the Guardian where they.. that was published there and the idea
there is that you turn the scientific publication process upside down, so if you
want to do a study the first thing you would do with the register report is, you
submit your design your study design protocol to the journal and then the
journal decides whether they will publish that before they see any result, because
then you can prevent publication bias and then you prevent the journals only publish
the nice findings and ignore the negative findings. And then you do the study and
then it gets published but it gets published independent of what the result
was. And there of course other things you can do to improve science, there's a lot
of talk about sharing data, sharing code, sharing methods because if you want to
replicate a study it's of course easier if you have access to all the details how the
original study was done. Then you could say "Okay we could do large
collaborations" because many studies are just too small if you have a study with
twenty people you just don't get a very reliable outcome. So maybe in many
situations it would be better get together 10 teams of scientists and let them all do
a big study together and then you can reliably answer a question. And also some
people propose just to get higher statistical thresholds that p-value of
0.05 means practically nothing. There was recently a paper that just argued which
would just like put the dot one more to the left and have 0.005 and that would
already solve a lot of problems. And for example in physics they have they have
something called Sigma 5 which is I think zero point and then 5 zeroes and 3 or
something like that so in physics they have much higher statistical thresholds.
Now whatever if you're working in any scientific field you might ask yourself
like "If we have statistic results are they pre registered in any way and do we
publish negative results?" like we tested an effect and we got nothing and are there
replications of all relevant results and I would say if you answer all these
questions with "no" which I think many people will do, then you're not really
doing science what you're doing is the alchemy of our time.
*Applause* Thanks.
Herald: Thank you very much.. Hanno: No I have more, sorry, I have
three more slides, that was not the finishing line. Big issue is also that
there are bad incentives in science, so a very standard thing to evaluate the impact
of science is citation counts for you say "if your scientific study is cited a lot
then this is a good thing and if your journal is cited a lot this is a good
thing" and this for example the impact factor but there are also other
measurements. And also universities like publicity so if your study gets a lot of
media reports then your press department likes you. And these incentives tend to
favor interesting results but they don't favor correct results and this is bad
because if we are realistic most results are not that interesting, most results
will be "Yeah we have this interesting and counterintuitive theory and it's totally
wrong" and then there's this idea that science is self-correcting. So if you
confront scientists with these issues with publication bias and peer hacking surely
they will immediately change that's what scientists do right? And I want to cite
something here with this sorry it's a bit long but "There are some evidence that
inferior statistical tests are commonly used research which yields non significant
results is not published." That sounds like publication bias and then it also
says: "Significant results published in these fields are seldom verified by
independent replication" so it seems there's a replication problem. These wise
words were set in 1959, so by a statistician called Theodore Sterling and
because science is so self-correcting in 1995 he complained that this article
presents evidence that published result of scientific investigations are not a
representative sample of all scientific studies. "These results also indicate that
practice leading to publication bias has not changed over a period of 30 years" and
here we are in 2018 and publication bias is still a problem. So if science is self-
correcting then it's pretty damn slow in correcting itself, right? And finally I
would like to ask you, if you're prepared for boring science, because ultimately, I
think, we have a choice between what I would like to call TEDTalk science and
boring science.. *Applause*
.. so with tedtalk science we get mostly positive and surprising results and
interesting results we have large defects many citations lots of media attention and
you may have a TED talk about it. Unfortunately usually it's not true and I
would like to propose boring science as the alternative which is mostly negative
results, pretty boring, small effects but it may be closer to the truth. And I would
like to have boring science but I know it's a pretty tough sell. Sorry I didn't
hear that. Yeah, thanks for listening. *Applause*
Herald: Thank you. Hanno: Two questions, or?
Herald: We don't have that much time for questions, three minutes, three minutes
guys. Question one - shoot. Mic: This isn't a question but I just
wanted to comment Hanno you missed out a very critical topic here, which is the use
of Bayesian probability. So you did conflate p-values with the scientific
method which isn't.. which gave the rest of you talk. I felt a slightly unnecessary
anti science slant. On p, p-values isn't the be-all and end-all of the scientific
method so p-values is sort of calculating the probability that your data will happen
given that no hypothesis is true whereas Bayesian probability would be calculating
the probability that your hypothesis is true given the data and more and more
scientists are slowly starting to realize that this sort of method is probably a
better way of doing science than p-values. So this is probably a a third alternative
to your sort of proposal boring science is doing the other side's Bayesian
probability. Hanno: Sorry yeah, I agree with you I
unfortunately I only had half an hour here.
Herald: Where are you going after this like where are we going after this lecture
can they find you somewhere in the bar? Hanno: I know him..
Herald: You know science is broken but then scientists it's a little bit like the
next lecture actually that's waiting there it's like: "you scratch my back and I
scratch yours for publication". Hanno: Maybe two more minutes?
Herald: One minute. Please go ahead.
Mic: Yeah hi, thank you for your talk. I'm curious so you've raised, you know, ways
we can address this assuming good actors, assuming people who want to do better
science that this happens out of ignorance or willful ignorance. What do we do about
bad actors. So for example the medical community drug companies, maybe they
really like the idea of being profitably incentivized by these random control
trials, to make out essentially a placebo do something. How do we begin to address
them current trying to maliciously p-hack or maliciously abuse the pre-reg system or
something like that? Hanno: I mean it's a big question, right?
But I think if the standards are kind of confining you so much that there's not
much room to cheat that's way out right and a basis and also I don't think
deliberate cheating is that much of a problem, I actually really think the
bigger problem is people honestly believe what they do is true.
Herald: Okay one last, you sir, please? Mic: So the value in science is often an
account of publications right? Account of citations so and so on, so is it true that
to improve this situation you've described, journals of whose publications
are available, who are like prospective, should impose more higher standards so the
journals are those who must like raise the bar, they should enforce publication of
protocols before like accepting and etc etc. So is it journals who should, like,
do work on that or can we regular scientists do something also? I mean you
can publish in the journals that have better standards, right? There are
journals that have these registered reports, but of course I mean as a single
scientist is always difficult because you're playing in a system that has all
these wrong incentives. Herald: Okay guys that's it, we have to
shut down. Please. There is a reference better science dot-org, go there, and one
last request give really warm applause!
*Applause*
*34c3 outro*
subtitles created by c3subtitles.de in the year 2018. Join, and help us!