*rC3 preroll music*
Herald: Welcome with me with a big round of applause in your living room or
wherever you are derJoram. derJoram is a science communicator. He got his
University education and his first scientific experience at Max Planck
Institute. And he will give you now a crash course for beginners to have the
best insight into the scientific method and to distinguish science from rubbish.
derJoram, the stage is yours.
derJoram: Hi, nice to have you here. My name is Joram Schwartzmann and I'm a plant
biologist. And today I want to talk about science. I have worked in research for
many years, first during my diploma thesis and then during my doctoral research. I've
worked both in Universities and at the Max Planck Institute. So I got pretty good
insights into the way these structures work. After my PhD, I left the research
career to instead talk about science, which is also what I'm about to do today.
I am working now in science communication, both as a job and in my spare time, when I
write about molecular plant research online. Today, I will only mention plants
a tiny bit because the topic is a different one. Today though, we are
talking about science literacy. So basically, how does the scientific system
work? How do you read scientific information and which information can you
trust? Science. It's kind of a big topic. Before we start, it's time for some
disclaimers: I am a plant biologist. I know stuff about STEM research that is
science, technology, engineering and mathematics. But there's so much more
other science out there. Social science and humanities share many core concepts
with natural sciences, but have also many approaches that are unique to them. I
don't know a lot about the way these works, so please forgive me if I stick
close to what I know, which is STEM research. Talking about science is also
much less precise than doing the science. For pretty much everything that I'll bring
up today there is an example where it is completely different. So if in your
country, field of research or experience something is different, we're probably
both right about whatever we're talking. With that out of the way, let's look at
the things that make science *science*. There are three parts of science that are
connected. The first one is the scientific system. This is the way science is done.
Next up, we have people, who do the science. The scientific term for them is
researchers. We want to look at how you become a researcher, how researchers
introduce biases and how they pick their volcanic layer to do evil science.
Finally, there are publications and this is the front end of science, the stuff we
look at most of the time when we look at science. There are several different kinds
and not all of them are equally trustworthy. Let's begin with the
scientific system. We just don't do science, we do science systematically.
Since the first people tried to understand the world around them, we have developed a
complex system for science. At the core of that is the scientific method. The
scientific method gives us structure and tools to do science. Without it, we end up
in the realm of guesswork, anecdotes and false conclusions. Here are some of my
favorite things that were believed before the scientific method became standard.
Gentlemen could not transmit disease. Mice are created from grain and cloth. Blood is
exclusively produced by the liver. Heart shaped plants are good for the heart. But
thanks to the scientific method, we have a system that allows us to make confident
judgment on our observations. Let's use an example. This year has aged me
significantly and so as a newly formed old person, I have pansies on my balcony. I
have blue ones and yellow ones, and in summer I can see bees buzz around the
flowers. I have a feeling, though, that they like the yellow ones better. That
right there is an observation. I now think to myself *I wonder if they prefer the*
*yellow flowers over the blue ones based on the color* and this is my hypothesis. The
point of a hypothesis is to test it so I can accept it or reject it later. So I
come up with a test. I count all bees that land on yellow flowers and on blue flowers
within a weekend. That is my experiment. So I sit there all weekend with one of
these clicky things in each hand and count the bees on the flowers. Every time a bee
lands on a flower, I click. *click, click, click, click, click*. It's the most fun I
had all summer. In the end, I look at my numbers. These are my results. I saw sixty
four bees on the yellow flowers and twenty seven on the blue flowers. Based on my
experiment I conclude that bees prefer yellow pansies over blue ones. I can now
return and accept my hypothesis. Bees do prefer yellow flowers over blue ones.
Based on that experiment I made a new observation and can now make a new
hypothesis: do other insects follow the same behavior? And so I sat there again
next weekend, counting all hoverflies on my pansies. Happy days. The scientists in
the audience are probably screaming by now. I am, too, but on the inside. My
little experiment and the conclusions I did were flawed. First up, I didn't do any
controls apart from yellow versus blue. What about time? Do the days or seasons
matter? Maybe I picked up the one time period when bees actually do prefer yellow
but on most other days they like blue better? And then I didn't control for
position. Maybe the blue ones get less sunlight and are less warm and so a good
control would have been to swap the pots around. I also said I wanted to test
color. Another good control would have been to put up a cardboard cutout of a
flower in blue and yellow and see whether it is the color or maybe another factor
that attracts the bees. And then I only counted once. I put the two data points
into an online statistical calculator and when I had calculated it, it told me I had
internet connectivity problems. So I busted out my old textbook about
statistics. And as it turns out, you need repetitions of your experiment to do
statistics and without statistics, you can't be sure of anything. If you want to
know whether what you measure is random or truly different between your two
conditions, you do a statistical test that tells you with what probability your
result could be random. That is called a P-value. You want that number to be low.
In biology, we're happy with a chance of one in twenty. So five percent that the
difference we observe between two measurements happened by chance. In high
energy particle physics, that chance of seeing a random effect is 1:3.500.000
or 0.00003%. So without statistics, you can never be sure whether
you observe something important or just two numbers that look different. A good
way to do science is to do an experiment a couple of times, three at least, and then
repeat it with controls again at least three times. With a bigger data set, I
could actually make an observation that holds significance. So why do I tell you
all of this? You want to know how to understand science not how to do it
yourself? Well, as it turns out, controls and repetitions are also a critical point
to check when you read about scientific results. Often enough cool findings are
based on experiments that didn't control for certain things or that are based on
very low numbers of repetitions. You have to be careful with conclusions from these
experiments as they might be wrong. So when you read about science, look for
science that they followed the scientific method like a clearly stated hypothesis,
experiments with proper controls and enough repetitions to do solid statistics.
It seems like an obvious improvement for the scientific system to just do more
repetitions. Well, there is a problem with that. Often experiments require the
researchers to break things. Maybe just because you take the things out of their
environment and into your lab, maybe because you can only study it when it's
broken. And as it turns out, not all things can be broken easily. Let me
introduce you to my scale of how easy it is to break the thing you study. All the
way to the left, you have things like particle physics. It's easy to break
particles. All you need is a big ring and some spare electrons you put in there
really, really fast. Once you have these two basic things, you can break millions
of particles and measure what happens so you can calculate really good statistics
on them. Then you have other areas of physics. In material science. the only
thing that stops you from testing how hard a rock is, is the price of your rock.
Again, that makes us quite confident in the material properties of things. Now we
enter the realm of biology. Biology is less precise because living things are not
all the same. If you take two bacterial cells of the same species, they might
still be slightly different in their genome. But luckily we can break millions
of bacteria and other microbes without running into ethical dilemmas. We even ask
researchers to become better at killing microbes. So doing more of the experiment
is easier when working with microbes. It gets harder, though, with bigger and more
complex organisms. Want to break plants in a greenhouse or in a field? As long as you
have the space, you can break thousands of them for science and no one minds. How
about animals like fish and mice and monkeys? There it gets much more
complicated very quickly. While we are happy to kill thousands of pigs every day
for sausages, we feel much less comfortable doing the same for science.
And it's not a bad thing when we try to reduce harm to animals. So while you
absolutely can do repetitions and controls and animal testing, you usually are
limited by the number of animals you can break for science. And then we come to
human biology. If you thought it was hard doing lots of repetitions and controls in
animals, try doing that in humans. You can't grow a human on a corn sugar based
diet just to see what would happen. You can't grow humans in isolation and you
can't breed humans to make more cancer as a control in your cancer experiment. So
with anything that involves science in humans, we have to have very clever
experiment design to control for all the things that we can't control. The other
way to do science on humans, of course, is to be a genetic life form and disk-
operating system. What this scale tells us is how careful we have to be with
conclusions from any of these research areas. We have to apply a much higher
skepticism when looking at single studies on human food than when we study how hard
a rock is. If I'm interested in stuff on the right end of the spectrum, I'd rather
see a couple of studies pointing at a conclusion. Whereas the further I get to
the left hand side, the more I trust single studies. That still doesn't mean
that there can't be mistakes in particle physics, but I hope you get the idea. Back
to the scientific method. Because it is circular, it is never done, and so is
science. We can always uncover more details, look at related things and refine
our understanding. There's no field where we could ever say: Ok, let's pack up. We
know now everything. Good job, everyone - the science has been completely done.
Everything in science can be potentially overturned. Nothing is set in stone.
However, and it's a big however, it's not likely that this happens for most things.
Most things have been shown so often that the chance that we will find out that
water actually boils at 250 degrees centigrade at sea level and normal
pressure is close to zero. But if researchers would be able to show that
strange behavior of water, it is in the nature of science to include that result
in our understanding. Even if that breaks some other ideas that we have about the
world. That is what sets science apart from dogma. New evidence is not frowned
upon and rejected, but welcomed and integrated into our current understanding
of the world. Enough about a scientific system. Let's talk about scientists. You
might be surprised to hear, but most researchers are actually people. Other
people, who are not researchers tend to forget that, especially when they talk
about the science that the researchers do. That goes both ways. There are some that
believe in the absolute objective truth of science. Ignoring all influence
researchers have on the data. And there are others, who say that science is lying
about things like vaccinations, climate change or infectious diseases. Both groups
are wrong. Researchers are not infallible demigods that eat nature and poop wisdom.
They're also not conspiring to bring harm to society in search for personal gain.
Trust me. I know people, who work in pesticide research, they're as miserable
as any other researcher. Researchers are people. And so they have thoughts and
ideas and wishes and biases and faults and good intentions. Most people don't want to
do bad things and inflict harm on others and so do researchers. They aim to do good
things and make lives of people better. The problem with researchers being people
is that they are also flawed. We all have cognitive biases that shape the way we
perceive and think about the world. And in science, there's a whole list of biases
that affect the way we gather data and draw conclusions from it. Luckily, there
are ways to deal with most biases. We have to be aware of them, address them and
change our behavior to avoid them. What we can't do is deny their impact on research.
Another issue is diversity. Whenever you put a group of similar people together,
they will only come up with ideas that fit within their group. That's why it is a
problem when only white men are dominating research leadership positions. *Hold on*.
Some of you might shout. *These men are men of science. They are objective. They*
*use the scientific method. We don't need diversity. We need smart people*. To which
I answer: *ugghhh*. Here is a story for you. For more than 150 years, researchers
believed that only male birds are singing. It fits the simple idea that male birds do
all the mating rituals and stuff, so they must be the singers. Just like in humans,
female birds were believed to just sit and listen while the men shout at each other.
In the last 20 years, this idea was debunked. New research found that also
female birds sing. So how did we miss that for so long? Another study on the studies
found that during these 20 years that overturned the dogma of male singing
birds, the researchers changed. Suddenly, more women took part in research and
research happened in more parts of the world. Previously, mostly men in U.S.,
Canada, England and Germany were studying singing birds in their countries. As a
result, they subconsciously introduced their own biases and ideas into the work.
And so we believe for a long time that female birds keep their beaks shut. Only
when the group of researchers diversified, we got new and better results. The male
researchers didn't ignore the female songbirds out of bad faith. The men were
shaped by their environment but they didn't want to do bad things. They just
happened to oversee something that someone with a different background would pick up
on. What does this tell us about science? It tells us that science is influenced
consciously or subconsciously by internal biases. When we talk about scientific
results we need to take that into account. Especially in studies regarding human
behavior. We have to be very careful about experiment design, framing and
interpretation of results. If you read about science that makes bold claims about
the way we should work, interact or communicate in society that science is
prone to be shaped by bias and you should be very careful when drawing conclusions
from it. I personally would rather wait for several studies pointing in a similar
direction before I draw major conclusions. I linked to a story about a publication
about the influence of female mentors on career success and it was criticized for a
couple of these biases. If we want to understand science better, we also have to
look at how someone becomes a scientist and I mean that in a sense of professional
career. Technically, everybody is a scientist as soon as they test a
hypothesis, observe the outcome and repeat. But unfortunately, most of us are
not paid for the tiny experiments during our day to day life. If you want to become
a scientist, you usually start by entering academia. Academia is the world of
Universities, Colleges and research institutes. There is a lot of science done
outside of academia, like in research and development in industry or by individuals
taking part in DIY science. As these groups rarely enter the spotlight of
public attention, I will ignore them today. Sorry. So this is a typical STEM
career path. You begin as a Bachelor's or Master's student. You work for something
between three months and a year and then *wohoo* you get a degree. From here you
can leave, go into the industry, be a scientific researcher at a University or
you continue your education. If you continue, you're most likely to do a PhD.
But before you can select one of the exciting options on a form when you order
your food, you have to do research. For three to six years, depending on where you
do your PhD, you work on a project and most likely will not have a great time.
You finish with your degree and some publications. A lot of people leave now
but if you stay in research, you'll become a postdoc. The word postdoc comes from the
word "doc" as in doctorate and "post" as in you have to post a lot of application
letters to get a job. Postdocs do more research, often on broader topics. They
supervise PhD students and are usually pretty knowledgeable about their research
field. They work and write papers until one of two things happen. The German
Wissenschaftszeitvertragsgesetz bites them in the butt and they get no more contract
or they move on to become a group leader or professor. Being a professor is great.
You have a permanent research position, you get to supervise and you get to talk
to many cool other researchers. You probably know a lot by now, not only about
your field but also many other fields in your part of science as you constantly go
to conferences because they have good food and also people are talking about science.
Downside is, you're probably not doing any experiments yourself anymore. You have
postdocs and PhD students, who do that for you. If you want to go into science,
please have a look at this. What looks like terrible city planning is actually
terrible career planning as less than one percent of PhDs will ever reach the level
of professor, also known as the only stable job in science. That's also what
happened to me, I left academia after my PhD. So what do we learn from all of this?
Different stages of a research career correlate with different levels of
expertise. If you read statements from a Master's student or professor, you can get
an estimate for how much they know about their field and in turn for how solid
their science is. Of course, this is just a rule of thumb- I have met both very
knowledgeable Master's students and professors, who knew nothing apart from
their own small work. So whenever you read statements from researchers independent of
their career stage, you should also wonder whether they represent the scientific
consensus. Any individual scientist might have a particular hot take about something
they care about but in general, they agree with their colleagues. When reading about
science that relates to policies or public debates, it is a good idea to explore
whether this particular researcher is representing their own opinion or the one
of their peers. Don't ask the researcher directly though, every single one of them
will say that, of course, they represent the majority opinion. The difference
between science and screwing around is writing it down, as Adam Savage once said.
Science without publications is pretty useless because if you keep all that
knowledge to yourself, well, congrats, you are very smart now but that doesn't really
help anyone but you. Any researchers' goal, therefore, is to get their findings
publicly known so that others can extend the work and create scientific progress.
So let's go back to my amazing bee research. I did the whole experiment again
with proper controls this time and now I want to tell people about it. The simplest
way to publish my findings would be to tweet about it. But then a random guy
would probably tell me that I'm wrong and stupid and should go f*** myself. So
instead I do what most researchers would do and go to a scientific conference.
That's where researchers hang out, have a lot of coffee and sit and listen to talks
from other researchers. Conferences are usually the first place that new
information becomes public. Well, public is a bit of a stretch, usually the talks
are not really recorded or made accessible to anyone, who wasn't there at the time.
So while the information is pretty trustworthy, it remains fairly
inaccessible to others. After my conference talk, the next step is to write
up all the details of my experiment and the results in a scientific paper. Before
I send this to an editor at a scientific journal, I could publish it myself as a
pre-print. These pre-prints are drafts of finished papers that are available to read
for anyone. They are great because they provide easy access to information that is
otherwise often behind paywalls. They are not so great because they have not yet
been peer reviewed. If a pre-print hasn't also been published with peer review, you
have to be careful with what you read as it is essentially only the point of view
of the authors. Peer review only happens when you submit your paper to a journal.
Journals are a whole thing and there have been some great talks in the past about
why many of them are problematic. Let's ignore for a second how these massive
enterprises collect money from everyone they get in contact with and let's focus
instead on what they're doing for the academic system. I send them my paper, an
editor sees if it's any good and then sends my paper to two to three reviewers.
These are other researchers that then critically check everything I did and
eventually recommend accepting or rejecting my paper. If it is accepted, the
paper will be published. I pay a fee and the paper will be available online. Often
behind a paywall, unless I pay some more cash. At this point, I'd like to have a
look at how a scientific paper works. There are five important parts to any
paper. The title, the author list, the abstract, the figures and the text. The
title is a summary of the main findings and unlike in popular media, it is much
more descriptive. Where a newspaper leaves out the most important information to get
people to read the article, the information is right there in the title of
the study. In my case that could be "Honeybees -Apis mellifera- show selective
preference for flower color in viola tricolor". You see, everything is right
there. The organisms I worked with and the main result I found. Below the title
stands the author list. As you might have guessed, the author list is a list of
authors. Depending on the field the paper is from, the list can be ordered
alphabetically or according to relative contribution. If it is contribution then
you usually find the first author to have done all the work or the middle authors to
have contributed some smaller parts and the last author to have paid for the whole
thing. The last author is usually a group leader or professor. A good way to learn
more about a research group and their work is to search for the last author's name. The
abstract is a summary of the findings. Read this to get a general idea of what
the researchers did and what they found. It is very dense in information but it is
usually written in a way that also researchers from other fields can
understand at least some of it. The figures are pretty to look at and hold the
key findings in most papers and the text has the full story with all the details or
the jargon and all your references that the research is built on. You probably
won't read the text unless you care a lot, so stick to title, abstract and authors to
get a quick understanding of what's going on. Scientific papers to reflect a peer
reviewed opinion of one or a few research groups. If you are interested in a broader
topic like what insects like to pollinate what flower, you should read review
papers. These are peer reviewed summaries of a much broader scope, often weighing
multiple points of view against each other. Review papers are a great resource
that avoids some of the biases individual research groups might have about their
topic. So my research is reviewed and published. I can go back now and start
counting butterflies, but this is not where the publishing of scientific results
ends. My institute might think that my bee counting is not even bad, it is actually
amazing and so they will issue a press release. Press releases often emphasize
the positive parts of a study while putting them into context of something
that's relevant to most people. Something like "bees remain attracted to yellow
flowers despite the climate crisis". The facts in a press release are usually
correct but shortcomings of a study that I mentioned in a paper are often missing
from the press release. Because my bee study is really cool and because the PR
department of my institute did a great job, journalists pick up on the story. The
first ones are often journals with a focus on science like *Scientific American* or
*Spektrum der Wissenschaft*. Most of the time, science journalists do a great job
in finding more sources and putting the results into context. They often ask other
experts for their opinion and they break down the scientific language into simpler
words. Science journalism is the source I recommend to most people when they want to
learn about a field that they are not experts in. Because my bee story is
freaking good, mainstream journalists are also reporting on it. They are often
pressed for time and write for much broader audience, so they just report the
basic findings, often putting even more emphasis on why people should care.
Usually climate change, personal health or now Covid. Mainstream press coverage is
rarely as detailed as the previous reporting and has the strongest tendency
to accidentally misrepresent facts or add framing that researchers wouldn't use. Oh,
and then there is the weird uncle, who posts a link to the article on their
Facebook with a blurb of text that says the opposite of what the study actually
did. As you might imagine, the process of getting scientific information out to the
public quickly becomes a game of telephone. What is clearly written in the
paper is framed positively in a press release and gets watered down even more
once it reaches mainstream press. So for you, as someone, who wants to understand
the science, it is a good idea to be more careful the further you get away from your
original source material. While specific scientific journalism usually does a good
job in breaking down the facts without distortion, the same can't be said for
popular media. If you come across an interesting story, try to find another
version of it in a different outlet, preferably one that is more catered to an
audience with scientific interest. Of course, you can jump straight to the
original paper but understanding the scientific jargon can be hard and
misunderstanding the message is easy, so it can do more harm than good. We see that
harm now with Hobbyists, when epidimi..., epidimio..., epediomiolo.., who are not
people, who study epidemics, who are making up their own pandemic modeling.
They are cherry picking bits of information from scientific papers without
understanding the bigger picture and context and then post their own charts on
Twitter. It's cool if you want to play with data in your free time, and it's a
fun way to learn more about a topic but it can also be very misleading and harmful
while dealing with a pandemic if expert studies have to fight for attention with
nonexperts Excel-graphs. It pays off to think twice about whether you're actually
helping by publishing your own take on a scientific question. Before we end, I want
to give you some practical advice on how to assess the credibility of a story and
how to understand the science better. This is now an in-depth guide to fact checking.
I want you to get a sort of gut feeling about science. When I read scientific
information, these are the questions that come to my mind. First up, I want to ask
yourself, is this plausible and does this follow the scientific consensus? If both
answers are "no" then you should carefully check the sources. More often than not,
these results are outliers that somebody exaggerated to get news coverage or
someone is actively reframing scientific information for their own goals. To get a
feeling about scientific consensus on things, it is a good idea to look for
joint statements from research communities. Whenever an issue that is
linked to current research comes up for public debate, there is usually a joint
statement laying down the scientific opinion signed by dozens or even hundreds
of researchers, like, for example, from Scientists for Future. And then whenever
you see a big number, you should look for context. When you read statements like "We
grow sugar beet on an area of over 400,000 hectare", you should immediately ask
yourself "Who is we? Is it Germany, Europe, the world? What is the time frame?
Is that per year? Is that a lot? How much is that compared to other crops?". Context
matters a lot and often big numbers are used to impress you. In this case, 400,000
hectare is the yearly area that Germany grows sugar beet on. A wheat, for example,
is grown on over 3 million hectare per year in Germany. Context matters, and so
whenever you see a number, look for a frame of reference. If the article doesn't
give you one, either, go and look for yourself or ignore the number for your
decision making based on the article. Numbers only work with framing, so be
aware of it. I want you to think briefly about how you felt when I gave you that
number of 400,000 hectare. Chances are that you felt a sort of feeling of unease
because it's really hard to imagine such a large number. An interesting exercise is
to create your own frame of reference. Collect a couple of numbers like total
agricultural area of your country, the current spending budget of your
municipality, the average yearly income, or the unemployment rate in relative and
absolute numbers. Keep the list somewhere accessible and use it whenever you come
across a big number that is hard to grasp. Are 100,000€ a lot of money in context of
public spending? How important are 5,000 jobs in context of population and
unemployment? Such a list can defuze the occasional scary big number in news
articles, and it can also help you to make your point better. Speaking of framing,
always be aware, who the sender of the information is. News outlets rarely have a
specific scientific agenda, but NGOs do. If Shell, the oil company, will provide a
leaflet where they cite scary numbers and present research that they funded that
finds that oil drilling is actually good for the environment but they won't
disclose, who they work with for the study, we all would laugh at that
information. But if we read a leaflet from an environmental NGO in Munich that is
structurally identical but with a narrative about glyphosate in beer that
fits our own perception of the world, we are more likely to accept the information
in the leaflet. In my opinion, both sources are problematic and I would not
use any of them to build my own opinion. Good journalists put links to the sources
in or under the article, and it is a good idea to check them. Often, however, you
have to look for the paper yourself based on hints in the text like author names,
institutions, and general topics. And then paywalls often block access to the
information that you're looking for. You can try pages like ResearchGate for legal
access to PDFs. Many researchers also use sci-hub but as the site provides illegal
access to publicly funded research, I won't recommend doing so. When you have
the paper in front of you, you can either read it completely, which is kind of hard,
or just read the abstract, which might be easier. The easiest is to look for science
journalism articles about the paper. Twitter is actually great to find those,
as many researchers are on Twitter and like to share articles about their own
research They also like to discuss research on Twitter. So if the story is
controversial, chances are you'll find some science accounts calling that out.
While Twitter is terrible in many regards, it is a great tool to engage with the
scientific community. You can also do a basic check-up yourself. Where was the
paper published and is it a known journal? Who are the people doing the research and
what are their affiliations? How did they do their experiment? Checking for controls
and repetitions in the experiment is hard if you don't know the topic, but if you do
know the topic, go for it. In the end, fact checking takes time and energy. It's
very likely that you won't do it very often but especially when something comes
up that really interests you and you want to tell people about it, you should do a
basic fact-check on the science. The world would be a lot better if you'd only share
information that you checked yourself for plausibility. You can also help to reduce
the need for rigorous fact checking. Simply do not spread any sane stories that
seem too good to be true and that you didn't check yourself or find in a
credible source. Misinformation and bad science reporting spread because we don't
care enough and because they are very, very attractive. If we break that pattern,
we can give reliable scientific information the attention that it
deserves. But don't worry, most of the science reporting you'll find online is
actually pretty good. There is no need to be extremely careful with every article
you find. Still, I think it is better to have a natural alertness to badly reported
signs than to trust just anything that is posted under a catchy headline. There is
no harm in double checking the facts because either you correct a mistake or
you reinforce correct information in your mind. So how do I assess whether a source
that I like is actually good? When I come across a new outlet, I try to find some
articles in an area that I know stuff about. For me, that's plant science. I
then read what they are writing about plants. If that sounds plausible, I am
tempted to also trust when they write about things like physics or climate
change, where I have much less expertize. This way I have my own personal list of
good and not so good outlets. If somebody on Twitter links to an article from the
not so good list, I know that I have to take that information with a large
quantity of salt. And if I want to learn more, I look for a different source to
back up any claims I find. It is tedious but so is science. With a bit of practice,
you can internalize the skepticism and navigate science information with much
more confidence. I hope I could help you with that a little bit. So that was my
attempt to help you to understand science better. I'd be glad if you'd leave me
feedback or direct any of your questions towards me on Twitter. That's
@sciencejoram. There will be sources for the things I talked about available
somewhere around this video or on my website: joram.schwartzmann.de. Thank you
for your attention. Goodbye.
Herald: derJoram, thank you for your talk, very entertaining and informative as well
as I might say. We have a few questions from here at the Congress that would be...
where's the signal? I need my questions from the internet - all of them are from
the Internet. Joram: *laughs*
H: So I would go through the questions and you can elaborate on some of the points
from your talk. So the first question... J: yeah, I will.
H: very good. The first question is: Is there a difference between reviewed
articles and meta studies? J: To my knowledge, there isn't really a
categorical difference in terms of peer review. Meta studies, so studies that
integrate, especially in the medical field you find that often, they integrate a lot
of studies and then summarize the findings again and try to put them in context of
one another, which are incredibly useful studies for medical conclusion making.
Because as I said in the talk, it's often very hard to do, for example, dietary
studies and you want to have large numbers and you get that by combining several
studies together. And usually these meta studies are also peer reviewed. So instead
of actually doing the research and going and doing whatever experiments you want to
do on humans, you instead collect all of the evidence others state, and then you
integrate it again, draw new conclusions from that and compare them and weigh them
and say "OK, this study had these shortcomings but we can take this part
from this study and put it in context with this part from his other study" because
you make so much additional conclusion making on that, you then submit it again
to a journal and it's again peer reviewed and then other researchers look at it and
say, and yeah, pretty much give their expertize on it and say whether or not it
made sense what you concluded from all of these things. So a meta study, when it's
published in a scientific journal, is also peer reviewed and also a very good,
credible source. And I would even say often meta studies are the studies that
you really want to look for if you have a very specific scientific question that you
as a sort of non expert, want to have answered because very often the individual
studies, they are very focused on a specific detail of a bigger research
question. But if you want to know is, I don't know, dietary fiber very good for
me. There's probably not a single study that will have the answer but there will
be many studies that together point towards the answer. And the meta study is
a place where you can find that answer. H: Very good, sounds like something to
reinforce the research. Maybe a follow-up question or it is a follow-up question: Is
there anything you can say in this regards about the reproducibility crisis in many
fields such as medicine? J: Yeah, that's a very good point. I mean,
that's something that I didn't mention at all in the talk because for pretty much
like complexity reasons because when you go into reproducibility, you run into all
kinds of, sort of complex additional problems because it is true that we often
struggle with reproducing. I actually don't have the numbers how often we fail
but this reproducibility crisis that's often mentioned - that is this idea that
when researchers take a paper that has whatever they studied and then other
researchers try to recreate a study and usually in a paper, there's also a
'Material & Method' section that details all of the things that they did. It's
pretty much the instructions of the experiment. And the results of the
experiment are both in the same paper usually - and when they try to sort of
recook the recipe that somebody else did, there is a chance that they don't find the
same thing. And we see that more and more often, especially with like complex
research questions. And that brings us to the idea that reproduction or
reproducibility is an issue and that maybe we we can't trust science as much or we
have to be more careful. It is true that we have to be more careful. But I wouldn't
go as far and to be like in general, sort of a distrustful of research. And that's
why I'm also saying, like in the medical field, you always want to have multiple
studies pointing at something. You always want to have multiple lines of evidence
because if one group finds something and another group can't find it, like
reproduce it, you end up in a place where you can't really say "Did this work now?
Like, who did the mistake? The first group or the second group? " Because also when
you were producing a study, you can make mistakes or there can be factors that the
initial research study didn't document in a way that it can be reproduced because
they didn't care to write down the supply of some chemicals, and the chemicals were
very important for the success of the experiment. Things like that happen and so
you don't know when you just have the initial study or the production study and
they have a different outcome. But if you have then multiple studies that all look
in a similar area and out of 10 studies, 8 or 7 point to do a certain direction, you
can then be more certain that this direction points towards the truth. In
science, it's really hard to say, like *OK, this is now the objective truth. This*
*is now.. we found now the definitive answer to the question that we're looking*
*at, especially in the medical field.* So, yeah.. So that's a very long way of saying
it's complicated reproduction or reproducibility studies, they are very
important but I wouldn't be too worried or too - what's the word here? Like, I
wouldn't be too worried that the lack of reproducibility breaks the entire
scientific method because it's usually more complex and more issues at hand than
just a simple recooking of another person's study.
H: Yes, speaking of more publishing, so this is a follow-up to the follow-up, the
Internet asks, how can we deal with the publish or perish culture?
J: Oh, yeah. If I knew that, I would write a very smart blog posts and trying to get
convince people about that. I think personally we need to rethink the way we
do the funding because that's in the end where it comes down to. Another issue I
really didn't go into much detail in the talk because it's also very complex. So
science funding is usually defined by a decision making process; at one point
somebody decides, who gets the money and to get the money they need a qualifier to
decide. Like there is 10 research groups or 100 research groups said that write a
grant and say like "Hey, we need money because we want to do research." And they
have to figure out or they have to decide, who gets it because they can't give money
to everyone because we spend money in our budgets on different things than just
science. So the next best thing that they came up with, what the idea to use papers
- the number of papers that you have - to sort of get a measurement - or the quality
of paper that you have - to get a measurement of whether you are deserving
of the money. And you can see how that's problematic and means that people, who are
early in their research career, who don't have a lot of papers, they have a lower
chance of getting the money. And that leads to publish or perish idea that if
you don't publish your results and if you don't publish them in a very well
respected journal, then the funding agencies won't give you money. And so you
perish and you can't really pursue your research career. And it's really a hard
problem to solve because the decision about the funding is very much detached
from the scientific world, from academia. That's like multiple levels of abstraction
between the people, who like in the end make the budgets and decide, who gets the
money and the people, who are actually using the money. I would wish for funding
agency to look less at papers and maybe come up with different qualifiers, maybe
also something like general scientific practice, maybe they could do audits of
some sort of labs. I mean, there's a ton of factors that influence good research
that are not mentioned in papers like work ethics, work culture, how much teaching you
do, which can be very important. But it's sort of detrimental to get more funding
because when you do teaching, you don't do research and then you don't get papers and
then you don't get money. So, yeah, I don't have a very good solution to the
question what we can do. I would like to see more diverse funding also of smaller
research groups. I would like to see more funding for negative results, which is
another thing that we don't really value. So if you do an experiment and it doesn't
work, you can't publish it, you don't get the paper, you don't get money and so on.
So there are many factors that need to change, many things that we need to touch
to actually get away from publish or perish.
H: Yeah, another question that is closely connected to that is: Why are there so few
stable jobs in science? J: Yeah, that's the
Wissenschaftszeitvertragsgesetzt, something that - I forgot when we got it -
I think in the late 90s or early 2000s. That's at least a very German specific
answer that defined this Gesetz, this law, put it into law that you have a limited
time span that you can work in research, you can only work in research for I think
12 years and are some footnotes and stuff around it. But there is a fixed time limit
that you can work in research on limited term contracts, but you're funding
whenever you get research funding, it's always for a limited time. You always get
research funding for three years, six years if you're lucky. So you never have
permanent money in the research group. Sometimes you have that in universities
but overall you don't have permanent money. And so if you don't have permanent
money, you can't have permanent contracts and therefore there aren't really stable
jobs. And then with professorships or some group leader positions, then it changes
because group leaders and professorships, they are more easily planned. And
therefore in universities and research institutes, they sort of make a long term
budget and say "OK, we will have 15 research groups. So we have money in the
long term for 15 group leaders.". But whoever is hired underneath these group
leaders, this has much more fluctuation and is based on sort of short term money.
And so there's no stable jobs there. At least that's in Germany. I know that, for
example, in the UK and in France, they have earlier permanent position jobs. They
have lecturers, for example, in the UK where you can without being a full
professor that has like its own backpack of stuff that has to be done, you can
already work at a university in the long term in a permanent contract. So it's a
very.. it's a problem we see across the world but Germany has its own very
specific problems introduced here that make it very unattractive to stay long
term in research in Germany. H: It's true. I concur.
J: Yes H *laughs* Coming to talk to the people,
who do science mostly for fun and less for profit. This question is: Can you write
and publish a paper without a formal degree in the sciences, assuming the
research efforts are sufficiently good? J: Yes, I think technically it is
possible. It comes with some problems, like, first of all, it's not free. First
of all, when you submit your paper to a journal, you pay money for it. I don't
know exactly but it ranges. I think the safe assumption is between 1.000 and
5.000$, depending on the journal, where you submit to. Then very often it's like
some formal problems that... I've been recently co-authoring a paper and I'm not
actively doing research anymore. I did something in my spare time, helped a
friend of mine, who was still doing research with some like basic stuff but he
was so nice to put me on the paper. And then there is a form where it says like
institute affiliation and I don't have an institute affiliation in that sense. So as
I'm just a middle author in this paper, I was published - or hopefully if it gets
accepted - I will be there as an independent researcher but it might be
that a journal has their own internal rules where they say we only accept people
from institutions. So it's not really inherent in the scientific system that you
have to be at an institution but there are these doors, there are these
pathways that are locked because somebody has to put in a form somewhere that which
institution you affiliate with. And I know that some people, who do like DIY science,
so they do outside of academia, that they need to have in academia partners that
help them with the publishing and also to get access to certain things. I mean, in
computer science, you don't need specific chemicals,but if you do anything like
chemical engineering or biology or anything, often you only get access to the
supplies when you are an academic institution. So, I know that many people
have sort of these partnerships, corporations with academia that allow them
to actually do the research and then publish it as well because otherwise, if
you're just doing it from your own bedroom, there might be a lot of barriers
in your way that might be very hard to overcome. But I think if you really,
really dedicated, you can overcome them. H: Coming to the elephants in that
bedroom: What can we do against the spread of false facts, IFG, corona-
vaccines? So they are very.. They get a lot of likes and are spread like a disease
themselves. And it's very hard to counter, especially in personal encounters, these
arguments because apparently a lot of people are not that familiar with the
scientific method. What's your take on that?
J: Yeah, it's difficult. And I've read over the years now many different
approaches ranging from nuts actually talking about facts because often
somebody, who has a very predefined opinion on something, they know a lot of
false facts that they have on their mind. And you, as somebody talking to them,
often don't have all of the correct facts in your mind. I mean, who runs around
with, like, a bag full of climate facts and a bag full of 5G facts and a bag full
of vaccine facts or like in the same quantity and quality as the stuff that
somebody, who read stuff on Facebook has in their in their backpack and their sort
of mental image of the world. So just arguing on the facts, it's very hard
because people, who follow these false ideas, they're very quick at making turns
and they like throw a thing at you one after the other. And so it's really hard
to just be like but actually debunking fact one and then debunking the next wrong
fact. So I've seen a paper where people try to do this sort of on a argumentative
standpoint. They say: "Look: You're drawing false conclusions. You say because
A, therefore B, but these two things aren't linked in a causal way. So you
can't actually draw this conclusion." And so sort of try to destroy that argument on
a meta level instead on a fact level. But also that is difficult. And usually
people, who are really devout followers of false facts, they are also not followers
of reasons or any reason based argument will just not work for them because they
will deny it. I think what really helps is a lot of small scale action in terms of
making scientific data. So making science more accessible. And I mean, I'm a science
communicator, so I'm heavily biased. I'm saying like we need more science
communication, we need more low level science communication. We need to have it
freely accessible because all of the stuff that you read with the false facts, this
is all freely available on Facebook and so on. So we need to have a similar low
level, low entry level for the correct facts. So for the real facts. And this is
also.. It's hard to do. I mean, in science communication field, there's also a lot of
debate how we do that. Should we do that over more presence on social media? Should
we simplify more or are we then actually oversimplifying like where is the balance?
How do we walk this line? So there's a lot of discussion and still ongoing learning
about that. But I think in the end, it's that what we need, we need people to be
able to just to find correct facts just as easily and understandable as they find the
fake news and the facts. Like we need science to be communicated as clearly as a
stupid share rolls on Facebook, as an image that - I don't want to repeat all of
the wrong claims, but something that says something very wrong, but very persuasive.
We need to be as persuasive with the correct facts. And I know that many people
are doing that by now, especially on places like Instagram or TikTok. You find
more and more people doing very high quality, low level - and I mean that on
sort of jargon level, not on a sort of intellectual level - so very low barrier
science communication. And I think this helps a lot. This helps more than very
complicated sort of pages debunking false facts. I mean, we also need these we also
need these as references. But if we really want to combat the spread of fake news, we
need to just be as accessible with the truth.
H: A thing closely connected to that is: "How do we find human error or detect
it?", since I guess people, who are watching this talk have already started
with a process of fine tuning their bullshit detectors but when, for example,
something very exciting and promising comes along as an example, CRISPR/Cas or
something. How do we go forward to not be fooled by our own already tuned bullshit
detectors and fall to false conclusions. J: I think a main part of this is
practice. Just try to look for something that would break the story, just not for
every story that I read - that's that's a lot of work. But from time to time, pick a
story where you're like "Oh, this is very exciting" and try to learn as much as you
can about that one story. And by doing that, also learn about the process, how
you drew the conclusions and then compare your final images after you did all the
research to the thing that you read in the beginning and see where there are things
that are not coming together and where there are things that are the same and
then based on that, practice. And I know that that's a lot of work, so that's sort
of the the high impact way of doing that by just practicing and just actively doing
the check-ups. But the other way you can do this is find people whose opinion you
trust on topics and follow them, follow them on podcasts, on social media, on
YouTube or wherever. And, especially in the beginning when you don't know them
well be very critical about them, it's easy to fall into like a sort of trap here
and following somebody, who actually doesn't know their stuff. But there are
some people, I mean, in this community here - I am not saying anything UFSA -
if you follow people like minkorrekt, like methodisch inkorrekt, they are great for a
very.. I actually can't really pin down which scientific area because in their
podcast they're touching so many different things and they have a very high level
understanding of how science works. So places like this are a good start to get a
healthy dose of skepticism. Another rule of thumb that I can give is like usually
stories are not as exciting when you get down to the nitty gritty details, like I'm
a big fan of CRISPR, for example, but I don't believe that we can cure all
diseases just now because we have CRISPR, like, there's very limited things we can
do with it and we can do much more with it than what we could do when we didn't have
it. But I'm not going around and thinking now we can create life at will because we
have CRISPR. We can fight any disease at will because we have CRISPR. So that's in
general a good rule of thumb is: just calm down, look what's really in there and see
how much.. or tone it just down like 20% and then take that level of excitement
with you instead of going around and being scared or overly excited about a new
technology and you think that's been found because we rarely do these massive jumps
that we need to start to worry or get over excited about something.
H: Very good, so very last question: Which tools did you use to create these nice
drawings? J: *laughs* Oh, a lot of people won't like
me for saying this because this will sound like a product promo. But there is.. I use
an iPad with a pencil and I used an app to draw the things on there called Affinity
Designer because that works very well then also across device. So that's how I
created all of the drawings and I put them all together in Apple Motion and exported
the whole thing in Apple FinalCut. So this is now the show like a sales pitch for all
of these products. But I can say, like for me, they work very well but there's pretty
much alternatives for everything along the way. I mean, I can say because I'm also
doing a lot of science communication with drawings for the Plants and Pipettes project
that I am part of and I can say an iPad with a pencil and the finishing designer gets you
very far for high quality drawings with a very easy access because I'm no way an
artist. I'm very bad at this stuff. But I can hide all my shortcomings because I
have an undo function in my iPad and because everything's in a vector drawing,
I can delete every stroke that I made, even if I realized like an hour later that
this should not be there, I can, like, reposition it and delete it. So vector
files and a pencil and an undo function were my best friends in the creating of
this video. H: Very good, derJoram. Thank you very
much for your talk and your very extensive Q&A. I think a lot of people are very
happy with your work. J: Thanks you.
H: And are actually saying in the pad that you should continue communicate science to
the public. J: That's very good because that's my job.
*laughs* It's good that people like that. H: Perfect.
J: Thank you very much. H: So a round of applause and some very
final announcements for this session. There will be the Herald new show and the
break. So stay tuned for that. And I would say if there are no further... no, we
don't have any more time, sadly, but I guess people know how to connect to you
and contact derJoram if they want to know anything more.
*rC3 postroll music*
Subtitles created by c3subtitles.de in the year 2020. Join, and help us!