Dear viewer, these subtitles were generated by a machine via the service Trint and therefore are (very) buggy. If you are capable, please help us to create good quality subtitles: https://c3subtitles.de/talk/176 Thanks!
Hi. We are Anna and Dominic,
and we're here to tell you something
about brain computer interfaces.
I'm a mathematician and he's a cognitive
scientist, so we are in this topic
first. We'll tell you why we are excited
about this topic.
Then we'll dove into the technical
details, how data is represented
inside of the brain, how to
read it out with the computer and how to
build your own BCI.
And finally, we'll have a quick
discussion and answer some questions.
So let's talk about communication.
Language is the most natural
way to communicate with your fellow
humans.
It's the most intuitive way of expressing
your thoughts.
And also the most common method
to communicate with your computer.
Obviously, the word is very powerful,
but which devices can we use to put
diverse from our minds into a computer?
Of course, there's the keyboard
compares to talking.
It's quite slow and
quite hard to learn, but extremely
accurate by design.
There are all kinds of motion tracking
tools.
The mouse.
The touchpad touch screen graphic
tablet, because we want text
input. We knew to use handwriting
higher information density
easier to use than a keyboard, but very
slow at actually transferring a message.
Then we have voice input.
Everybody knows that dictating as much
profit and writing
since a couple of years, even our
computers understand what we say.
This could become very interesting in the
long run because
talking raises the transfer speed limit
considerably.
Have you thought about the information?
Have you thought about the information
transfer that's really happening here
because the actual chain of information
looks more like this?
It's the brain that sends the information
to a body part, which interacts with
the device that sends information to the
computer and at both
and both ends of the communication can
deal with much higher data rates
than usually five to 15 bytes per second.
That's the measurement.
So what would happen if you cut out
the middleman and connect the device
directly to the brain?
So as for language spoilers,
it would be surprisingly underwhelming
thinking birds isn't that much faster
than speaking them when you read out the
words from your brain.
The data quality is horrible.
There would be a lot of misunderstanding
and you would have
and you would have to sorry,
and you would have to teach the software
where the words are in your brain.
So you need a couple of hours before you
could get even started.
But of course, that's such an interface
you could also use something else than
words as input.
For example, let's look at art.
When I imagine a picture and we
want to show it to other people,
I can just pick up my canvas and
take a paintbrush and start painting,
or I could also use a graphics tablet to
paint with a word or a brush.
Or I could use a specialized mouse
to put together 3D objects and
render the scene.
But each of these strategies is going to
cost me several hours.
And even if I actually had some knowledge
in painting, the end result would never
look exactly like my imagination.
Here's why a brain computer interface
interface is starting to shine because
it can access the image in your head
entirely and immediately.
There's not just arts and
language, there's also gaming,
there's music, there's interfaces
everywhere, interfaces
that are hard to learn and even harder to
master interfaces that
put a limit on our credit creativity.
Having a really good brain computer
interface would make lots of
these devices redundant.
And as soon as a computer can guess what
we want to do, a lot of frustration
starts to disappear from learning.
And since there's more to the brain than
just hearing and seeing, maybe
it makes sense to transfer data from
other senses as well.
Unfortunately, this is still very much
a vision into the future,
but we have already today
some serious possibilities of controlling
a computer with our minds.
It's not even that expensive, but I'll
come back to that in the last part of our
talk.
Before we can build a system that can
read from our brain, we need to know
how the data is represented in there.
For that, we need to take a look at the
very basic map of the brain
we see and imagine with the back
of the brain.
It's called the occipital cortex.
Hearing them talking happens in the lower
side areas directly next to the ears,
which makes sense.
The top of the brain is responsible for
skin and muscles and movement in general.
And what the front part does is most
difficult to understand and even for
scientists. So let's just say it does the
actual thinking.
So let's start with a simple example,
let's imagine a Red Bull.
First, we look up the meaning of a
red ball in the meaning library
down here.
Then we send this object to
the appearance library in the front.
This area is a library of visual objects.
Resolve the object code into its color
and shape codes, which
are immediately sent downwards.
Color and Shape Center then convert these
property codes into the visual meaning
and build up a link to the correct
space in the visual buffer.
As soon as this connection is
established, we see the image of
a red ball in our minds.
This red ball is, of course, storage
somewhere in the brain to along with
millions of other datasets.
All this information is stored and the
connections between neurons.
Neurons never work alone because they're
too weak. I'll explain that in a minute
to create a meaningful signal.
They have to work together, and
they do that in groups of hundreds of
neurons, which is called a mini column.
Each mini column stores one
unit of information, together
with 100 other million homes.
They form one memory unit called a hyper
column organized in
like. This column forms.
Each one of these hexagonal memory units
is incredibly flexible.
They can remember everything about Red
Bulls, how it looks, how it feels,
how it sounds large, and even your
emotions toward Red Bulls.
But because we only have two million
hyper columns, it would be a ways to
use a separate one for every ball.
That's why your memory unit for Red Balls
is the same human unit as for green,
black, blue and green and
yellow balls as well.
So what happens when the hyper
column of neurons recognizes the
special ball from your childhood?
It activates the mini column.
It stores the emotion of your childhood
before all the neurons in this minute
column, but just idling
their activity was low, random,
slow and completely synchronized.
But as soon as the activation signal
arrives it, they generate waves
of activity.
Now, each one of these yellow dots on the
bottom is the activity of a single
neuron because single
neurons can only generate a short
flash of activity.
They have to work together
and produce these waves.
These waves are the typical data format
inside the brain.
Waves can encode more information by
changing frequency and phase.
The amplitude, however, is mostly the
same.
And in this case, this wave is part of
the memory of the red ball of your
childhood.
In case you were wondering,
this is how hyper
columns communicate with each other in
slow motion.
This is a slow motion part
of the actual cortex cortical surface
stimulated from a rat cortex.
You'll see that there's some structure
and some clear loops, but it's
all flowing, shifting and
never really repeating.
There are some nodes, there are some
loops, and all of these loops up
forming in the later stages of the brain.
More complex and more
and larger loops.
And these are what we perceive
as information.
This is how all these electrical waves
are traveling through the brain.
So before further, I'd give
throughout my warning, I will use some
formulas.
Please don't be.
Yeah.
Don't be alarmed.
Yeah, it's
OK.
They won't do anything.
And what we now have to do is to find the
origin of this electrical waves.
So how do we do this
without physical access to the brain?
Well, the easiest and already established
way is to use an EEG.
This is a device that measures electric
fields with the help of some electrodes
there are used for research or
medical purposes.
That's some of this
and also smaller ones for consumers
like this one on the other side.
They differ mostly in the price and the
number of sensors.
You have probably seen one in the media
already publicly to
explain how this work.
For now, it's so efficient to know that
you measure electric fields.
The big question is now how does this
all work?
The electric field from each neural
cluster is strong enough to be measured
outside of the head.
It is usually between 200 and 500
millibars.
The electrodes of an EEG are
placed all over the head.
We have to place the electrodes directly
onto the head face because air
connectivity is very low.
Then we combine all the electrodes
electrode voltage so that we know the
voltage for each point on
a head. Surface
neuron clusters also produce a magnetic
field at the same time.
This could also be measured from a small
distance.
But the device for measuring
very weak magnetic fields are far
too big to carry around and also very
expensive.
They also have to be isolated
from other strong fields.
Even regular watches
create and magnetic fields that magnitude
stronger than the whole brain.
The research for portable devices
are already starting, but currently
are the only handy devices for custom
usage.
Since the electric fields is cost
somewhere inside of the brain,
we have to look at what happens on the
way outside our head
consists of different tissues,
which all put up some amount of
resistance to electric fields.
So instead of traveling straight
through to head, the electric field gets
distorted.
It could be imagined like this
when jogging.
You have different speeds on different
kind of surfaces
when you are dragging on asphalt.
You are quicker than on sand.
It's the same with the electric fields
in our head. For example, our bones
have a very high resistance, and the
liquid around our brain is very
conductive.
And one brain tissue even has different
conductivity for different directions.
So it's got a little bit complicated
here.
We are finished with the electoral
physical set up, active neurons create
an electric field inside of the head.
We know about the conductivity of
different tissue layers and the
voltage on the heads of face without
any other knowledge. You're stuck here,
like in many occasions, for solving
problems.
Now we are required a mighty toolbox
of mathematical science to find a
solution.
What we want to do now, we want
to translate our setting
and our problem too
into the language of mathematics.
Solve the problem. Then translate the
mathematical solution back into reality
and to warn you beforehand within the
mathematical world now, and
as I said, we will need another
translation into something a computer
can solve.
The first step
in translating into a mathematical
language is also calls to model the
problem.
Building a mathematical model uses
principles, laws and concepts
from other fields of science.
In our case, we use a modified version
of the famous Maxwell equations, which
describe the connection between magnetic
and electric fields and forces.
They were first described by a physical
scientist named Jesse Maxwell
in 1865.
Those equations can be simplified and
adapted to our application.
The information we get as a result is
written here.
First, we look at the left side.
The ex describes any point in the head
you have X is the electric
future at the position X and
when you derive that you get
the current at this point.
Now your fault is with the electric
connectivity and compute the
divergence.
That's what's happened mathematically.
Well, what you can imagine happening
in the left side is the difference
between the incoming flow
and the outgoing flow of the electric
field at every point x.
The right side is all about communal
activity, was this Britain, there
is the so-called so-called current
dipole.
This means that the
whole equation is zero everywhere, except
at the position of the activity.
Together, this means for every point in
the head, the same amount of electricity
flows in it flows out, with the exception
of the active neurons.
So that was the equation.
Let's talk about solving this equation,
the usual way to solve this
is to assume that it had connectivity
and neuronal activity.
And now
the head connectivity and the location
of the activity are already known.
And you would start calculating you had
currents.
This is called the forward problem.
But of course, we don't know the
current location that because that's
something we want to compute.
So.
We want to calculate the activity
location from the conductivity and the
measured surface wellhead for
just the equation needs to be
of turned around.
This is called the inverse problem and
it's surprisingly difficult.
We encounter the next problem when
we look at these things we measure,
we want to know where the activity inside
the brain is located,
so we measure the electric field
on the head surface.
But we cannot measure inside to
have that something different over there.
Medical, you have probably around
220 sensors,
so you measure electric field
only has a resolution of one hundred and
twenty points.
This gives us much less information
than we actually need.
We can deal with such kind of less
information if you make some presumptions
about the location of activity.
Usually, we limit ourselves
to places in the brain with actual
neurons, namely the cortex.
But of course, the resolution of our
estimation gets better, the more sensors
we have.
Once we have faced these problems,
we now can convert the equation into a
computer friendly format,
a computer cannot deal with equation
that describes something for everyone
in the head.
That would be infinitely many points,
and infinite memory is still quite
expensive.
So fortunately, some
clever mathematicians develop
algorithms for solving the problem,
which are computer friendly.
We do not compute
for every point in the head.
Instead, we select only
some points of the head, for example,
one million and compute results
only for them to interpolate
the areas between our selected points
when we need to evaluate their.
Mathematically speaking, this selection
process is called just fertilization.
The continuous structures of the head
are divided into small cells and simple
of simple geometry
the continuous structures
of the head.
Usually we use tetrahydrate or IRA
or a mixture of both.
This works well, and it can be
mathematically proven that the error
from the disk fertilization is small
enough for certain settings.
For this approach to work well,
we need a good 3D model of the hat,
and we haven't talked about the
connectivity.
That's the last part that's missing for
our model when building the
3D model.
We need this model to represent the
connectivity well enough.
That's the next step in the scientific
way for getting connectivity
values is to put the health of choice
into an MRI.
That is a big machine
which can take High-Resolution images
of the inside of the head.
Then we take such
a good picture from this head and cover
this picture into this little geometric
cells.
There's only one connectivity value for
one cell.
If you want to be very exact,
you have to use a different head model
for every person because every head
is unique.
And because the brain can move around
inside the head, you should always,
always measure your person in the same
position.
But when you don't need to be precise,
just a simple rule.
Any head model is better than no have
model.
Then we are ready to estimate the
location of neuronal activity.
We take the connectivity model
of our head, the location of
the sensors,
all the locations of all cortex points
and put all of this into this machine.
This machine first puts all of this
information together into one head model.
It made sure that all these little cubes
and a sense of position and the cortex
share the same coordinate system.
The last dataset makes sure
that we only estimate locations of
activity inside the cortex
when putting up these restrictions.
You have to be sure of two things.
First, the relocation of activity is
actually in the cortex.
And second, that there is no activity
outside of the cortex.
With the head model, the machine then
compute something in an intermediate
step called transfer matrix or lead
matrix, the transfer
matrix describes what the census would
measure for every possible electric
activity in the cortex.
Building this matrix is a very time and
memory consuming process.
But the nice thing about this whole first
part is that you can hear
it offline,
so this means you only have to do it once
after the first huge calculation, only
the last step requires some brain
activity over there.
Since this last step can happen in a few
milliseconds.
This is perfect for real time
application.
And if you always use your
gene in the same position on your head
and your cortex has enough points, there
is no reason to change the transfer
matrix.
The next step takes this transfer
metrics and the measured electric
field
a good solver for the invest problem,
then decides which was the most
likely activity location that cost is
tens of voltage.
There are different ones like Manaus
Loretta, which give reasonable
results, but still
this algorithm a little to time in memory
consuming to do this in a good
online time.
So in the end, you get an activity map
that shows where the active clusters
are.
And to conclude my park,
yes, we can estimate we are
neurons are active.
Our estimation would be much better
if we had millions of sensors instead
of 120.
It would be much better if the sensors
were inside of the brain instead of
outside and
still need a ton of computational power
and memory.
But it is possible, like Dominic will
explain in a few seconds.
All right. We are finished with all the
theory and be coming back to
the real world now.
Don't worry, it's not as hard as
it sounds like
in the beginning. You can actually
deal with it actually and deal
with not having the whole localization
part and deal with sensor
data only.
But you will see that you run
into a wall quite soon because your
resolution isn't high enough.
So the whole once
you get there, you have to consider the
part that any talk about and
actually create an activity map.
So what do you need to build your own
BCI?
Two pieces of hardware and two pieces of
software.
I'll start with the hardware.
And E.J. consists of two core parts,
an amplifier and an analog digital
converter.
You need both of these parts for every
channel, since these chips are
relatively expensive.
More channels automatically also means
more expensive.
We want a noise floor of at least 40
decibels for our amplifier to have
a reasonable signal to noise
ratio.
The same is true for the 8C.
The absolute minimum sampling frequency
is around 100 carats, and you should
go for 200 hertz for
a reasonable minimum state
of the art research uses.
Sample rates between thousand and two and
a half thousand hertz.
Not because the brain actually generates
these frequencies, but to get a better
position for the frequency faces.
Finally, since we are dealing with
microwaves at the first stage, it's very
important to care for proper shielding
shielding in the first steps.
So this is the current spectrum of EEG
devices in a sensible price range,
excluding all the medical devices here
in our community, we have two big open
EEG project schematics and
different implementations readily
available, and it's already more than 10
years old.
Then we have the Open BCI
project, which is much newer and
sponsored by dopamine.
So you want to choose a camp
and we start with 40
euros for the cheapest and worst
version of an open EEG.
It's so cheap because it uses the AC
in your sound card and your computer
only has two channels, and these channels
will be quite bad, but it
will work.
The better alternative is this only
makes open energy intensive
has a reference electrodes,
which means you won't get channel drift
or saturation problems with your
amplifiers, and its ADC
is much better than the one
you use for your microphone in the
computer.
So this is the one the cheapest device
actually recommend for beginners.
The best solution depends on your budget.
Of course, what you pay for is the number
of channels and the low amount of noise.
The best device you can get right now is
the open BCI, with 16 channels
in the double configuration, but
and forty euros will only get you the
passive electrodes.
Passive electrodes are cheaper, but they
send micro walls through very long
wires, so they are very noisy
and you need conductive gel to make
contact with the head surface.
Active electrodes have the amplifier and
the ADC directly next to the sensors.
That's why active electrodes are more
expensive. You need separate electronics
for each channel,
but the noise levels are much lower and
not having to put gel in your hair all
the time, it's nice to
have. The emotive, emotive
epoch is the most compact
package and trim. It's what transmits
wirelessly and encrypted by default,
but it's completely proprietary.
I've heard that the transmission
has been hacked.
Actually,
it's big weakness.
It's the sampling frequency.
It only can do one hundred and twenty
five hertz of the maximum
and then be coming to the software and
the software needs to do for
basic things become on the left
side.
First, it needs to acquire signals from
the e.g and put them into data
packets.
Second, it needs to estimate the location
of brain activity,
so the estimated activity needs to be
processed at a signal level.
I'm talking about frequency filters,
Fourier transform formations and so on.
For some BCI paradigms,
these are the small programs that
actually know what brain activity is.
Let's start on the left field trip.
It's based on MATLAB developed in the
Netherlands and most feature complete.
It's a very versatile.
It's used in science a lot,
but it's real time. Support is quite
weak, so you have to encode a lot of
things.
Neuro Nurofen is a plug in for a field
trip. It's the only software right now
that can deal with the highest quality of
had models, so it's mostly used in
cutting edge science.
But if you're interested, you can do this
only with free trip.
The most user friendly solution is
probably open vibe.
It doesn't do localization, but it has a
Google. I supports many types of
e.g., and it's very well documented.
There's even a plethora of examples out
there and then this Mushu.
This is on internships running on Python,
mainly for people who don't like who like
Python or dislike MATLAB.
So when you have decided which software
to use, it's time to choose a BCI
paradigm.
I will give you a short overview of
the most popular paradigms today.
Maybe this is a good start for you guys
to get into something more advanced.
Steady state evoked responses are
a brute force approach to the brain,
you put in a certain frequency directly
through the eyes and hope that
at least some neurons pick it up and
convert it to electrical activity.
The classic pattern is a flickering
chessboard before
show it to you.
Here's a little warning everyone has a
history of epilepsy.
Please look away for five seconds.
Really?
OK, go.
Works quite easily, right?
So 12 hours and 10 hertz,
both checkerboards were flickering at a
very comfortable frequency for most
neurons in the brain.
They are usually usually at the 10
to a 60 hertz range.
This frequency come in at the back of the
brain, along with everything else you
usually see when
you select the back of the brain with
your EEG software.
You will confine this confined back
to this exact frequency in your
spectrogram.
Now, the cool thing is that it can pay
attention to one checkerboard and ignore
the other.
And in your measurement, the frequency
band of the attendu checkerboard becomes
stronger and the other one becomes weaker
when you do that.
Was this trick?
Your software can detect if your
attention is on the one checkerboard or
in the other.
In practice, this paradigm works like
mental buttons.
You just press a button by looking at.
They don't even have to be big, and you
can run up a thousand, take a bottle
at the same time.
So next paradigm P300
or Paul paradigm
in psychology, these are two different
things, but in this case, they're
combined.
The basic idea behind the oddball
paradigm is that your brain always
creates the same pattern under certain
conditions.
The three hundredth signal I showed
in the next slide is generated
when we see something unfamiliar or
something that doesn't quite belong into
our group.
The signal is the strongest.
A few hundred milliseconds after the
odd ball.
The strongest pick is positive, which is
what the P stands for.
And usually the upward paradigm works
with images or sounds.
But of course, you can do it with other
senses as well.
How do you analyze that?
On the left, there's a reference P300.
This is how it usually looks like when
you get get rid of all the noise.
Detecting the video in a continuous
signal isn't completely straightforward
because your incoming signal would be
contaminated by other brain activity.
So if you use a simple amplitude
threshold, your you will trigger
all the time.
A good strategy, therefore, is to use a
reference signal.
You take the last five hundred
milliseconds of your continuous signal
and correlated with the reference
wave.
When the correlation is above 50
percent or so, you know that the correct
shape was actually in your signal.
With this tool, you can even type on a
keyboard with your mind.
This is currently used for locked in
patients, for example, or quadriplegics.
Just look up the P 300 speller.
Last paradigm, the event related to
synchronization.
This is a very popular one,
event related to synchronization is
really easy to set up and very reliable.
You imagine moving one of your limbs,
for example, of wiggling your right foot.
You can actually do it, but it's not as
impressive as this.
Imagination causes a certain neuron
clusters to synchronize or to
synchronize. You probably remember the
waves that spread out in the beginning
and in the other spectrum,
the synchronization will create a gap
in the 10 to 13
hertz range of
the analysis.
For this is very simple select the region
from the top right side of your brain or
left side. You have to consider that
the left foot is connected to the right
brain half and vice versa.
And take the
Fourier transform and
look at a few frequencies between eight
and 13 has four amplitude changes.
You know your software triggers whenever
you think of a moving foot or waving
hands.
And when you look for two regions of the
brain at the same time to left and right,
your software can actually detect which
hand you're waving in your mind.
So in summary, by an
EEG headset.
Choose your BCI software and get started.
We are nearly finished.
Just do a quick recap.
Certain clusters in your brain
do always the same thing at the same
place, which is important
for selecting a location.
Memories are stored in the connection
between neurons.
Memories and neurons communicate with
electric waves and certain frequencies.
We want to find the clusters that do what
we want, or
we already know how to where to search
from. For example, from my talk or from
any other literature.
But either way, we select an area on the
activity map.
We put the signal into our BCI paradigm
and enter the internal feedback loop with
the user.
Have a lot of fun. That's it for today.
Anyone with questions?
Please line up. There are two microphones
here and the idea there, if you
leave, please leave quietly and
there's a camera already running for the
guy or crossing the picture.
Other questions from the internets?
Yes. No, maybe no.
Then number two.
OK, if you want to to steal some device
with a with a from the activity,
what is the advantage of first
inferring? Rather, activity really comes
from instead of just training
your algorithm on the basis of the raw
input signals of the edge anyway.
Can you repeat the first part of your
sentence? What is it?
Yeah.
So what is the advantage?
What is the advantage of inferring
the location of brain activity
instead of directly using the raw egg
input vectors as training
the right?
Exac-, no, it's actually not really
raw because it's a mixture
of several types of
neural activities from the from different
parts of the brain.
So you get a mixture, and
when you get this mixture, you, for
example, get a lot of visual activity or
much preparation.
It's all very, very much
a convoluted and with the localization,
you can actually split them up into
different regions so that you only get
the one signal that
did what you wanted to do.
So what is your example?
Is controlling something with
this imagining your
right foot, for example, and you would
select you'd take the electrode that's
around here. Then you will also get
something from some visual also
get something from audio.
It's not really actually a signal, so it
won't be as noise
free as when you do with the localization
before.
OK.
Number one yeah,
do the when you're motivated
using it for visual
input, you gave some numbers about
it being, I don't know, like 300 features
per minute or something like that.
How do you come up with that?
I'm waving. Oh, right.
And number two,
I assume the option
of putting sensors in the
brain is too expensive, and that's
why it wasn't discussed
in this.
I think you're right, but
I've always wondered about some kind
of middle ground, like
as you showed in your slides, that
the skull is one big barrier between
the brain signals and their sensors.
So why hasn't anybody
proposed driving
needles partially or completely
through the skull to survey the same
thing or?
Or anything along those lines?
That's like relatively
easy and cheap to do.
OK, I know what you mean.
It's actually being done.
These approaches were one of
the first step in neuroscience
at the very beginning because you,
you see you, you, you, you,
you've got a skull is a big
barrier and you need to go somewhere
around it. So it's the easiest to
actually put the sensors directly onto
the brain. This is called
electrical cortical activity.
It's it's done widely in
basic research.
But the problem is that you need a
person person that's ready to open up
their skull, which is usually
only done when there's something wrong
with their brain and it
needs to be operated or something.
So all the basic
research is only getting maybe half an
hour per person of research time.
And the thing with needles in the brain
isn't so far fetched.
It's actually being done with partners
and patients, for example.
The main problem with needles in the
brain is that they are
really foreign bodies in the brain.
They have a hell,
have a median lifespan of about six
months, and then they slowly start
stop working because the brain reacts
to metal objects within their
midst and
and starts to cover all the metallic
surfaces with with fat.
So it isn't such so
much of an electrical conductor anymore.
Yeah, but that's true.
Even if you like,
like, say, driving the needle
nine tenths of the thickness of the skull
towards the brain, that helps
nothing at all. Or that?
How does that work? That would help,
of course,
for 90 percent of the of the variance.
As far as I know, there are some people,
you know, put
things like on their head, you know, as
piercing or whatever.
So.
Yeah, actually, you could very well
go completely through the brain
as a car.
It doesn't matter so much because they're
still the dual motor, which separates the
brain from the skull, and they're
in my very weak
knowledge about your, uh,
your clinical stuff.
There's no really no real information
in there, but it could work.
Yes, it hasn't been done before because
this type of egg
hasn't become so mainstream that people
do it for fun.
But it's very, very, very possible.
Yes.
Questions from the internet.
So question number one
in this comparison, the last to,
uh, suppress the show
didn't have the localization options.
And what can't you do if you don't have
that
last part again, you
know?
Oh, OK.
What can you do if you are missing a
little PlayStation feature in your
software?
OK? You can't actually decode both the
different types of activity that
come into one electrode, which in
practice means your whole signal will be
a mixture of different activities and
your activity you are searching for is
only one, maybe one third of the
whole activity, which means you have
a higher practical noise level, which
makes it harder for your algorithms to
actually determine what, what, what
happened to this part.
Number two in the back?
Well, actually, I have two questions.
The first question is, you said the
data points you get, the better the
quality is. Well, look, the basic thing.
Did you ever think about trying to
enhance the conductivity of hair,
like put some graphite shampoo
in it or something to to?
I don't know. Maybe it's possible
to to use it as a sensor wire because
your hair only needs in German.
Well, you know, the better
hair is in the in the in
the skin, it goes a little bit into the
skin. Hmm.
So maybe you can use it to get a
real good grip.
OK, so the skin isn't our big barrier?
No. But you get a lot of data points.
Just, I mean, it's already in there.
You just need the cabling to attach a
kidney to your arms.
I don't know.
Just due to an issue,
they don't conduct electricity, you know?
So at some graphite or something,
graphite?
OK. I don't know.
Oh, that's actually not bad.
There's a lot of
there's a lot of chemistry going on.
Is it a barbershop? I don't know.
So if they make curls from it, they they
really do a lot to the chemistry in the
hair. So maybe you can apply it in
there. Just go to some somatic.
OK. I don't want to be the person that
connects that it connects the wire to
every hair, but I think it would be
possible. Yes.
But you
just suggestion.
And the second thing is, you showed
that if I want to move my food,
I have to think about it and you can show
that as a signal.
OK?
There is also a lot
of tension to my body that keeps me
upright. I don't think about
and I think that is
done somewhere in the back here in the
motor cortex. No, no, no.
The tension thing is happening in your
spine.
OK. Roll it back. I have narcolepsy, so I
have something which shuts
that off, OK, and I don't want it.
So what I try to do, I
want to do is to put
my brain in the loop, diagnose when this
tension goes away and give
some, let's say,
yeah, give some electrical impulse
to to my muscles to regain
that, OK, which the second part
is basically proven to be possible.
And now I know the sensory input and
put in the loop.
So can I.
Can I find that if you lose
the tension of all your muscles,
can I can I sense
that with you?
That's a basic question
you want to sense if you lose control of
your muscles right now. Yeah.
Like in sleep when you have a paralysis.
Mm hmm.
The easiest.
I'm not really completely familiar with
this basic, really basic motor
stuff, but my easiest
approach would be to use an electrode
actually on your muscles.
So on your forearm, for example,
because the muscles in contrast
to the head can generate a
lot of electricity, like 12 or
so, and you
can really easily measure that.
And when the swatters goes down, then you
have your signal.
OK, so there is a basic activity I always
have in my mouth.
And it's it's a it's it's the frequency
and the frequency goes up and
down.
And it's when you when you relax and the
frequency is low and when you
move it, then it becomes higher.
OK, thank you.
The internets and then number one in the
back.
The question. There was a question about
the organization of the hyper
columns, are they really strictly
hexagonal or is that a simplification?
No, of course not.
It wasn't very idealistic.
Every civilization, but
this hyper column
organization only applies for
what's what's for the few
flexible regions in the brain.
They have a lot of fixed purpose stuff
in the brain, and hyper columns
are used
for for a mixture of storage and
computation in the general purpose
areas.
And there they are organized like this,
but in other areas that can be, I don't
know, all over
the place.
No one in the back.
So everything we
talked about right now was about
giving visual signals and then recording
them somehow, right?
But can I can I
skip the eye and give those signals
without wasting
energy on the you know that
the entry
sensor?
Basically, there's you want to
ask if you can replace our creepy eyes
for something better?
Well, yeah, that could be
possible. Use case.
Ah, yes, of course.
Yes, you can do that.
The problem, again, is that you can't
really stick some needles in your in your
brain because they won't have
a long lifetime.
And the back of our eyes
is actually considered part of the brain,
a completely same organization.
It's the same cells.
Technically, it's just an extension of an
extension of the brain, so the retina
will perform the same way like
the rest of the brain does.
There's actually some research going on
with blind people,
but hopefully blind people
like vision impaired people
who have to low contrast in
their in their retinas.
They get implants which have put
directly onto the retina.
And when you put some water on there,
then they can see black and white images
again. Funny thing, actually, we don't
know how color works here,
but why actually
did they use a mouse to go forward
and backwards for your presentation?
Or how precise are your
recognizes of what you are thinking?
So why didn't you use your left and
right thumb?
Um, you mean for your
media should have used an EEG interface?
OK. Actually, maybe we forgot to stick
with us.
It's precise enough.
It would work.
I would be precise enough to work.
It wouldn't be 100 percent accurate.
But since we did in the presentation, we
actually have the possibility of
correcting.
We would somewhat
be somewhat comfortable to do this, but
we didn't dare overregulate.
Number two,
I've been looking for the possibilities
of outside the university
brain computer interface for some time.
And basically, you're the first team
who speaks positively about
the emotive
because I was focusing on
the open BCI and I wanted to even
create a hackathon, maybe
in what might be in Berlin somewhere
in Europe and invite
the open BCI people because
I saw no,
no good progress about a motive
and basically the right electrodes.
Uh, can you?
Have you
experimented with them?
And do you have any
expanding experiences you want to share?
You could share with us?
Yes, I can.
I actually bought an amateur book when it
came out. I think it was in 2008
also, and it
was the cheapest and the
most performant device at the
time. And until mid
of next year, it still is.
The my major issue with it
is that it's not such a great
build quality.
And the company
behind it seemed to have
regretted their decision to to make
it available. Somehow, I'm guessing it
was very expensive to develop.
They brought it out as as a
gamer accessory.
They wanted people to
code some games and use it as some
all-purpose kind of
input device.
This didn't really work out because there
there is a
API that's behind the whole system
is not really that great.
So it was used
a lot for research because it's the only
mobile headset right now.
It's not. It doesn't produce a lot.
It doesn't have a lot of channels.
I mean, in research, we're using more
like 128 or 256
channels. But in the comparison, 40
is the noteworthy.
And I'm
the epoch hasn't really
progressed in the last five years
anywhere. I'm not sure this
it has any future.
But other than that, I have
made quite positive experiences with it.
Besides that, it only runs on Mac and
Windows, which is quite graphic because I
want to do it on my smartphone and
it has a relatively
nice data quality.
It has actually electrodes, which is
also nice for some mobile applications,
and it has a reasonably
OK C++ API,
which you can use to just get it out
of there and get it into something into a
software package that actually can deal
with brain signals.
OK. Number one Yeah,
OK.
This is sort of related
to the first question.
Do you know, are there any attempts
to make the,
uh, the imaging more precise by
using biofeedback for the
actual calibration
of the abuses?
Um, biofeedback.
Um, can you
clarify that?
Yes, there was an art project
a few years ago that I heard about Code
Brain Avatar, which used
the direct outputs of the e.g.
electrodes fed into
a water basin, which was then
projected through an overhead projector.
So you saw the the
wave interference patterns
that were produced by the U.S.
and that had some quite interesting
sort of biofeedback effects, too, so
people could actually
try to influence the
the the patterns
produced by their brain activities.
And this this system used,
I think, eight electrodes or something.
And I wondered if it's possible to
perhaps by adding
another layer which is less directly
accessible, perhaps increase the
the overall sort of
granularity of the
yes, actually just
a real time activity map is one of the
easiest problems
that you can you can do with a such an
EEG.
It's it's I think it's even built in in
most, most software packages.
Um, you biofeedback.
It's actually something of a more
complicated. Setup, because
you have a feedback loop to the computer
and you, for example,
use them as a measure of your
of your alpha waves or something
to influence something within
within a computer and then displays your
back if you're doing OK or not.
In today's research,
this is already quite advanced.
It's Buford PRETEXTING.
You can learn to meditate before that.
And my most amazing
story was that you
can actually learn how to shoot a ball
without actually learning how to shoot a
bow. But you in the scanner and you try
to move one point towards another point.
And when you have done that, your brain
has has sufficiently changed
so that you can actually sort of a
small number two.
When I watched a string of popular
questions and I
just wanted to know if you have
an outlook about the ultimate privacy
intrusion by reading
what we are thinking about
or something like that have any timescale
to say, maybe next five
or 10 years,
OK, unless we solve
the whole censorship in the brain issue,
we are not nearly close to the
reading thoughts stage.
We actually need nano
sensors, ideally
at each mini
column, which is around, I
don't know, several millions of
sensors streaming
in real time towards a very
advanced brain computer learning
interface that it has
to be trained first.
What its neurons of your what is
neurons of your brain is actually meaning
its output.
So it's it's a completely
deliberate approach.
You have to opt into this because
otherwise the technical limitations are
huge. And this is the time
I'm speaking of a timescale of at least
at least 20 years.
OK, number one.
OK, great.
So I think it would be
awesome if we could read
something more out of our brains.
But just the movement of
limbs or things that
we see.
And what
would increasing the number of sensors
say from now, hundreds to thousands
helping this?
Yes.
No. Not yet.
I mean, hundred 120
are the medical. And yet you can
only buy one with
256 256 now.
Blood? Yeah, OK.
And if you increase the number of
sensors, every sensor would help you to
get a better result.
So thousands would be yeah.
But the more the better.
So why are they all the systems that
we see just with these sensors
every few centimeters and not every
millimeter
is the main reason is actually price.
It's just so simple.
An active electrode one
tunnel costs around five hundred
and twenty euros for medical reasons.
Economy of scale.
Yes, and I'm expecting that it becomes
cheaper over time. But the problem is
that we have to be against
magnitudes of more sensors and
the price is only going to it's going
down slowly.
OK. Number two
Yeah.
Given the plasticity of the brain
and the tendency to kind of
move functions around to see
when parts of the brain is damaged,
these columns are they're easily
identifiable and are identifiable
by the signals or.
No, not at all.
You're not. The resolution of today's
ages are nowhere near identifying
single columns.
We still need 15 years for that and
the approximate according to sensor
increases.
OK, number one in the back?
Um, just to continue the question before
you said, you can make them make
people that are blind, see
black and white images kind of vision
impaired vision vision.
Sorry, but
can I show them pictures
that are not those that they see?
Yes.
So I already know how, how it's encoded.
We know how the retina and we know
how the back of the brain is encoded,
which would
be a match, and we know how to create the
signals
that we need. It's also no problem.
We don't know how color.
Let's see.
Yeah, but that's a side effect if
you want to actually do that.
If you actually want to input signals
into the brain, then you shouldn't use
your retinal. You should use the back of
your brain because there's much higher
resolution there.
Thank you.
OK.
Number two So I had
a conversation with a neuroscientist
about a year ago
about much of the same issues,
rather two of our technical difficulties
he encountered.
One is that the data
from decent is really high,
too is.
That's in twice with that's
where they implanted electrodes.
The areas
on the boy intended to move
around, so the needles
that you stuck in one day didn't exactly
correspond didn't exactly communicate
with the same area anymore after
my OK because the area itself moved.
So I don't know, um,
how,
how well this is.
That's just the waiting
areas themselves and humans
at least don't really move that much, but
single neurons can move up to
centimeters per hour.
So I would
say that it is an issue, but
with our current technology,
what we do is we stick at most
64 electrodes in the brain
and this at a grid of one millimeter.
It shouldn't be that much of an issue.
If it's an issue, then you have to
recalibrate after every
week or so. I don't know.
OK.
OK, thank you. I want to get to know.