Dear viewer, these subtitles were generated by a machine via the service Trint and therefore are (very) buggy. If you are capable, please help us to create good quality subtitles: https://c3subtitles.de/talk/62 Thanks!
Thanks a lot.
First of all, yeah, it's
really nice to be here, and
it's quite yeah,
it's for me a little bit exceptional.
So usually I tend to talk a lot,
you know, give presentations a lot, but
this is something that also
gets me a little bit excited this event.
So the Chaos Communication Congress
and I hope now for the next
hour or kind of forty five minutes, I'll
tell you something you find interesting.
And today I want to talk to you about
towards a cognitive quantified self.
So activity recognition for the mind.
Some background about myself.
My name is. As mentioned,
I'm
enjoying living in Osaka,
in Japan for one and a half years now.
And beforehand I did my
Ph.D. in Paso and Pervasive
Computing.
And what I want to talk to you
today about is first,
I'll give you a little bit more
background about my previous work.
So in physical activity recognition
and why I believe that it's slowly
becoming mainstream, so more
and more people are using it.
And then I will talk
about a new kind of research directions
or towards activity recognition
for the
mind.
And we in
Osaka picked a specific topic out
of the cognitive tasks.
And this is reading, and I will give you
a small introduction why we picked
just reading tasks and
then get into some of the questions
we ask, you know, kind of how much
or what are you read?
What are you reading, how
to do
reading traffic or tracking
of reading habits during
everyday time?
And then hopefully at the end?
And this is also why we took a bit longer
to set up. I can show you some demos.
Seemingly, the iPad doesn't like the
VGA connection to the projector right
now, so I have to hurry a bit.
My previous research, or a lot of the
things I'm doing, is in the pervasive
computing area.
And in this case, we really believe
that if we are getting more and more
personal with computers and have more and
more devices around us,
the interface has to vanish because
the limiting factor is no longer CPU
speed or memory or
even battery life.
But the limiting factor, if you have
computing very close to your body
or in your environment is
actually human attention.
And so the interface
needs to vanish and computers or
computing needs to become proactive.
This means computing needs to sense
what people are
doing or what they're doing
to enable context aware systems.
And one of this enabling technologies,
this activity recognition.
And if I talk about this, I
just want to give you probably a small
application scenario just as a video
relatively old one.
Sorry for the
um video quality
of a
maintenance worker, and this is kind of
usually how he does his job.
You have the manual and
the
expert comes to a machine.
The machine might be 20 or 30 years
old.
Although he is an expert, he does not
know, you know, the inner workings of
it. He has to read the manual.
Usually it's a 500 page instruction
set, and he has to always switch has
to do this context switch between
the manual
reading what he needs to do
and then
his actual work.
And what you could imagine already
today is just using a heads up display
and then
you get rid of this context switch.
But still, you have to manually advanced
instructions and browse the
instruction set somehow.
So what you really want to have is a
system that directly
supports
the worker doing the
doing the
maintenance work and in this case
recognizes an error.
You get feedback on the display that
the cabling error
happened and as soon
as the
maintenance worker fixes the error, the
display turns a normal again
and also advancing in the instruction
sets or so on are done automatically.
And this is not only a nice kind
of scenario what we mean with living
with computers or computing helping us
in everyday life.
But this was a user study.
We did that size with 19
trained professional maintenance
workers that were no volunteers, so
they were quite critical at the beginning
using this kind of system.
Most of the sensing was not really
implemented. This is called kind of
Wizard of Oz study, so we focused on
the interface.
But at the end of the study, we really
asked about, you know, when can we use
the system? And at the time, I was a
Ph.D. student and I had to tell them, you
know, it's kind of research, and I'm not
sure.
What's coming out of it?
So if you're interested in this kind of
work, there was a talk five years back at
the twenty five sixty three where I
talked about this scenario and a little
bit more. But if you want to implement
this, we need to have sensors
embedded in the environment, you know,
something like a 3D camera like the
Kinect or
microphone array, or also something
I specialized on
sensing on the body.
And here you see also some old recording
about, uh, doing some exercises
in the gym with accelerometers.
They're mounted on the lower arm,
on the
upper arm, in the trouser pocket.
And what you can already recognize in the
signal is kind of in both
signals. You can count
the repetitions and if you would see, you
know, different machines, it would be
possible to distinguish different
machines and so on.
So you could do tracking, although you'll
also see some problem.
So you see, you know, these bandages,
which
don't look
pretty cool, and also
that the signal, even though on the same
kind of body part or is slightly
shifted body parts or look
for an accelerometer distinctly looks
different and you have to deal
with this somehow if you want to
recognize activity.
And also, as I mentioned, you know, all
the cabling and so on,
it's not, you know, the nicest thing
to wear. And can you imagine wearing
your grandmother, your grandmother
wearing this kind of sensors,
huh?
Yeah, maybe if
you're doing a Ph.D.
in wearable computing,
that's by
the way, my grandmother actively
supporting my research.
But you want to have the sensing
integrated in everyday
objects in general, at
least
if you actually get back to my
grandmother
later on.
Um, so today now we really
have this case where we have sensing
embedded in everyday objects or our
smartphones right now have
the sensing capabilities.
We used five or 10 or so years
back in the lab, the dedicated devices,
the accelerometers and so on in
gyroscopes, in inertial
motion sensors, a really high quality
in these devices.
And we get more and more of these and you
see the first
applications that do activity tracking,
uh,
commercial applications and commercial
devices, especially in the health
and fitness area
and
kind of from simple step counters to
also some iPhone applications
that tell you how much cycling you did
or how much you use your car to
go to work and things like that.
And we see you also already some
commercial devices
that move towards
physiological sensing.
So one example is the Withings
Pulse that has a small
heart rate monitor.
This is also a step counter and has just
a small lady in the back and the heart
rate monitor so you can monitor your
heart rate.
Also, there are a couple of sleep
trackers, so honorable
mention is for the CEO sleeping coach.
The company, I think, went unfortunately
bankrupt two years ago,
but this was kind of a nice device
because they were very open to hackers.
So you could just use
a modified USB
USB
cable in the back of the device and
you could connect it to a USB host
and then the USB host could life
treat your sleep cycles and stuff like
that?
Um yeah.
Now I
got also excited about, uh,
this, uh uh,
Kickstarter campaign about the sleeping
mask. Uh, neuro on.
This is a poly face sleeping mask
that measures using EEG and other
sensors, your sleep faces.
And we slowly approaching
a time where we can extend this
to cognitive tasks.
And in research, we want to be a little
bit further out,
uh,
from from commercial appliances.
So I thought, you know, can we do
something like cognitive task tracking
and,
you know, the first approaches or, you
know, there's also work in cognitive
science. And if you look at it, you know,
the obvious approach is can I directly
tap into the brain like a brain computer
interface or so on?
Now, however, if I looked at the devices,
the drawback is a lot of them are quite
obtrusive. So if you talk fMRI,
uh, you know, you need a whole room and
it's quite expensive, expensive.
So nothing you can carry around.
And also you were talking about
really strong magnets
and
then more interesting, more interesting.
I found
where wearable
devices like EEG or f
nurse.
Um, but
still there you have to do a lot of
processing, and at least my experience is
so far with
it, although I'm quite naive,
uh, towards
this technology is that it's very user
dependent and also especially
each reacts quite bad
to motion. So if you have motion
artifacts and other things
in there and
and the other approach is to use
secondary sensors like, you know,
girlband skin responds or also maybe
motion sensors and one that
seems to work especially well is using
AI gaze.
So using some kind of eye tracking,
there are
two fairly
prominent ways to do this.
One is electro oculus
lock Raffi Yochi.
Now this is more or less using the
same principle as EEG, so you attach
electrodes to the eye.
But in this case, you don't really
measure brain waves, but you measure
the eyes the deep so you measure
the eye movements with this is a
good reference if you want to go into
this, and I think you can use the same
hardware people used to build
EEG devices, uh, to
set this up, and the other is optical.
So here you see one of the optical
eye trackers we use, usually with
infrared light. This is an in
my mobile eye tracker, one
of the uh,
devices we use also lidar for
most of the experiments
I will refer to.
And what you see is it
uses active
infrared light sources.
If you see the six or five in each,
eye tracking is quite accurate and quite
high. The problem with this is, of
course, these experimental devices are a
bit expensive
and
also you need, of course, more processing
in comparison to your team.
And then if we have these kind of
devices, then the question comes, where
do we start?
And we picked.
Reading habits, trying to track reading
habits out of the following
reasons.
There are a lot of cognitive science
studies that try to tackle reading,
but there are very few in situ studies
that explore reading
for in quotation marks
normal humans.
So there are a lot of studies with
infants or also with
people with dyslexia or
mental illnesses,
but very few that give you
information about what a healthy reading
habits when you know I want
to really remember something,
when should I read
it?
And also, if you know, I
mean, reading is one of the primary
learning or knowledge acquisition tasks.
And we have these studies that show
that people who read more seem to
have a higher vocabulary skills
and seemingly also higher general
knowledge. But a lot of these studies
rely on questionnaires and
so on. And I just wondered, you know,
also maybe there are some.
I see at least one or two Ph.D.
students here.
And I also wondered, you know, what
happens if I copied reading habits
of my thesis advisor?
And will I
become a better researcher?
Or maybe
not.
And so on.
So the first thing
we started
kissing the
head of
our institute started with the reading
LifeLock. So we have some character
recognition and also document image
retrieval techniques in the lab
and
document image retrieval.
It kind of
works.
Character recognition in the images
is an unsolved problem.
That document image retrieval works
relatively well.
What you do is you register the document
beforehand and then if you use a scene
camera, you can easily recognize the
document and very fast
way in, even for a large set of documents
or even millions of of
pages are not a problem.
And if you
have a large enough server or a big
enough server, you can kind of deal with
Google Books, size copies
or so. And so this is not a problem, and
you can attach eye tracking to it.
And what you see here is some recording
of a student of
us is reading
the
document.
You get the retrieve page over here
and then can match.
I trace because you have the same camera
from the eye tracker as well.
So you can kind of also stabilize the
eye trace over here and
you can really
record each word.
Uh, the
student is reading in this case, or
maybe later on, we have some device
that will record every word
we read during the
day,
but we also wondered What can you do
without using the same camera?
So what can you do with ETRADE's
alone? And also, if
you don't have something like document
image retrieval?
So these are the three questions I try to
handle during the remaining
talks. Or can we deal
somehow? So maybe I should be a bit
anticlimactic.
So we haven't answered any of these
completely, of course,
but we try to to to work towards
this and they give you all the tools
or try to provide you with tools that
you might be able to do.
Reading tracking, reading
habits during your everyday life.
And so how much are you reading
for this days?
We implemented using, I guess, a word
or meter so analogous to
the step counter, which is sort
of step counter helps for your fitness
life. Maybe what the meter
can help you for your knowledge life.
And we using Outraise, we estimate
the number of lines.
So first you do a reading recognition, a
reading detection and then you estimate
the number of lines and then you can
estimate, of course, the number
of words read.
Here you see again, you know, the eye
tracker. We use mobile eye tracker with
the infrared camera.
And here you see the
projected I
trace on to the document.
This, uh, this little
uh,
circle's fixations and
the connections between fixations as
seconds.
So rapid eye movements and then
always the fixations of the words
and what you usually have, uh, for
a line is you have, you know, a number
of, uh,
fixations and then
along
page towards the new
line.
Japanese is a bit more difficult because
you might have or in general, Asian
languages or Chinese as well, because you
have changing,
uh, reading
directions.
Uh, but in general, it's relatively easy
to do a reading detection
and recognition.
And then afterwards, you just have to
look for these longer decades.
You have to do some little bit of
filtering.
But in a experiment
with 10 users, around 14 documents
the natural reading, we got an error
rate between eight and 13 percent.
So the current that.
Pedometers are around also
five to seven percent, so it's
comparable, so it might help
you
with your knowledge, life and getting
some more information about
how
much reading you're actually doing or how
you compare with other
people.
And then the
next question what are you reading?
We already saw the document image
retrieval with the eye tracker
together, but
can we now also distinguish document
types using, I
guess.
So in this case, we also recorded,
um, some
10 users with five different document
types. So he is the
man, the another magazine
textbook and a newspaper.
And as long as the document types
are different enough, so
text layout and the number of images
as well
as, you
know,
text or reading direction
are different enough.
Um, we can do a fairly good job.
So we are user independent with 74
percent and user dependent with 98
percent. This is kind of a sliding window
approach. I showed you a little bit
later.
Uh, what kind
of features we used.
Interestingly, why we are so bad and use
it in the
user
independent case is for the magazine
case. We had two male subjects
that were just reading the magazine,
the fashion magazine, because they were
not really interested in
the
pictures, so they started
reading
it line by line.
So next time.
And also, if you're working with, I guess
it's important that you have to have the
goal in mind.
So also, our eye gaze is
very, very directed towards the goal.
So if you tell somebody, just read it,
it's maybe not a good idea.
So in this case, this this is also why we
couldn't really use a lot of the fixation
features.
And these are the examples
for
the different, I guess, patterns.
So here you see in
the textbook, this
is an
I guess pattern for the textbook, for the
novel, for
the fashion magazine, for the newspaper
and also for the manga.
And as you can recognize, they're
fairly different.
And you know, if you have more in images,
of course you
have a, uh,
longer fixations and also reading
direction and so on is different.
And now I want to quickly switch to
a small demonstration.
I'm not sure how many of you are familiar
with with item
and oh, quite quite
a lot. OK, I paid a notebook as well.
Yeah. Oh, good.
Yeah. So more or less I
ran into it,
um, a
couple of years back.
And uh, since
then,
oh, I have to switch to the
to the project.
Since then,
I'm using it before and I used mostly
MATLAB.
And since then I'm using it,
um, for the
document type analysis.
So, uh, if you're familiar
with it, you can
use markdown
to write
an introduction or so on or what you're
actually doing.
And then just Python code and you can
execute it in a browser.
Kind of, uh uh,
you can execute it by a shift enter
and you can in
this way explain your code quite
nicely. You can also join
our code
or, uh,
um uh, JavaScript
code. And here I'm currently reading
in, uh, three files
or one for the manga, one for
the novel and one for the textbook,
uh,
raw data. So this is kind of how the raw
data looks plotted directly out of the
file, what you get from that as an eye
tracker, this is just line by line
the coordinates.
And if it is a fixation or affected
and what you want to do, you want to
clean it up a bit because what you
always have, if you if you fixate
somewhere, you have really small eye
movements. So you have fixations
around the point, you don't really fixate
one point, but you fixate around it.
So you want to combine all these small
fixations around the pointed to
a larger one.
And then if you do this,
uh, the filtering.
By the way, the code is also up on
GitHub, so you can later
on check out the code and you,
you,
you first get this.
This is kind of a clean data, but I don't
include the duration of the fixation yet.
And then if I
include the duration of the fixation, you
get the nice uploads,
uh, we
had before on the side.
And then you can also recognize or
distinguish to the reading types
and then to do to do classification,
you need kind of to do a sliding window.
In this case, I did a relatively easy
time to use this approach.
Sliding window, uh,
counting
to
just
discard directions or scarred
up, scarred, left, scarred, right,
scarred down.
And if I do this
for the
three document.
What you can see is, of course, the novel
is up here because it has a lot of sarkar
down because the reading direction is
different and so on.
And of course, this is a very simple,
dumbed down version
of, uh,
what we did when I can get
to. Oh, here.
OK.
Um, but it gives you
an impression of how to do
it. This analysis, and I will give you
some, some, uh, other
info and hints about Peyton
Notebook and what kind of packages
we are currently using.
And also some more general
hints about how to do research and how
to to organize, especially data
folders, because it happened to me quite
often that I overwrite, you know, data
at the beginning, overwrite some
important data with aggregated data.
So, you know,
didn't keep the raw files and
this is
bad. So you shouldn't do this.
And
then the last question I want to tackle
today is the most
interesting, but also the most difficult
one. This is how much do you understand
while reading?
And here we
try to estimate reading comprehension.
So we gave people just a standardized
comprehension test section.
Uh, and
from the two week to week is the
Japanese equivalent to
the truffle.
And people read it with a mobile
tracker to answer questions afterwards
marked difficult words.
And we were
not
able yet to estimate
the toy scores.
We want to do that because then later on,
you don't have to answer questions more.
If somebody asked you about your language
expertize, you can just read a text
and afterwards you know how proficient
you are in the language.
The sounds a bit scary, but let's see
if we can get there.
But what we could do is we could figure
out difficult words.
And this just worked on the fixations.
So just using
again, red fixations and
blue that card.
Uh, and
then we do a horizontal projection
towards the
line.
Uh, the user
read.
And then just the histogram of the
fixations,
uh, tells you, uh,
which word is difficult?
In this case, more was not understood
by the reader.
Uh, so we are working on that.
And I think also by looking at the text,
especially English, is interesting
because, um, we had just
Japanese readers
and
you see with
the, uh, not
so good Japanese
Japanese readers of English text
that they
mimic
the Japanese sentence structure.
So they read the sentence first and
then they come back to the subject
and they will look at the verb at the
last part, because in Japanese, the
word comes last.
And this is interesting.
So I'm wondering also if you can do
classification of
a
personal background by looking at
how you are doing, reading or how you
look at something in general.
I mean, this is not limited to reading.
And then for future work,
maybe this is I just took the fifth bit
page and changed it a little bit.
Uh, so you have, uh,
words
that maybe you know and you can share
with your
friends, uh, how many words you read
during the day
for the
quantified self people?
I'm not so sure if it's such a nice
vision to have and see what type
of reading you're doing.
And maybe the professor can see,
you know how much mangas and how much
science papers the students are reading.
Uh, and
or concentrate on reading or how much
in the different languages you do.
I think that's slowly possible with
this technology.
And there's one problem, although
this is eye tracking hardware, so far,
it's quite expensive.
But you know, if you
look at patterns, especially from Apple,
Google and Samsung,
maybe the next version of Google Glass or
similar similar product will have eye
tracking already implemented into it.
Now you might say, Oh,
you know, I'm not
so sure if I want to run around with
Google Glass, and it might be a bit dorky
and so on. And I promised you a comeback
to my grandmother, you know, even my
grandmother was wearing it.
So I think it's it's OK and
safe.
By the way, my grandmother and
grandfather had it for a day
and um, and used
it, and they really liked it, and they
have a good track record for technology.
So I was quite surprised to do like the
S and the WI and
the iPhone and the iPad so far.
So yeah, let's see.
I'm not so convinced, but let's have a
look. So I have one of the devices
actually here with me.
Um, and I can actually
I can show you a small demonstration if
you want to. So one thing is, if you
look
at it, um, you can
always see if it's on.
And if I know, um.
So, OK, Glass,
take
a picture, take
a picture, you also see, you know, that
I took a picture and you maybe also heard
the sound. So at least privacy
wise, I think you can get
spy
cameras
on Amazon, you know, way easier that
can take
way more
creepy pictures than having somebody,
you know, that takes pictures with this.
But I kind of understand also.
I mean, at first when I saw the device, I
thought, You know why?
Why put the camera on the outside?
I want the camera that points towards the
eye. But for one thing, though,
um yeah.
I mean, this is just my personal
preference. So I understand
why they put the camera on the outside.
Um yeah.
And also wanting
you what your warranty if you do stuff
like that. But uh, big kudos
to the people at Google.
A lot of them are also from the wearable
community, so that's why it's close
to to,
uh, uh, to
people I know or also kind of, you know,
from from conferences and so on.
And they made it really easy to
to hack it so you can unlock the
bootloader.
You can easily get root on the device
and you can,
uh,
flash easily the
firmware,
the uh, the
uh, factory firmware back onto it
if you nearly bricked it, which happened
to me twice. And it was kind of a little
bit sweating before the all because I was
not sure if it's working.
So things there, but said
kind of, you know, why did you put the
camera into it?
I wanted the camera that was just the
eye and also then
I made a little privacy in hand.
So while I wear it to the Congress
because I know people are, you know,
uh, privacy aware here, and
I don't want to be running around
pointing a camera at somebody.
So later you'll see me.
If I wear it out, you'll see
me wearing this.
It's done with a 3D doodler relatively
quickly. And the problem is you need
to spell out a little part at the bottom
because just the light sensor for the
display and otherwise you cannot
see anything in normal lighting
conditions on the display.
But yeah,
now I want to quickly show you a
demo that, uh, Sony
hack
together
and uh,
which first.
Yeah. So.
First, I have to give you a screen, so
this might take
just a
bit
here.
So.
Yeah, if you if I
if I
do this, you see my screen,
so
um.
OK, let's take a picture.
You see the interaction and you have this
sliding view.
So out of the
way.
Uh, the sliding views.
So this is kind of my Google
now things
or weather forecasts and stuff
and back you see kind of
pictures I took.
And what I want to show you now
is they have actually something pointing
towards the eye.
It's just a proximity sensor
that it
if you root your device and void your
warranty and might break your device
and so on.
Um.
You can.
Hopefully it won't.
Oh, yeah,
you can run.
Why is the census showing?
Maybe I have to wait a bit, so what does
this currently doing is counting my
blinks.
It has a little proximity sensor pointing
towards the eye, which is
just, yeah, infrared.
Sometimes it's not working so accurately.
And also, if you give a talk or so on
your blinking frequencies, higher are now
it shows the graph.
It's relatively chunky, but
if I blink, you see the spike
down.
Yeah, and it's kind of it's working, you
know, OK.
And I've already thought about
applications if, of
course, if I move my head and especially
if the lighting condition changes, you
know, you will also see some recognition
of faults blinks.
Where'd you get the idea?
And I think this is a good point why you
should wear glass while driving.
Because if you have an app that can
detect if you close your eyes
for a longer time, I think this is quite
useful and it should not be banned
from driving.
But people should think about, you know,
kind of interesting applications you can
do with it.
So that's that.
So now I can get back to the slides.
And, you
know, I'm also wondering what now
that you have an IMU?
So now you have an accelerometer,
gyroscope and magnetic field sensors
strapped to the heart of a lot of people,
at least not so many, but maybe later
on more. And also this proximity sensor,
you know, kind of what kind of things can
you do towards the cognitive
task tracking?
And then I also want to mention because I
was with
the eye tracking
and how expensive it currently
is, at least some eye tracker and other
commercial solutions.
There are a couple of open source eye
tracking efforts out there.
Um, I haven't
tried those, but you know,
I just wanted to give you a list of
things I know about.
And of course, also you might,
you know, you can build your own hardware
and
3D printed versions and so on.
Already out there,
you can also mount one and you can also
mount on Google
Glass,
a small webcam.
But why can't we use commodity
devices to do eye tracking?
Maybe some low fidelity eye tracking?
Because a lot of the devices here
have a front facing camera,
and maybe we can do something with it.
Not as nice as
the infrared stuff, you know the infrared
cameras, but maybe.
And now I want to to ask you if
you can maybe give a demo about
what he's currently working on?
OK.
And I should mention social, yeah, it's
uh, uh, last year, Bachelor in
Osaka was like a perfect union with a
hole.
Hello, my name is Joya.
I'm a student. Okay?
And today I want to show you the
demonstration about our appreciation,
which tells us where we are
looking.
Although.
Oh, it's very difficult.
I.
It doesn't like pets, I wonder about
this idea.
I.
It's not about book, it's a feature.
But.
No.
Oh, no.
OK, OK, OK, OK, OK,
thank you. So I will show you the
demonstration.
It's very easy to get.
Oh no.
Oh oh,
oh, stop.
This is
OK.
I had a shoulder, I dragging on it, but
now he's carrying this with them
and looking at the bottom
of application with nothing my,
I guess, status.
I think you are succeeding.
The point is my I gave point.
Now I am looking for a the image.
I'm looking at the second paragraph.
Can you see?
Oh.
Oh, oh,
yeah. So now I am looking to
buy the image.
So if we want to try by yourself,
please catch us up, stop Congress.
Thank you.
So I think so, yeah.
Um, although, you know, with technical
difficulties, but you can also later on,
try it out yourself, and we are really
trying to source this after after
some point in time.
Currently, it's working relative.
It's really working very well, for sure.
Yeah, and a little bit for me, but we
have to extend it for two more people.
No, the
trouble is actually not.
So detection works really stable and
good, but the problem is doing
the estimation on screen and especially
if you move your head and also if you
move the device there a lot of tricks we
can do. But you know,
this is something we need to work more
a little bit more on.
So I had a backup video just in case the
demo fails. But you saw this and
this kind of brings me also to the more
or less to the end of the presentation.
And it just included one slide
to thank all of the people that were
involved with the research or in the
community. Just to give you also a
feeling for I mean, I'm just now
standing here presenting this stuff.
But uh, new
ideas don't come out of a vacuum
and it's always through
discussions and through
living community.
And I think especially in the pervasive
and variable field, we have kind of a
good community and it always feels
also similar
to what
I have here, you know, kind of visiting
the Chaos Communication
Congress, also just listening
to the talks here.
I always feel energized and
motivated. And sometimes,
especially if you look at the world
news,
uh,
you know, you can feel depressed
at some point in time and just think,
you know, why am I doing this?
And a
lot of
or if some people take
the technology and use it for,
you know, yeah,
mass surveillance or other things
that were not intended by
maybe the people that envisioned it and
so on. So this gives me energy
and another kind of special feeling
sorry for the two.
Thank you slides. But this is important
because these are the people that
actually did the work.
So, you know, I'm just presenting here
and especially Katsumata an
cannot try and show you.
You already saw on the on the floor.
They're excellent students and also
excellent hackers.
They're also doing a lot in iOS and
mobile hacking.
And you should google their names, maybe
and check out some of their their private
projects and so on, but also
to
the
eye tracking students or
yoshimura,
uh, Nybakken
and
Karachi.
They know anything about mobile and
stationary eye tracking and how to filter
and enhance
eye gaze and classify it and so on.
And then because I'm
I'm German, I also
want to leave you with
a German sentence, you know, feel
for some gadget you might know,
and I wish you
that
you have fun with technology
the next year or the coming years.
And also thinking about,
you know,
how people
could misuse technology.
And if I'm talking to you,
you know, I'm talking to the people that
are the living lab. You know, if
we think about something or people
publish something or do research, you are
the first ones who try it or, you know,
even research yourself or build
stuff. And you are also the first ones
that come in touch with problems
that this technology might have.
And you have to speak up and
tell us or other people also
what you found.
I think it's important that we exchange
and also that the research community
sometimes listens a little bit more to
what's going on in the hacker community
to prevent these
bad or dystopian futures and have
a brighter one where we can really have
fun and with technology.
So this brings me to the end of the talk.
So now if you have
questions,
remarks violent descent,
um,
there's also my email address,
or especially if you're interested in,
um uh,
maybe speed reading or knowing a little
bit more about your reading habits
on tablets or so on, send me an email.
We might have a prototype
maybe middle of next year or
so on about the speed reading app we
developed together with the University of
Stuttgart, Tilman Dillinger and
Niels M.
Tenser, who was, I think, also here
somewhere.
Thank you, Victor.
And um,
and
also to shameless advertisements.
So there's Augmented Human Conference in
October this year, and I just included
this because we have,
uh, a keynote talk
with exoskeletons.
So that's pretty cool.
Nothing to do with the
talk, but you know.
OK, thanks a lot.
So now if there are questions, also catch
me, I'm here the whole time during the
conference. And if you want to talk about
things, so thanks.
OK. Yeah, thank you very much for the
talk. I have one question about what you
can already see from your data.
Is there a relation between reading
skill and comprehension
of the text? So do people who read faster
have less words that they find difficult
in the text?
Yes. So I mean, I cannot talk too much.
I mean, we just started with the
research, and I cannot talk too much to.
One thing we saw was, of course,
people who have trouble with text
comprehension usually also go back.
And then what I talked about was
just this effect of the Japanese
speaker that they really search for
specific words or so on.
And what you see also that people who
have good text comprehension.
So for the first reading part,
if you wanted to estimate a toy score
which didn't work really well,
one a good
estimate was just time.
So how fast is somebody reading
and the people who read relatively fast?
We are also
the better students.
The problem, then, was that our spread of
topics core was relatively limited
because we had four students and they
have a certain kind of score,
so it's hard to tell.
So we had to try to find also people who
are a little bit worse or really, really,
really good and not to have native
speakers, because then you get this
trouble with cultural background because
I think it matters a lot in which
language your your native and in
which you are not for this task.
But yeah, this is something where we want
to get at an especially if this is
this interesting stuff that gets towards
the speed reading EP.
You know what happens if you read faster
and faster and faster?
At what point do you lose text
comprehension
and how
can you get a reading pattern that is
really, really good to do some type
of reading? So if you want to understand
a specific fact, you know what type
of reading pattern should you use to get
this? So this is something I find
intriguing, and especially also how to
measure text comprehension because it's
not so easy, you might say, Oh, I just
asked questions afterwards.
So then the problem comes, you know, if
you, for example, take a Dan Brown
novel, you
know somebody you read one Dan Brown
novel and you give him a second Dan
Brown novel to read and ask him questions
about it. He might be able to answer the
second questions, but yeah, you know,
so there's this this trouble?
Yeah, yeah, rather
technical question, I guess,
on the word or meta thing.
I was wondering where you have to
estimate the words because it looked
like in the text that the person
stays with their eyes on
the first part of the word longer and
then jumps to the next word.
So I was wondering if you could also take
like the jump to count the words
and not the lines.
Yeah. So the problem with this is
this depends highly on the skill
of the user.
So first, we try to use fixations,
but if you have a fast user, a fast
reader, you just have to fixations for
light.
And then you have this trouble.
So it's highly depends on the the reading
skill of the user and as well.
Also, we had some trouble
beforehand, so the video I showed you was
really nice because you could kind of
follow
along the
words read, but usually depends
also on the reader how good the
resolution is. Sometimes it's shifted up
or down, and then you really have to work
with the line breaks and then you get
into really difficult situations
where you have people rereading
sentences.
And then we also are not sure what to do.
You know, should you count it again?
Or is it, you know, just trying to
understand the sentence and stuff like
that? So, uh, just this issue as
well.
Oh.
A short
questioner. First, a great talk.
I write a lot and
then with the slides, be online,
the other sites will be online.
Oh, sorry, I couldn't put them up.
I'll put them on my website
and
maybe later tonight,
and
I'll also put them up on the the
conference
and the speech reading, why didn't
you put a bigger focus on that in
your
head? Because this is kind of this is
something we just starting
to work on. So I don't really
have too much insights, but this is
something I'm personally interested
and also something the relation
between text comprehension or
comprehension and learning in general and
sleep. This, you know,
there was the reason I showed this to
you, a sleeping coach. And these things
and I'm wondering also about the poly
facing sleep or sleeping in different,
not eight hours a night, but kind of
three times three, two times three
hours, something like that.
And what it does to to learning in
general. But as I said, I mean, I don't
really have any results yet.
So I wanted to to
get people
maybe even a little bit excited.
But also what I can tell
you so far is,
uh, more.
There is not much science towards speed
reading out there so far.
What I would also be interested in is
like how how
attention changes
the results. Like
if? If you check for
people who do meditation
there, they are known for being able to
concentrate for longer times.
Yeah. And uh, whether they they
would have a higher comprehension than
over maybe two hours, three hours or
so of that would be interesting.
I mean, I know this.
You know, maybe this, uh, ten thousand
hour rule from this, uh, bestseller
list or so on, they
don't know what the book name is.
Uh, um, anyway.
So there's this rule of ten thousand
hours. If you study something ten
thousand hours, they say you can
be an expert in that people.
I'm not really sure if I should trust
this, but I was reading the papers that
he cited in the book, and one was really
interesting because they showed
violinists, expert violinists
in a in a
music school and they had,
you know, kind of three rings.
One is soloist player.
This is the highest skill, one is
orchestra player and then teacher.
So these are the three levels.
And he says in the book that the only
discriminating factor between the three
is the the number of hours they train.
So soloists three trained more
than the other two
way more. But if you read the papers,
there's another thing that is different
for the smallest players that sleep,
so they sleep off the training more.
The nap during the day.
And I wonder, you know, they're kind of
two to causation
if it can be two reasons.
One is they train really hard,
so they feel really tired after training
or that, you know, sleeping
after learning something really helps
your memory. And there's some evidence
towards this. And but it has never been
really studies, so
we better get our sleep after the lecture
now. Yeah, of course you go to bed.
So one more quick,
one more one. I mean, OK,
so you also encouraged us to think about
consequences for society.
So during your talk, I was imagining
that my teacher would actually make me
wear those glasses and give me a novel
and then a check if I actually read
a novel, which, you know, in school, many
books are pretty boring.
And then they if they figure out, I just
spend one second for each page, then they
would probably give me a bad mark because
I didn't score well, whatever.
And so my question is, can I convince
you to come up with a license that would
prevent teachers from doing that?
And yeah, it can.
That's that's
tough because I actually had
a teacher in
a talk I gave at a conference in the
audience and he was coming up to me
afterwards and said, Oh, can we use this?
You know, I take it for the classroom
because they are already using iPads.
And he thought, You know, I want to know
how fast people are reading and so on.
But I'm also wondering a little bit about
if you can fake the
to
the reading pattern.
And somehow if you can kind of
manage to to, uh, just
look at the screen to fake the reading.
It's hard. I think it's not really
possible.
But yeah, I'm wondering
about if you can come up with a good idea
about the license to prevent this stuff.
Would it be up for that?
Also, I would be really up for, you know,
if you mentioned you saw what you can do
with the small camera and so on, and
you're not always sure if the camera is
on. So how to prevent that
people stream your guess data somewhere.
I mean, I'm already feeling uncomfortable
having my email or on out there,
but can I get one more or
is it we are out of time or
sorry, OK?
Yeah. So maybe
later. Yeah, probably stick around and
you can ask your questions in private.
So thanks again very much.