*36C3 preroll music*
Herald-Angel: Good. Ladies and gentlemen, we have here a talk by Sebastian Staacks.
Do I pronounce this well? Sebastian Staacks: Yes.
Herald: Yes. Staacks. Staacks. [In German] Ich musste das mal in Deutsch sagen. And
he's related to the University of Aachen. He did a PhD physics. And he was in a team
that developed a fantastic application, as I mentioned earlier on. He developed the
app phyphox. Do I pronounce this well? Staacks: I would say phi-phox, physical
phone experiments. Herald: Okay. Yep. Of course. I'm sorry.
I'm not in that kind of department. But this application actually gives you all
the possibilities off your the usage, off your smart smartphone. Really? Really
extending certain borders, to my opinion. So please give a warm, warm welcome here
to Stefan.
*Applause*
Stefan: Thank you. Thank you for the introduction and welcome everybody to my
talk. Yeah. As you've just heard, I'm a physicist from the RWTH Aachen university
where I developed the app phyphox. Phyphox is an app for those of you who do not know
it already. That uses the sensors in the smartphone for physics teaching. So the
idea is that students can use their own phones to do experimentation in class, in
the lecture hall. So for schools and universities. I should explain. That in
contrast to some other talks by me. This one will not be that much about education
because it is the chaos communication Congress and this is the hardware track
here. So I tried to tell you a little bit about the app, a little bit about the
sensors that we have on our phones and. Yeah. Would we love to get in touch with
some, especially people from maker community and from open source communities
to find some connections, how he can get many open source projects together?
Because I've got so much feedback from teachers and I think I could also use some
feedback from other developers as well. So I would like to start with a short
explanation of what we actually do. So yes, I said I come from a university and
there we have this introductionary lecture for physics students, which is called
experimental physics one. And it's typical lecture. Looks like this. We have a fancy
new lecture hall by now, but the situation is the same. We've got 300 I think 370
students this year sitting in a lecture hall and doing no experimentation at all.
There's only one guy experimenting and that's the professor. And the students are
sitting there and enjoying the whole show like they would enjoy a YouTube video and
maybe they are mildly amused if something goes wrong. OK. And we thought we could
change this by using the sensors in the smartphones. We're not the first ones with
the idea to use the sensors there, but for some reason we decided to write our own
app, which turned out to be quite successful then. So in contrast to the old
version where students just had to look at and I'll get the assignments where they
can do their own experiments with their own measurement devices. And to give you
an idea of what this looks like. I would like to start with the first experiment.
Which is about centrifugal acceleration or centripetal acceleration depending on your
preferred frame of reference. So the idea is from a rotation movement, we want to
measure the radial acceleration as a function of the angular velocity. So the
rotation rate. To do this we take a regular smartphone, this is an iPhone 8 in
this case and we put it into a salad spinner. Okay. We get some rotation in
there and whoops let me just place it in there. Sound is not important, but it
sounds nice. I have been told. So here we get the live data from the phone already.
Acceleration on the y axis and angular velocity on the x axis. If the salad
spinner is actually moving. And what you see is the faster I rotate the spinner,
the farther on the right you get your data because that's angular velocity and also
the radial acceleration increases. If I'm not going too fast because then I do not
get any data at all anymore. Let's slow down again and we can fill up the gaps
there by going really slow and filling up this path. And in the end, if so, who here
has a physics background some more than expected. Great. Because those of you who
just raised their hands would not be surprised that we expect a square
relationship between the radial accaleration and angular velocity. Those
of you who do not know will believe me from this plot where on the x axis we've
got the angle of velocity squared and on the y axis the radial acceleration we get
a straight line and that's what you would expect. So besides the physics, because
this is not that much about the physics. This is a simple experiment all our
students could do and actually they ge, we gave them this assignment. We gave them
also a bonus point if they created a video. Don't worry. Their consent to that
we use the video was not related to the point, they first got the point and then
we asked for their consent to use the video. And we learned two things from
these videos. A Our students do not really have salad spinners. they've got bicycles
and office chairs, but b and that was the most important thing. It looks like I
mean, these are from this year where we got almost 100 videos they we actually
could trigger them to go out, search for something where they've got the
rotationary movement and they could repeat this experiment. Ok. Another example which
actually changed just the course of the lecture a little bit is a situation where
we first give the assignment before we actually let them, before we actually
discuss the theory behind this, which means in this example, this is a little bit
older because we did not get there yet this year, we assigned our students to build
string pendulums. They look very similar because we were very precise about how
they should build them. And then we had an online form where they could submit the
length of their pendulum and the frequency they received from it that they measured
with the pendulum. They should do this for three different cases. And the idea was
that we did this assignment long before we discussed the pendulum in the lecture so
that they have got a little bit of research experience. And after we
collected all the data from them, then the lecture would discuss the pendulum. So the
physicists were there now. We do a small angle, approximation solving differential
equation. All this theory stuff. And in the end we were done, we could tell our
students, well, we do not have to do this experiment on stage. Now, because all of
you did this experiment and we simply can compare the theory that we just arrived
with your data. And it worked out quite well. So you see most of the white points,
which is the data from the students matches the theory, which is the orange
line, except maybe for those three who should proceed on a career of theoretical
physics. But yeah, so this is all something got nice feedback from and this
is in principle how we use the app and what it's designed for. There are also of
course many applications in school by now. More teachers use this in school than we
use it at the university. So we take this into consideration as well. But that's the
reason that I am standing here talking about the sensors in the smartphone.
That's the reason that I am trying to access them. so let's have a look at what
sensors we actually have in our phone. I think the first one that most of you would
think often talking about sensors besides obvious stuff like the microphone would be
the accelerometer. So I think yeah, I think I first explain how the
accelerometer works. OK, so the accelerometer in your phone is actually a
so-called MEMS device. MEMS is M E M S stands for Micro Electrical Mechanical
System and it looks roughly like this. It's a simplification. If you search for
actual MEMS devices, simply search for M E M S and accelerometer and you find some
pictures. They usually are a little bit more complicated, although the
accelerometer is not that much more complicated. It consists of an orange
case. Yeah, well so far so obvious, but also two contacts. The blue and the red
one and important part is this silvery structure here or the metallic structure
which is under etched its bit hard to see on this picture, but it's actually
floating. It's only attached to the sides, you see light in between here. So if you
move around, the accelerometer the inner path, can actually move. So let's do this.
So at each point where the device is extra riding in one direction or the other
direction, due to inertia the that the metallic part in here is distorted, moved
into one direction and we can measure the amount by which it is deflected by this
movement with the two contacts by measuring the capacity between these
structures. So that's the principle of the accelerometer. One thing to mention at
this point is that it's in the sense of physics. It does not really only measure
acceleration. It measures acceleration you see in this image of the device
accelerating. We get some data, but if you imagine we take this device and rotate it
like this, then of course you also get a deflection of the of the metallic part by
gravity. So gravity is pulling it down as well. And that's the main reason the
accelerometer is in there because the developers and manufacturers of the phones
are not really interested in measuring acceleration, at least there aren't that
many use cases for it. But instead, what they want to have is an indication on
which direction is down or which direction is up. So when you rotate the screen of
your phone, actually they can rotate the content of the phone as well or with this
you can also then control video games by tilting your phone and stuff like this.
Because gravity also deflects the accelerometers. Earth's acceleration,
which you try to avoid because from didactic point of view, this is a
nightmare to distinguish these both. But the point is that we can detect rotations
like this and this is pretty much in every phone. I mean, this is not really a
statistic. This is just the first pie chart we have about availability. I have
never encountered a single phone or tablet that does not have an accelerometer. So if
anyone ever encountered some special device, some very unique device that
doesn't have one. Let me know because I would be interested in this at least. I do
not know of any device on which phyphox actually runs, which doesn't have an
accelerometer. A bit more interesting is which data rate we can achieve. So most
accelerometers have several hundred samples per second. Actually the fastest
ones go up to 500 hertz and but there are also many devices that only do one hundred
hertz That's 100 values per second. These are mostly the cheaper Android devices and
all the iPhones. So I think the internal accelerometer will do more on an iPhone.
But I have to admit, at some point I can understand why they might limit this. But
on an iPhone, you get 100 hertz. That's the limit. From the API, what you can get
there. But this is actually quite a lot. I will later see what we can do with this.
And one other point about calibration of this thing. Actually for all the sensors
to get reasonable units from the system so the acceleration is given in meter per
square second. I just realized that if I get the units, that's something I would
really tell my students. But yeah. So on the x axis, it's a meter per square second
and you see that as a wide range of values that you get there. So this data is from
our sensor database. I would mention it later as well. This is contribution from
our users what data there this. This only absolute value that we get from resting
phones and we would expect nine point eight one meter per square second for
earth acceleration. There are some local variation, but not on that scale. So do
not expect your sensors to be well calibrated. Also, if you've got any app
that tells you you can push a single button and then calibrates your sensor,
don't trust it. It's not that simple. These sensors may have different errors on
each axis. They're all 3D sensors we've got an X, Y and z axis. These errors can
be linear errors so you have to multiply a correction. It could be an offset. So it
would have to add an correction. And on top of this, the entire device could be
tilted within your phone. OK. So actually, if you look into the data sheets of the
accelerometer, they have some tolerance on how much they might already be shifted or
rotated within the package. And when soldering it into the phone, I would
assume there will be an additional error. I've seen so many different errors on
different phones. It's not that easy to simply calibrate that. But let me give you
an example of what you can do with it. Or just a quick look first. So we see in our
app. Yeah. So this is phyphox. OK. Thank you. Got this. You have an entry
acceleration with G. That's the extra raw data from the sensor or as raw as we get
it. If I started you see if I shake it, you get some readings there. It's fast.
It's already great. You can apply to pendulum and measure the acceleration of
the pendulum like this. But something I want to demonstrate is that we can also
get the frequencies from this data by doing a fourier transform and calculating
the frequency spectrum of this exploration data and to demonstrate this I brought a
little device a old hard disk drive. It says it's broken, but it's still rotating and
that's important part for us. So if I place my phone on top of it, start the
measurement. Turn on the hard disk drive. And then you see a peak showing up in the
spectrum and it settles at 120 hertz. If you don't believe me. Unfortunately, we
don't have a camera here right now. You can later have a look. It's supposed to
run at seven thousand two hundred RPM, which is 120 hertz. We can even get a time
resolution of this. So if I turn it off again, you see how the frequency drops
down. And if I turn it on again. There it comes up again. OK. So this an example of
what you can do. It's great for students that can check if the washing machine at
home is working properly or they can check other things. But usually I do not
like to bring washing machines to talks. So I used the hard disk drive here. One
other thing you might have noticed before is that we've actually got acceleration
with G and acceleration without G. The second one is actually a sensor that
removes Earth's gravity. So if I start the one with G, you will notice that down here
on the Z, the axis you still have the 9.81 meter per square second, which is great
because if i rotate the phone. This contribution goes to other axis and we can
determine the orientation of the phone. But this is bad actually for dedactics
because actually the phone is resting. It's not moving at all. There's no
velocity involved. There's no acceleration. So luckily, there's also an
acceleration without G, which gives us roughly 0 an all axis unless I actually
accelerate this thing. Problem with this is this is only a virtual sensor. This is
a sensor that's fusing the data from the accelerometer with an additional sensor
like the gyroscope. So we can actually distinguish between rotating the phone or
accelerating it in one or the other direction. Usually you only get
acceleration without G. If you also have a gyroscope in your phone, I've seen two or
three devices that offer you acceleration without G, even though they don't have a
gyroscope. This case, don't trust them. This is merely guessing. OK. So it's. They
probably have only low frequency filter on top of this, or they're averaging out
your movement and this doesn't really work for anything. Yeah so that's the
accelerometer or one other thing I want to mention is if you look into the API to
access the sensors yourself for some reason you will notice acceleration
without G is usually called linear acceleration in our app since it's made
for teaching. We decided to call it with and without G. So if you find
accelerometer, that's the one with G and linear acceleration is the one without G.
If you look at other apps or the API. Okay. Next up, I already mentioned this
one is a gyroscope. If you have, some physics background. Then when you think of
a gyroscope, you're thinking of a device that's spinning fast so it has some angular
momentum and then usually you want it to be heavy and to have the weight at the
large radius. We've got a strong moment of inertia so that you get when it's spinning
fast, a strong, angular momentum and due to the conservation of angular momentum.
These spinning devices can keep an axis regardless of rotating the frame around
it. That's what I was thinking about, a gyroscope of what I think is a gyroscope.
When you just give me the term out of context, of course, a heavy, huge, fast
spinning device is the last thing you want in your phone. So that's not what's meant
with the gyroscope when people are talking about gyroscopes in your phone. Instead
there again you have a MEMS device. So again, micro electromechanical system. You
notice this looks almost exactly like accelerometer. If you look for real
devices, those are actually much more complicated because they need some
specific geometry to make sure that they do not act like an accelerometer. But the
principle is easy to explain with the same geometry. So we again have this floating
metallic part and we've got 2 contacts. So again, we've got a part that can wobble in
this direction here. But on top of this, we've got the motion that's perpendicular
to this. So this is now not depicting the motion of your phone, but this is
depicting a vibration that the gyroscope does by itself all the time. So there are
different ways to build them. Some have a rotary motion, some have this linear
motion. Also, the way to create this motion makes this device so much more
complicated. But in principle, it's a similar structure which is vibrating forth
and back and now if you add rotation to it. It's a little bit hard to see it as
it's rotating the inner part now suddenly gets deflected. That's changed, right?
Frame of reference. So let's get the camera in sync with this device. What you
now see is that the inner part is moving left and right, although the device itself
is only moving up and down. And the reason is I don't want to deduce it entirely
here, but most of you probably have heard of it. This is the Coriolis effect. So,
yes, in fact, your phone is determining the rotation rate of your phone, not the
actual angle, but the rotation rate or angular velocity due to the coriolis
effect, which is just mind blowing if you do some of the calculations. There are
some manufacturers on the Internet which claim that they can detect a movement of
the order of magnitude of a single atom. And I believe them because we use similar
structures in solid state physics. So that's possible. If you want to try it,
just turn on the gyroscope on your phone. And do slight rotation like this, which is
about the Z axis, one perpendicular to the display, you can detect really slow
rotations with this. And think about the fact that this is done using the coriolis
effect and it's just mind blowing I think. So this sensor is a bit more available.
Actually, almost 80 percent of the phones have them. This has become significantly
more since Pokémon GO. The reason is when this game came up, suddenly people
noticed that there's a device called the gyroscope. And if it's not present, they
did not have this AR mode where you can actually take pictures of the nice cute
Pokémon and so on. So this is when the many people noticed it and the
manufacturers decided, OK, let's just throw in the gyroscope as well, because
it's not that expensive, in fact, usually it's on the same chip as the
accelerometer. Then they're sold as one thing it's an IMU - Inertia Measurement
Unit not important at home, but so it's quite a common thing. And the sensor rates
look pretty much the same. You mostly notice the dip in the 100 hertz regime
because those are the real cheap phones, which then also don't have a gyroscope.
But most of the phones achieve higher rates. Again, since we were laughing
before the iPhones also are here again at the 100 hertz. Wouldn't make sense to have
the gyroscope faster at this point. Yeah, but that's it about the gyroscope you've
seen it in action in the salad spinner. And that's one of the sensors you do not
really see that often directly, but were just mostly there to assist other things
that you do where you need to get smooth motion like controlling games, AR . And
actually removing the Earth's acceleration from the accelerometer. Next up is a
magnetometer, which I think is a more obvious sensor because that's your compass
in your device. So when you're doing navigation with a GPS in your car, it's a
simple thing. GPS gets a position, you get a sequence of position as you going and
from the sequence of the positions you get, the direction you're moving in your
car and your phone is attached to the dashboard at least i hope so. So it's
pointing in the same direction you're moving, everything's obvious. But if
you're standing on an open space looking for not sure a train station or anything
and you wondering which direction you want to go from point of view of GPS, it's
always the same position it doesn't get an orientation. You need a compass, which is
the magnetometer. How do we get a compass on your phone? This is usually a hall
sensor. A hall sensor is in principle just a conductor with charge carriers so these
are the nice shiny white balls here drifting from one side to the other so
it's just an electric current. And if you apply a magnetic field to an electric
current or to any electric charge, then there is an effect. You might know from
school, which is called the lorentz effect. So there is a charge going one
direction, you get the magnetic field perpendicular to this and then the charge
is deflected into a direction perpendicular to the flying direction. And
yeah, that's lorentz effect the older guys, of you would know it from CRTs. If
you bring a magnet close to a CRT, the entire image is messed up due to this
effect. And that's what we're using in hall effect sensor or hall sensor you've
got this electric current and if you bring a magnetic field close to it, the charge
carriers are deflected to one side or the other. And therefore, if you're measuring
the voltage perpendicular to the flow of the count, you get. Yeah. You get an extra
voltage that's proportional to the magnetic field. That's the hall effect.
That's how your phone is able to determine the magnetic field. This one is even more
common than the gyroscope simply because it's used for navigation and people start
to notice if it's not. If it's not present and they do not get an orientation in the
navigation software. But the actual rate of the sensors is much slower than for the
accelerometer. Most of them are running at 100 Hertz. It will be important in two
more slides. Besides that, there's not that much strange about the availability
of this, but it's extremely sensitive because it's supposed to measure Earth's
magnetic field. Earth's magnetic field has the strength around 50 micro Tesla. This
is not much actually if ever carried the magnetic magnet with you. Did you fear of
some force from the Earth's magnetic field? Of course, it didn't need to build
some compass where the needle is floating on something like this to actually get a
rotation. It's a very weak field and that's good news and bad news as well,
because on one hand, it's very sensitive. downside is it's very sensitive. Which
means it saturates very early. If you want to measure the magnetic field of an actual
magnet. Don't even try it will saturate right away. You do not get anything to
demonstrate how how sensitive this actually is. I've brought a flashlight, so
a very simple one. And I switch to a modus where we've got an s.o.s signal. That's
coming up, a point in this direction and I place it next to the magnetometer in my
phone. And yeah, you see right away so much of his seeing the lights are pointing
in this direction. You see the s.o.s signal popping up in the magnetic field
reading simply because of the current going through the LED. So that's what we
call an Oersted-field. This is just the typical magnetic field you get from any
current flowing. So I stop it. We got a nice SOS signal over there. Three short, three
long and three short signals. And it's just coming from this simple flashlight. And
this is also a good indicator on how sensitive this thing is. I mean, if you
place your phone in a case with a magnetic some magnetic closing mechanism, compass
wouldn't work anymore. If you're not careful when paying your clothes and you
place your phone on the big magnet that removes the theft protection from the
clothes, something in your phone would get magnetized and would certainly be stronger
than Earth's magnetic field. For the rest of the day, your compass would be pointing
in the wrong direction. Okay. Luckily, usually the phones are able to notice this
and they recalibrate the phone to simply subtract any constant fears. That again is
bad. If you want to do absolute measurements because you have not much
control over the recalibration mechanism, you can access the raw data value. So if
you folks there's a checkmark where you can disable the calibration, but then you
have to do everything by hand. You will certainly have some background that's
annoying. And one other thing, you should also take care and notice where your
actual magnetometer is because in most phones it's on top left corner, top right
corner, top center. And this Pixel 3 is a very strange one. It has it on the right
hand side, but it's never dead center. I think because of all the currents in the
phone, I mean, you're charging your battery with three amps. How much you
charge them now? This would yield a stronger field than a flashlight and you
would see it in the magnetometer again. Now for what you can do with this. So as
little homework for all of you who came by train yesterday, when I came here on the
ICE, I turned on the magnetic spectrum, the same thing as the acceleration
spectrum you just seen. And when you're doing it on train, you would see a peak at
16.6 hertz. It might depend on your actual seat. You might move it around a little
bit. But so far I usually always saw this peak. This is the electrification
frequency of the German railway. So you can simply check if it's working properly.
You should see 16.7 hertz. Okay. One other thing that some of you might get in your
head right now, that you could do this with simple electrical outlets. There you
would get a problem with the rate. So that's what I mentioned, that the rate of
the sensor is quite important. I also got something via Twitter yesterday. Just as a
response to the other one, I thought, well, I was looking for an example like
this for this talk talk, so I just put it in. This is a measurement of an American
power outlet which is run at 60 hertz. But this guy is seeing 40 hertz and he was
wondering about this. That's what's called aliasing. So the alias effect, sort of you
might notice this from computer games. They usually use it in slightly different
context. The idea is if you're measuring a frequency that's higher than half of the
data acquisition rate of your sensor. So this one is runningat 100 hertz like most
of the phones do. Then half of this frequency is what's called the Nyquist-
frequency. And you notice that the spectrum goes from zero to the Nyquist-
frequency. This is simple math, not simple math, but its maths. The roots of the
fourier-transformation, you could say so. And if you try to detect a frequency
that's higher than this, so an American power outlet with 60 hertz, actually the
higher frequency is showing up as on the other side of this upper limit at 40
hertz, even if you go to a higher frequency, it would shift down further and
further until reaching zero and then it would shift up again. So if you're
interested in this. Check out some articles about aliasing. If you're not
that interested in this. Just keep in mind, if you're measuring frequencies that
are higher than half your data acquisition rate, you will not see the correct
frequency. OK. Then one of my favorite sensors, the pressure sensor for this one
I need. Again, the phone. That's not on a wire. Let me before before I show
anything. Let me demonstrate what it can do, because that's something I find quite
surprising. Let's turn on the measurement. By the way, those who are wondering how
this works. There's a function in phyphox, we call it remote access. It's basically a
web server running in the app which provides the data so we can simply access
the data on the phone to demonstrate or to control the measurement. And now here we
see the pressure sensor. Right now, just mostly noise or what I do now is I hold it
up. And if we wait a few seconds, you would see that the pressure's actually
dropping. It has dropped far enough. Then I place it on the ground and the pressure
is rising again. So actually, your phone, if it has a pressure sensor, has a
pressure sensor that's sensitive enough. So we turn it off to measure a change of
pressure of a distance like this. OK. And that's again, when I first tried this, I
repeated this test several times before, believed it was just not by accident. And
how do they do this? You have got another device that actually has a cavity. So
below the bluish gray part, there's a cavity in there which is covered by a
silicon membrane, which is the bluish part. And if you change the pressure this
simply moves it like you would expect from a membrane just in small. And to detect
this movement, here is some material on top of this which changes its resistance.
Or resistivity depending on the strain created by morphing, dismembering. And
unfortunately, this sensor is not that much available. So about a third of the
devices that we know of have the sensor. Of course, there's some bias in there from
the users that submit data to us. This means that, yes, these are usually the
more expensive devices. So my rule of thumb is if it's an iPhone, they usually
have the pressure sensor except for the iPhone SE or some older models. If it's an
Android, if you payed half as much as you paid for an iPhone, then you have a good
chance that you have to pressure sensor as well. But OK, that data rates? Yeah.
Varies a lot. So the iPhones, like you just saw the rate of about 1 Hertz. Most
Android phones are on five, ten or twenty five hertz. I've never had a device like
this in my hand. It does 100 hertz. I don't really believe that this makes sense
because I already noticed on my phone that I think it does 25 hertz. Just handing it
because of the sealed casing introduces more noise than you can actually use, at
least for these small distances that I use it for. But you can do other funny things
with this. So this is something I received by Dianna Cowern. You might know her as a
YouTuber called "The Physics Girl". She used a pressure measurement on the flight.
It's something you should do anyways, because that's the way you can figure out
how much air you get to breathe up there. It's much lower than you might expect.
But she saw something else. So at some point she saw the drop in the pressure and
increase again. And she asked her followers, what could this be? And I'm not
asking the audience right now. I just give you the solution. She wasn't lavatory and
she flushed the toilet. So when water and air gets sucked out, you can actually
measure this. And then about a month ago, I found someone else who allowed me to use
his measurement. So this guy, Phillip Smith, was on an airplane again. But he
did not actually go to the lavatory. He stayed on his seat and he just checked
when people were flushing the toilet. So as he sat, there was there were
turbulence. So they couldn't go for a while. And then there was the rush while
the toilet and he was plotting it. So just for those of you that came here by plane,
just a hint as a conversation starter next time, when the guy next to you goes to the toilet and
he comes back, tell him exactly all of the head to flush the toilet and ask him why.
Okay. And you would enjoy the rest of the flight. Some other example that we
actually use is measuring the movement of an elevator. So this is a lift in Aachen.
We have the accelometer which measures the acceleration of this thing, gets the total
height difference of the elevator from the again, from the pressure sensor, a
barometer. That's a pressure sensor. And the velocity of the elevator as well from
the change in height. OK, so next time you enter an elevator, I want to see you all
to take out your phones and measure the distance that the elevator is traveling
and the velocity at which it does so. OK. So these are, in my opinion, most
important sensors, some honorable mentions. Almost all phones have a light
sensor as well, which controls the display brightness depending on the ambient light.
Unfortunately, there is no API on IOS to access this. So if there are apps
that seem to access a sensor like this, they usually use the camera instead, which
is which also works quite well. But it's slightly different since the difference
between illuminance and luminance, which I do not want to go into detail here. And on
most Android phones, they are badly calibrated or do this so much difference
in the quality of the sensors. We have to check it on your own phone if it's worth
anything. But it's a bit difficult. This proximity sensor, which is the one that
turns off the screen when you hold the phone to your ear when you're
actually doing your call. Sounds interesting, but unfortunately it only
distinguishes or has I know it distinguishes between between the near and
far value, which is the difference between five centimeters. So I do not have that
much use for it. There is the temperature sensor, maybe if they are officially there, then
they usually come along with the humidity sensor, but that's the sensors in your
phone. So you should be a little bit skeptical about this. You're mostly
measuring the heat from your battery or from your device. They tried to compensate
for this, but that's a difficult thing to do. So if you actually, one, need a
thermometer, take a thermometer. They're not that expensive. OK. You might see some
temperature sensors that are not official. Which phyphhox can pick up. Those are usually
temperature sensors that are part of the pressure sensor to compensate for
temperature effects. So they're not even designed to get an outside temperature.
OK. So I wanted to mention this. While the information about where we got the
information about the sensors from, so in our App at the very bottom, does this entry
submit to a sensor database which tells you to leave the phone resting on a table?
It also checks if you're actually doing this, doesn't let you submit it before it
is happy about the error rate or the standard deviation of the accelometer. And
if you submit it, we collect the data on phyphox.org/sensordb and that's
where I got the statistics from so far. So if you're interested in what a new phone
that you're about to buy can actually do. Of course we don't give you any guarantee,
but you can check up or check out all the data, all the phones. At least those that
are already in our database. And of course, I'm happy if you contribute
statistics about the census in your phone as well. So you might want to play with
this later. And then finally, the last thing to finally conclude is some
information on how you can access the sensors. Of course you can write your own
APP. I think here quite a few who can do this. Just have a look if you can write an
App. Have a look at the API. They're not too complicated. It's easy to access the
sensor data. If you're not interested in designing your own app, but you want to
include sensor data in some other projects, there are three ways you can use
Phyfox for this, which I want to introduce, because that's something that's
one of the reasons I wanted to connect here. Don't hesitate. Phyfox is free. You
can get for free on Google Play and on the Appstore. And when I say it's free, I mean
it's really free. So it's open source. The GPL and you can also get an afterwards we
assured of code running on your phone is the code that you see. And we have
three versions how you can.. At least they are categorized into three versions. How
you can access the sensor data. First thing is you can implement something in
Phyfox yourself. So I've got this editor, visual editor of all file format, which
allows you to take a sensor, place on mathematics. So this is just adding stuff,
but you can apply a Fourier transform or anything and then assign it to a graph.
Alternatively, and of course a bit more powerful. You can have a look at our XML
format, which defines all the experiments. So actually all experiments to see in
Phyfox are not hardcoded, but they are defined in our own file format you can edit any of
them to your needs. And when you're done you can transfer your data with the QR
code. Do not try to scan this QR code just from your QR code app. You have to scan
it from within Phyfox and if you do, you'll find a nice little experiment which
uses our file formats to implement a Turing machine that's counting binary up to 256.
So this is the proof that all file format actually is Turing complete. So you can do
a lot with it. Okay. I'm not suggesting that you're trying to implement doom on it
or something like this because you won't be able to. It's not efficient that way.
It's not designed to be Turing complete. It just happens to be Turing complete. So
if you want to do something more, you can connect to Phyfox via a network. You've
seen one example with the salad spinner. When I said that there is a
a web server running on the App. You can use this to access the data directly from your
preferred programing language. There's an example where I'm using Python to read out
the sensor data and control a synthesizer. So what's running on the web server is
basically a rest API. So yeah. Just visit our website and learn how to do this. So
you can read out the sensor data of a network and control your project with it.
An alternative to this is a new network interface that we have, which is more on
this XML side or the design of our experiment configurations, which is meant
to collect data from many users and not life data. So we had this lecture. So this is
the new lecture hall, by the way. So we had a lecture where every student got a
spring from us and there was supposed to build a spring pendulum and we collected
the data from all students and the lecture hall in realtime on the big screen to
determine the dependency of the frequency from the mass of the pendulum. And another
example. Just a few days ago, we during the winter solstice, we asked our
international users to point their phone at the sun. So we get an angle for the
elevation of the sun and the azimuth from the magnetometer with a compass. And this
way we could trace the path of the sun across the earth from all the users. What
each black point with the line is a contribution from a user. So, yeah, from
this we could, for example, determine the tilted angle of the earth's axis. OK, so
just example, what you can do is this network interface, as long as we're able
to set up some server to receive the data, you can use this network interface. We're
still working on this network interface. So far it can only do HTTP requests, get
or post. But we are also planning on implementing Mqtt and other protocols like
this. And the third option is a Bluetooth connection, which is mostly designed for
sensors. So if you want. If you have some Bluetooth low energy sensor that you want
to read out, you can use Phyfox. So there's an example of a Texas Instruments
sensor tech, which has a software which is not designed for Phyfox. But our file
format is flexible enough to simply tell Phyfox how to read all the
data and suddenly we've got the sensor that can run independently from the phone.
And of course you can include your own projects like this. So there is an example from
actually my institute, because originally I'm in solid state physicist. So we're working a
lot with graphene and this is a demonstrated we create that was an ESP 32.
So this is another version of an Arduino, or Arduino compatibel. What we're doing here.
We're reading out a graphene Hallsensor and so. It's all similar to the holecenter
of phone, but based on graphene and we can get life measurements in Phyfox with this.
And so if you have an Arduino project with which you want to.. from which you want
to send data that is plotted in Phyfox, you can do it with a bluetooth low
energy interface. But if you have some patients and maybe wait two more months,
we are working on Arduino library to make this simpler. So this the entire code, you
would need to read out the analog input from an Arduino and send it to Phyfox to
be plotted. OK, so this is working right now. If you cannot wait, you can check it
out on our website. So this is already available, although it's a work in
progress. The interface will change a bit still. I would prefer if you want to
start right now, if you contact me so we can get some feedback and maybe even
design the library also to your needs. So that we get an idea. So with this, I'm about to
finish. So just a short summary what I'm hoping I can trigger. Yeah. So if you were
mildly amused, mightily entertained by this by this talk, check out our Web site
or check out our YouTube channel or Twitter. We can get some more examples,
what we do with the sensors in the phone. If you are a teacher, are teachers here?
Quite a few. That's great! And if you want to use this in class or in a lecture,
check out our Web site phyfox.org. We've got a database of experiments that you can
do: phyfox.org/experiments .That's then actually about physics and less about the
hardware where we also demonstrate the experiments and how they work. If you are
a teacher and has a specific project in mind. Check out our editor to design your
own set up with which you can do something specific for a very specific experiment.
phyfox.org/editor. Then if you are working on arduino project and want to plot
something, you can visit Phyfox.org/arduino, where you already can access
our library. Although it's not complete as I said. So maybe wait a little bit or
contact me first. If you have a Bluetooth low energy device that you want to use or
integrate. You can visit phyfox.org/ble. If it's about a device that you did not
design yourself, you probably need some background information about bluetooth low
energy. Should know what a GATT server is and how characteristics and services, new
ideas and all this stuff and bluetooth energy works. And it's good to get some
documentation or to be good as reverse engineering, but in principle I haven't
seen many devices so far which could not work with phyfox easily. Then if you want
to read the values for another project via network, visit our website, the wiki on
our website. phyfox.org/wiki, where you can get information about the rest API and
on your network interface. And finally, something I would really love if you want
to contribute. If you can write some apps, I mean you can use a lot of things. The iOS
app is written in swift. The Android version is written in Java. Our webserver,
of course, has web development and Html in JavaScript. So if you want to contribute
there. Visit our Web site at a phyfox.org/source. And we would love to
see some help in development. With this I finish my talk and I'm looking forward to
any exchange we will have later and any questions. And I'm just thankful that it
was allowed to talk here and get so much attention. Thank you.
*Applause*
Oh, by the way, since it is up there. One bad news, unfortunately, I can only be
here today. So if you want to talk to me, try to catch me today. You can also call
me. I actually brought a DECT phone, but, uh, sorry, only today.
Herald: Oh, my God. So quickly, though, we have questions now, 15 minutes, then 15
minutes, I think. And then afterwards, you have to find him and catch him. Thank you,
Sebastian. Questions. Shoot. There is one. Question: You mentioned aliasing affect
during.. Is it possible to change or modulate the sampling frequency to actually find
out our frequency above the sampling frequency? Sebastian: Yeah, that's that's a good
question. Not only because of the of the alias affect, but also because some
projects also want to reduce the sampling frequency. It's a little bit tricky
because on both APIs and both IOS and Android, you cannot specify a target
frequency, you can only specify a frequency that specific for certain use
case. So for example, you say I need the accelerometer data, that's which at a rate
that's reasonable for UI changes or at a rate that's reasonable for games. Right.
Right. That's as fast as possible. So if you do it for UI, you get something like
let's say two three hertz. We heard something like this or you doesn't waiting ages before
the screen rotates for games. It's 25 50 hertz something like this. So we can
control the game and fastest is the data I've just plotted. And Phyfox always
request the fastest we can see and in Phyfox we have a setting, we can limit the
frequency. Unfortunately, if your frequency is not simply a multiple, no,
the other way around is. The frequency given by the device, is not a multiple of the
frequency that you gave. It's not easy to break it down to the target frequency. So
you usually see some odd cases where Phyfox tells to group the sensor events
along this to get near this frequency. So it might not work that well. And
especially if you're looking for the alias effect. This might really mess up their
alias effect, so you might need to try a little bit which frequency looks good to
do this. But of course in principle you can average about multiple values in this
way or simply pick only every end value. And this way we'd use the frequency. And
yeah, this can be done to our editor or to the main screen. There's a plus button
with which you can simply expand which already allows you to set this simple
frequency. Just keep in mind that you cannot really always get to the
target frequency, right? Herald: Right. There is another question.
No? Yes. Please. Question: Hi. Thanks for the cool task.
It's a great app. I love using it in school. I was wondering if those cool
animations how to sensor types of working are available.
Sebastian: Sorry. The animation scene. Yeah. I think I wonder how to do this
best. Before that, I was already thinking about sharing the slides. Actually, my
talk is space it's just written in HTML in Javascript it's not easy to control for everyone. That's
why I did not simply upload it. I would if I would check later, if I can, upload the entire
talk in some way that makes sense either on our website. I'm not sure if it makes
sense to upload it to the system of the conference. Still, after the talk, I would
check it, but I am not... I want to share the slides, but I probably need to add
some documentation on how to use them because they are not Power point PDF or
Latex generated PDF. It's handmade.
Herald: You can always cut them out of the video getting streamed and La la la la la.
Yeah, right. Question: Just a quick question of the the
axis of the phone. They're like like that and that distorts us.
Sebastian: So it's not for most phones. The X-axis is reading directlon. The
Y-axis is upwards along the screen and Z access, Z-axis depending on your dialect is
perpendicular to the screen. I'd say in most cases because officially the X-Axis
at least I think I've written this documentation for Android is along the
natural reading direction of the device. So if you've got a huge tablet which you
naturally would put in horizontal alignment, not portrait mode, it might be
that the X-Axis is the long Axis. I have never seen this myself, but I'm a little bit
careful to say that all these devices have the same axis, but Z is definitely always
perpendicular to the screen and X and Y are than the other ones and they are fixed and
usually the short side is the X-Axis. Herald: Ok. There is one more question
there, please, sir. Take the microphone. It's next to you. You got it off the
... Question: Hi, you mentioned the necessity
of the magnetic sensor to to determine the content orientation. Can you not use past
G.P.S. data and then integrate over the gyroscope data to get the current
orientation? Sebastian: *Lauthing* Mathematically, your
correct, problem is integrating sensor data is not as simple. I'm often
surprised on what some software can actually do. If you do it naively
right now I only have an example in mind for the accelerometer cause it could also
say you can integrate the accelometer data to get velocity. You can integrate the
velocity to get the displacement of the phone of the location. If you do this,
we've got a very simple example in our wiki. Very naiv even one without any
filtering, then just the noise means that's if there's little arrow, you summit
up integrations, nothing else but suming up in small steps. You get an
offset error in the velocity. If you integrate this again, you get an error in
the location with which is growing with the square of the time. So if you do this
for location and try it out with our naive approach your phone is supposed to
be 100 meters upwards after about 10 seconds. If you do this for the gyroscope,
it's a little easier because you only want integration. But still there will be
some drift. I'm not sure about all the techniques the manufacturers imployed to
filter out any errors. I mean, obviously the gyroscope is self calibrating
otherwise, it would be pointing in different direction all the time. And on
some phones I've seen it jumping when it recalibrates. But if you simply
integrate this, you will certainly get drift, there's no way that you can get
a fixed position. What I think what they probably do for most cases, they use the
gyroscope to immediate direct rotation and then try to fuze it
in some way with the magnetometer information to keep it fixed so that at
the end you're not pointing the wrong direction. But the gyroscope itself, only
on its own, is unfortunately only giving you the rotation rate, not the absolute
rotation in contrast to an actual gyroscope. The big one that's rotating. So
it's at least not that easy. That's all I can say.
Herald. Whow? What the bunch of information, Sebastian? I really love the
.... There is someone else with a question. I really love your replication.
Actually, it was really immediately fun to, go.
Question: Thank you so much for a great application. And my question is, just very
short. Can you also integrate external sensors through Wi-Fi or is it only to be
early? Sebastian: No. That's what I meant with
the network connection. Network usually has Wi-Fi in this case, I'm not sure if it
would work on a conference like this into the cable. So now you can get the data
through our REST API. Might not be the fastest thing. Maybe we will add to our
network, our new network functionality, something that will keep open apart and
push the data in there, so far the best thing to go is with our rest API.
Question: I was just thinking about the external sensor connection.
Sebastian: So external? Sorry, I was thinking a different direction. Actually,
that's a good question. That reminds me of that, that there's something I wanted to
add. You can use the REST API in theory to push data in there, but that's only a
parameter in the Url. It's simple a Get/ push off a single value which doesn't get
get you far and which is quite inefficient. However, within you network
interface you can do requests to other devices so you can GET request and already
is able to respa Json packet as a response us to interpret the adjacent
packet as a response. And that's where adding Mqtt and stuff like this, this is
supposed to go in both directions. But this is really new. So if you've got
something specific, try if it works or contact me if it's not working, if you
need some help, if you find the bug. So but it's supposed to work on your network
stuff. That's there in the configuration. So the idea of the workflow of all this
connection with specific devices have something set up like this. You create a
configuration for Phyfox, which in the end is supplied to the QR code. For example,
the user scans the QR code. And this all the information, how to communicate with
the device is already supplied. You can also do this for Bluetooth. That the
device itself provides it to Phyphox. But in the end it's these configurations
and for the new network interface, it can also receive data from the network. But so
far only via HTTP. Question: OK. Thank you.
Herald: I have maybe a last question if no one else has. What's the next step? What
is your next goal? Because this is a tremendous successful thing. And you see
the educational purposes. So that's fantastic, actually, isn't it? It's not
only on university level if you're using it, that's all around in Germany.
Sebastian: That's not in Germany. It's by the way another thing you could
contribute. If you're speaking a language that has been translated into Phyfox is
translated by volunteers and it's already available, I think in 2010 and 2012, 2013,
14 languages, something around this. So yeah, but next step I think will be using
the camera because that's another sensor, broadly speaking, which we are not using
at all, which can do a lot, but we haven't yet started on this. So lot to do in this
project. Herald: Super. I'm looking forward to see
you next year then. *Laughing, Applause* Sebastian Starks, thank you very much. An
honor and a pleasure to have you.
*Postroll music*
Subtitles created by c3subtitles.de in the year 2020. Join, and help us!