Dear viewer, these subtitles were generated by a machine via the service Trint and therefore are (very) buggy. If you are capable, please help us to create good quality subtitles: https://c3subtitles.de/talk/1099 Thanks!
[Music]
and uh I'd like to wel welcome you to a
talk titled media disruption led by the
blind which is by Daniel Sabio Daniel is
originally from Puerto Rico is now a new
media artist and other things which he
probably will talk about as well in
Berlin and one of the topics he will be
talking about is infinite Observer which
is this uh the first narrative VR
experience designed from the ground up
by the blind for the
blind hi um I'm Saia or the Glad
scientist um and yeah there's my full
long name um
and uh yeah I want to talk to you about
some of my projects but uh first I want
to uh to try a little experiment with
you guys so uh I'd like for everyone to
close their
eyes and as we all close our
eyes I would like for just 30 seconds to
listen and listen deeply to the room
around
you and as you're listening
create an image of the room around you
create an image of how many people you
think are in the
room an image of all the people who may
be walking into the room right now where
they're coming
from and just do
this
okay now you can open your
eyes and as you open your eyes imagine
those 30 seconds as 30
minutes or 30
hours or 30
years and then imagine this as your
status quo pers perspective of the
world now imagine the thousands of years
spent attending to visual
Persuasions and how you'd personally
interpret everyday Computing with this
newly gained
perspective I hold on to that for a
little bit because we're going to go on
a tangent um so real quick I just want
to give you a long story of my life it's
basically like three
conversations um so permission is
granted please you know if you feel like
you need to interrupt you can do that um
so when I was 5 years old I told my mom
I wanted to make video games she was
like okay whatever um I was just super
into duck hunt um and you know 10 years
later she's like what do you want to do
in life you know it's that time I said I
want to make video games so that led me
on a crazy path and I ended up going to
Georgia Tech studying computer science
and then I ended up in Barcelona met
this guy who changed my life who I
literally on the hall earlier and I said
hey man you changed my life and he was
like yeah okay
um but basically what he taught me was
that I could make art with code um and
that literally just blew my mind and I
realized that everything around me was
code and that everything around me was
you know manipulable and everything
could be an art project or an art
anything whatever you want to call it
and it was like this crazy achievement
became unlocked in my life um so I spent
about eight years after that um didn't
really even care about video games and
just wanted to make stuff with code
installation art VJ projection mapping
blah blah blah all that stuff and then
you know we're talking about VR though
so then I realized one day oh yeah
there's this immersive art that I'm
making all the time I'm making this
crazy projection mapping stuff there's
all these Logistics involved what if I
just make a VR experience and put people
into this wow and it hit me so late that
that also blew my mind um and then I'm
also a musician so when I got the VR
headset I wondered if there was a way to
make music in VR so I figured that out
um and I laughed a lot while I was doing
it and I performed on stage a ton doing
this um but that's not really what I'm
talking about as I was doing that people
would come up to me after every show and
say Hey you make music for VR right I'm
like no I make music in VR but I
realized that hey I had to learn a new
skill set because people are going to
continually ask me about this so I quit
my job um I moved to Berlin there's a
bunch of stuff about how to make stuff
for sound in XR technology if you're
interested slides will be up all that
good stuff and I moved to Berlin and I
took a course with someone who's
actually sitting in the audience and
learned all these different Technologies
about producing sound in a spatial audio
system and I had this question of how
does a group audio driven VR experience
look so I created an experience with a
group of amazing people where we had
four people blindfolded and they're in
the monome space which is in Berlin and
it's a 4D sound center so there's
spatial audio and you can walk up to
sounds and they exist in the space
around you and you can really interact
with sound in a different way and it was
a good opportunity to take people into a
sound focused and tactile focused VR
world so I did that in that setting but
but the issue with that setting is that
it lives there it lives in monome and
you can't take that space anywhere else
but what happened during that and lots
of gratitude to everybody that I worked
with um is that I realized something
that I just kept on thinking about what
it feels like to be blind and how I
could never really fully empathize with
that
experience
and so I want us to close our eyes eyes
again and I want us to just try one more
time but this time as you close your
eyes I want you to just
feel I want you to feel your fingertips
wherever they're laying I want you to
feel the bottom of your elbows the
insides of your knees the bottom of your
feet and I want you to see if you can
detect vibrations around you and see if
you can feel the actual reflections of
the those vibrations in the
room
okay now open your
eyes and you can imagine the same things
again and again and again that these are
minutes and these are hours and these
are
years but at the end of it all what we
come to is that that's our perspective
from our limited experience in this and
that we need a deeper expected uh
perspective so people often tell tell me
what VR is people in the VR Community
try to shove things in different places
say that VR has to be this VR has to be
that and well honestly I'm kind of like
f that noise there's a lot of different
things and we're creating our own paths
in this world um so it's not really
virtual reality or visual reality it's
virtual reality so if you think you need
a headset necessarily to create a
virtual reality experience maybe we
could think about it in a different way
and I came back to this question again
and again and what I realized is
that it transformed into what would a VR
experience be like for the Blind and can
this experience actually challenge
visual
culture and that led to the creation of
infinite
Observer so infinite Observer is a VR
experience experience designed for the
Blind and as I started this project I
thought okay I'll make a VR experience
that's for the blind but also can be
experienced by the sighted as well um
and you know people in the VR Community
are just sitting there calling me crazy
nonstop um and I'm like
okay uh so now we're going to get to a
lot of slides with just text because I
actually feel like I want to embody more
so a less visual tone for this part um
so there's a lot of research to do as
soon as I really seriously decided to
pursue this project I went to
uh so there was that realization I went
to actually live with a uh a blind
couple in the Netherlands um Yanni and
Tina Visser and they run a uh a
foundation called the walk-in foundation
and they actually bring people into a
dark experience which we could say is a
multi-sensory enhanced version of the
Glimpse that you guys participated in
during this presentation and so they
have this multi- room experience that
you go into there's literally no light
your eyes try to search for it for the
first few minutes and then they
literally just get tired of firing the
the cones and rods or whatever that is
um and then you give in and you start to
use your other senses and you start to
discover all of the powers that you have
Beyond just site and so I spent about
five days um living with them just you
know night and day really learning so
much um and what I ended up learning
from them was first of all these people
are incredible um Yanni and Tino were
both fir Walkers teaching people how to
walk on
coals uh Tino was teaching people you
know ancient songs for years and Yanni
actually was a hacker he you know was
building his own Ham radios um and has
built multiple Linux machines and also
has his own distillery in his house
which his liquor is really good and as
they shared their experience and their
their blind experience with me I also
shared with them a ton of VR experiences
which you know I thought okay I'm going
to take the best audio um VR experiences
I'm going to show them to them they're
going to have a response they're going
to say yeah this is good yeah this is
bad um honestly there's zero ux
accessibility for people who are blind
in VR at all there's like nothing to do
with voice menus there's no thought
whatsoever um and yeah it made me it
made me pretty sad at first um but I
also realized um that there's a there's
a great opportunity to to make a change
and I think that as I was with them uh I
really got inspired for The Narrative of
the piece as well because of the the
background and as we were talking they
were saying well what if we had this
group of you know Underground hackers
who are interested in overthrowing the
dominant visual
culture and that seemed to to resonate a
lot um so yeah there's a little
reminder um so that really sparked the
uh the story
building and this is a oh you can barely
see it cool that's fine um so this is
the the basic image that sort of LED our
our guidance in that it's Smoky there's
a unknown figure and you're welcomed
into this group of hackers who want to
subvert these dominant forces of society
and they want to use vibrational forces
to do so so instead of you know typical
ways they may use echolocation to
navigate they then use molecular
vibration to actually teleport and then
they use by mapping the tectonic
shifting because they're actually
underground as well well um they're able
to detect movement above and also detect
things which sound can actually disrupt
visual um vibrations in the visual
Spectrum um so these were some of the
things that we talked about and actually
became parts of the game and we're able
to implement already some of these into
the into the experience so as I
mentioned echolocation we also have a
frequency matching gun where you're able
to to hold it up to different materials
and match a frequency and that's able to
explode open up new Pathways um in this
underground
World um and really what what the
narrative is about is kind of telling a
meta narrative and how how can this
actually include a real Society of blind
hackers potentially through just this
experience in this art so how can we
take a site specific installation for
instance that takes place at a specific
festival or at a specific Community
event um focus on the Blind and have
this merging of okay at the end of The
Experience someone in the narrative is
inviting you to join this group and as
you leave the experience maybe maybe you
have a a text on your phone with a link
to a website that now is access to this
community um these sorts of interactions
and then how is it feasible to create an
audio only group Community platform that
would be easily navigable um by the
blind so these are a lot of the things
that we're thinking about in this
project now um so the technical side of
things is pretty interesting um so I
teamed up with a audio designer who
specifically um specializes in spatial
audio and he works at Magic leap and
with him we've been building out the
audio environment and so we're using
wise which is a popular spatial audio
sound engine for uh for Unity and unreal
and uh because of the plugin that they
have it's able to actually detect the
real time uh game geometries and create
realistic reflections of the sound in
the space so as we test this we're you
know looking very strous at how
realistic this sounds for the blind like
is this passing you know the
test um and so far it's it's been
questioned um and we're wondering you
know is technology not there yet is this
something where we have to continue uh
deeply researching more and more and
maybe use some Cutting Edge research
institutions to reach that point
um and yeah as I mentioned we're working
in uh in unity and originally we're
focused on deploying it to headsets um
but I think that with the small enough
market of the the headsets that blind
users actually have access to um it
makes more sense now to uh to look at
both PC users as well as mobile users
and in using some of the the wise
plugins in a game engine setting it's
easy to be able to to deploy it to both
um and be flexible in that
way so those are a lot of the the
different things that we're considering
um
and can an audio version only exist um
can we create custom solutions that can
travel um that maybe combine all of
these into installation formats as well
um these are all different things if you
haven't noticed this is a in progress
project um and so this is the team um
that's Yanni and Tina and that's Kar and
this is Chris who's been guided ing a
lot of the the interaction and uh
creative um along with
me and really the goal of this project
is to empower the Blind and create a
heightened appreciation for our senses
Beyond
sight and through the narrative
hopefully it will instill some clarity
around the dominance of visual culture
and the ways that all of these Tech
Giants have been using visual culture to
greatly manipulate um people and it may
Inspire some new ways to circumvent this
um and in a bigger sense I would love to
create a voice for the blind as we
develop new technologies um I really do
see immersive Technologies as being like
the next big way that we interact with
computers for the next 10 years and I
would love to see this you know be
potentially a wakeup call or along with
the slew of other projects be a wakeup
call that this needs to be done ahead of
time of projects coming out um or sorry
Hardware
and these these Cycles coming out um and
then yeah creating a community for for
interaction um between real life blind
hackers to be able to discuss
anonymously and even maybe practice some
of uh some of these uh techniques
together that are inspired by The
Narrative um yeah and I do have a
video um so I'm going I'm going to show
that
now
so this is a uh a video of uh an early
[Music]
prototype our members usually find us
track us through the Echoes but you
you're
different you are here for something
special grab the
freaker
I know you're wondering why we're all
here so let me break it down for you we
have never had a voice we're always just
an afterthought and they think we are
powerless and until now we believe them
but look that's not a problem that is
our advantage they've made us invisible
so they'll never see us
coming we will start with the US by
finding their single point of failure we
can break down the greater
infrastructure look the US controls the
entertainment Market if we are able to
Infiltrate The System and corrupt it
without detection and insert our beacons
we can start to find others like us and
accelerate our bigger initiatives across
the
[Music]
world
[Music]
he
[Applause]
[Music]
so that was a video um so yeah um I uh
I'm happy to open up to questions now
and I actually have a few questions for
you guys cuz I know that probably I'll
just say all of you are smarter than me
um
so um very I'm very interested to know
if anyone has experience uh using sound
or noise in the context of packing um
and what people think of the community
extension of the
piece and uh if there's a research out
there that could possibly help this
project that I'm not aware
of if you have a question please walk
through the one of the
microphones mhone number two hi um have
you given any thought uh to the fact
that currently as far as I understand
you need a big installation right so you
cannot have this experience at home
where uh if you just you know if you do
um if you're visual you can just buy the
headset and that's it have you given any
thought into building a uh audio VR
headset yeah so this is something that I
was trying to actually talk to Oculus
about doing um and basically
extrapolating like the sensors that they
have um into using it for head tracking
because when you're in like an ambisonic
world the main thing that lets to
interpret it as uh 3D audio is the the
sensors being able to know when you're
when you're turning your head or when
you're moving within that space um they
kind of uh just brushed me off and and
didn't didn't give it much thought um
but it is something that I'm considering
and also that's that's kind of why I
mentioned um looking towards like some
some AR uh focused libraries where it
has the same spatial audio features and
you're actually able to scan a room um
with your phone and that may just be a
more readily accessible to where okay
someone's wearing headphones and they're
able to scan their room with their phone
and then get audio cues and also have
voice recognition where they could guide
um their character through the the space
as well which is something all of these
people are super familiar with is is
talking to their phone and having those
interactions thank you okay microphone
number
one so thank you very much for for doing
this and for the talk it was really
interesting um I have some experience of
listening to 3D noise 5D music they call
it as well which is easily done with
Ableton in a normal headset and uh my
question to you would be um since I I
think it's not developed yet you don't
have a final product um if you ever
thought of doing something that actually
blind people could learn how to listen
their reality in a better way um
increasing some ways to pay attention to
some kind of noise or feedback that
would only require a nor normal headset
because closing my eyes once and
listening to to the sounds of the guy
shaving your head like the the the VR
experience of that it's quite amazing
what you can do with normal headsets you
know yeah um that's a really interesting
idea to to have like an educational um
element in there
um yeah that's super cool and I love
that you brought up the uh the the
binural uh Barber Shop uh example cuz I
actually showed that to Yanni antina and
it was one of their favorites you know
beyond all the the VR and ambisonic
stuff there like that one it was crazy
um so binaural definitely has has its
place and even talking with like Kar
who's like this expert in you know
spatial audio he's like I don't they've
been trying to explain it but there's
some psychological Phenomenon with uh
the way that binaur works with our
brains and it just kind of
psychologically fakes us out in a in a
cool way yeah so thank you thank you and
if you if you never watch watch The
Barber uh shop
for um audio experience on
YouTube okay again microphone number
one what kind of a equipment have you
used for haptics or have you played
around with haptics yeah so all that
we've used so far is just the the basic
Oculus touch controllers um I've applied
to a couple of Grants with with haptic
research teams um to look into that more
um we were going to talk to hardlight
and then they went under um hardlight uh
hardlight is like uh it's like a full
suit um for haptics which also has its
its pluses and minuses um especially in
an installation setting like switching
over between people that can be like
pretty crazy um but something that I'm
looking into is something more simple
that's like a wristband that creates um
basic vibrations um that actually like
send impulses to all of your hands so
you feel like you're feeling things um
and then the ultrasound stuff looks
super cool um have are do you have
experience working with haptics um yeah
I played around with like hap trators um
they basically create create an illusion
on your skin that you feel like
something's like for instance dragging
you through the air um there's a
psychologist who's working with u
visually impaired people called Vincent
hward um he has a lot of really crazy
experiments um basically it's like a
vibrating motor um that um it's hard to
explain you have to feel it but if you
hold that thing it just like kind of
drags you through the air and you can do
a lot of really crazy stuff with it cool
um but yeah just Google Vincent hward
this crazy guy awesome thank you before
we switch the microphone number two
there are questions from the internet oh
cool
Angel Sim en reality even if it
isal what about all the
otheres yeah definitely um that's why
ideally um this is a an installation
based exper experience where you are
able to
integrate even real life objects um as
haptics and also scent possibly taste um
which is something that I did explore
with the the Blindfolded experience um
that I'd love to bring into the
installation format um but yeah a lot of
it is technology becoming more
accessible like with the
haptics um and the more that those
devices become more prevalent the more
that they can kind of enter into the
work
okay microphone number two hey um thanks
a lot a really inspiring project and um
I would really love to hear a bit more
about the collabor collaboration you
have um with the three other people how
do you work together are you at in one
place um and can you share a bit more of
these experiences yeah definitely um so
we had the the initial um idea which was
kind of in my brain and then I uh I
emptied my brain to Yanni and Tina and
we really started to to go crazy on the
the conceptual side of stuff um and I
talked with Kar um about it who had made
a VR experience um for the blind a few
years ago um and then we started
developing he lives in in San Francisco
um and so we would work over the
Internet together and then Chris I knew
from living in Atlanta and pulled him
onto the project and it's kind of been
me traveling to all these different
places for really intense like
hackathons um and then us working
remotely as we can
yeah next question again for microphone
number
two um because you ask for research that
could help you with your project um I
you probably know but uh just to be sure
there is an existing community of audio
games and people who make those and
there's a researcher in Vienna who's
called Mel uran who does research on
them um that might be interesting for
you cool yeah I'll definitely look him
up thank
you okay any other questions signal
Angel do we have new questions from the
internet no
questions do you have new questions for
the audience
I um I just want to actually get us to
do this one more time if you guys are up
for it
[Laughter]
so this time as we as we close our eyes
I want us to to really take a moment
to imagine ourselves having been this
way before as we have experienced this
before and I want us to use feeling and
listening and scent and
taste and to try to create an imaginary
world around us and to have that feel as
real as
possible
okay thank you so much for for your time
and all your everything and uh enjoy all
of
[Applause]
[Music]
this
[Music]
oh