Dear viewer, these subtitles were generated by a machine via the service Trint and therefore are (very) buggy. If you are capable, please help us to create good quality subtitles: https://c3subtitles.de/talk/423 Thanks!
And we're very happy to have you with us
tonight, Christian author, so please give
it up and have a lot of fun with the
talk.
Thank you very much.
It's really a privilege to be here.
This is an amazing event.
So we're going to do the project and talk
a bit about the scientific aspect behind
it, as well as sort of the journey we've
taken over the last three years that
we've been working and how our
methodology has kind of evolved over that
that whole process.
So really, the the main question behind
the project that we've kind of been
addressing the whole time is what would
the world be like if you could see
through the eyes of someone else?
If people who are in conflict on
different sides of a war could experience
what it's like to be someone else, their
story, their family and so forth, how
would that affect our ability to connect
and resolve conflicts?
So really what we're looking at is the is
the relationship between identity and
empathy, how you construct your identity
and how that inhibits or facilitates
empathy.
So here is an example of some research
that is happening in this
movement we're seeing in neuroscience
in the past 10 years in the
embedded tradition and around empathy.
This, for example, is a study that showed
that there was a stronger
correlation between the
neural representation of threat.
So you do
a brain scan of a person
being threatened to be hurt
and and you would report
a missing person looking
at someone that was being threatened.
And when there was two people,
two persons were close.
They had were like family ties
or friends, there was a stronger
correlation on the representation of
threat. So in a way, familiarity was
promoting sort
of blurriness of self
was a representation on the on the
neurological level.
Yeah. Basically, the difference like
between you and the other person
neurologically is not that different than
if they're being threatened and you feel
close to them.
So we have also memories.
A lot did was in investigating
mirror neurons and mirror neurons are
specifically specific neurons
which fire when you both perceive an
action and when you take that action
yourself as well.
And there's a nice quote from him there,
we're going to kind of go a bit fast
because we have, I think, too many
slides.
So one of kind of
illusions that you can induce
is a very on one hand
illusion that when you see
plastic hand being touched and you
feel your real hand is why it's hidden
behind the blanket, rather, at the same
time, in the same area, you can induce
the feeling of owning a plastic hand.
And this kind of effect is
further investigated by
person. I've been in Karolinska
Institute in Stockholm, Sweden,
using replacing visual
feedback, uh, through virtual reality
headsets and camera.
So in this sense, in the example
they were studying how owning plastic
body.
So using the same technique of touching
the two feet at the same time with affect
the perception, for instance, of the
world around you. So, yeah, the size
of your own body determines the perceived
size of the of the world.
So it's interesting that with that,
it's basically that your sensors don't
operate independently of each other.
There's the effect each other all the
time. And so from that, you can hack your
body's perception about itself and
override your own sense of body
ownership. So we see also
another event lab in Barcelona
led by Mel Slater. And they've done some
really interesting research of what that
means. Also socially, how you can change
your perception about stuff using
embodiment effect and in this
case, virtual world.
They've seen that if you put a white
person in the skin of a black and
avatar with black skin, that actually
reduces their implicit racial bias.
So this is not the kind of bias that you
see you have, but it's the bias that is
that they detect through some sort of
rigorous testing afterwards.
So this is really the inspiration behind
the project. We decided to take this
and put it in a different context, put it
in an artistic context, investigate the
ethical implications of it and other
possible uses.
So this is an early slide
or an early picture of how we started
out. We're using a very
low budget stuff, things we bought from
China, virtual reality glasses
from the 90s that we kind of got from the
university somehow and really started
experimenting with these ideas and
experience, trying to recreate them and
see what would happen.
So the kind of system that we
have been using, there's been further
iteration after that was kind
of clearly inspired by the research
we're showing before, except
that we try to replace a virtual
environment by sort of a
human element.
We're not using virtually generated
environments through
computation, but rather using another
person's sensory
motor contingencies to
to transfer to the body.
So we here
we're using the Oculus Rift. We're using
other headsets before, which is sent
to, um, um,
a mounted camera which follows the head
movements. And the performer would be
copying the movements of the of the user.
And we use identical objects and
identical spaces so that they can get the
appropriate touch feedback.
So really what we're looking at here is
how to use this in how to develop this
project in different context.
So one of the first places that we
started doing this was in the which is a
performing arts center outside of
Barcelona. And they have an interesting
association with the Indignados movement
in Spain. So there's a lot of different
community groups and activist groups
within this place.
So the idea was to find a methodology
that would allow us to investigate this
without hierarchy. So we're not just
saying, oh, we're the technical people
and we're going to tell you how to do
this, but rather say here's how the
system works. Here's the technology.
Here's the scientific aspect.
What would you use it for?
What kind of stories or situations would
you like to explore using this?
And so working with anthropologists
and community organizers,
women's groups, dancers,
hackers, a bunch of different people, we
tried to kind of create a bunch of
different experiments and situations and
see what would come out of this.
This is one of the first ones here.
You see basically the two identical.
Oh, yeah.
Here you see two identical spaces,
identical objects.
And then, as Arthur was saying,
the user will reach out and pick
up an object that they see.
The object is there and
at the same time, it's in the other
space. So they can they
can feel that. And then at the same time,
this is associated to narrative.
So we would have prerecorded
narrative of the person that they're
embodying associated to different
objects. So you can branch the narrative
then in sort of embodied theater
kind of way in non-linear form.
So that's really the first iterations
that happened.
There are other experiments we did
maybe want to talk about.
Yeah, so that was Sara Ann and
her mother, Anna, who's an artist.
And they they came up, they can they
reach out to us and talking about
the kind of difficulties that had in
their relationship and they were trying
to figure out a way to try to
do some kind of performance around that.
And Sarah made a very beautiful
story about it, was very
poetic and very
interesting metaphors for such a young
kid. She was talking the story was called
the the girl with the red tears,
the blood tears
and talking about through the story about
her difficulties relating to other
people. That's cool.
And her mother was so and what
she was telling the story. She was
drawing it and her mother was kind
of like copying the movement to get this
embodiment effect into
the world and story and
feelings of her daughter to
hopefully get a better understanding and
work in works in both ways, I
think both as a
sort of self-improvement empowerment tool
for for Sarah and
Anna together, like a
sort of insider connection and to that
that story.
So it's really like the kind of
experiences that people I mean, we tried
the experiment. I was like, oh, this is
interesting. But once we started sharing
it with people and they were creating
their own experiences, it became
it was very powerful.
They came back weeks later and were like,
wow, that's amazing that she's doing
better in school. We have a better
relationship now. And this kind of stuff
was really what motivated us to keep on
working because we weren't being paid.
This was entirely self-funded.
And we're like, OK, well, let's see what
other things we could do.
So one of the other people we worked with
was Yousef, who was an illegal immigrant
in Spain who had come over illegally on a
raft and had been living in the street,
had learned to read, learn to write,
and eventually became a dance instructor.
And he wanted to share his story with
people in the local community where he
lived. So we worked with him over the
course of several weeks and figured out
different protocols and recorded some
audio and had different objects that he
could interpret that people could interact
with. And then we presented this as a
public performance for people in the
in the town where we were.
And it was amazing.
Like, you see, this is really, I think,
what became interesting.
It wasn't that the technology was like
providing something that that
was needed. You know, like you don't need
the technology. What it is, is it
provides a context where you're in this
liminal space, you're not quite yourself,
you're not quite the other person.
And what happens in that space is
the starting point for dialog afterwards.
That's really how we try to use this
technology, like whatever video game
equipment and stuff, and bring the
co-opted and bring the focus back towards
something human. When you take when they
take it off, they're actually facing
the person that they just were.
Before that, they were just interacting
and experiencing themselves from the
perspective of the other person, and
that's really what seems to be a powerful
experience after taking it off, the
people who don't know him, you know, he's
big tall guy, you know, like just wanted
to reach out and hug him. And it was kind
of these sort of experiences which kept
on confirming to us that
we're sort of on to something there.
Yeah. We've been also working with an
integrated dance company
which featured both people
in wheelchairs and able bodied dancer
dancers. And through exchanging with
them, one of the, um,
suggestions that came out from
wheelchair users, what they wanted to see
themself dancing, standing up,
which was like kind of thing where
I mean, I think would have come to our
idea to propose that because it seems
kind of weird. But after that,
we developed some kind of choreography
language where using his
hands, he could, um, there was an
improvization happening with a dancer
which was used to, um, create
to get through at the end
of the rest of the body.
And so they could see themselves dancing
through the camera of the other person
standing up.
There's all these videos online.
We don't really have much time to show
them now, but you can just look online
and find documentation about these
projects. One of the interesting things
about this is how it relates to
see if we were doing this in an academic
institution or scientific institution,
it'd be very, very difficult to get an
ethical approval for this because like
some of these people had never worked.
What if they had a traumatic experience
or they became depressed afterwards?
You know, it'd be very difficult to get
ethical approval to run this kind of
experience or experiment.
But when you're working in artistic
context, it's a lot more flexible.
And also it allows for this kind of
horizontal collaboration that's
more egalitarian. You don't you're not
they're not your research subject.
They are creating the whole experience
with you in that process.
And then, yeah, we have this
this video, which, uh, kind of
a lot
of attention, like quite unintentionally,
uh, I should I should say,
um, we just posted the video and
it went viral.
It just happened when we thought the
project was actually about to end because
we were all so
we all working together in Barcelona
was with four of us at the time.
And we all went to a
different country to, um, to
make a living.
But in the meantime, we
we have a lot of people can start and
reaching out to us to, uh, after
seeing this video. And it kind of
served as an introduction to all the work
we had been doing before.
And yeah, it also happened through
the IDF and came out of the success.
And uh, uh, some people.
Yeah, we were in one of the open
because a lot of the stuff happened
during this open collapse.
And so one couple came to us.
Actually, there are two friends, I think
male and female and like, oh, we really
want to kind of explore each other's
bodies. Can you just give us the
equipment and we'll go off into the room
on our own? I'm not sure. Okay, no
problem. Then they came out afterwards
and they were like, wow, that was totally
amazing. Like, OK, so how can we.
Thank you.
So we thought, OK, how can we expand on
that so this setup is different than the
other one. Rather than having a user and
a performer, you have
two users. Basically, they're swapping
perspectives. And in order for the
embodiment effect to work, they have to
stay in sync. So that means that all
the actions they're doing have to be
agreed like kind of nonverbally.
And it creates like this interesting
dance of agency and consent, you know,
wherever you like, you know, I mean, you
know, whatever. The stereotypical thing
that you might think is like, oh, I
Bressan, I can go like this, but I mean,
like, if you do that and they don't want
to, you're not going to see it, you know,
you're just going to be an asshole.
So it's.
So the idea is like it just changed
the whole relationship then between
between the two people who are experiencing
it. And and yeah, that was that was
really interesting. So we got loads of
attention from this unexpectedly.
We didn't even promote it, just got found
out in somewhere and and then the
programs that we had to actually show it.
And insulations. Yeah.
So they invite us to do an installation.
I was like, we don't normally work in
installation, but we did, we started
developing stuff and we've been invited
to a bunch of different events.
So we went to some medical
hacking day thing and started then really
because lots of people from different
fields contacted us, like cognitive
science researchers, performing arts
people, people involved in conflict
resolution, all this stuff.
And so we started saying, well, actually,
this idea, this concept, this sort of
stuff has applications in lots of these
fields. How do we expand this?
So, you know, of course, it's open source
and stuff. We thought, well, let's just
just share it and try to go to all these
places where they're inviting us and see
what we can get.
So this is one of the experiments
that was some sort of
pre experiments.
We're trying to see if the design, the
system could be used in this way is
kind of like a replication with our
system of the virtual
environment, therapies for
burn victims where they found out and
it's actually using hospitals sometimes
that's for severe burn victims.
When you put them in the video game with
they like throw snowballs that penguins
and polar bears, it kind of distracts
them from the pain, like from pain.
That's chronic pain that they cannot get
rid of. And with this kind of distraction
work. So we we have been looking at
this kind of so we put
we measure the pain tolerance with
the like how long you you can stand
with your hand in a bucket of ice with
control, temperature
with and without the system.
So doing this kind of respiration
exercise, focusing on your body while
being another person,
it's in the semi
control.
Yeah. This result and environment
results seem to indicate that that was a
positive effect.
Yeah, it was in collaboration with two
different neuroscientists at different
universities. This is like a pilot
study. So in addition, we also went
of course, to some different art
exhibitions. This is in Eindhoven, that
art exhibition called Gender Blendr.
And then we realized that showing the art
in these contexts provides us an
opportunity to do to
do user testing in a way, you know, like
you're exposed to a much broader section
of the population than you would if
you're in research institution where
you're dealing with mostly 20 to 30 year
old white males with a technology
background. So if all of a sudden we're
being invited to all these places, we
thought, well, let's go and show it and
really be there and do it as kind of a
technical and anthropological research at
the same time.
So this is some workshop at a recent
conference in September in Berlin
where we had a workshop with people
showing them how to use it.
Yeah, and then there's a series of
pictures.
This is in Sao Paulo in Brazil as well.
This is in Spain, again, working with
dancers. This time there's lots of
different iterations of the project that
we've sort of gone through, but there's
more documentation online.
Yeah, this was interesting.
This is one of the people who contacted
us actually after the video went viral.
It works in the United Nations
and the Alliance of Civilizations, which
is a small kind of department that
funds and researches experimental
projects and conflict resolution.
So you'd actually have the same idea
using neuroscience embodiments,
virtual reality, storytelling performance
to address conflict specifically.
But he hadn't found anyone to work on it,
didn't have the skills himself.
So when he found us, he was like, wow.
And we're like, oh, cool. This is
amazing. So I yeah, join the team, you
know, like, it's, you know, it's open.
So since then, this was a project
that we we sort of participated in.
It was a retreat for a Somali diaspora
storytellers. So that would be
journalists and artists and poets and
writers and activists and
documentary makers looking at intertribal
conflict. So this is of course, it's a
tricky situation. You go there and you
don't want to be like, oh, we're the
white European guys with some technology
that's going to fix all your problems.
You know, you have to kind of go there
and say, all right, we just have the
stuff we've been working on.
What do you think of it? How would you
use it? What's the context?
How do we create a context together that
this might be useful? Maybe it's not
useful, but you need to try to have this
this dialog.
So I think that's really like
where we're at now.
We're you know, we've
been invited to MIT for a year to do some
research there as well.
We're also got invited to two events in
Israel. And so we decided, well, we're
going to Israel. Let's see if we can
connect those two events and stay in
Israel for a month. And then we're like,
well, if we're in Israel, we might as
well. Try to go to Palestine as well, OK?
There's no funding. How do we do that?
But let's run through with the ideas.
We're going to run two concurrent
simultaneous sort of workshop development
periods with members of the community,
activists, artists, hackers in Israel and
Palestine, and try to network these
people and interconnect the results
across the border as well, looking
everyone with the same common goal of
like how do we address conflict and just
see what comes out of that.
Maybe it's to do with the machine, maybe
it's not. But using it as a tool to
really build networks of people who work
and collaborate on a horizontal level and
find the tools for communicating
in an interdisciplinary manner.
It is really what we're what we're
interested in doing with
the future of the project.
So, yeah, we have we have,
like we're talking about before, open
open source platform.
We have this three
printed hardware that was designed for us
by Barcelona.
We always work with really low
budget technology.
So this is the jacket you're
seeing is a bunch of our three printed
pieces, uh, PlayStation cameras
are doing in the back the really
most expensive piece of hardware is the
computer in it, I guess, and and
the Oculus.
And we
invite all the quarters
in in the room to check out our GitHub
and reach out to us if they have
any idea how they would like to
contribute. There's little things
that we need help with, uh,
on the technical level as well.
And, um, yeah, I think
yeah, that's yeah.
I guess like also just thinking about
this sort of context.
And we we're working we're also
interested in other people who are maybe
working maybe with different projects,
but also in a disciplinary manner, like
how useful is it like to design stuff
like if you're working with like PGP
adoption or something like that, you
know, working directly with users and
stuff or people in the field to actually
create these tools and methods of usage.
Maybe there's people working on that
here. Maybe there's stuff that we've been
doing over the last few years which can
contribute to these kind of things in
terms of formalizing a methodology for
kind of collaborative development of
projects that are interdisciplinary.
Yeah, I think that's that's basically it.
So thank you very much.
Yeah, thank you, Christian and Atha, the
good news is we still have like 10
minutes for questions and answers, so
please line up at the microphones
here in the hallway.
We start with a question from the
Internet.
Oh, hi.
Um, the question from the Internet is
more related to,
um.
Yeah. How you can use the technology
also in other fields.
Like, would you say you could
tell people with body image issues
like anorexia or something to
recalibrate their perception
of their own body?
That's part of the thing.
We have been talking with some, uh,
neurologists that it's working.
Exactly. And that this
fucking medicine event in Madrid,
that was a possibility.
They could do that in virtual reality.
Right now, the way it's done is to
assess if someone has a bad, badly
assessed body image.
Is that just show a bunch of pictures to
a person.
And and
that on every picture they say, oh, yeah,
this is bigger than me and thinner than
me. Bigger than me.
But you could very easily
imagine doing that, uh, with with
someone else rather than giving the
agency and like that.
Really the strong thing about the
the machine is like we're using the real
world and the it's actually
I think so far it's, uh,
strangely my singing, it's it's more
convincing to use a video feed with,
like, uh, some movement with maybe
some approximation and some latency, then
just a virtual environment.
It's easy to imagine how it could be
better, um, quickly.
But then the flexibility of
of using just really simple technology
and the and the real world is, I think
allows us to to to explore very quickly
different things.
And that was, uh, something we,
uh and we possibly we actually
met with someone who found the project
and was saying, I've been on record for
20 something years and I would like to
come and meet you. So they actually
traveled to meet us at one of the events
we were doing. And we're able to try the
system and make some recommendations.
I mean, you know, we said, well, we're
not doctors.
We don't we can't promise you anything.
But like, no, no, I just want to go and
participate and see what I can
kind of offer in terms of that.
So there was like there has been some
initial kind of dialog about that and
working with people, you know, patients
and this kind of horizontal sort of
structure.
Do you know any medical papers on that?
Medical papers on anorexia and
virtual reality?
Yes, there could be.
I can't think off the top of my head, but
if you want to email us or whatever, we
can talk later as well.
All right. Thanks for the input.
Next question from microphone three.
Yeah, hi. Um, I've read that
there are sort of two different ways that
different individuals process
visual input and reconcile it with how
they're where they are in space.
And that one of those modes makes people
more prone to things like motion sickness
from virtual reality equipment.
And it happens that those things break
down across gender lines.
It's more common for women to be
motion sick from VR technologies.
And I was wondering if you have seen
much of that in your observations of your
project, if you have
many women involved working with you
regularly. I know you commented about
the typical white dudes in technology
problem.
So if you could speak to that, I would
find that really interesting.
Yeah. In terms of
we've not necessarily done
a quantifiable analysis of
everyone, every participant that we've
worked with. I've worked in this field
before as well. And yeah, you're right,
there's differences in how this is
perceived by people also across age as
well. I was working in VR rehabilitation
for people with stroke, which is mostly
elderly as well.
So, yeah, there's definitely a big
difference in terms of motion sickness.
I don't think we've observed any kind of
effect. But that's that's interesting.
I didn't know there was there was
happening.
Do you think that that could be related
to the fact that you're actually
coordinating with another person and so
that the timing maybe works out better?
Or is it just you
have no idea because that's a totally
valid and sometimes there's
some very technical stuff about why we
haven't had too many issues with motion
sickness. We actually cut off.
We're changing the orientation of the
camera so we have greater up and down
orientation and less side so that
there's there's less motion sickness
because it's not your entire field of
view that's that's covered in this case.
So but.
All right, thanks. Next question for
microphone two.
All right.
Yeah, this will be a question about a
setup we've seen in the slides.
Um, it
looked like there was just one camera,
essentially.
Did you get a proper stereoscopic view
enabling that perception
on the Oculus?
We have been working with a bunch of
cameras.
This setup we were talking about.
Yeah, this is Modisane.
Yeah, we is duplicate the signal.
But on one of the issues with using
stereoscopic cameras in an installation
setup is that you have to kind of
calibrate per user with enter pupillary
distance. So it makes it a lot simpler if
we just, you know, if we're getting
people in and out or whatever and it's
chaos, then, you know, it's easier.
We do have some that's like a mechanical
way of actually adjusting the cameras.
And then you can also do it with software
to.
But we don't generally use that in
whatever we do on this one,
for example, we had a bunch of TV
cameras. This one was trying to build our
own was a bit complicated.
This one we're not available are not
available anymore. But we have we've had
some. Yeah, it really makes the immersion
stronger, of course, especially
in close quarters.
And you can put your hand and your body
in this like sort of less of a difference
in like after a couple of meters.
But yeah, it's actually important.
Definitely.
Thanks.
OK, it seems to me that there's another
question from the Internet.
Yes. Yes.
There's so how well do you think
the immersion controllable by the user,
what's like the risk used for
this technology, like for torturing or
anything in this direction?
I think there is.
That's one of the main reasons that we're
interested in kind of taking
the project ourselves to these different
contexts. Like we've been to ARC things
have been to documentary.
We're now at MIT, which doesn't have a
very ethical record and a lot of ways.
And it's like it's interesting to go to
this context and kind of sort of
bring the ideology that's kind of brought
the made the project what it is so far
into these contexts and see how we can
sort of keep that going.
I think there's obviously uses for
torture with this or whatever, but I
mean, we're not so worried about
that. I think the cooption that could
happen is that, you know, you have
this like as
just an immersive experience that you
have on your own. Really, what we're
focusing on here is bringing creating
a context that brings you back towards
dealing with someone. In reality, you
take off the the
gear and then the experience
is really then the meaningful part
happens in that in that way.
That's really what we're looking at, is
creating a kind of a social context
where this is just the starting point.
All right. Next question for microphone
three.
How do you measure increased empathy
and what kind of methodological issues
have you seen arising with that
food?
Yeah, we haven't actually started
quantifiable empathy is.
Yeah, that's not something that we've
been doing so much at the moment.
We've been working at
setting up some different collaborations
with some other universities, doing
comparative studies with
this and other things regarding virtual
reality. And I think now the
environments we're working in are quite
chaotic. So having, you know,
measurable signs of it doesn't really
work. So there's methodologies that we
can use. Some of them don't necessarily
work, but some of the things we try here
I think are useful to bring back into
maybe a more clinical environment and
investigate there. That's one of the
things we're going to be doing while
we're in at MIT, for instance, is trying
to make a more rigorous set up of
some of the stuff we've tried in the
field.
And the question from the Internet as to
what it's actually like, one would know
how long the longest experiment
with to people thinking was.
And the other is whether there will be a
demo or workshop that's
used the next day.
Actually, a good question, because we
forgot to mention we're are running a
workshop here, but we haven't
been able to book it because the wiki was
down. So if you want to check our
Twitter, then we can we
will have that organized at some point.
I'm thinking tomorrow or at some point.
But if you see us around and there's
people who want to hack and you have a
space already at your desk, we have all
the gear here already. So just let us
know when we can get stuff going.
In terms of the other question, which was
I forgot in the first
part of the question is there I can get
the question first part, how long the
sinking of to people
lasted?
Yeah, I think the longest one we did was
we kept one of our research,
one of our team members in there for two
hours, I think.
But it wasn't with one other person in
this case. It was lots of other people
swapping with him and he didn't take it
off the whole time. So he was quite
disoriented by the end of it.
All right. Since you mentioned your
Twitter account, maybe you want to go
back to the last slide so people can note
it down. And I think this would be the
very last question for microphone two.
OK, thanks.
Um, my question is about you
said this is more like an art project
of yours, but you already mentioned
some university.
So are you aware of some
scientific research
already in the making?
Right. I was just set up or are
there already papers?
Maybe we have some people
actually explain then like how to
replicate.
It's inspired by, like
kind of research by Erson Lab in our own,
but it's still in the mix in the making.
There's other research that's similar and
some ways, like we mentioned, some of it
there in terms of using our set up
specifically, we the first one
that we're working with is at MIT.
There's other universities that we can't
really mention yet because it's not
official. So but we're
already in talks with researchers within
there who are seeking funding for
having us there and developing the
project and using it as a common
platform, maybe for sharing research
between institutions as well, since it's
open source or whatever.
And if people are making some advancement
with the interface or with the technology
and one thing, it can be fed to the
broader open source community.
That's one of our motivations for working
with these large technical and scientific
universities as well.
All right. Thank you so much also for
taking the time for Q&A.
And thank you for this.
Interesting talk. Thank you so much for
being here. Thanks.