Dear viewer, these subtitles were generated by a machine via the service Trint and therefore are (very) buggy. If you are capable, please help us to create good quality subtitles: https://c3subtitles.de/talk/376 Thanks!
Um, yes, so today my talk
is about, uh, hacking ethics in education
and, um, I
sort of on purpose, made the
title a little bit, um,
uh, have different meanings.
So I'll be talking about hacking and
education first.
So, um, a little bit of background, why
we do hacking, um, and
what kind of hacking we do.
Um, then I'll be talking about
hacking ethics in education, how
to um how people feel
about the hacking, um
and how they work on hacking.
And then I'm talking about hacking
ethics into education.
Um and I'll be talking about the
different viewpoints, we'll be talking
about the viewpoints from the students,
uh, the teachers and
most importantly, the administration of
the university.
Um.
The hacking and education we have a
system in network engineering master is,
uh, where I'm a teacher, um,
and this started at the University of
Amsterdam in 2003.
So we have been doing this for 11
years.
Um, there are
so this deals with various subjects
of a system and network engineering.
Um, we have networking experts,
but we also, um, educate
forensics experts.
Um, so we
have various security related, uh,
courses too, that, uh, throughout the
year, um,
and there are many different
research projects which are very
intensive research projects where the
students work on a single topic for one
month. Um, and they really
delve deep into, uh, into
a particular subject and
some examples of what they do within
a month or six weeks.
If this is during the course, um,
one of the first acts of the of a chip
cards, uh, the Dutch public transport
chip card. And that, um, happens
by students, by our education.
Um, we
did some hack on Tinder, which
I may talk about if I have some time.
We have found vulnerabilities
in mobile banking apps, um,
that students worked on.
And, uh, also the
the Dutch passport tip,
um, was cloned by a student of ours.
Um, now if we talk about
hacking ethics in education, um,
what students think they do when they're
when they're hacking, um, they
feel they're very powerful and they can
hack anything.
Um, they want to break stuff and they
think they have the knowledge
and they are really into
hacking and they want to
scrutinize the security of products.
And their first aim is to break
stuff.
They don't want to see if something is
secure or not. They want to break stuff.
Then if you look at what the teachers
think they do
is they're shooting these, um,
most of them are not very convinced yet.
And most of these things are reasonably
secure and it's well thought out.
Um.
It's interesting that the students learn
about, uh, looking at the security
of different products,
um, and it's good that they start
shooting at it and see how that
feels like.
Um, but most of the things aren't
really effective.
That doesn't mean.
So the security of the hacking itself
may not be effective, but the experience
for the students is very effective.
Um,
uh, what the administration
thinks that they do is, is different.
They're very scared.
Um, they think that these students are
all very elite hackers and
are breaking stuff and doing this in very
nasty ways that the teachers
don't see.
Now, the problem with this
is, um,
even if you're shooting these, sometimes
you shoot at the right spot and you
break it off.
This is what happened with the mobile
banking security. We thought, well, we
can let the students look at mobile
banking security and they probably
have their have their act together.
Um, and, uh,
it was even audited.
Um, the mobile banking app
was audited by, uh, very
professional company.
But, um, the students looked at
this for a day or two and they they
it was completely broken.
Um,
um, they could, uh, they scared
the banking official by showing
him his his, um, pin
number when they did the demonstration.
Um, but even
even if you're not breaking stuff, if
you're shooting at somebody, uh,
they can get very annoyed,
um, even if you're not
really breaking stuff.
Um, and then if they get
annoyed and then, um,
very often the old boys network kicks
in, um, and
the staff, the upper staff
of the company contacts the upper
staff of the administration and
they panic and freak out.
And um
and that's when, um, we
started doing the hacking ethics in
education because what.
So the hacking ethics in education.
Um, because we wanted to see
that uh we wanted to show the
administration that they they really
are ethical hackers, that they're
thinking about what they're doing.
So, um, the administration, they
freaked out and they did what they always
do when when university administration
freaks out, they formed a committee,
um, they formed an ethical
committee, uh, for the computer science
faculty.
And this was for the whole faculty,
um, uh, and it
included all nonstandard research.
So the standard research that
each of the, uh, research groups
do that doesn't have to go through the
whole ethical, um, evaluation
procedure.
But if you do anything out of the
ordinary, then you have to, uh, um,
or you should be doing something
with the ethical committee.
Um, the problem a little bit with
this ethical committee was that it's
formed by, uh, group leaders
of the computer science faculty.
Um, and the problem with group leaders is
that they have a very busy agenda.
So the Ethics
Committee doesn't meet very often and
they don't have a lot of time to review
projects. And, um, like
I mentioned earlier in my talk,
we have a one year program and it's a
very intensive program.
And we don't have a lot of time if
students are doing projects to review
those projects, especially not
for, um, uh,
talking about, um, the the
review time in in a matter
of months, that doesn't work with
our education.
Um, and finally, there was also a lawyer
on the Ethics Committee, um,
but there in this Ethics Committee, there
is no ethical um, there's no ethicist,
um, which is a bit strange, I think,
because, uh, I wonder how
this makes it an ethical committee, but
they think that this is an ethical
committee. Um,
um, we so we found
we had to deal with this because this was
the ethical committee for our computer
science faculty.
Uh, and they were imposing us that,
uh, all of our project had to go through
them. And then we said, well,
um, maybe we can hack the process.
Um, uh, and
we form a committee ourselves.
Um, so we formed an ethical committee
for the Master Education,
um, and, uh, we wrote a
procedure how to make
the students think about, uh, the ethical
aspects of their security research.
Um, when they do a
research project or a security project,
the first thing they have to do is they
write the proposal, what they were
planning to do.
Um, this is so that we know,
um, we have some idea that they
know what they're doing, um, that
we think that this is a valuable project,
um, that they that this is interesting
enough for the course that they can get
a passing grade at the end of the course.
Um, but we changed this so that they
also now have to write an
ethical paragraph into that project plan.
So, um,
at the end of the project plan, they have
to say, what are the the
ethical impacts, um,
of this project?
What is the, uh, who is going to get
harmed if you if you actually break
stuff?
Um, uh, are you impacting
on the privacy of user data?
Are you getting access to user data
and these kinds of things they have to
write down and they have to
defend this?
Uh, to the teacher,
then the teacher of the course
with the Ethical Committee of the
Master Education is a very small,
very small ethical committee.
Um, and they meet, uh,
and they evaluate all
of the projects that the students do,
uh, handed in.
And we, uh, first of all, we check
whether the and the ethical paragraphs
that they handed in is actually complete,
that they identified all the major issues
that they identified, um, uh,
um, all of the things that could go
wrong, all the actors that may get
involved.
And then, um,
uh.
And then we give this a traffic
light status, we go
from green to red,
green means that there is basically no
ethical considerations involved.
This is an evaluation of a
security tool or an offline analysis
of something where
they don't get into contact with any user
data, that it doesn't have
an impact on our user privacy,
etc..
Um, yellow is if they may get
into contact with, uh, um,
with personal data, but
in a very confined way or,
um, we don't think that they
will break stuff.
And if they do, it
will not have a very high impact on
society.
Um, orange is where things get,
uh, a little bit serious, where they
have access to a lot of they
possibly have access to a lot of user
data privacy impacting
data.
Or if they actually manage to break
things, then
this is going to have a very high
security impact.
Um, and red is
something that we, um, it's
a status that we use for projects
that that actually cross the line but
may be important
to do anyway.
Um, or
not.
I mean, sometimes you have product
and you're going to cross
an ethical line, but it may
be that the problem is so important
or so politically relevant
that you think that this is a project
that has to be done anyway.
But then we agreed with the
Ethical Committee for the Faculty
that any project that is going to be
read, uh, we're going to report to them
before we start doing the project, then
we get permission to do this.
Um, and this
has worked very well over the last
year.
Um, from the
student's standpoint,
we've looked we've now, uh, forced
the students to think about ethics
when they're doing the research.
And this has led to very valuable
insights for them.
And it's been a very good experience for
them also.
And it also means that we force them to
to think about the ethics before they
start doing the research.
And, um, they they,
um, do an ethics by design
in their research plan
and how they set up, um,
experiments and they think about privacy
when they're setting up experiments with
user data.
Um.
And this is something that wasn't
so we had some ethical courses
and ethics classes during
the year, but then it was mostly abstract
and we we started the discussions.
Can you take a computer home with you?
If you if it's an old computer
at your company, can you can
you take it? Can you look into user's
email boxes and you do the discussion,
but it's still a little bit at an
abstract level that doesn't really
touch the students at that moment.
And then it's hard to see if you
really made an impression about the
ethics of of that
of that issue.
But now this is
by incorporating it completely into the
research, they're forced to think about
this. And this impacts them in a very
profound way because they have to think
about their own research that they're
doing right now and
have to think about this and how do I do
this in the right way and what is
the right way?
Um.
Of course, they're still students, so
we work with them to
help them see these kind of issues.
And even like
I mentioned, the traffic lights
status also means
that the orange projects get more
supervision by the ethics adviser,
that they get a
little bit closer interaction, that
we monitor what they do and,
um, that we take care that they
don't cross the line.
Um, finally, at the end
of the of the line, if they do
find issues, we do
do a coordinated disclosure
or responsible disclosure, as it's also
called.
Um, and this is done by the teachers
of the education.
Um, this is a conscious choice because
we find it coordinated or responsible.
Disclosure is not really accepted
that much yet.
It's getting better, but it's not there
yet and it usually takes a very long
time.
Um.
We've, uh, we had some projects
before the summer, and it took more
than two months to finish all of the,
uh, coordinated disclosure project, and
this is just time that the students don't
have or if they're doing a project
at the end of the of the year,
they're basically gone.
And we have to we have to make sure
that the procedure is still, uh,
finished.
Um,
I have some time for the extra slides.
Um, like I said,
uh, Tinder, um,
this was a very interesting research.
And we we showed very
much how the, um, the
privacy by design and ethics by design
worked very well in this research.
Um, little bit of
background, um, in
July 2013.
I mean, everybody knows Tinder, right.
Who doesn't know Tina?
OK, Tinder is
a dating app.
And it's a very shallow dating app.
This is the interface, and
the way you manage the interface
is you swipe either left or right,
saying you like this person or you don't.
And if you both like each other, then you
get into contact with each other and then
you could start chatting and, um,
uh, and maybe go on a date or something.
Um,
the power of Tinder is that it uses tips
to find matches near you.
It doesn't really make sense if you get a
match that's in in America
and the chance that you actually meet is
very slim.
Um, so they have to have the GPS
coordinates, two of the users
in order to find matches near
you.
The problem was that in July 2013,
somebody looked at the traffic that they
was sending and they found
that Tinder was actually sending the GPS
coordinates of other users to
your phone in
order to measure the distance of between
the two users.
Um, this is,
of course, a little bit of a privacy
problem.
So they fix this.
And then in February to February 2014,
somebody started looking at this again
and they, uh, they fix this by sending
the exact distance to
the person.
Um, but the problem with this is that the
API that Tinder exposes
is actually open.
You can just use, uh, write the scripts,
query the API.
Um, so you take three different points
and then you do try illustration
and then you still find the exact
location of the user.
So this is this was
all external research.
And then in May 2014,
our students started looking into this
because the the the way the Tinder
tried to fix this is because
they implemented a, uh,
rounding. So you get, um,
if you if the matches are
very close to you, they say that
this is this user is when within one
kilometer.
But the API actually exposes,
uh, lower boundary, uh,
that you can set very much lower
than one kilometer.
And if you do so, you get
different matches.
So they started looking into this and it
turns out that they may say
that this is within one kilometer, but
because you're using
the lower lower bound,
it's actually within one 500
meters or within 300
within three hundred meters
or something like that. So you do
a little bit more querying, but you still
find the user, the user location.
But then how they research
this is they did privacy by design
in their research plan.
And at first they started
I mean, the naive approach is just to
match everything, store everything, and
then start comparing the results.
And we didn't want to do that because we
wanted to preserve
the privacy of the users.
But they use the test user,
but.
They were at the university and at the
university, students are very,
very enthusiastic users of Tinder.
So it was very hard to find their own
test user.
There were too many users.
So what we
ended up doing and, uh, I work with
the students to think about the problem
and think about how to solve this
in a privacy friendly way
and finally resolved by doing this,
that we only stored the hashed idea of
users that
they found in their location and no other
result.
And in this way, you can find
you can find the matches and you can find
you can verify results
without impeding on
the privacy of the users.
So this was a very good example of doing
the privacy by design and
the ethics by design in a in
a security research.
And that is actually what I wanted to
say and what I wanted
to tell you about with our ethical
committee.
Thank you very, very much for this
interesting talk.
Do we have some questions?
So please line up at the
microphones now.
We have some time for a Q&A
microphone to please.
Did your institution, I guess in this
case school, did they see any value
in what you were doing or do they only
perceive you to be a threat?
It took a while, so we started this
was last year and it took
a while for them to see the value of this
approach.
But now that I'm very that
I was enthusiastic and that we get
actually very good results, they see
some value in this.
But it took a very long time.
Yes.
Thank you.
Microphone three, please.
Do you have any tips for
organizations that are refreshing their
security curriculum to take
into account?
I would take I would take
if you're refreshing your curriculum
and you're doing project, then I
would very much recommend taking
this procedure into account.
That you start with the students
with ethics in their project
immediately.
There is some question from the I.R.S..
Yeah, I have actually
to sort of catch up.
Both Yeah.
OK, so the
process of ethics is natural.
To what extent does the ethical
come into conflict, the norms or values
on the students?
And the second question is,
to what extent can students
act even if their norms and values
are not in line with the common T..
OK, um.
Um, we try not to impose.
Sorry, and we try not to impose
too much of our values into the students,
but we are I mean, the
university is a very public organization
and has its reputation to think about
also.
Um, so we have to be careful
with that. But we do,
um, we do do a very open
uh, we have a very open debate about
the norms
and the evaluations
that the students do.
And this is in a discussion,
um, so we try not to impose this.
And the second question was.
Um, uh,
to what extent can students act, even
the norms and values are not in line with
the committee,
um,
in the end, the Ethics Committee
decides and then the teacher
decides what what we can do.
And if that doesn't align and
we don't agree that this is an
important subject to do,
then the students are not allowed to do
it during the curriculum, is free to do
it in his own time.
But it won't be, uh, great work
at the microphone to please.
Um, are you considering
an ethical code of conduct?
A student could, uh, take an oath
when they graduate from the university.
Similar thing is happening in the
University of Groningen, the physics
department, that a student can take an
oath. They don't work in their arms race
and history and work on.
Uh, well, you know what?
Um, something similar.
I think if I sent some code of ethics.
Yeah. And if they,
um, I
forgot the name. There's a code of
ethics for a system administrators.
Um, and we we try to,
uh, we educate them about that one.
Um, but we don't
we don't enforce this on onto the
students after the curriculum.
No, I'm not saying I'm forcing no.
Indian groaning And this is just an
optional thing you can do when you take
your ball.
Go, go.
OK, no,
we haven't really thought about it.
It's something that we present and that
we present as a good thing,
because it also helps you a little
bit with, uh, if
you were forced to make hard choices,
then you can point to this public
document and say,
this is an oath that I abide by.
Um, and
this is not something that I thought of,
but this is something that that exists
and that other people have thought about.
Yeah. Um,
so that's what we do.
Yeah.
I think you might
make number
three.
I first let me say, uh, it's really great
that you're doing this.
And I was at the University of Amsterdam
also ages ago, and we had some
extracurricular classes on ethics, on the
faculty of computer sciences.
Um, but what I'm questioning
is why don't you,
uh, expand that? The whole ethics
question also into the design phase
is this is just security research,
but there's also ethics questions in
picking libraries or making
design decisions on networking equipment
or software design.
Um, I think it would be great if it
could be expanded.
A lot of flaws that we seen presented
here on the CCC come from poor design
decisions or because people think, well,
I can get away with it.
And that's, um, it's
something that we've been
subconsciously working already with.
Um,
uh, but it's not something that we
explicitly have been doing, but
maybe something for the future
then,
uh, it's working again.
So there are some more questions from the
Internet.
What one, um, that
the lawyer even into win or do
anything like.
Did you understand the problem?
He was presented with,
um, the the lawyer and the
ethical committee for the faculty
is actually very constructive.
And, uh,
she helps with the, um,
discussing the law and what the
boundaries of the law are.
Um,
I'm not completely sure whether she
actually stopped anything
yet, but it is sometimes
helpful to understand what the what the
actual boundaries are
and what you have to think about
at microphone to please.
You talk about the the traffic light
was red, green, yellow.
Um, the parameters
you have to classify which one
is red and yellow and green.
Do you have that public somewhere?
Can I have a look at that because.
Yeah, sounds interesting.
And so this is still
very much a work in progress,
but the whole procedure is online at, uh,
the website.
If you click through there, the info and
then ethics and you see the, uh, the,
um, the evaluation procedure
and some of the examples that we have
for each of the different
classes. Yeah.
So as I can see, no more questions.
Thanks a lot. Again, give
them the applause.