Dear viewer, these subtitles were generated by a machine via the service Trint and therefore are (very) buggy. If you are capable, please help us to create good quality subtitles: https://c3subtitles.de/talk/579 Thanks!
Hi, everyone, thanks for coming.
And it's so good to see familiar faces
again at this time of the year.
And today I'm going I'm
going to do our annual tradition
on stylometry talk.
And I'm from Princeton University.
My name is Eileen as he introduced
me. And I'm currently a post-doctoral
research associate.
And we have been presenting
at CCC for a few years now.
And they have been mostly about
stylometry. But the original
on the first day she gave the alternative
keynote, then it was Nunc telemetric
this time. And I'm going to keep the
tradition alive and talk about stylometry
and machine learning today.
So what happened since last year?
Last year I talked about the anonymizing
program, which for about like 15 minutes
this year, Giana, the anonymization
just became easier.
And that's kind of equivalent to the fact
that there are more privacy
concerns, concerns for programmers
now and also open source software
developers.
And today we're
going to talk about stylometry and
machine learning. But at the same time,
we released our most current paper on
this. It's on archiving on my
website. And if you want
to read a summary of this talk or the
paper, you can also check our blog,
Freedom to Think.
Let's start talking about stylistic
fingerprints.
Why not, if you're not familiar with
stylometry, I'll give you a brief
introduction.
Stylometry is the study of individual
style. Most of the time it has been
researched in writing style,
but we can see stylometry in fine
arts. For example, artists can be
identified by their brushstrokes
and in music, musicians
can be identified by
the by the tones or rhythms that they're
using. And three years
ago, we presented that stylometry
is also present in unconventional
text and by unconventional
text I mean underground forums
where sometimes cyber criminals or
a variety of people engage
in. We can identify
them as well.
And we have looked at translated text
to see if translations can
anonymize you.
And we saw that even when you take
your English writing translated
to German, then to Japanese, then
back to English, we can still identify
you. And that sounds like a serious
concern for someone who would like to
remain anonymous.
And now we started investigating
Source Scott, because if you are going to
investigate style in language,
we can think of source code as another
type of language.
It's a programing language.
And today I will give you the
improvements in our source code,
authorship, attribution method.
And at the end of this talk, we will see
that style that's expressed in code can
be quantified and characterized.
And that's kind of the answer to our
research question.
So what happens with supervised
stylometry, I say supervised stylometry
because I'm going to talk about machine
learning today, we can identify
style in some type of
personal data or writing
by using machine learning methods.
And I will give you a very common
setting.
Let's say that you have a set of
documents with unknown authors
and some with no notice, and
you would like to find out who these
anonymous documents belong to.
So what you do is you take a machine
learning classifier and then
you train it based on the documents
whose authorship is known.
And then you create a model for everyone,
for the people with
known authorship documents.
And after that, you can use your machine
learning classifier to test and see
who this anonymous document was written
by. And let's think about the common
scenario for this.
There's Alisdair Anonymous blogger and
Bob, the abusive employer.
So Alice is blogging about abuses
in Bob's company.
And Bob, as he is abusive, he's
going to go ahead and collect everyone
in his company's writing and
then he's going to train a classifier so
that he can identify who this anonymous
blogger, Alexis and Bob,
can do this by using stylometry and
machine learning.
I will give some other motivating
or scary
examples. For example, there was this
case with a person called
the Koner or hurt her or his
user name was the Konner on Twitter.
And she tweeted that Siska just
offered me a job, not I have to weigh the
utility of a further paycheck against
the daily commute to San Jose and heating
the work. And then to Mileva,
who is the channel partner advocate for
Cisco alert. He saw this and
that wasn't very good because you
can then identify who the Connery's
and her job offer
might be in danger at that moment.
So what if Cisco takes all
the cover letters that were submitted to
Cisco and after that train a classifier
and try to find who the corner is by
looking at her tweets?
Because you can also identify people from
their tweets?
But that wasn't necessary in this case
because since you can find some
cached information online, she
was identified as
Connor Riley and unfortunately,
she lost the job offer after this.
So this is one example where this
might have been applied.
So you need to understand, like when we
are talking about machine learning
methods that make us make
it possible to anonymize people, there
might be some dangers associated
with this, or you might want to be
more aware of how you're
sharing your information online, keeping
in mind that you can always be re
identified.
And what happens with sars-cov-2, for
example, there was this recent tweet
and it says, I just heard from an
Internet couple that they disallow her
from contributing to open source on her
own time. That's illegal, right?
It's probably illegal, but Apple
can probably find out if someone
is contributing to open source code
repositories by looking at the code that
they have at Apple and then just compare
any suspicious code to maybe we identify
who this contributor is.
And because of that, we're going to talk
about the anonymizing programmers with
code stylometry today.
This has been joint work with my great
collaborators and some of them are here
with us today.
And why do we want to do source code
stylometry? Like how can we first start
doing this? First of all, we know that
any language source code as a programing
language is learned on an individual
basis and as a result, you
develop a unique coding style and
that can potentially make you
identifiable.
So if you want to investigate, if that's
really possible, do we leave any
fingerprints in source code that might
make us identifiable?
And why else would we do it as like maybe
we want to gain some software engineering
insights. For example, we want to analyze
how coding style changes over the years
or the differences between coding
styles of more advanced
and less advanced programmers.
Or does your coding style change when
you're trying to implement more
sophisticated functionality?
And the main goal,
or like the most motivating goal here,
would be to identify malicious
programmers who are maybe trying to
contribute malicious code or like back
doors to open source software.
And let's think about a common scenario
analysis, analyzing a library.
And there is malicious code in the
library. And Bob has a source code
collection and he knows
the authors. So Bob is going to search
his collection machine learning to
find out who Alice's adversary is.
And a second scenario about plagiarism.
So you're a college student and
you get an extension to your programing
assignment. And Bob, your professor
wants to know if you have plagiarized or
not. So he's going to train a classifier
on all the submissions
by the other students and then he can
check if there are extreme similarities
between coding styles similar
to that doesn't really belong to your
former coding style.
Unfortunately, these two examples for a
kind of security infringing
sorry, security enhancing
examples, but source code stylometry
could also be very privacy
infringing. So you have to be very
careful when you want to use that.
For example, or maybe some
of you remember him from last
year's top.
He was sentenced to that because he was
identified as the website programmer
of a porn site.
And unfortunately, the Iranian government
found about this and he was sentenced to
that. But he managed to get out of this
entire thing because, like, he was also a
resident of another country.
But this is a dangerous case where
in like oppressive regimes, your court
might put you in a dangerous situation.
So I'll start talking about a little more
technical stuff and show
you how our work improves
the state of the art and bring some novel
contributions, first
of all, this is a list of
comparison to related work.
And there is not much related work, as
you can see.
And the only difference in
the previous features used to represent
coding style and ours is syntactic
features. So we use structural features
to represent your coding style.
We don't just use your function.
Names are variable names or spaces
and types that you use.
And as a classifier, we use a random
force and I'm going to show you why these
are making very big differences
in the results.
So in the past, the highest accuracy that
has been reached to the anonymise
programmers was 97 percent.
And this is today.
Anonymise talk to programmers correctly,
but we can anonymize
two hundred and fifty programmers with 98
percent accuracy.
So we are beating the highest accuracy
in the past. And we have a much more
difficult machine learning problem
because we have a much larger data set
of hundred and fifty programmers.
The largest dataset that has been used
in the past is four to six programmers
and they get seventy five percent
accuracy.
But after last year's talk, we were
able to scale our approach to
1600 programmers
and we get 94 percent accuracy
incorrectly identifying 14500
source code samples of these 1600
programmers. So this is large scale
authorship attribution.
This is large scale, the anonymization.
How do we do that? So we have our general
machine learning set up, first of all, we
need data with ground truth.
So we go to Google Kojm.
It's an international annual programing
competition.
And we collected a data set and it was in
C++ because that was the most commonly
used language in this competition.
And we had about 100000 users
from different years.
We have our data set.
So now what we have to do is we
need to find the features
and properties that are going to
represent the coding styles of these
people.
So we preprocess the data, set
the source code, and
we get the abstract syntax
tree of the source code with using
a fuzzy S.T.
parser.
And after that we extract the features
that represent coding style and then
repeat these properties that are going
to represent individual style in
a random first machine learning
classifier.
And then we do our classification with
major to all of these like 300
random forestry's.
Why did we use Foja, so we have data
from 2008 to 2014.
Now we actually have 2015 status.
And the most important
point was that everyone in this
competition is implementing
the solution to the same programing
tasks.
So they're implementing the same
algorithmic functionality.
And the only thing or the most prevalent
thing that can differentiate
the source code samples are the
right or the coding styles of
these programmers.
And at the same time, they have to
implement this functionality
in a very limited time, which means that
they don't have a chance to go back to
the code, improve it, make
it nicer, and then copy paste some
stuff from stack overflow.
And as
contestants are able to complete
RANTZ, the problems gets harder.
So we kind of have a control
of when someone is implementing
a more sophisticated functionality
and as a result we can infer
which programmers are maybe more
advanced.
And as I said, C++ was the most common
language. So we decided to go ahead with
C++ in our experiments.
How do we do this? How to represent
personal coding style?
First of all, we have the source code
sample here. We see this is a just
five lines of code.
And from that we can look at
lexical features, for example,
integer and and is a lexical
feature because you chose it as
NP and then BA and full
function names, you chose those.
So those are lexical features that come
from personal input.
And then we have layout features like
the spaces and types and where you
put the curly brackets and things like
that. But the important thing
that was able to represent
coding style in a very strong
manner in our experiments was that
we use structural features
and we can do that by converting the
source code sample to an abstract
syntax tree. And then you get the grammar
and structure of the source code
and here you can extract
a rich set of features.
And these are also more
difficult to change as opposed to just
changing foo and
because these are kind of embedded.
So we extract features from the
source code. We extract features such as
edges, nodes, term
frequency, inverse document frequency
or the average depth of a
like statement node, for example.
And then we built our feature set
to represent programing style
and write it be user random.
For first of all, random forests by
nature are multiclass
classifiers, for example, as opposed
to common support vector
machine, which is a two
class classifier.
Random Forest is more
successful in classifying
many classes as opposed to support vector
machine, classifying just two
classes, making a binary classification
problem and also random.
For since they use decision trees and
information gain during the
training process, they avoid overfitting.
So we want to make sure that we are not
overfitting to a bias in the dataset
or to someone's very peculiar
property.
And what we do is we get our data set.
We extract all the features that we do
called cross-validation, which means
that, for example, for each programmer
we have nine source code samples.
We train on eight source code samples
from each programmer, and then we try
to test the ninth one from all of
them and see who it was
written by.
And then we validate our method on a
different dataset to see, like if
the features that we obtained are really
making sense.
Let's talk about the general cases, and
here I will talk about
how we were able to improve the method.
For example, there is this general case.
Who is this anonymous programmer?
This is programmer authorship attribution
program or the anonymization.
And maybe this can be applied to Satoshi
Nakamoto, who was the founder of Bitcoin,
and we don't really know who is.
What happens here, so we have 1600
programmers and each have
nine code samples and
we do a ninefold cross-validation and
we extract features from their core
samples.
And once we train our classifier
and test on the fourteen thousand four
hundred samples, we get 94 percent
accuracy.
How can we do this with Satoshi?
So if we had a suspect set for Satoshi,
what we do is we take the suspect
sets previous code
samples.
We train a classifier on that.
And after that, as test data,
we take Bitcoin's initial
get commit the first original
Bitcoin code and then we try
to see who this was written by
in the suspect's set.
And many people ask us, so who is
Satoshi?
And the thing is, we have a suspect set,
but unfortunately, the main suspect
in our set doesn't have any formal
code samples.
So we are just leaving this life like
this.
What happens if someone tries to
obfuscate code, so why do people
obfuscate what they would do that
to make their code become unrecognizable?
Maybe they're plagiarizing, maybe it's
malicious code, or maybe they're trying
to be anonymous. But we are going to show
that this is not going to make them
anonymous. So our authorship
attribution technique is impervious
to off the shelf source code.
Obfuscatory.
And this is one example.
This is a commercial off the shelf
obfuscated called Senex.
And what it does
is like it's available.
You can go online, buy it and like it
works for many languages.
What it does is if you look
here, it will take all the lexical
features, like the function names,
variable names and all the comments and
also all the spaces and refactor
them. But it's not going to make
any difference in
the structure of the program.
So like all the spaces are wrapped,
everything is refactored,
characters are refactored
with hexadecimal ASCII
representations.
The same goes true for the numbers that
are here.
But since the structure of
the program remains unchanged,
we still get the same
accuracy in the anonymizing programmer
programmers.
Once we obfuscate them with such
a of the common off the shelf
obfuscatory, which is not changing the
structure of the program.
And the example here is like we use 20
C++ programmers obfuscated their
code. You are able to anonymize them with
99 percent accuracy with their original
code, and we are able to anonymize them
with 99 percent accuracy from their
obfuscated code because we have
the structural features which are
very powerful, but
happens if we try to use a more
sophisticated, obfuscatory here.
Our example would be Tigress.
It's a visualizer and it
enables you to apply many various
kinds of obfuscation methods.
So we have code.
It's like 14 lines and we obfuscated.
It becomes like 800 lines.
It's completely unreadable.
So I don't think like if you're open
source software developer, I'm
not sure the
people in your project would be happy
if you contribute code like this.
But this works better at anonymizing
your coding style.
So we took C programmers
for this experiment and then again,
we had 20 programmers.
We are able to do anonymize them with 96
percent accuracy. But when we
obfuscated with Tigress, changing the
structure of the program as well, but the
functionality remains the same.
We get 67 percent accuracy.
So there's almost a 30 percent drop
in accuracy. But compared
to the random chance, which is five
percent, that's like the five
percent chance that you can correctly
identify someone just randomly
six to seven percent is a very high
number, which shows that your code is
certainly not completely anonymized
once you apply this obfuscatory.
So it kind of gives the answer that
obfuscation is not the solution to
anonymization in source code,
but advanced coding style throughout
years. So if you want to see coding,
style is consistent because if that's
the case, we can take someone's code
from 10 years ago and then try to test
on code from this year.
And for this, we took 25 letters from
2012 trying to classify
them. And then our test data came from
2014 and we were able
to correctly identified this these
with 96 percent accuracy.
And if we were trying to do this with in
2014, the accuracy was 98
percent. So there's a two percent
change in the accuracy, which shows that
coding style is somehow still persistent
throughout years.
And if you want to generalize our
approach and we wanted to do this very
quickly because we would like to see
how feasible this is and like, is this
a general approach that can be applied to
other programing languages?
And how easy would it be for someone else
that wants to use our code just for a
different programing language?
And for this, we only use structural
features and we use the E
generator that comes with
Python to generate the highest use
of Python source code.
And we were able to do anonymize 229
program, which just from this
abstract, abstract syntax,
three features and structural features
with 54 percent accuracy.
And if you do top five relax
classification, which means that you
do classification, I would classify
you would return your probability saying
that, OK, this is the most probable
person and this is the second most
probable person.
And then if you look at the first five
probabilities, if the program is
correct, programmer is within
that set, then that's considered correct.
And in that case, we are able to
increase the accuracy to 76 percent.
And if you do this for 23 programmers, we
get 88 percent accuracy.
And with top five relax classification,
we get close to 100 percent accuracy.
And whatever you do, relax
classification. Let's say that you have a
huge dataset and maybe you want
to you're willing to do some manual
analysis. But first, you would like to
start with reducing your suspect
size so you can just relax
classification and then maybe look at the
top 10 manually to understand
it better and.
In our results, we see that we are
bringing a new principal method with a
robust syntactic feature
set for the anonymizing programmers,
and this shows that there is
a serious concern for anonymity
when you're trying to be open source
software developer or just a programmer,
because we will soon talk about
executable binaries.
Where there is no source code and
for future work, we are planning to look
at multiple authorship detection, for
example, get repositories, can we find
multiple alters and can
we identify which part was exactly
written by whom?
And we would also like to look into
anonymizing source code because we saw
that obfuscation is not the answer to
that and then but about stylometry
and executable binary.
So executable binaries are compiled
code. When you compile code,
the coding style features still
persist to the compiled version.
So this is what happens.
We have source code. It's like 20 lines.
I don't have all of it here.
And then once you compile it, you get
binary zeros and ones like thousands
of them.
And I don't think I can personally
understand anything from this.
I don't think we can anonymize it by just
looking at it like this.
So now I'm going to talk about the second
part of this talk and
what happens when you compile
code and you try to anonymize programmers
from their executable binaries.
And this is the paper that just went
public today. If you want, you can look
at it on my Web site.
Why would we want to do that?
First of all, the research question, does
coding style exist and binary code?
And is there a
threat to privacy and anonymity
can be the anonymized programmers
from compiled code and maybe at
the end, can we use this for malware
family classification?
This is the approach they had in related
work, so since I've shown you the
machine learning workflow that
we had, I think you you're kind of
getting the idea how machine learning
would work. You have your data set.
You need to extract features that are
going to represent a class.
Once you do that, you feed
it to the classifier and then test
to see which class one
sample belongs to and in related
work took a executable
binaries. And then they disassembled
them with reverse engineering methods
and they obtained the control flow graphs
as well. So they extracted features
from the assembly instructions and also
the control flow graph.
They used some information
gain methods to find the most
prevalent stylistic features and then
they use the support vector machine.
And I would like to remind you that we
don't use support vector machines that
much for multiclass classification
problems. That's why we use reenforce.
And then the idea anonymized programing.
And what we do is we take our dataset
and we use the same dataset with them so
that we can make a real comparison.
We disassemble it with reverse
engineering. We also compile it
and the compilation
we can get the source code
representation of the binary.
And again, we can apply all the source
code feature extraction methods such
as the abstract syntax regeneration.
To this we also get the control
flow graphs and then we run information
gain on these to see what
feature belongs to an alter
instead of just being a random property
of the code. And then we do
classification to down to the
Nanami as a programmer.
And some features for this would be,
for example, we have our common AFSC
features coming from the abstract syntax
tree that's obtained from
generating the structural properties.
And these are things like just
not anagrams in the abstract syntax tree
estie Bagram's or edges.
And we do similar things for the control
flow graphs and we get the unique grammes
Bagram's. But remember that like since
this is decompiled, called the
abstract syntax tree and control flow
graph is like 10, 20 times longer than
the original one.
So we get a lot of features and they look
very similar to each other because they
have been reversed, engineered with the
same tools.
And for example, when we have 100
programmers, we extract their features
from their 900
binary executable samples
and we get about 200000 features.
Once we get all of our features,
once we are on information gain on this,
we see that only four hundred and twenty
six of them in this particular dataset
represent coding style.
So we focus on these four hundred and
twenty six features.
And what happens when we try to do
Anonymise 100 program or so,
we wanted to see how much training data
we need. First of all, like how many
binary samples do I need to accurately
anonymize a programmer?
And with one binary
sample and 100 programmers,
you can still re identify
them with 20 percent accuracy.
And once we use eight training
samples, we got 78
percent accuracy in the anonymized and
100 programmers.
And it seems like if we had more
source, more binary samples,
you would be able to increase our
accuracy further.
But our dataset wasn't really
letting us do this with 100
programmers and
relax classification, which I
mentioned towards the end of the first
part with 100 programmers.
When we relaxed the classification
to a set
of size 10, we get 95
percent accuracy in reducing
our suspects that size.
So let's say that you start with 100
programmers and their binaries and you
want to focus on 10 of them with 95
percent accuracy.
You're the correct programmer would be
within that set
of ten people.
And we also wanted to see what happens
with a smaller dataset size.
So here, when we just
sit and relax classification,
we can get close to 100 percent accuracy
after, like, just relaxing
it to a suspect that size of four.
And we're getting certainly 100 percent
accuracy at the suspect that size eight.
And we also wanted to
see what happens when we just use one
training sample for trying to program.
So we 20 programmers, if you just
have one sample from 20 programmers, you
train on 20 files
to generate 20 classes.
And then once samples are given to you,
you can correctly identify those samples
with 75 percent accuracy.
And that's kind of scary.
So if you have just one binary out there
that belongs to and that is known,
and if you're in a suspect set size
of 20, there is a 75
percent chance that your
anonymous binary would be identified
as it belongs to you and
you want it to scale this up.
So we went from 100 programmers
to 600 programmers.
And in this case, we see that the
accuracy is gradually decreasing.
And with 600 programmers, we get
52 percent accuracy.
And here I would like to mention that,
for example, in the previous part we had
1600 programmers.
But since we are using the same dataset,
we had to compile the source code
so that we can
use it in a controlled setting with the
same compilation
options.
And we couldn't obtain
1600 binaries after
that compilation. Some code just did and
compile and like some programmers,
did not have enough code because they
were missing. So we had to get rid of all
of those programmers.
So there hasn't been much work done
on this area.
There is one major paper that has been
published by Rosenblum and it's a great
paper. It's the previous workflow
that I have shown you in the beginning of
the second section.
And for example, with
20 programmers, they get 77 percent
accuracy and they use more training
samples than us.
It's not very clear, but the least
number of training samples they use is
eight and it goes up to 16.
And we kind of know that when we use
more samples, we are going to get higher
accuracy. But in our case,
with 100 programmers, we
get 78 percent accuracy
and.
For example, when we look at
the year 100 or
data set, we see that they get 61 percent
accuracy and we can get 78
percent accuracy. And again, they are
using more training samples.
And at the end, we are able to scale
our approach to 600 programmers.
But their largest dataset is almost
200 programmers and there
are 200 programmers.
Data that gets the same accuracy with
our 600 programmer dataset, which
was a more difficult machine learning
problem. So this is a great
improvement in accuracy.
What happens if we optimize code?
So is that kind of like the translation
of obfuscation in binaries,
like are they going to anonymized code?
So for that, for the first time
in in the literature,
we wanted to try compiler optimization
and stripping the symbols.
And we saw that with 100 programmers
without any optimizations, simply
compiling code, we can anonymize
100 programmers with 78 percent accuracy.
But after conflation, once we
stripped the symbols from
the winery's, we got 66 percent
accuracy.
And once we start applying more
optimizations like the common level
one optimization level to optimization,
which like which is cumulative, it takes
the previous optimizations in it
as well.
And it makes the program
more efficient and maybe
faster and smaller.
We see that the accuracy is not
decreasing in a tragic
way. With the highest level of
optimizations that we tried Level three,
we get 60 percent accuracy incorrectly,
the anonymizing these 100 programmers.
So compiler optimization
is not our solution to anonymizing
binaries.
We also wanted to see how we can find
out features that are remaining in
binaries that represent your coding
style, because since binaries are so
cryptic, it's difficult to tell
what's going on, what kind of
transformations are happening after
compilation.
So for this, we came up with a
machine learning setting
where we have the same code samples
and we have numeric
representations of these code samples
for the original code and the compiled
code. And what we tried to do was by
taking the decompiled
code, can we predict the features
in the original code that has
not been compiled at all.
And once we did that, we
generated the predictions for the
new features in the original code,
and we wanted to see how
similar this is.
And there is not a very simple way to
make a direct comparison between between
these predictions.
So we looked at cosine distance
similarity and we saw that the new
predictions were 80
percent, 81 percent similar
to the original code features
which are set.
And also we did one more experiment.
So we just took the original code
features and we took the decompiled
code features and we looked at the
similarity between those two and
the cosine distance was zero
point three five. So it was about thirty
five percent similarity, which
is much less.
And this is kind of showing that like
coding style and properties in compiled
code are certainly getting
transformed.
But this transformation is not wiping
away all coding style features.
So somehow they still remain embedded
in the binary.
And this might be a concern for the
anonymization and remaining anonymous.
We also wanted to see if we can
gain any insights.
And again, we looked at
the differences between the binaries of
more advanced programmers that were able
to advance to
more difficult rounds.
And we saw that even in the binaries,
you can see when a programmer is
more advanced as opposed to
other programmers with a smaller
skill set.
And for doing this in the data
set, we generated
two subsets.
The first one was with people that were
only able to compete,
complete the same seven problems.
And the second one was people who
were able to complete 14 problems,
including the seven in the first subset.
And we just used the seven samples
that were same between these subsets
and see how well we can anonymize
these programmers.
And for the more advanced programmers,
we got 88 percent accuracy and correct
with the anonymizing, they
binaries this to program programmer data
set. And for the less advanced
programmers, we got 80 percent accuracy.
So somehow there is more
coding style present
in the source code of the
more advanced programmers, and it's
getting transferred better after
being compiled.
And to validate this, we tried the same
setting with the six problems, having
a subset that was only able to
complete six problems and a subset that
was only able to complete that was able
to complete 12 problems, including
the six in the first subset.
And again, we see the same results.
So the accuracy is lower because
our training samples are less.
So our machine learning model might
be less accurate than as opposed to using
more samples.
So we get 87 percent accuracy incorrectly
identifying those, whereas we get 78
percent accuracy in the anonymizing
the less advanced programmers.
So we have been working on Google
Kojm, which is a very controlled
environment for running these
experiments, for the reasons I explained
to you, and we get the question
from many people asking that is
it is this the anonymization
being so successful because
of your Google called data set?
And we wanted to see what would
be the difference if we tried
to do the anonymization in the wild and
we try to collect the data
set from GitHub.
So for this we
get help and we find single alternate
repositories.
And these repositories had
at least 500 lines of code
and they had to have like at least 10
stars. And these people
should have several
number of repositories on GitHub.
So we had some requirements and
after that we ended up with four to nine
programmers. After all of our
restrictions, you can refer to our paper
for the details of these datasets.
And we had one and seventeen
repositories. But unfortunately, again,
GitHub code is sometimes very difficult
to compile. So after compiling
these, we ended up with 12
and 50 binaries.
And on this
dataset, we are able to anonymize the
programmers with 62 percent accuracy.
And we tried to generate the exact same
dataset from Google, could
use the exact same number of binaries for
each programmer, and we were getting 68
percent accuracy.
So it shows that, like, this is
a very promising result for running
program or the anonymization in the wild.
And for future work, again,
we would like to look at anonymizing
executable binaries because here we are
showing your problem like there is a
privacy problem here.
We're able to anonymize programmers with
very high accuracy in a
very simple machine learning setting.
And we can do this on the large scale
and for the executable binaries which
show that optimizations are not the
result, the solution to
anonymization.
And we would also like to look at the
anonymizing collaborative binaries
that have been written by multiple
people. And we would really
like to find out if we can extend this to
malware family classification.
But for that problem,
we need a dataset with some
ground truth. So if anyone
in the audience is interested in that
and has some ground truth data,
a data set that we can work with at
the end of the tunnel, please come and
talk to me. That might be amazing
for us.
And we have some available tools on these
projects, so you can find all of them
online, you can send us emails like if
you want to run things on different
settings. We have announced these
in our previous talks as well.
So we start with the main one program.
So the source code program or the
anonymization one is on my GitHub
account. So you can just like Google for
it and like find it and run it.
And we also have an
authorship attribution framework where
you can have some authors with known
documents, documents with known
authors, and then you might try to
identify an anonymous document
and it will give you many machine
learning options to do that and different
ways to generate different features and
so on so that you get a hands on
experience and like running this
machine, learning setting and see how
anonymization works.
And on top of G.
Stila, we have an animal that's built
so anonymous users stylize the
back engine and it's a framework that
will help you anonymize your writing
style. So once you give
it a suspect set and you're
also in the suspect's set and you want to
make sure that in that aspect that you're
anonymized, you can
use Anonymous so that
it will identify all the
all the alters in the suspect
set, and then it will give you certain
recommendations and suggestions so
that you can make your writing
more anonymous.
And this might
be a very helpful tool for
people, for example, in oppressive
regimes are like who really want to make
sure that they would like to write
anonymously so that they don't want to
get in trouble.
Because even, for example, let's say
that you are as with
the like Alice and abusive employer
Bob example, even if you are writing
or blogging through Tor
and you think that you're anonymous, your
writing style would make you
identifiable.
And I would like to thank all my
collaborators for this
great work, without them, it wouldn't
have been possible and.
If you like, I have some backup slides,
but if you think that you have a lot of
questions, we can just go to Cairney
now.
And thanks for coming
in.
Thank you.
Thank you very much.
We have about 20 minutes for Q&A, so
there's plenty of room for questions.
The first two questions go to the
Internet and the people who feel that
they have to leave. Please do so quietly.
OK, um, the first question is whether
your technique also works on, uh,
Shell Command so that I can see who
actually did something in my
system.
Can you repeat the question if it also
works on what? Can you speak a little
louder?
Oh, I'm sorry.
Does it also work on command?
So if you type or look at a session
of someone locked in a computer
to analyze with it.
So I'm not sure I understand the
question, are you asking
about sanctions
now if you have a lot
of unique command lines that you see that
someone has entered somewhere?
Can you also try to find who is the
author of these command lines?
Oh, just just command lines.
So as long as you
find the correct features for that,
I believe you can do it.
But this doesn't research this
current research does not exactly
apply to that.
But since we can do this on
various kinds of textual
data and we have shown that
like we can do it on like many other
things that have other than
mentioned in this presentation, I believe
it might be possible
to do that.
OK, next question from the Internet,
um, do you also look at
comments like, uh, style
of writing comments or position
of comments, support
these datasets to make sure that there
was no personally identifiable or
identifiable information left
that would just bias the classifier we
removed to comments?
So we don't have comments in this,
but we also looked at it with comments
and it usually makes it just like the
anonymization accuracy just increases.
Before moving to questions from the room,
please do remember that questions are
so short sentences and we have a question
mark.
First question over here.
What do you think the chances are of
having something like a compiler switch
to automatically anonymized code
by doing structural refactoring switch
then like your mouth thing that
you know, that's a very
good point.
And exactly. It's kind of a similar thing
to do with Anonymous because you're just
trying to, like, convert the
features that make you an unmissable.
And that might be a good experiment
to try in the future.
So I will keep that in mind when we are
like starting our anonymization work.
Next question.
Over there, yes.
Thank you for the talk in the
beginning. Let's say I am
writing code for
I'm in Iran, Iraq code for a
porn website.
And I want to
obviously complete the anonymization is,
uh, not possible.
Anonymization, uh, can
I, uh, produce uh,
can I maximize my chances to,
uh, to
not be identified and
executed?
Um, so first of all,
with the site Molik, for example, he was
identified because his name was
on the court. So that was like a direct
identification.
But as you suggest, if you are very
careful about being anonymous, maybe
you can try to follow very
strict conventions and make sure
that everyone in that project is also
following the same conventions so
that all of you look very similar and you
cannot be identified, you cannot
be discriminated
from each other in that case.
So that might be one solution so far
or it might at least help you.
OK, next question up there.
My question is, I assume
all of those numbers that you had
where you had a set
of people that you programed into the
system and you were giving them one
sample of a person that you knew
was in that set.
Have you ever tried giving them
a completely unrelated code sample?
Is the software able to
tell that that's someone else who
is not part of the reference set?
And if so, at what probability?
So you can't look at the
slide.
And this is kind of a verification
problem in machine learning.
And this is a one class, two class
kind of classifier that might
help you verify if the
anonymous code sample comes
from someone that is in your
suspect set or like that's in the set
of programmers
that you trained the classifier on.
And for the setting, what we did was so
we have Mallery and Mallery claims
that this code has been written
by her and we want to find
out if it was really written by her.
So we have two classes,
the one on the samples
from Mallory and the second one has
samples from random people, which
represents the outside world.
It's anyone but Mallory.
And in this case, we can take the
code that Mallory is claiming to have
written and see if it
was really written by her.
Like is it going to be attributed to the
outside world?
Who is not Mallory or is it going to be
really attributed to Mallory?
And for this case, we get ninety one
percent accuracy with
each repetition of such a setting.
And if you want to find more details
about the probabilities of how we can
like threshold the verification,
look at our first paper.
We had the details of that or we can
chat later.
So. But you haven't tried
when you don't know who it could
have been. If you say, OK, we have
like a suspect list of 50 people, we
haven't tried seeing if the
if it would be possible to determine that
it's not one of those fifty people.
So with verification like
so instead of this two class setting,
let's think about the fifth class setting
with 50 programmers.
If a sample is being attributed
to one programmer, A
is below a certain probability,
then you might be able to say that this
looks like this person, but this is
sketchy. Maybe this is not this person
because this is not a very confident
classification.
Does that answer?
Oh, I think I'll just talk to you in
person. OK, let's go to the next question
over here.
What about codings? Diago Most languages
have a devout codding Steiglitz and you
have cording formatters.
You can run automatically.
Would that help?
Oh, so, yeah, that's I think the
previous question was similar to that.
If you follow strict conventions and if
you everyone does that, that would
normalize, that should normalize your
writing style to some degree
or coding style.
And but we saw with
the compilation case that compilation
is also kind of a normalization over your
coding style because it's converting
everything to a set set
of rules and it becomes very similar.
And even in that case, we're able to do
anonymize programmers, but with lower
accuracy. So that might help you
be more anonymous.
But I'm not sure if it would be
the exact solution, but no numbers how
much it would change.
So the problem with that
experimental setting was that we don't
have such a dataset to test this.
But like if someone from the industry has
a dataset with very programmers are
following strict conditions.
And so we should be able to
get more answers to this question,
I think to that may have a few more
questions.
Yes.
Can you use your technique to actually
forge code to look like it was written
by the programmer?
Uh, so, yes,
this is possible with machine learning
because like, for example, with
anonymized case, that's where you
anonymize your writing style, not coding
style.
What we do is like your witness aspects
that and you try to do anonymize yourself
in that suspect set in.
What you do is try to
like bring in features that do not
represent your style.
But if you bring in features that belong
to someone else. Exactly.
And you can see what those features are,
then your style would be more similar to
that person.
So once you have this framework
for source code,
that should be possible.
Next question up here.
Hey, you said there's a difference
between advanced and less advanced
programmers.
So my question is, when you have a given
set of binaries, can you tell which ones
were written by advanced programmers?
That's a very good question.
We haven't tried that, but I think you
can come up with an experimental setting
to test that and
we haven't done that yet.
OK, thank you.
Hi, two questions.
First question, have you considered using
SCAP Grams where and
like Skip and Grams and
is greater than 30, then you get more
support? Second question, how
about unsupervised learning?
Oh, yes, I'll answer both of
them for the skip gram or like
multiton grams.
You can do that. We tried it.
It just takes a very long time to extract
those features and then try
to classify it like that.
And we saw that when you get
more in grams or skip grams or like a
variety of powerful features,
it doesn't really help, especially the
source code, authorship, attribution,
accuracy, so much because it's already so
high. We also would like to avoid
overfitting to the data set by
generating very detailed features
that might bias the classifier
at the same time. Very lucky that we have
Amazon Research Grant where we can
run our experiments on Etsy too.
So it's making things much faster.
But other than that, I wouldn't always
want to extract
like thousands and maybe millions
of features if you're going to go into
such details.
And for the second question, yes,
you can do unsupervised learning.
You can just try to cluster these
based on certain properties found
in court. And these properties could
also be coding style properties.
And that's possible.
But we haven't gone through
that setting.
Thanks. Yeah.
Hello, so have
you thought about researching
stuff of using
meta metadata of code
to actually classify it?
So let me get a get
canids, commit messages or
workflow or even set of used
libraries or something like that.
And, of course, how to protect
from that.
So in the beginning, I started this
project by using exactly
a data set, like you describe it, like
all the get comments messages and it's
possible, yes, you can
anonymize those programs and it probably
makes it more powerful.
But then with Kitab,
it kind of becomes a multi altor problem.
So it relates to our future part where
we would like to look at multi other
source code, authorship, attribution
and the same for binaries.
And we are working on that currently.
Thank you. Yep.
I, uh, using your approach,
do you think it's possible to abstract
these models even more so that it becomes
possible to train on data
sets that are constructed from
one programing language and
then do classification on another
programing language or with
text or samples from another programing
language?
And did you do experiments in that
direction?
Thank you. That's a very good question.
We haven't done experiments on that.
Again, like it's difficult to find
ground. True data, for instance, like
even with Google Culture, some
programmers write in multiple languages,
but that's a very small set.
So we haven't looked at that
yet. But like when, for example, compare
C++, yes, you can do a cross training
and testing because like the nature
of the two languages are very similar
when you you are coding.
Next question goes to the Internet.
Yes, yes.
Would that also work on assembly
because you do not really have something
like variable names.
So with assembly,
I will go back to the site where we
oops, I just
closed that just a second.
We have a bunch of important features
that come from assembly and.
It's good and it's very difficult
to understand what they actually mean
because, oh, it's close.
Oh, I see.
OK, here we go.
With assembly futures, here,
we get assembly futures from
two different disassembles to make it
stronger and richer and we don't
get that many, but we have like close to
like one hundred and something.
And it's very difficult to tell, like,
what those exactly mean, because, like,
some of them are just anagrams and
it's very it looks very overfitting.
But since we show in our reconstruction
experiments that somehow Preserve's
coding style in its somewhere is about.
Does that answer
the question on my online, online
over that I,
um, do you think it would be possible to
combine your research with, for example,
a social graph analysis to find
cooperation between programs or groups?
Um, I guess you can do that.
You can just add that as an extreme
machine learning feature and just like
improve your classifier.
But in this research, we are particularly
interested in finding coding
style and quantifying coding style.
So if you wanted to exclude like any
other irrelevant information,
which might make them more identifiable.
Did you do any research in the direction
of different compilers of the same
language, so, for example, GCSE
and is better than 11am, or
can it help anonymize my code if
I use different compilers for different
binaries or projects?
Uh, and mix and match might help
a little.
We haven't investigated that question
in particular, because when we look at
related work, we see the compiler
detection is kind of a small problem.
So as long as we know
the compiler because like it it
can be detected, then we can
come up with a setting where
a source code has been compiled in that
setting or like try to get rid
of the properties that that compiler
would bring in.
But as you said, if you like, start
mixing and matching, that might help you
anonymize it a little.
And that might be a good way to start our
anonymization experiments.
Thank you. Thanks.
Hi, I have a question regarding the real
world application you talked about,
what is the statistical statistical
probability that this is just based on
pure luck? I mean, you have like 70 to 70
or 60 or 70 percent of the anonymization
for 12 programmers.
How high is the probability that this is
just random?
Yes, it's difficult to talk about
statistical significance in this case
because like, we have a smaller dataset,
but at least we can kind of compare it to
the Google Kojm dataset where we know
there is statistical significance.
That's why we think that it might be
possible. And so in our future,
work to like apply these to larger
real world datasets where we can talk
about statistical significance for sure.
And that's a very good question that
we have been thinking about.
Thank you.
What may be the last question goes to the
Internet.
Oh, well, yes.
So, um, the question is
what violence, where the most important
and the random forest analysis.
Let me open that slide.
We should have a slight word, so the most
important ones are Bagram's, but
like, what do you mean real variables?
Or are you talking about features?
This is not the.
So for source code, authorship,
attribution, the most important feature
is for the random first
R word anagrams, first of all,
and Virginny Grams are things like your
function name or your integer or
double choices or
things that come from your
source code.
And we have the Bigram
that come from the abstract syntax tree,
even though they are percentage in the
information game dataset was less
than the Virgini grams.
Their information gain is almost
equivalent to this entire
information gainsaid.
So in this case, the trigrams
are the most important.
OK, thank you very much.
Time's up.