Dear viewer, these subtitles were generated by a machine via the service Trint and therefore are (very) buggy. If you are capable, please help us to create good quality subtitles: https://c3subtitles.de/talk/589 Thanks!
So who if you saw the
talk about politicians speak this
morning?
Nobody.
OK.
Yeah, it wasn't German, so maybe
I wanted to respond something to
the people who did, but yeah,
apparently now, uh,
talking gibberish in
human understandable language.
Uh, you didn't hear about that
today, but, um, talking gibberish in
electronic languages,
you are probably familiar
with that. So
then here is
a security researcher with
checkpoints and he will talk
to you today about deejay's.
So, um, algorithms
that produce gibberish.
But, uh, they got a bit smarter in the
past. And
he will tell you something about how
to detect gibberish,
which somebody some people
might want to have for politicians, too.
But you have to use reason for
that. And he will give you an idea about
how you can do that, for example.
OK, give a warm round of applause for
Ben-Meir.
Um, is this on it is.
OK, first things first.
If this slide makes any amount
of sense to you, then I'm sorry to have
to tell you this, but you're probably a
robot. Uh,
so, uh, well, the good news.
That's the bad news. The good news is
that you have come to the right lecture
because once this is done, you'll be able
to detect gibberish just like the rest of
the humans. You be able to blend in and
no one will know a thing.
Uh, so first,
I'm going to refresh your memory a bit
about what D.J.
is and what the problem is that it
was trying to solve.
Let's look at a regular scenario
and basic scenario where an infected
system has been infected with malware
and it wants to converse with its command
and control server. That's what malware
does nowadays. In the past, it may have
just done its own thing without receiving
any commands. But today,
uh, malware usually
waits for commands and operates
based on commands that it, uh, receives.
So, uh, in this
basic, uh, usual scenario, the
malware came with a built in
DNS address. It's hard coded.
And malware queries that in a server
with this hard coded address and receives
a response. This is the IP address of the
server, not the infected system contacts
the use of the Internet and the server,
the census server very excitedly
responds, yes, I have another machine
under my sway and the connection
is complete, not infect the system and
the server can converse.
So all of this is fine and good until one
day the powers that be the
maybe do forty's.
I know they
find out about all of this and
they talk to the people in charge of the
DNS ever. That's probably right,
but not necessarily.
And they tell them, well,
there is this there's been this shady
activity going on and it's making use of
your DNS servers.
Would you kindly make sure that it stops
and the people in charge of the DNS ever
do not want any trouble?
So they removed the record, pointing to
the IP address of the server.
And now the infected system, just as
before, makes the DNS query to the server
and asks, OK, where's the IP server
of? Where's the IP address of
my senses server?
And the server basically responds,
go.
Now the since the server just stender
fully functional, waiting to
send commands and it's Tenzer and
it waits and it waits and it waits.
And that's not very good for the
campaign.
Now DGA
is basically a mechanism that campaign
managers came up with.
They looked at
this problem, the ease with which in
a state known can happen, and they said,
we want something better that won't be
taken down as easily.
So I could stand here for a lot
of time and talk theoretically about and
how it works. But I think a practical
walkthrough of just how it works in
practice is going to be more productive.
So let's
see how it actually works.
It begins our story begins with
this. Since the server and it has access
to a pseudo random number generator,
which is basically a creature that takes
in a small amount of entropy, randomness
and outputs a large amount of entropy
randomness.
Now, this is a random generator
specifically taxin that publicly
available, such as the date of today
or maybe the headlines of today's
newspaper? I don't know.
The important thing is that it should be
publicly available to everybody.
Now it is a customized algorithm that
takes in this particular variable
a small amount of information and output
outputs a large number of
domains.
Uh, these domains are not
very understandable.
And this is typical.
And this is basically what this lecture
is about.
But now what the synthesis over does
is take one
of those domains at random.
Typically that's what it does and
register it with the DNS ever to
point at the IP address that's
relevant to the IP address that
in fact the machine can contact.
Now, what happens to the infected client
site? Something similar, the fact the
system also has access to the same store,
the random generator that came bundled
with the malware, and it has access
to the publicly available seed because
it's publicly available.
So it consults the pseudo random
generator and asks, what are the domain
of today? What are the domains available
to me today?
And it gets a list of how many domains
are there. It varies sometimes 15,
sometimes it's 200.
There's a lot. That's what I'm trying to
say. And now the infected
system knows that the since the server
had registered one of
those domains to point at the IP address,
but it doesn't know which one.
So what is he going to do?
There's really only one solution contact
all the addresses, it's going to iterate
over all the addresses one by one
and make DNS queries
asking for the relevant IP address.
Now, most of those domains
had not been actually registered.
So what results is
a very peculiar sort
of conversation across the DNS protocols,
the protocol that kind of
resembles the cheese shop sketch
by Monty Python.
If you're not familiar with it, it's a
sketch that involves a guy walking into
a cheese shop and he tries to purchase
various kinds of cheese.
And as the sketch progresses, it becomes
increasingly clear that the shop
does not actually hold any kind of cheese
at all.
The guys, do you have any Parmesan cheese
shop on and says no?
Well, how about Retno and so
on and so forth.
And the DNS
conversation going on resembles this
exchange greatly, because what happens
is, is that the infected machine
unsteadiness over.
Well, do you have the IP address for this
jibberish address?
The NSA responds no.
Well, how about this jibberish address?
Knew about this gibberish address?
No, sir. Sorry about this gibberish
address. No, not today, sir.
And this goes on and on.
Here you can see a traffic upcher
depicting this process.
You can see the repeated no such name, no
such thing, no such name.
The dinner service says, What do you want
from me? I have never heard of any one of
those domains. Please stop bothering me.
And but recall
that the since the server had actually
registered one of those domains,
one of those domains is a valid domain
that points to the IP address of
the since the server. So eventually
the infected system is going to make the
golden query and the DNA server is going
to excitedly jump up and down.
Oh my God, I know this one.
It reaches down to the drawer and pulls
up the IP response and the infected
system is related.
Now, it finally has the IP address of
this, the server as before and contact
as before. And all is well with the
world.
Well, you notice that I kept
saying as before.
As before, as before what all this work
just before all this bloated mechanism
of the. Just to get the same result as
before?
Well, not exactly, because think what
you now have to do if you want to try to
take down this infrastructure
infrastructure.
Now, let's look at the domains generated
in one day by this algorithm.
That's a lot, and if
you're trying to take down this
infrastructure and you do not have access
to the pursuit of random number
generator, the algorithm, basically all
of this is random, too.
So every day you're going to have to
chase down and hunt all of those
addresses being queried
all over the world.
And that's not going to be easy.
Now, if a
takedown happens at all, here's the more
likely scenario of how it's going to play
out. First, you have your victim on your
victim gets infected in some
internal enterprise or something, and
it contacts the server.
And there's that exfiltration sound
going on. And eventually someone
wises up to this and this hot
potato gets thrown over to incident
response.
Now, incident response,
and they do
whatever they can with this to try to put
out this fire.
And maybe if we
are lucky, they want to
draw conclusions from this
and make sure that the information that
they've obtained is relevant and can be
used later to prevent further
attacks of this nature by the same family
of malware.
So if you're lucky, this
incident response is buddy buddy with
middle management and some security
vendor. So let's get bus over
to middle management at some security
vendor which ever burns down
somewhere where the sun don't shine or
if you're lucky, it passes
the middle. Management passes this over
to some reverse engineer who is
going to spend like a few
weeks or maybe a few months poring
over this file in either pro until
if you're lucky, this thing results
in a report the report
ever lays down at the bottom of the
Internet, and no one pays attention to
it. But if you're lucky, over a
streamlined process or some kind soul
is going to take this thing and make sure
that the data about how this pseudo
random number generator works
is incorporated into a firewall
somewhere, which will actually, once this
process is down, block any
future traffic based on the same domain
generation algorithm.
And is a streamlined process, what could
possibly go wrong,
but suppose that
you had the ability to automatically
detect the DJ.
Now you could aggressively cut out
a lot of those middlemen.
Now, a lot of the links here have
better things to do with their time
middle management place there so they
have better things to do with their time.
Invest engineers, too.
If you had the ability
to automatically detect deejay
traffic, you could theoretically put
it straight in the firewall.
The firewall is going to see the outgoing
digit traffic and aggressively it's going
to shut it down after like four or five
years and say, hey, I'm sorry,
this traffic looks shady.
It looks like you're not getting through.
So automatically detecting digits
is useful and it's cool.
And as a consequence, there
have been past attempts
to solve this problem.
And we're going to look at some of the
past features that have been suggested
to identify G.J.
And we're going to talk a bit about
how those features do not necessarily
work 100 percent well
all the time and how they could
be improved, which is what we did.
OK, let's talk about
some ways to detect.
One way involves looking at character
frequency. It's a well-known fact that
some letters are more common in the
English language than others.
So if, let's say, I take the
common letters in English and
code them as green and the borderline
letters I call the code is yellow and the
rare letters I know code is
red, and they take five
words that I randomly picked out from the
dictionary and five
jibberish segments of comparable
length. You can at a glance
tell which five words came
from the dictionary, in which five words
didn't. So that's a useful feature.
Another useful feature is
along the same lines.
It it's based on frequencies
of pairs of letters instead of singular
or some pairs of consequent letters
in English are more common than others.
T is common.
It is not so common.
And the
another feature is
called The Longest Meaningful Substring.
It's been suggested in some researches
into this problem.
It involves looking at your input, your
domain name, and seeing what's the
longest substring in there that you can
actually find in the dictionary.
So bug me not contains the words bug and
not so the longest
meaningful suffering is of long free.
Amazon is actually a word in the
dictionary. So this is definitely going
to register is not gibberish by this
metric. EBay contains the word BAI,
so it's like three quarters of gibberish
and this actual gibberish is really
gibberish. I can't find any word in the
dictionary in this thing.
So another useful feature
and the last feature that has
been suggested in the past that
I want to talk about is the next domain.
Remember, just like the cheese shop,
there are the repeated no, sir, no, sir,
not today. So no such thing, no such
name, just counting.
These is a useful feature.
So we have all of those that
suggested useful features for detecting
T.J. And we are done right.
The problem is solved and we can go
shopping.
Well, not exactly
as you might have imagined.
And the first issue
with what I just said is what
I like to call the Tumbler conundrum.
And I mean, let's look at Reddit.
Reddit is like there's the word red and
there's a word that's for most of, you
know, the little dot from our school.
And there's that.
If you look at Google, it contains the
words go and my God, we lucked
out. OGU is the word.
So the longest meaningful substring
criterion is going to look at Google
and say, OK, go ogo
seems legit, but
let's look at Tumblr.
What is Tumblr?
You're not going to find this in the
dictionary.
I mean, Tumblr is not the word.
No, substring of it is whatever.
Tumblr is not word unblurred, not a word
blurr still not or whatever.
And so
there you have one issue, because
the longest meaningful subject
is going to look at this and say this is
gibberish. And if you got a human
to take a look at this, the army is not
going to be so hasty to say
that this is gibberish and we're going to
just touch later on.
The reason why that's
one issue.
The second issue is quite Djibo
now. Quite Yarbo is
digga engine that has surfaced
a few years ago.
And it draws its name from an incident
in an episode of The Simpsons, where Bart
is playing Scrabble against Homer
and Bart is stuck with the list
of letters that you see up there at the
top of the slide.
And he doesn't know what to do until he
says, well, you know what?
I'm going to put on the board the word
quite EUBAM.
And he plays that word.
And of course, it's for four billion
points because he used all these letters
and on the triple word square and so
forth and so on.
And Homer is not happy, and
they asked about what is this word and
Bart, without blinking an eye, he says,
well, quite sure it means a stupid North
American yellow ape.
So
much like Bart was
able to pass this under the radar because
you both sounds like a word, even though
it's not a word.
Quite Yarbo. The generator passes
domain names under the radar because they
sound like words, but they're not words.
What quite Yarbo does and this is
stupidly simple, it makes sure that
every other letter in its
output domains is a vowel.
Now you're sitting there and thinking,
Ben, look, just this
every other letters of all and all of
the features that you talked about
earlier are now suddenly useless.
Well, let's
look at the letter frequencies.
Earlier, the gibberish generated
by the main generation algorithms
contained lots of letters,
X's and these injuries.
Now, if you average out the frequencies
of letters you're going to encounter,
suddenly it looks much more peachy
because you have vowels everywhere
and vowels are common letters.
You're telling me, OK, let's look at the
pair of letters.
The Bagram's is the same thing.
Pairs of letters with vowels in them
are very, very common.
Not all of them, but a lot of them.
And three letters are going to run
into more or less the exact same issue.
So the letter frequencies
approach is now going to be significantly
weaker than it was before.
And how about the longest meaningful
substring?
Well, you can play a game and
you can look at the domains listed here,
which I swear I pulled randomly from
Equatable Page and start
looking forwards by skimming
this. I found give and nope.
And I swear I did not plan this in
advance. Gaited, which is very
appropriate for this conference.
So the point is that now that
your features that seem so strong before
they're no, they're
half useful.
And if you take half useful features
and fit them into a machine learning
algorithm, you're going to get the
result. That's a total loss.
So because of
all of those problems.
OK, let's improvise.
Because of all of those problems, we came
up with our pretty idea for
a solution, a pretty nice theoretical
idea that involves looking
at the input and deciding how close
it is to a concatenation
of words from the dictionary.
So now, Tumbler, that was a
complete nonsense before Ken, but
with just one edit turned into Tumblr,
which is actually a word from the
dictionary.
We have several here at.
That was Youthwork.
Oh, excellent.
So we inserted just one letter in
Guatemala, which is a word Google to
edit and becomes Google the word from the
dictionary. That's not a coincidence.
This is the word that inspired a company
named Reddit with two edits becomes
read it and now we can finally make sense
of those domain names.
And as for quite uibo, quite uibo
generates strings of gibberish.
And sometimes it's going to lock out
and create a world that
that's out of the dictionary, but it's
not going to successfully create
concatenation of actual words.
One word might be in there, but not
you're not going to be able to look at
the whole thing and make sense of it
through the lens of this material.
So the way forward
seems clear.
Step one, we measure the distance of
the input from a
concatenation of dictionary words and
concatenation as long as we can reach it
using the criterion at a distance.
I skipped over this, but the added
distance between a word and another
word is the minimum amount of insertions
of letters, deletions of letters and
edits of letters that you need to get
from that word to the other one.
So we look at the minimum edit distance
between our input and a concatenation of
words from the dictionary, and the
miracle occurs here.
And then we profit because this new
criterion is going through the fifth
grade Yarbo and mitigate the tumbler
problem because the word tumbler is now
going to suddenly make sense to us and
we're going to win the day.
So we were super happy and then we
actually tried to implement this thing in
practice,
why we ran into trouble.
Why did we run into trouble?
Well, let's look at how you actually
performed this this
computation of at the distance.
The canonical algorithm to do this is
called a flood search.
What you basically do is that you take
your input and you perform a breadth
first search in the space of
possible strings by performing every
edit that you can think of, which is
basically a stupid brute force.
So as the amount
of edits that you're willing to search
for grows and grows, the size
of the space that you're going to search
for grows exponentially.
If you have input of size eight
and you are actually willing to search it
out exhaustively and see how close to the
dictionary it is, then
you're going to need to do.
I don't remember the number by heart, but
it's a large number of lookups.
So let's say that one look up
into the dictionary is going to take one
microsecond. I imagine that's in the
right ballpark.
So we take the number of lookups and we
multiply it by the number of seconds
that we can expect.
One lookup to take.
And now we reach the conclusion that
in order to get our answer
for how close to the dictionary
is the input that we have
on our hands, one input, one domain
name, we're going to have to simply plug
it into the algorithm and sit and
wait for two and a half days.
Two and a half days.
Oh, by the way, that's a lower bound
number of lookups that I mentioned
earlier. It's not the actual number of
lookups. The actual number of lookups is
greater. Every calculation you see
in this presentation, it's the back of
the envelope combinatorics calculation.
Yes. It's not exactly precise, just to
give you an idea.
So that was a lower bound about.
Oh, and by the way, we were implicitly
comparing words against at
least a set of all
possible concatenation of dictionary
words. And there's an infinite number
of them laptop, paper,
concatenation of words, bottle
chairperson, podium
style concatenation affords there's an
infinite number of them.
So the problem is that
Google's infinite service project isn't
going public in another half a year.
So we're not going to be able to take
this infinite database and fit it
anywhere.
So at this point, we realized that
we're going to have to improvise.
So we needed to
come up with some ugly hacks and cut some
corners on this insane computation to
make it actually feasible to compute
this thing.
So the first idea that we came
up with, it's it was like a first aid
solution is to use a greedy
algorithm. And instead of trying to model
the whole input against the
set of all possible concatenation
of dictionary words, try to look for
prefixes of the input and match them
against the plain old dictionary.
Now, first of all, by using
this imprecise approximation, you
can expect your running time to drop by
a lot because the length
of the input that you feed into the
distance, the distance algorithm
has an exponential relation, as I said
earlier to the time you can expect
the search to take.
So that's first of all, you can see the
numbers here. The difference is drastic.
But the more important thing is that now
we're actually comparing against
the infinite dictionary and that's
progress. Now, it's not all sunshine
and rainbows because the expected
time using the back of the envelope
calculation here is still, what,
like an hour and something
for one traffic capture that's
unacceptable is unacceptable.
We're not going to sit there for an hour
and wait for the computation on a single
traffic capture to go through.
So we need the more ugly hucks.
The second ugly Hucks hack
involves and
looking at the classical approach of
asymmetric search that I talked about
earlier, that's just stupid.
The breadth first search.
Now, what would happen if I took
the dictionary that we have and I
bloated it to contain
all strings of letters that are
within an edit distance of two
of the original dictionary?
What's going to happen?
Well, you're going to have a larger
dictionary. But the interesting thing
that's going to happen is that you will
now be able to do a much
smaller flood search for the same result,
because think about it, if your input is
at a distance of four of the
original dictionary, then it's within
at a distance of two and
within something that's within
a distance of two of the original
dictionary.
So with just two edits, you're
going to be able to match up against that
something. Now, your flood search, the
input to flood search is the
same length, but the number of
edits that you're expected to make is
much smaller.
So you cut the exponent and
now suddenly the running time is
drastically reduced. But the
downside of this is that now your
dictionary is larger, much
larger by the back of the envelope
calculation shows like by a factor of
10000. So it's a it's unwieldy,
but we can still live with it.
So that's still not enough.
Let's go on to the ugly huc.
The first agritech is not so ugly,
actually. It involves looking
at the distance measure that we are
using. It's the added distance.
Now, what would happen if we disallowed
in place editing of letters?
We only allowed insertions and
deletions.
Now, on the face of it, the metric that
you're going to get is not much
less legitimate than the distance,
actually. There's no reason to think that
it's less legitimate.
It's actually bound within the
insertion deletion.
Distance between two strings of letters
is bound between the distance and twice
the distance. The proof of this is left
as an exercise to the reader.
And now
we switched one criterion
for another criterion.
And the two criterions
are more or less, we think that they're
legitimate to the same degree.
But now a flood search is going to
contain much less options to iterate
through because all the in-place edits
are gone. You don't have to worry about
them anymore.
So all of those aggregates are nice.
But the nice thing really
is not the individual like Linux,
but the way that they come together to
form something greater than the sum of
its parts.
Now, let's look at.
The symmetric search that I talked about
earlier and how it can combine with
insertion deletion distance, let's
look at the not a word spox.
We can remove the pin, get Shook's, and
then we can remove the X and get.
And if you look at the word shout, we can
remove the T and get chow
and we can remove the O and get Shiu.
And those two words
after a fashion have now met in the
middle. Uh, so what we
have now proved is that the distance,
the insertion position distance between
Spook's and Shout is at most
four because you can text books and
remove the P and remove the
X. And now you can go backwards
and add the O and add the
T and sporks and Sharlto now connected
for this. Now there's a peculiar
something about this computation.
You have noticed that we didn't have to
actually insert any letters.
We only deleted letters.
So we deleted letters from spox and we
deleted letters from Shout.
And eventually we reached
this kind of lowest common denominator.
And the truth is that it's possible to do
this for any two inputs.
So really, we can forget about symmetric
insertion deletion and just talk about
symmetric deletion.
Now,
this is a nice thing, because now
we're only left with deletion out of in
session and deletion and editing that we
started with before.
Now, let's combine this
with the blow to the dictionary idea
that we saw earlier.
We can now keep a dictionary containing
all the reduced forms that
you can get from a word in the original
dictionary by removing letters.
So now you can take your input and
start deleting letters and comparing
against reduced forms.
And eventually you're going to be able to
meet in the middle just like books
and shouted.
And the expected time
to carry out this computation is much,
much lower because now you don't have the
crazy flood search anymore.
You only have to delete some letters.
Now, the to really get
this algorithm all the way through, you
would have to look at all
the deletions that you can perform when
your input.
But we decided to limit it to taking
away free letters because
really the performance gain was very
nice from this. And otherwise, what
word in English, you know, that you can
take away free letters and there's still
something sensible left of the original?
There are some, but we thought that it
was a good tradeoff.
So the limitation
that we got out of this was that if
actually meeting in the middle requires
more deletions than the number of
deletions that we were willing to make,
that we said that to be free,
free and we won't be
able to do the meeting, the middle thing,
but otherwise this thing is going to
work. The important thing is that
insertions we get for free all
insertions that need to be made in
order to get to the word in the
dictionary are going to be detected.
Even if you need like eight insertions or
something, it's still going to work
because you're going to find the reduced
form that you got by deleting those eight
letters from the word in the dictionary.
So how good is this thing?
How quick is this thing now after we have
applied all of those ugly?
Well, obviously, we
need to implement it to find out.
I was going to implement it in Perl, but
just the other day I heard the Spanish is
horrible and you shouldn't write anything
in Perl.
So I
went back to the drawing board to write
everything in Python.
And there's all sorts of
calculations here of how many atomic
clock ups in the
Deletions Dictionary.
The bloated dictionary that we created
are going to be required for this test
run and how much time the tests
took, including, I'm going to say,
the including
degeneration of the actual gibberish
to be tested.
And you divide this by that and you
multiply this by that to see
how long it's going to take to do the
same feat that we tried to do before,
which is to see how close
to the dictionary a string of eight
characters is.
And earlier it took us two and a half
days. And now by this calculation,
it's going to take us a quarter
of a second, a quarter of a second.
That's an improvement by a factor of like
nearly a million.
So we're very happy now.
And in the wise words of Hannah
Montana, we can now enjoy the best of
both worlds.
The dictionary size
is hefty, but it's not prohibitive.
And our query time is improved
by a really drastic amount.
OK, yes, we have this deletion, this
limitation on the number of deletions
that we can deal with.
But I explained why.
I think it's worth the tradeoff.
And this feature is now something
that can actually happen in the real
world.
So we decided that
now what we need to do is an experiment.
So let's recap the
three features that we decided to
extract from our pickups
to see how close this
pickup is to being
DGCA generated traffic.
So there's a maximum of the requests
that got the same response, whether
that's an IP address or an accident
for 410 of the involved request.
We calculated this feature that I
just spent like 20 minutes explaining
how it works, of how close this domain
is to a concatenation of words from the
dictionary and for
all of the involved request.
Again, this was computer
for ten of the involved requests just to
improve performance. I think that's
enough. And finally, we computed
the frequency based on Begum's.
I explain this feature earlier, mainly
for comparison's sake, to see
how our metric first against
this and also they can work in unison
to maybe detect
jointly things that each one of them
alone could not.
So now we're going to
see the resulting classifier
classify a traffic capture.
Let's hope this works.
OK, now.
We're going to run this on air traffic
capture. Now, I set this
log to basically pass
the first free time.
Something interesting happens and then
it's going to just blow past this.
So we're not going to be standing here
forever.
And OK, we
have the traffic after and we started
to analyze it and we're
going to extract the pictures, the
features from it.
Obviously, now we find
that the most common, Dena's response
that was received during this traffic
capture is an exclamation.
So we're going to look at the request
requested domain's that got this response
and now I bet you were getting
a little suspicious
looking at those domains.
It's a list of 28 domains,
and now we have our maximum
domain collusion features.
This is the maximum number of domains
that mapped to the same response.
Now we're looking at the longest,
longest, because they're actually all the
same length, so they just sorted
alphabetically, actually relevant
request and we're going to start
analyzing them, using the features that
we talked about before.
Now, we're going to start with what's
called the pronunciation device.
And see, that's just a fancy way
of saying that we are going to look at
the pairs of letters, the big ones, and
see how frequent they are
and how this compares to what we would
expect from English.
And so we're going to start looking
at the first domain name and now the
algorithm looks at this input,
the domain name, and says, OK, it starts
with C.
How much am I surprised
by the fact that it starts with C.?
Well, as you can see, the answer is five
point six surprized, more or less.
Don't ask me about the measurement
units on this, because this is a we'll
talk about probability theory and we
don't have the time for that.
And the same
happens now because after this, the
C, there's an AI and the algorithm ask
itself, OK. So I see how surprised
am I seeing an AI after the see.
Well it's four point three surprise
less than before and
then this goes on and on.
Now we are looking at the AI until
finally we have a bag full of
surprises and we are very surprised.
We have a number and we normalized it by
dividing by a factor of the length of the
input. And we get a score of how
surprised we are generally by
Peagram and wise pairs of letters wise
by this input domain name.
So we move on to the next domain name.
The same thing happens.
We get a general idea of how surprised
you are by it.
And then same for the next domain name.
We went for for domain names and I was
going to go woosh. We went over all
the domain names and we average them out.
Now we've got a measure of how surprised
we are by the big Rum's, the pairs
of letters in all of the
relevant domains here generally on
average, and the answer is zero point
nine surprised.
And we
keep on going and now we're going to
calculate the lexical
divide and see, which is really just a
fancy word for the feature
that I spent 20 minutes explaining
how we're going to put together the
closeness to a
concatenation of words in the dictionary
as approximated by a host of Ugly
Hex.
So we're going
to start calculating it.
And in order to calculate it, the
algorithm is going to iterate over every
possible prefix and see which prefix
looks the most promising to take away
from the input and say, OK, I imagine
this more or less is my most promising
candidate as something that used
to be a word in the dictionary but got
mutilated somehow.
So we performed the look up
on the sea
and we can delete nothing and
stay with sea, or we can delete
the sea and stay with nothing.
And it turns out that
both of those
options are in the
dictionary.
So obviously it's better to
have to delete nothing and just say the
sea is in the dictionary, it's a
candidate, and we can
just take it away and
say, OK, that's a word from the
dictionary. The downside, of course, is
that sees just one letter.
So it may be in the dictionary,
but it may not be the best candidate to
take away from the input because we want
a longer candidate to cut the input
length. This is how the greedy algorithm
operates. It wants to take
away the longest the best candidate.
And that also depends on the length.
We want to make the input smaller
as we work on it.
Now we look at the perfect sky
and the same process happens basically
with every prefix
that that is
in this input until
finally the algorithm makes the choice.
OK, I think taking the first letters
S.I is the best choice here.
That's the closest thing I have here
to a word in the dictionary that I can
take away.
Now the same thing happens again
and it takes away the prefix ally
and it takes away the prefix Kidwai
and so forth and so on
until finally it
reaches a score
for similarity to the
two, a concatenation of words in the
dictionary for this domain
name.
Now we get the
same calculation for the next
domain name and the next domain name
and the rest of the domain names.
And are all averages out to find
the final measure of the closeness
to concatenation of dictionary words,
of all the domains that were relevant
to ten domains that we expected and
decided to take a look at.
So we now have our
final list of features, we
have 28 domains pointing the same way.
We have electrical devices of zero point
six and ability for opponent's
ability. The fancy of nine word lexical
devices, I remind you, is the feature
that we built here and the president's
ability devices based on their pairs of
letters and how frequent they are.
Now, the algorithm
is going to be looking closely at those
features and it's going to look at the 28
domains pointing the same way and
say, well,
that's too much.
Twenty eight domains pointing the same
way is really too much.
Everything more than five raises alarms
already.
So it says that's excessive.
Now, as for the
closeness to a concatenation of words in
the dictionary, it looks at the value.
Later, I'm going to tell you where the
parameters for the classify came
from. It looks at it and says that's
also excessive.
It's not close to
concatenation of words in the dictionary
at all.
And finally, it's going to look at the
pronunciation of the how
likely the pairs of letters that appeared
seem to be.
And it's going to say that it's
actually reasonable
because we got the value of zero point
nine and anything
up to one point five is actually
reasonable. So lots of the mines,
they look like gibberish to our future.
But the big Romare future
looked at this thing and said, OK,
seems fine.
Why is this?
I imagine some of you have guessed it's
quite possible this picture was generated
by catchable.
So now this classifier is going to look
at the domain collusion and lexical
defensive, the fancy and say,
OK, Mr. Pronounceable, you
said that this was reasonable, but I have
another feature now that I can rely on,
and this is definitely Dejiang.
So this
was the short demo of how
this thing works.
And now we're going to look at
some pretty graphs.
And now we
we took 10000
pickup's out of
checkpoints just to see how
the data looks if we map it across the
features that we have created.
And these are your 10000 pickups
mapped across the closeness
to concatenation of dictionary words
and the maximum domain name collusions.
The number of DNS requested got the same
response. Now, if the lump there
to the upper right seems suspicious to
you, then you're probably right,
because when we took some test samples,
like a hundred samples just to test the
waters and we
labeled them by hand, the
samples were in the lump on all the clean
samples aligned neatly
across the vertical
over there to the left.
And if you want a visualization
of the classifier itself
that I promised you, then I'm
going to tell you how it was generated,
actually. I tried all sorts of machine
learning algorithm and tried Gaussian
mixture models and also that.
But so far I got the best result
by just looking at the testing data
myself. So really, this is a case
of machine learning, a subclass
called Been Learning, and it kind of
bummed because I really wanted the
proper machine learning algorithm to make
sense of this data and I'm still looking
for it. This is a classifier
that it was not generated based
on any of the data that you see most
against the classifier.
That's the test data.
I mean, it's the 10000
pickups that we took.
It generated the parameters
to generate misclassify based on our
traffic captures.
Of course, you don't test your classifier
based on the same samples
that used to generate the classifier.
Then you're going to have overfitting and
that's not cool.
And this is how the
classifier looks.
And now I'm going to talk a bit about
the future of this
project and what needs
to be done further on it.
Now, first of all, more testing because
testing on 100 samples is nice, but
I bet that there's a lot of
surprises that the JSF have
up their sleeves that will require
this project to evolve and
get better features and fine tuned
features to be better.
Actually, we're going to touch on one of
those in like two slides next.
As I said, more machine learning I
want.
I want to be using an actual
machine learning algorithm, even though
this finely tuned tune by hand classifier
has worked, when I say I think you saw
the graph, you saw that it was a very
reasonable conclusion to draw from the
data.
And finally, there is gibberish
detection, one of three, because
as it turns out, the main generation
algorithm, some of those at least,
have already anticipated this kind of
analysis that we have performed here.
And you have deejay's, such as the one
used by the Matsuno and malware
that generates random
domains by concatenating concatenating
words from the dictionary.
So the PIGRAM feature in our features
is going to do nothing to detect
this kind of, uh, how
could we evolve to defeat
this? Well, I'm just trying an
idea out there. Maybe we could take a
look at the words from the dictionary
that you actually found and make some
kind of semantic comparison to see how
likely they were to actually appear
together in the same domain name.
I mean, Panter and asphyxiation.
I guess I could see it a good name for a
rock band.
And so
how about undetectable gibberish?
Can this arms race eventually end
in undetectable gibberish?
Well, my personal opinion
is that an undetectable lump
of gibberish is a contradiction in terms,
because the idea with undetectable
is that it resembles the legitimate
distribution of what you expect to see in
the real world of every conceivable
feature. Now, if something resembles
the real world across every conceivable
feature, it's not going to look like
gibberish to you.
But that doesn't mean that we're off the
hook, because while I think that
undetectable gibberish is an
impossibility, undetectable
auto generated domain names are very much
a possibility.
I think that in theory,
Grothus could eventually create an
algorithm that generates random domain
names that are very
much like domains that you see out there
in the real world across every feature
that you could conceive of.
And it's not going to run
into any issues, because even if
you put that set of constraints,
eventually it's going
to be the space of possible domains
is so large, even
with all those constraints, that there's
plenty of space, therefore, to generate
the domains to prosper and flourish.
But, you know, that's all theoretical.
And in the future,
in practice, you have the total
gibberish just
running amok. Today, you have quite you
were running amok today.
And at the top end of the year, you have
the dictionary concatenation basically
just running amok today.
You know what? Let's first force all
the judges to actually use undetectable
gibberish. And if we do that, I believe
that we will have done enough for that
thing. Yes.
Then we can think and what we can do
from then from that point on.
Now, your next question regarding
this nice classifier that tells you
whether traffic cultures are containing
T.J. or not is can I have it?
The answer is yes.
There is the address of the GitHub
report. And I really hope that the
people try to use it and say,
oh, my God, Ben, it doesn't work for me
at all. And it's useless to prove it
because I want to prove it.
I want this thing to work and be
available for anyone who wants to
detect deejay's in their pick-ups.
So we are we can
now summarize this whole journey and
we can say that the deejay's, our pain
and the automatic detection of the J
helps. But if it is done safely
and it gets confused by strategic
placement of vowels.
But if you do it Snavely, it gets less
confused by strategics basement AVOs
and becomes equipped to 201 domain names
like Tumblr and
undetectable auto generated domain names
may be a possibility in the future.
But you know what? First, let's focus on
the juice then, and then we can see what
we can do next.
So thank you. And are there any
questions?
OK, thank you to Ben
for your very nice talk,
very interesting talk, if you have
to leave now, do so quietly,
please, so we can have a nice
and informative Q&A session.
Thank you very much.
So we have a few minutes of time,
a few not not so few minutes.
We have a lot of time for Q&A.
So if you have any questions lined up
at the microphones
and we will also take questions from
the Internet.
So leaving
is OK. Talking is not unless
you step up to a microphone.
Thank you.
So we will start
at the front from
my side. Left from your side.
Right microphone.
Hi. So thanks for your talk.
I was wondering, you had lots of issues
with the dictionary because of its size
and so on, and then
you built all these hacks.
But I wondered whether you thought
about getting rid of the dictionary
altogether by including sellable
information, for example, because
I also think you're rather interested
in possible words of the English language
and the words that are listed in the
dictionary. So, for example,
Mandalorian or something like that
is probably in
a dictionary about Star Wars, but
probably not in the dictionary used.
But it is a possible word of the English
language by the rules
for the syllables.
So, yes, I just thought maybe
you could get rid of the dictionary by
using those syllable information,
so.
Well, actually, there have there
has been in one paper
on the subject that used exactly the
approach that you described.
Well, more or less by stemming words
and trying to look at morphemes
and so forth and so on.
It was very interesting.
Their particular attempt didn't
go so well, but I think it's
a good approach. I don't know about
getting rid of the dictionary completely
because I'm not very convinced
about how the more the
word stems or the syllables or the
items that you are proposing are
going to be at reconstructing words from
the dictionary.
But you know what?
I think it's an avenue worth pursuing.
If actually this could help
downsize the dictionary, then it's
something that I'm interested in looking
into and doing, because right now the
dictionary where is like ten
point seven gigabytes and anything to
make it smaller is very welcome.
So thank you.
OK, to the left front
microphone, please.
I was wondering how you managed domains
like electricity or things which are
valid, but which aren't in the
dictionary.
Things that you mean like
an SkyCity, because it's not
even like a word in the
dictionary at all?
Yeah, that's that's a
really good question.
You're not not every domain
name is going to fit into this part
of domain names that are
like a word in the dictionary,
which is why this thing still needs
to be tuned.
And more approaches
for recognizing
valid domain names need to be added
to it. Looking at the top domains.
I mean, that's a given.
And you see what
let me ask you a question.
What sort of approach do you see
that would have foreseen from first
principles that exclusivity
is a legitimate domain name?
That's kind of an issue.
So I think if things like
exclusivity there will ever have
to be manually
waitlisted or you could have a
specific I don't know.
I really don't know how you could see
in advance that it is legitimate
except, you know, taking
this apriori knowledge and applying it.
OK, the real rights microphone,
please.
How are you dealing with Kuniko demands
as well as
some demands in the U.S.
and other countries that are look like
English or gibberish?
China uses a lot of strings
that we wouldn't necessarily recognize.
And have you looked at other properties
of these two domains, such as
time of first use, you know?
Has anyone gone to these domains before
as well as how close,
how more recent?
Everything registered?
OK, so first of all, regarding the
languages issue, this project
was specifically about English.
It is a work in progress and the proof of
concept. I specifically physically fought
about that.
And if this thing is actually going to
be used and is going to function
properly, I am interested in expanding
it to be able to handle other languages.
Now, can you remind me if you have a
question?
So yeah,
pwning code or
domain strings that are English
characters but are used in countries
or popular in countries that it's
not English words.
That's what I said.
Then there are other characteristics
of these domains.
For instance, had they, had they ever
been
right, you know.
Right. And only registered.
Yeah. Yeah I remember now.
Well plenty of projects trying
to solve this.
This problem in the past have relied
on this sort of thing, keeping
sort of ongoing intelligence
operation that will
be able to tell you whatever the
domain is, grieving and very suspicious
based on this sort of intelligence.
When was it registered?
When was it taxes and so forth and so on?
These are all very useful features.
I specifically decided that
there were a lot of scope for this
project and this project should join
hands with any
sort of engine making use of
an ongoing intelligence operation instead
of reinventing the wheel and implementing
that sort of approach.
Thank you.
OK, Angel, other questions
from the Internet.
I'm just looking at them right now,
but it seems
the Internet the same question as
that's convenient.
So I guess the next question
from microphone right up front,
it sounds like the
DJ always with some output,
some sort of stick poetry
that will make your system have to
identify today's poetry, but also
in like 400 different languages,
such as the dot com.
So will allow you to register it.
That would be a problem.
So my question is, in
what place in the DNS ecosystem
do you believe that your system will
have? I mean, where exactly
do we have a use for this?
OK, so actually
this was conceived in an entirely
different context than what I described
in the first few slides.
This was conceived as a feature that
you can compute on a sample in
a sandboxing context so you can
do machine learning operations and
so forth and extract a feature
that you can look at the sample and
say, OK, it's deejayed.
That's one of the features that I can
look at and use.
But as I mentioned
in one of the first slides, I think
that the pinnacle of what this film could
do is if the code were optimized for
performance in that kind of context,
which it currently is not, it could
sit on the firewall and throttle
traffic before it manages to get out of
the network.
OK, and that's another question on the
rear right Smitt microphone.
Yes, you mentioned that you tried other
machine learning models for this.
Did you try a Markov chain classifier?
Actually, that one I haven't tried.
Markov Chains are saying, OK, I'm going
to try that tomorrow.
I'm aware of some other research that
does DedÃ© classification using Markov
Chains. And if you're there's a couple
of advantages to this.
So if you're concerned about Bagram's
not being enough, you can get extended to
a third order or fourth order.
Markov chain assumption with smoothing.
And you can also, you know,
look up time is is linear,
so it's pretty fast.
OK, I'm going to have to take a look at
that.
Yeah. Thank you.
OK, on the left rear
microphone. That's another question.
Did you try using using filters
for nougats because they might you'd use
to look up time significantly to
use what Blum Footies
Bloomfield's.
No, I have not looked into that to
lots of useful suggestions to this.
I am hopeful that.
OK.
Oh, we have time for another
couple of questions, so I'm looking at.
Nothing from the Internet.
OK, last question, maybe to wrap
it up from the right's
front microphone.
Thank you.
I liked your presentation very much.
I'm all for data science and
not experience with the network
perspective, OK?
I think it looks like a cat and mouse
game. And I think the from the
methods perspective, I think it's
a losing game. So I would like to hear
more about the motivation at the
beginning.
The the the standard process that
is described, how it is handled with
the middle management and the reports
that it was done in reverse engineering
of their statistical methods
involved to today that are
the same cat and mouse game that you are
only playing it better.
Or is it just
the scheme, which I believe personally is
a solution?
I agree with you completely.
That's what I said. In theory, it really
is a losing game because all forms
of deejay's are going to evolve
eventually.
They haven't the will.
And as I said, my personal
analysis, and it seems that you agree
with me, is that eventually,
theoretically, they win this game.
Now, security is full of
such games where the attackers
theoretically eventually win because
the mission, the burden on the shoulders
of the defenders is like some
eventually in the worst case,
worst case, it's equivalent
to some ENPI complete problem
or is generally in theory, in the worst
case, impossible.
But while you are correct,
I believe that we should be focusing
on what's happening right now.
As I said, OK, it's a losing game right
now. But besides
trying to look aside and think, OK,
what game changer can we bring in
here to make this ultimately not a losing
game? We need to do our best to push
forward and make it less of a losing game
temporarily, even if eventually
we know that in theory
we're going to lose.
That's what I believe.
OK, Ben.
The rest of us give a warm applause to
Ben, and thank you very much for your
talk.