*36C3 preroll music*
Herald: So, have you ever wondered how to almost perfectly fake an email? Then you
might be actually in the right talk here. We have our next speaker. Andrew, who is
currently working for the National CERT of Latvia as a security researcher. And he's
going to talk about e-mail counterfeiting and strategies for modern anti-spoofing.
Stage is yours.
*Applause*
Andrew: So. Greetings. I'm Andrew and I work for Latvian National CERT. One of our
current goals is improving the state of email security in our country and which we
mostly do through raising awareness about this issue and communicating best
practices. And of course we are not the only organization that is doing that.
There are many more CERTs in other countries and there are various
nongovernmental organizations that are doing the same. And commercial entities.
However, so far, frankly speaking, our collective progress has been quite
underwhelming. So for example, here is the one stat which is the usage of one
specific technology, DMARC, which as you will learn in this talk, is quite
important and I hope that everyone will start using it. So on the left. There are
twenty thousand domains across all the world which are important domains for
important organizations that truly should know better. And on the right side we see
the top 50, top 500 EU retailer domains and across both of these groups two thirds
haven't even configured DMARC yet. And out of those that have configured majority
hasn't enabled strict policy yet. So if there is just one key takeaway from this
talk, I hope that it will be that everyone should start using DMARC. It is important
to use it even for domains that are not supposed to send email. So, one
explanation for these low adoption rates, I think, is that, there are seemingly too
many competing technologies. This is the contents for my talk. And I really tried
to do my best to trim it down. But as you can see, there are three abbreviations, well and
SMTP and out of this, SPF, DKIM and DMARC actually two are i don't even remember the
whole name for them. But still, they are all important. And of course, this problem
that there are too many buzzwords, too many technologies, and it's not clear
which one which ones we should use, it's not specific to email. And we have this
across the industry and, ah, security industry, i think by now we have found at
least one way to solve it. And it is penetration testing. So when the
penetration test has been run properly and the results have been published, then we
can start talking. We can shift the conversation from talking about whether
your organization prefers technology A or technology B we can instead start talking
about the questions that really matter, such like: Is it possible for someone for
some third party to spoof your organization's e-mails and to send such
e-mails to your, for example, customers or your partners or to media organizations in
such a way that they will think that the emails really came from within your
organization? So that's why penetration testers are the key audience for this
talk. However, I hope that any blue teamers in the audience also will find
this talking interesting. I'm sure that you already know all the basics about the
email and about these technologies, but looking at the problem from the different
side from attacker's perspective sometimes can really put things into perspective. It
can help for you understand what you should focus on when protecting your
environment. And finally, the SMTP protocol. The technology that runs
underneath our e-mail conversations is actually relatively easy to understand.
And so. And also the lessons learned from all of this journey from SMTP, how it
became and how it's possible to spoof it and all the technologies that are trying
to prevent spoofing. I think it's a interesting case study and it should be
interesting to follow even for people who are new to e-mail. Um, finally. Threat
landscape. So email security is quite a wide topic. And so today I will only focus
on one small part of it, which is successful spoofing of e-mails. Tampering
attacks. And I know that many, penetration testers already, incorporate some part of
phishing or spear phishing, emulation into their engagements and. But as far as
I know, they mostly do it from the, social engineering perspective using such tools
as a social engineering toolkit, for example. And it's, uh, I don't want to
argue, though, that it's important to do that and to demonstrate to the customer
that what risks are in regards with social engineering. However, I think you're doing
a disservice to the customer if that's the only thing that you are testing from the
email perspective, because from the customers, from managers perspective that
are reading your reports, if they only mention social engineering attacks, then
the logical conclusion is, that the best way to mitigate these threats is by
educating your personnel, especially those that are least technical, as you will see
in this talk. There are quite a lot of attacks and many organizations are
susceptible to them, which are much better than that. And no amount of user education
will help here because we can't expect users to check headers, for example,
manually. So we actually need to improve our e-mail infrastructure. No way around
it. And finally, before we move on to actual technical stuff, there's a little
secret, which I think might help people that are not working in the email industry
understand why we have such problems and is that, for email admins historically,
um, they value availability of their system and reliable reliability much more
than security. And that's because that's not an ideological decision. It's a very
pragmatic one. So, for example, if you are an e-mail an email admin in an
organization and some of your customers stop receiving invoices, your management
will find you and will inform you about it and will ask you a really nicely to fix it
as soon as possible, even if it's not your fault, if it might happen that the problem
is on the other side of the email. Not on your server. And the for example, if,
other example, if you, if some of your, some of your employees can't receive
e-mail soon enough, for example, to restore the password or to verify the
email or to use multi factor authentication token and they can't log
into some important systems again, they will find you on though you will need to
solve that. But if your system is has some security vulnerabilities, if it's assessed
susceptible to spoofing attacks and so on, then not users, no management will
normally notice it. You might not not notice it, but you are. You have this
vulnerability. So that's why obviously penetration testers are important. Okay.
Now we can finally start talking about the technical stuff. So and we will start with
the short introduction to SMTP protocol. SMTP is the protocol that underlies all
email communications and it's actually pretty easy to follow. So here's a data
flow of what's happening when one person sends e-mail to another person. For
example Alice is sending to Bob and they're using different they are working
for different companies. They use different domains. So what's happening
here is that both of them would say use email clients such as Outlook or
Thunderbird. And Alice is sending email. It's going through this protocol SMTP to
Alice's mail server. But important to note is that this is an outgoing e-mail server.
Usually organizations will have two types of servers, one for incoming transactions
and one for outgoing and for smaller organizations it might be one server, but
again, it's important for penetration tester to think of this as different
systems because they will have even if it's physically one machine, it will have
different configuration for outgoing mail and for incoming mail. So as a penetration
tester you need to check both of them. Okay. Now, when Alice's server tries to
send email to Bob's server, there is sort of a problem in that the server needs to
somehow automatically find what is the other server to send the email and it is
done through this blue box MX which is DNS specific DNS record type MX. So that's
something that is maintained by Bob's organization. So Bob's organization, if
they want to receive e-mail, they create this DNS record. And I say that. Okay. If
you want to send e-mail to us, please use this particular server. So it should point
to Bob's server. And Alice's outgoing server knowing Bob's incoming server
address can communicate to that. And then later, Bob, will receive its e-mail. So
the part that we as penetration testers will be trying to breach is actually
between Alice's server and between Bob Server. And then we need to think about
the second example, which is the opposite way. And you might think that it's a
pointless example because we are just basically changing the direction of
traffic. But the important part here is for us as penetration testers to
understand that our client only controls part of this transaction. If our client,
let's say, for the rest of this presentation is Alice or Alice's
organization, then in the second example when we are sending mail from Bob to
Alice, then we'll be sending emails only. Basically, part of this transaction will
go through Alice's servers. In the first example, if we were sending email from
Alice to Bob, it wouldn't be so. So if it's a bit confusing, that's okay. We will
return to that a bit later. And finally, there is a third example which looks
similar, but not quite. And that's if Alice is communicating. Alice is our
customer. And if she is communicating with her coworkers, which are using the same
organization, same e-mail server, same domain. In that example, again, there will
be to at least logically two email servers, outgoing server and incoming
server. But both of them will belong to our customer. So right now, if you are not
familiar with e-mail, you can. It's just interesting to try to think which of
these scenarios, three scenarios, which of them are easier to protect? And a bit
later we will see how it's actually happening. Okay. And then we need to look
at what actually is being sent, when email is being sent. So again, it's using SMTP
protocol and it's really nice protocol you can. As you can see, it's just text. So
it's plain text protocol and it's very easy to play around because you can just
open telnet connection to the right server and you can try writing down the commands
just with your hands. So you can try mangling something or modifying or trying
different, different, different types and see in real time how it was going on. So
on the left side we see here two parts which are defined by SMTP. So first of
all, there comes SMTP envelope, which basically you connect the server, say
hello, then you say what. Specify the sender of email and recipient. "mail from"
is sender. Recipient is Bob, for example. And then the second part starts with data
and ends with quit. And that's the part which is called Content/Message. So just
if you want to play around with it, a bit more, this is defined by a different
standard, which is not that important for penetration testers but if you want to
look into details and it might be important. And this internal message,
which is called either Content or SMTP message, it again, it contains two parts.
One is headers and another is body. And I think some people might not be familiar
with email, but probably everyone is familiar in this audience with HTTP and
this looks quite, quite the same. So easy to understand. But the interesting part
here is that you might have noticed that we have Alice's and Bob's addresses twice.
Right. For example, Alice's is specified on the second line "mail from". And then
we have the same address. alice @ her organization in "From" header. The red
ones are the headers. And the same goes for Bob. So why is that? Well, it comes
down to how we see e-mail. I as a normal regular person who has used email in
past quite a lot, i usually see them as described on the left side, which is a
sort of postcard. So on a postcard there is someone who has sent it. The sender.
There is the recipient. That's usually me. I'm receiving. And then there's some
message. So at least that's how I perceived it before I learned a bit more
about it. But email admins and the standard bodies, they see this situation
as the one which is shown on the right, which is. There is an envelope and inside
the envelope then there is this message or a postcard maybe. So you have two
addresses in this scenario. You specified the address from and to whom you are
sending the envelope, which is the part that post office, for example, will look.
But post office won't look generally inside your envelope and inside the
envelope there is another message, and that is the internal message is actually
meant for a recipient. So actually, you could do even more and you could even put
the whole envelope with the message of the postcard inside another envelope. And this
sounds crazy to me as a regular person, but actually e-mail allows that. And in
the RFC the standard document, there are some examples why that would be necessary.
Why why such why such things are allowed. But but they are confusing. And so as a
result, it is the here in this first example, we see that we generally we are
specifying the same address twice. But as a penetration tester the question that
we should be asking is: So is that required, actually? Is that always true or
is it just like a wishful thinking? And it's actually wishful thinking. So
standards specifically do not say that you should be specifying the same address for
recipient or for "From" from the sender on the envelope and inside a message. So you
could actually tweak them and send different, different stuff. So, actually,
there are much more headers than what I showed. The ones I showed I think are just
the ones that we all have experience because even if you are just using e-mail,
that's usually the stuff that you see or see the date, you see the subject, you see
who has who sent you something and to whom it was sent. Usually yourself. And there
might be, of course, more recipients. Oh, yeah. And the question then another
question is: Which one is actually, if we have specified for some reason by accident
or especially if we have specified different addresses in this envelope in
the message which one the user will see the recipient, it's actually the header.
So inside that the message is the one which is intended for the user. OK. So and
as I was saying, there are actually standards allow a bit more headers. And
actually 3 headers "From", "Sender", "Reply to" which are semantically really
close and in the standard it's actually explains when you should be using which
one. And the funny thing for me is that, for example "From" header, which is
usually the one with that we see it might contain . By reading the RFC you will see
that you shouldn't have more than one such header, but the header itself might
contain multiple addresses. Personally, I've never received an email which would
come from different people, but that's allowed. But the important thing to
understand here again is the backwards compatibility that I mentioned before. So
even though standards explain how you should use the each header and that you
shouldn't have more than one of each of these headers in practice actually can
send malformed email. You could send email with multiple headers, the same header
"From" header multiple times, or you could send header which does not contain "From"
but contain "Sender" according to RFC that's incorrect. But in practice it will
work. Most organizations, most e-mail service will try their best to pass your
completely malformed email because they really are concerned about lowering the
support costs. So if something does not work, then you will come to them. So it is
better to make that everything is working most of the time. Of course, for
penetration testers that means that you can play around with this because there
are different implementations and it's exactly which header, for example, if you
have two headers, will be shown or will be used for some algorithm. It depends on the
particular implementation. So because there are so many implementations, they
are interconnected in different ways. You could and you should as a penetration
tester try various things, for example, add the same header multiple times. OK.
Now that we have covered these basics, let's actually look into how you would try
to spoof an e-mail, for example. Yeah. And here we are again, we are coming back to
this diagram that we have seen before. And for example, in the first example about
Alice is sending email to Bob. Let's say we are, Chuck. So we are a third party. We
are penetration tester licensed, we have an arrangement that we are allowed to do
this and we are trying to send spoofed e-mail to Bob. And in this example, we are
trying to spoof Alice's message. So our intention is that Bob wants Bob receives
email. It should look to them, to the Bob, that email was sent by Alice. So risk for
this. Okay. I will not cover the risk. I think you can imagine that. So, for
example, you could do fake news is one of the problems that we have seen in Latvia.
It's one this was used against government bodies. And when someone sent a fake news
e-mail to other people, organizations and so on, and were trying to impersonate some
some government person. And of course, you could could imagine yourself how it's not
a good thing if you if it's possible. But the interesting thing here is that even
though Chuck is doing attack, it depends on your perspective. It might look like
attack on Alice or on Bob. But in this case, email won't go through Alice's
systems. As you can see, Chuck is sending e-mail directly to Bob's incoming
server. Now, there is a second type of attack that will be looked at. If we are
sending e-mail in other direction from Bob to Alice. And our customer is Alice. So we
are testing Alice's server. And in this case, we are trying, again we are Chuck.
We are sending e-mail. In this case, e-mail will go through Alice's systems. So
interesting question is, which is easier to protect. It might seem that since in
the second example, e-mail is actually going through Alice's systems, that means
that Alice has more power to do something, to do some additional checks and balances
and so on. But actually, as you will see in the future, it's easier to protect the
first example. So even though our customer is Alice, we're trying to protect Alice,
but it's easier to protect in practice this example where someone is selling,
sending e-mail, trying to impersonate Alice. Okay. Oh, yeah. That there is the
third example, which is if Alice is communicating with her colleagues inside
the same organization. Again, we are Chuck in this case. Again, we will only send the
e-mail to Alice's incoming server. Not to outgoing server. Right. So important thing
to note. And again, in principle, this third example is the easiest to notice,
because Alice's organization presumably knows that her e-mails always should come
from this particular outgoing server. Right. Like if we are sending e-mail from
Alice's colleague, then incoming server in principle should have all the power, even
without any standards and stuff like that. But in practice, sometimes actually quite
often there will be a specific whitelist for Alice's own organization. So some
checks won't happen if incoming server for Alice is receiving email, which is coming
from, again, Alice. And by the way, there's this example. We've seen that for
the past few years. I think it's not specific to Latvia. So here, for example,
is Canada and others,if you can see. This are these emails which are fake like
ransomware stuff. Basically, they are telling you that they have hacked your
computer or your email. In this case, and they have arranged all sorts of financial
activity or have some blackmailing you. And please send them the money. Your
money. I mean, your money in bitcoins to their address. So, these e-mails.
Interesting part about these e-mails is, that they are usually in order to prove to
you that they have access to your e-mail account. They are sending e-mail from your
address to your address. So and for many people, that works. So they see that
someone has hacked their account, obviously, because they've received e-mail
from themselves. So as you will see a bit later, it's actually easy to spoof such
e-mails if there haven't been any protections, haven't been put in place. So
the important thing, I hope that now no one in this audience is falling for such
scam. But if you have some friends or colleagues that have contacted you and
told you about such e-mails that they have received. But one of the things besides
checking the passwords is starting using more effective authentification on is a
just maybe you could tell them that they should contact their email administrators
or IT team and ask them about anti spoofing protection, because obviously if
they are able to receive such e-mail and it's not filtered, something is wrong.
Okay, and now let's see a spoofed SMTP conversation, so that's example similar to
previous one. But in this now we are actually Chuck. So this is sent by Chuck
to Bob, but we are pretending to be Alice. The question is, can you see the
difference how this is different from from the previous one? And it's hard to see the
difference because there is none difference. That is the same conversation.
So the point here is that SMTP protocol by itself it actually it doesn't have any
protection. So, yeah, you could just for example, if you are that guy that is
sending the fake ransom letters, you can just write down this text and just dump it
to telnet and it will work for many organizations. Not for all. And of course,
the email admins know this stuff, know that SMTP is not very reliable in this
regard. That's easy to spoof and so on. And there have been many attempts to add
some protection, just like ad hoc way. So no standards just to ransom, add some
additional filters and stuff into your own mail. And some of these protections
actually break RFC. If you read it, but who cares? Like RFC is not a sacred text
or it's. I absolutely approve this, for example. So yeah, go on. But the problem
is that there is not enough information. So if you think back here, if we are Bob
and we are trying to protect our systems. So we are Bob, some system administrator
probably or Bob is a sys admin and we are trying to add some additional rules and
stuff, then what actually can we do? So one example that I listed here is doing
this SMTP callback, and that means that we are just the when we receive e-mail from
Alice, we actually check does that email exist at all? Because many spammers, what
they will do, they will just send e-mail from non existing emails and it will work
by if you are just running raw SMTP server. So SMTP callback is basically you
are when you are receiving email from, for example. Alice, you are trying. You are
running, spawning a separate process which will try to connect back to Alice, etc.
And it will try to send email her. If a server says that. Yeah, that's okay. Such
email exists and so on. You are not like, you actually stop the conversation. You
don't continue with sending email, but then your system can automatically find
that actually this e-mail really exists. So another way to do this is through
checking this "Hello". And this is the first line and the first line, it's,
normally it should tell you the hostname of the server that is sending email.
Interesting part. So according to RFC again, you shouldn't check it that you
shouldn't verify. And if it doesn't, if it's a random thing, you should accept
email still. But what many servers will do is they will try to verify that. First of
all, this hostname, which you are telling that you have this hostname. First of all,
that it really points to the same IP address and then they do the opposite. So
they will take IP address and try to run a reverse DNS PTR query and they will try to
find whether that IP address really responds to this hostname. So again, as a
penetration testers we should be aware of these protections, ad hoc protections,
because they are if you don't know about them, you will try running something and
it won't work for you. But they are easy if you are aware of them and if you have
to identify that this organization uses them. They are easy to bypass so that they
don't offer good protection. They are meant to protect from mass abuse from
spam. OK, so SMTP, as we've seen, by itself does not do does not offer any
protection. So which additions to the protocol actually can we use to protect
ourselves? One of such protocols is SPF. And what SPF does is it's trying to be
like mirror MX system. MX system is the one which basically Alice can use to
Alice's server can use to automatically find the server that belongs to Bob and
its incoming server. So. SPF is the opposite of that. So that's an idea is
here to run the system automatically on the Bob's incoming server. And now when
Bob receives the e-mail, they can run again DNS query and they can find what IP
addresses actually should belong to Alice's outgoing server. Right. So it's I
think it's easy to understand it's actually a meaningful way. It sounds
meaningful addition. And the one field that is checked in this example is this
envelope sender. OK. And here's an example of minimal SPF syntax and the as we can
see. I think it's easy to understand, even if you don't know the syntax is it lists
IP address, which is IP, should be IP address of outgoing server, legitimate
outgoing server. And then it says this "-all" which again, is easy to understand.
In this case, it means that that's the only one. So if you receive a message,
message comes from this IP address. That's cool. I accept it. If it's something else,
then just drop it. And there are multiple ways to specify the IP address. You could
just specify the IP address. You could specify IP subnet, you could specify DNS
hostname. So it's just for admin. So basically for a penetration test, it
doesn't do much different, for admins it's just easier to maintain these systems. And
then there are these qualifiers, qualifiers. This is what's something which
you put before the methods. For example, here in this example, IPv4 before doesn't
have any qualifier. There's no plus or minus or something. That's because plus is
assumed by default. So by default, everything that is listed in SPF record
will should the match some legitimate SMTP server, outgoing server. However. There
are other options you could use minus which is fail. And that means if something
matches this record, for example, minus all is the one which is the most often
used, it means if it matches this one, so that's usually the last one, then please
drop the mail. It's not real. It's it's fake mail. And then there's this third
option, which is softfail, and that's meant for testing period. So when you are
just starting to implement SPF, there might be some. So the problem is that you
might forget, for example, to add some SMTP servers. So because you haven't done
it before, maybe you think you have only one SMTP actually outgoing server. But in
fact, you have multiple of them or multiple ways to send e-mail. So in that
case, if you were to start set that SPF record with "fail" strong policy, then
your users won't be able to send the message anymore. So that's why testing is
good. However. Here are some other examples, a bit more complicated. One of
them is was include. So instead of defining the policy yourself because
you're using third party, for example, Google in this example, and then you will
just include whatever Google has published. And the interesting thing is
this usage of SPF. If we just if we just look at the amount of domains that have
defined some sort of policy, that the number looks pretty okay. I guess that's
for example for most popular domains that's around 70 percent. But the problem
is that the majority of them are either poorly configured or they just use the
softfail option. And what softfail practically does is nothing. You still can
even if there is policy with softfail, you can in most cases you can spoof your email
and it will still go because the recipient side will think that it's just in the
testing mode. You shouldn't drop e-mail automatically. Yeah. So. Actually, the
percentage isn't that great. However, the most important thing for us as penetration
testers is to understand. So what do we do when we see this SPF. That means that now
we can't spoof mail and. No, it does not. That it's game over for us. We can do some
stuff. So first of all, is this softfail that I mentioned. And that's basically you
have some rules, rules, rules, and then in the end, you are putting typically just
this softfail at all. So if we as a penetration testers will try spoofing from
some unknown IP address that hasn't been listed in the previous rules. Then do
nothing. Do nothing. I mean, don't drop email. That is good for us, right? That
means that we can actually spoof just in the same old way and it will mostly go. So
the one great one note here is that some systems are you are not using just this
binary classification, whether something is good or bad, but they are trying to run
some scoring. And then it might be that even if you have this soft fail, they
won't automatically drop your e-mail, but maybe they will add some like suspicious
level to it. But important thing is that it's not automatically a game over.
Another thing is this include. So include is it very convenient when you are using
third parties. But the problem is that it's not what it sounds to some people, at
least even in the standard, it mentions that it was a poorly chosen name. And the
reason for that is that it's not a macro. So to understand what's happening when
this included, you shouldn't just copy paste everything from inside recursively
to the top level. It's not how it works. It will try running all the checks inside
this include. But then if it fails, it won't automatically drop the message. It
will go to the one level top and it will try running the other rules. So the
problem with that is that two cases that are the most common is that either if you
just forget to add this minus all to , or your system administrator who has
forgotten to do that. In that case, even if they include has minus all, it won't
work because I mean, it would because when the recipient will be checking it minus
all inside include does not mean the same as it does on the top level. And the
second would be if they have added all but did softfail all. And some admins might
think that. But that's okay because I'm including GMail and GMail has this hard
fail. Doesn't work that way. And then one, which actually is I think maybe the most
common case, is that something often you actually see this type of SPF records, but
there is lots of stuff inside there is IP addresses. There are these A records,
there is a MX. There is a pointer. Basically, everything that the admins
could think of and the reason is that the most commonly, they are just not sure how
it works. They're not sure what they should put inside. So, for example, one
thing that the point that out is if there is a MX record inside the SPF, most
commonly most organizations, unless they are very small and just have one server,
they will have different servers, different IP addresses for outgoing mail
and for incoming mail. That means there is no practical for this organization,here is
no practical reason to include MX into SPF because no, no mail should go out through
their incoming mail server. And another case might be that the admins understand
how it works, but it's really, truly their architecture is really messy and they are
sending emails from many, many different points, which is good for penetration
testers. That means that they are not well organized. OK. And then there's another
flaw, which is that granularity isn't very well suited. So the only thing you can.
There are multiple this record types. But all they do basically are resolve the IP
address. But the as you can imagine, in many cases, IP is not linked just to mail
server. So on one IP, there might be mail server and web server or database or
something else. And that means that as a penetration tester, you can exploit this
something else. Not mail server itself, because mailserver usually is pretty like
low key. There's not many vulnerabilities there. You just patch them and that's it.
But those other systems, for example, web, it's easy to exploit. In most cases. So
then you can elevate like in some sort elevate privileges by gaining access
through some other server on that IP address or IP range. You can start sending
mails. They will pass all SPF filters. OK. So one example is shared hosting, which is
the very common case and the problem with shared hosting is that. In this case.
Okay. You have IP address of SMTP server. Maybe that's server only used for sending
mails. But the server itself works not just for you. It works for many domains,
maybe hundreds of thousand domains. That means as an attacker, again, you can
exploit at least one of them, or for shared hosting you can just buy. You can
become a customer of that shared hosting. You don't even need to exploit anything.
And then you can potentially start sending email, which will look good as far as SPF
is concerned, just like their own. So. And the another one is this checking wrong
identifier. And this is probably the worst, worst problem with SPF. It is that,
as I mentioned before, the one there are at least two identifiers. Typically
envelope sender, the outer one, which lists the sender, and then there is
internal one, which is usually "from" header. But out of those two SPF only
checks, if SPF is the only technology that you are using, SPF only checks the first
one: envelope sender. And as I mentioned, in most cases, actual users that will
receive the mail, they won't see envelope senders. They will see this and this other
one "from" for example, or one of the other headers they mention. So this
behavior is fixed actually by DMARC, which is the technology that I mentioned. But
the majority of SPF installations, domains that are using SPF do not have DMARC, so
they are not protected by this. So even if their SPF is completely great for
attacker, it means that you only need to, what you need to do to pass SPF is a to
set envelope sender to something else. For example, your own controlled address,
which will pass all SPF checks. But then inside the "from" you can show the header
that will match this organization that you want to pretend to be. Okay. So then there
is another technology which is supposed to fix this and it's DKIM. As we have seen,
SPF is not enough. So DKIM. Sorry, the wrong letters, Domainkeys identified mail.
That's the DKIM and you don't need to remember the long name, just the short
name. And what it does, basically, it uses cryptography, which is nice, right? It's
math. It's hard to break for attackers. And what it does is it signs every mail so
every mail that is going out through the DKIM enabled server will get signature,
which you can, as a recipient, you can cryptographically verify. So as you can
see, how it looks is actually pretty hard to see because it's not meant to be
processed by humans. It's cryptography. It's meant to be processed by computers.
But the important part here is basically the yellow stuff is this cryptographic
signature. But the green part is what's called domain identifier. And the red part
is what's called. I don't remember how it's called *laughs*. But basically it's
idea is that you can have multiple keys for your organization, for example, your
organization might be sending mails from your original SMTP server, then you might
have a backup one or you might have might be sending some messages from Google or
some marketing campaign and so on. And then each of them might have different
"red", this parameter. The problem is and then the recipient will need to run DNS
query, which is the second example using this combination of green and red one. And
then they will get the public key and they can use this public key to verify the
signature. So it's sounds really nice. The problem here is no, another problem yet.
So how to use it? I think it's easy if you understand the public cryptography. So on
the sender side, you need to first generate public and private keypairr. Then
you publish the public part in the DNS. Then you use private key to sign each
message. Now recipient does sort of the opposite. They once they receive the
email, they figure out from this red and green part they figured out the correct
DNS record to run, run it, get the public key and then compare whether this public
key corresponds to the signature. So it sounds really nice, right? What's the
problem? So customers. Selectors, that's the name. So the problem with that is that
the selectors there might be multiple selectors as a DKIM when you are doing
configuration, you can select as many of this custom selectors as you want, and the
recipient doesn't know whether you actually should have used a selector and
what selector you should have used. So the problem is that while, if we are talking
just about the vanilla DKIM, modifying existing signature is hard for penetration
tester or for an attacker. But it's easy to just remove it because if you have
removed DKIM at all the header, the recipient doesn't know that it should have
been there because in order to check, they need to. So here, for example, in order to
check the signature, I need to know this green part. This domain identifier and the
selector which are part of this header. Right. So that's a huge problem. And that
means that. Yeah. That means that we can actually while we can't spoof DKIM itself,
we can just trim DKIM, send the message without it. And if the DKIM was the only
thing which protected this system, it will work. So it might not get the green
checkmark or whatever, but it will get to the recipient. So. And another thing is
this domain selector. Why do we even need to set that? Because the best practice, of
course, is that you have envelope sender equal to "from" header equal to this DKIM
domain selector. Right. So if you are if I am sending from Alice, then all three
should be Alice.org or whatever. The problem is that it's not mentioned in RFC
that that should be the case. So what exactly happens when it is not that way?
For example, on the right side there is some real domain which was using Gmail,
Google Apps, Google suite, and in that case the default by default Google suite will
sign all messages. But if you do not do your own configuration, it will sign them
with domain it controls, which is this "gappssmtp". And what it means is that
although technically something has been signed with DKIM, it wasn't signed in the
way that you can trace back to your organisation. It's something completely
else. What exactly recipient should do in that case? Should they just ignore it?
Should they reject the message or something? So the correct way would be not
to reject it, but just consider it not valid, at least not not a valid DKIM, but
it actually depends. So some validators will just see any DKIM, will validate it
and will say that's cool that matches RFC. So but now the interesting part. Modifying
DKIM, which I don't have time for. But the idea is that in some cases this is not
always but sometimes you actually can modify. The easiest part to modify in the
messages are headers because DKIM, since it's placed in headers itself, it does not
automatically sign old headers. There's like a chicken and egg problem. So by
default it only signs one or two headers and you can specify more headers that need
to be signed, but it doesn't happen automatically. So the easy part for
attacker is to add another header. If that's somehow helps you in your like
plan, then that's easy to do. You just add another header. An interesting part is,
although the RFC, as I mentioned before, mentions that some headers such as
"subject" or "from" should only be present in one copy. Actually you could add more
than one for example "from" header, and what happens in that case is pretty
interesting. DKIM will match if you have told to DKIM that "from" header should be,
for example, signed, then it will match and sign first "from" header from the
bottom. But quite a lot of software in our software email clients will actually only
display to the user first from the other side, from the up side. So what it means
is that the attacker can mangle or overwrite headers by just adding new
headers to the top. And the this actually problem is mentioned in the DKIM RFC and
the protection that they propose is this code Over-Signing or you can go and read
the RFC. But not everyone is doing that actually. And however, that only goes to
the headers. So sometimes that is good. Sometimes that's not good. Modifying
message body is actually much harder to do. Basically the naiv way do it through
cryptography, which we don't want to do. And another way is through this one
parameter, which is body length, and that's actually like questionable
functionality that DKIM has. Sometimes you can specify that the hash like. For
signing purposes, we shouldn't consider the whole body, but only first something
bytes. So that's actually useful in some cases regarding was a mailing list, but
for the most part that's not useful. And in practice, most email software does not
do this. If it does, then it is susceptible to potentially to this
overwriting body as well. You could add another mime type and then then modify
headers to show that different mime type and it will pass DKIM. So in this case, it
actually will show, for example, the green button or whatever, because DKIM, it will
be valid. So now there's the third technology, which is called DMARC. And
again, there is the full name, which is long, but in this case actually it means
something. There are two key words: reporting and conformance. Reporting is
the one which most admins are familiar with because that's how DMARC I think
often is being sold to them. Reporting means that when you have some problems in
this case, you actually get get to tell other side what to do in that case. So
basically you tell them to send you reports either once per day or every time
and so on. So for penetration testers, it's not that useful. Potentially we could
use that to understand what sort of configuration is running on the other
side. But the currently this functionality actually is not that widely implemented.
However, the other part conformance, it's actually really, really, really powerful.
What it does, that it corrects these major flaws that I mentioned in SPF and DKIM. So
first of all, DKIM had this massive problem that if you just strip down the
header, then the recipient has no way of knowing whether you whether there was
should have been DKIM in first place. If you are using DKIM alongside with DMARC
that fixes the problem, because DMARC specifies just that you have DMARC itself.
It means that you're automatically at least one of the SPF or DKIM should pass.
So automatically DKIM is like measure problem solved. The other thing that
changes is, it changes the semantics for SPF. Now, SPF, if you have both SPF and
DMARC, it means that SPF should be checked against "from" header. And as I mentioned,
that was the major flaw with SPF, because if you're using SPF itself, even, it is
the hard to fail mode and so on, it means that attackers can modify "from" headers
still and the recipient won't know any better. So a minimal example of DMARC is
really, really small. And I think it's easy to understand. You have just a DMARC
reject. You need to like find out the right place to specify. But it's easy and
all you have to do is create this one DNS record. And the benefit for that is even
if you don't have DKIM and DMARC, if you have created. Sorry if you don't have SPF
and DKIM, but you have created DMARC, effectively what it means is that this
domain should not send any mail because for recipient to consider a mail valid at
least SPF or DKIM should be valid as well. If they are not, then they can't be valid.
So in fact what it means is that most domains out there should consider enabling
DMARC. That's just the right thing to do. OK. So there are more tags. So in the
wild, these DMARC records might be much longer, but it's not of much use to
penetration testers. So important part here is again, this is this policy which
can be three values "none", "quarantine" and "reject". And if it is "quarantine",
that means if the, if there is a failure, the message should go to the spam folder.
If it's "reject", it should be rejected outright. And if it's "none", it means
it's in investing mode. So and this is the picture that I showed in before, which
shows that actually even though DMARC is really like the best technology out of
these three, it's not really widely used, unfortunately for defenders. Quite a nice
fact for all penetration testers out there. That means that you can, in fact
spoof most of the mails out there. Okay. So how do we work around it? Sorry. So.
What happens if actually someone has implemented DMARC? Does that mean that now
penetration testers can't do anything? You don't don't even need to do any research?
No, it doesn't. So in practice, if someone has implemented both DKIM and DMARC, but
not SPF, so they have only two of them. That's a really cool combination. DKIM is
pretty powerful and the major flaw that it had DMARC solves. So this combination is
really cool in theory. In practice, the problem is that in order to protect your
own mails, the recipient side should validate both DKIM and DMARC and
unfortunately, quite a lot of software still does not do that. One such software
is Microsoft Exchange. And even if you are not running Microsoft Exchange, chances
are good that some of the partners that you are communicating with are running
Microsoft Exchange, and by default it doesn't have any functionality to parse
DKIM. So in fact, most systems still need to enable SPF just for practical purposes,
which is good for penetration testers because if SPF and DMARC as enabled by
default together, then again that fixes one of the major problems with SPF, but
does not automatically fix other problems because there's not enough granularity and
the potential for misconfiguration. So. And the interesting fact is that DMARC
only requires that one of the other technologies SPF or DKIM is passed in
order to consider email valid. There is no way in DMARC, even though there are many
others like selectors. There is no way to specify that both of them should be valid
or that DKIM should be preferred to SPF. In practice, what it means is that for
most systems that enable all three of them, which is a good practical solution
from penetration tester side we can just ignore DKIM outright and just focus on SPF
because the SPF is the weakest link in this situation. Okay. So just a minute for
recap. I'm not sure if I have any more time. Not many time I have. Okay. So
sorry. Yeah. So one really important note is, when you are testing the systems,
consider both scenarios. So don't focus just on send. If you are, for example,
testing Alice. Alice is the organisation that is your customer. Don't just focus on
testing emails sent impersonating Alice, but also as the other side. Because in
this here you can see that it's easy to implement for example, SPF and DMARC
because for both of them only you only need DNS configuration. Just one record
per each. However actually testing them like well validating them properly is
harder. For the first you need the software support, you need to configure it
correctly as well. So in practice it might be that many of organisations that have
enabled DMARC or SPF on the DNS side for outgoing mails, they are not actually
properly validating it. Yeah. Okay. Sorry, I don't have time for that. So probably.
That's it. Sorry. Maybe some questions.
*applause*
Herald: Thanks, Andrew, for this nice talk. Sure. We have time for a couple of
questions. So there I already see one person, microphone number two.
M2: Hey, thanks a lot. Do you know some good tools to monitor DMARC reports that I
get sent by my recipients? A: Yeah. So this is a really good
question. We as a CERT, we are really suggesting everyone to enable this tool,
but unfortunately, as far as I know, all the tools that are popular on the
Internet, they are collecting some data on you. So they are using it for marketing
purposes, do they are not very good for privacy, if you are concerned about that.
So you need to implement something yourself or you need to look at some,
start some open source project maybe. Herald: OK. Microphone number one, please.
M1: Thank you for the good talk. Me myself, I would consider myself an mail
administrator. I sometimes get advised to shorten your SPF record because if it's
too long, it gets dropped anyway. For that, I sometimes get advised to drop the
PTR record. But in your talk, you say the PTR record is useful for reverse DNS
checking, which I find very useful as well. How are you about shortening your
SPF and how are you about the PTR record in general?
A: Well, it really depends on your particular use case. So it might be the
case that some organizations really need this longer SPF and there's not no way
around that you could do. What you could do is include this, include use includes
because they won't be they are not macros, so they won't get expanded. They do not
like your record doesn't become longer if you include and use many includes. But the
problem, which I would suggest to you is actually reconsider whether it's a really
whether you really need that many records if it's still long, because they're a very
common problem, is that unless you are Google or something like that, you don't
really need that long SPF. It's probably some problem with some. Yeah. So it's
probably an error for most organizations. Herald: OK. Well, very. Just briefly.
Number 1 M1: On the PTI rocker record. I heard that
it's dropped. Not dropped from the standards, but it's not in the standards.
A: It is in the standard. No. PTR record by itself is if it's really your use case.
I don't I'm not aware that it will be automatically dropped somewhere. Shouldn't
be a problem. Herald: We have a couple of more
questions here. So number six in the very, very back.
M6: Thank you for your talk. That's not directly related, but even it should be
related. If mail server accepts because DKIM, DKARC and SPF, everything is fine,
but especially Google for a lot of organizations, the mail is delivered but
classified as spam. It means on the inbox of the recipient, it is not displayed.
Have you a solution to solve this problem against Google?
A: Yeah. OK. So I have like different opinions about that because one thing
which actually enables which we actually should be doing. Thank you Google. Is
that they are so strict because that's the only reason that we even have this high
percentage of even improperly configured SPF. The only reason there are 70 percent
websites are using SPF is because that they need to communicate with Google. And
Google won't accept your mail if it doesn't have even SPF on the baseline. So.
I actually I enjoy it as a job that I do. I've. I would prefer that Google does what
it does. But I understand the real admins which have this problem. Google has the
tool. You probably know about it. Where you can check what it considers about your
domain. So you need to consider this problem on a case by case basis. Quite
often what happens is that even though you have this DKIM, DMARC and so on, it's not
configured correctly. So that's what the talk was about. So you have it. You
probably think that you have configured it correctly, but there are some errors.
Herald: Okay, let's give priority to the Internet.
Signal Angel: We have one question from the Internet. Well, attempting to verify
and address how to handle no reply email addresses.
A: No reply, I'm sorry. Can you read it again, please?
Signal Angel: When attempting to verify an address, how to handle noreply Email
addresses. A: Maybe it was about the noreply header ?
Or not existing IP addresses ? Signal Angel: How to handle email. No
reply email adresses. A: I will try to get an answer to how I
understand it. So what often happens is that what often happens is that the email
will be sent from nonexisting addresses. So maybe that's what the question was. For
example, there is "no reply", and it's not the problem itself. No reply. The problem
is that it's not an real address. There is no such address. Right. And so I don't
have an answer for that because according to RFC, you should you should still accept
it. Practically, as I said, lots of mail systems already are dropping this
addresses if you're sending from not existing unless you are Google or
something large, so you have been put into whitelist. You just won't be able to do
that. You won't be able to send email from non-existing address. So if that's your
situation, create the address, make it like a remove all the email that comes
there, but create the real address so that your acceptable. If you are on the other
side. So you are receiving this email. It depends on this particular use case. So
just check what's going on. If you can contact them, contact them. If you can't
contact them, then you should decide what is the risk, if you are dropping these
addresses, are they important for you? So according to RFC you should receive and
process this addresses. Herald: Okay. Microphone number four,
please. M4: Hey, thank you for this talk. Do you
know about effort to solve problems with big email senders like online booksellers,
which are very great because they don't seem to have their own SPF records, for
example, in in control. A: Yeah. So in many cases you can just
contact them. So it's just the question that they haven't thought about it. Or
maybe no one told them what to do or maybe they don't know how to do better. Right.
So that's one of the parts that we as a CERT we are doing. If you have some some
this problem with some large company in particular country, I would suggest to
contact CERT. Even if it's not a government organization, for example, in
Latvia, if that will be a latvian company. We would do the triage. We would try to
try to talk to them, explain to them why they need to change and so on. So that's
maybe one option for you. But the practices that if something looks to you
as a third party, as a wrong configuration, that is one I couldn't
mention in this talk. If something isn't perfectly secure, it doesn't mean that
it's wrong. There might be actually business case why it should be this way.
Right. Because, for example, if it's a large I don't know, Amazon and some for
something like that. And if they have tested and they know that when they enable
very strict configuration, some percentage of their emails just doesn't come. Not
because of their problem, because of someone else's problem. Right. But then
there is actually a real business case that they they are not. It would be stupid
for them to enable this, you know, to strict configuration, knowing that it will
damage their business. That makes sense, right?
Herald: Okay. We are unfortunately running out of time for those who are on the
microphones. please just line up with the speaker next to the desk. He's gonna talk
to you. Perfectly sure. And.
*applause*
*36C3 postroll*
Subtitles created by c3subtitles.de in the year 2020. Join, and help us!