We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Email authentication for penetration testers

00:00

Formal Metadata

Title
Email authentication for penetration testers
Subtitle
When SPF is not enough
Title of Series
Number of Parts
254
Author
License
CC Attribution 4.0 International:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language

Content Metadata

Subject Area
Genre
Abstract
Forget look-alike domains, typosquatting and homograph attacks. In this talk we will discuss ways of forging perfect email counterfeits that (as far as recipients can tell) appear to be coming from well-known domain and successfully pass all checks on their way. Prime focus of this talk will be modern anti-spoofing strategies and the ways around them. Join us as we try to figure out answers to questions such as "Isn't SPF enough?", "Do I *really* need DMARC?" and "Does ticking all three (SPF, DKIM, DMARC) provide the best protection possible?" (answers to these questions are "no", "yes", "no" by the way). Email security is poorly covered by a contemporary penetration testing curricula. In this talk I will argue that it leads to underreporting of email-related security issues during regular penetration tests or red team assignments. Getting clicks from (at least some) users is usually fairly easy, even with obviously fake domain names and email addresses, so penetration testers rarely need to do anything more fancy in order to achieve their objective. While this highlights the need for user education, it misses common misconfiguration issues that might lead to much more devastating compromises and could instill false sense of security in (rare) cases that regular phishing attacks fail. Technically inclined users (such as developers, tech support or even SIEM analysts) are less likely than others to fall for phishing email originating from fake domain, but they are actually more likely to fall for email seemingly originating from real known-good source due to overconfidence. In this talk we will see just how easy is it to send spoofed mail from arbitrary source address due to lack of protection for this scenario in original SMTP spec. We won't stop there however and our next object of focus will be contemporary anti-spoofing technologies (SPF, DKIM and DMARC). We will discuss motivation behind them, their technical limitations, weaknesses discovered in recent years as well as common misconfigurations. Attendees will gain knowledge about relevant protocols and technologies that should be applicable for identifying weaknesses in the architecture of their own email systems.
Keywords
AuthenticationEmailEmailLevel (video gaming)Strategy gameInformation securityCovering spaceComputer animationJSONLecture/Conference
IntelDirected graphLattice (order)Information securityState of matterDeterminismEmailSelf-organizationSet (mathematics)Latent heatArithmetic progressionStatistical hypothesis testingSoftware testingContext awarenessConfiguration space1 (number)Content (media)Domain nameData conversionKey (cryptography)Data storage deviceGroup actionWordTheory of relativityBit rateStatisticsResultantLecture/ConferenceComputer animation
Integrated development environmentPerspective (visual)EmailSelf-organizationData conversionResultantHypermediaStatistical hypothesis testingKey (cryptography)Different (Kate Ryan album)
System administratorEmailPasswordAuthenticationError messageInclusion mapDensity of statesServer (computing)Normal (geometry)Social engineering (security)MereologyClient (computing)Statistical hypothesis testingEmailDataflowServer (computing)IP addressDirect numerical simulationQuicksortConfiguration spaceRow (database)Latent heatDatabase transactionSelf-organizationBitAuthenticationPhysical systemCommunications protocolVulnerability (computing)Data managementInformation securityPasswordDifferent (Kate Ryan album)Type theoryDomain nameDecision theorySystem administratorEmulatorCASE <Informatik>Perspective (visual)Service (economics)Cellular automatonCivil engineeringPresentation of a groupDirection (geometry)CuboidMixed realityMultiplication signVirtual machineCondition numberTelecommunicationObservational studyThread (computing)Traffic reportingComputer animation
Normal (geometry)Server (computing)EmailMessage passingStatistical hypothesis testingSystem administratorLinker (computing)SpeicheradresseAddress spaceCellular automatonConnected spaceTelnetType theoryDifferent (Kate Ryan album)Server (computing)Communications protocol1 (number)BitMereology2 (number)Real-time operating systemEmailMultiplicationImplementationAddress spaceSpeicheradresseMessage passingMultiplication signRegular graphStandard deviationOffice suiteSystem administratorQuicksortResultantDressing (medical)Arithmetic progressionDisk read-and-write headMoving averageStatistical hypothesis testingService (economics)Metropolitan area networkLine (geometry)Right angleArithmetic meanContent (media)Hacker (term)Product (business)AlgorithmSelf-organizationDecision theoryPattern languageComputer animation
Linker (computing)SpeicheradresseEmailAddress spaceMessage passingServer (computing)MathematicsPasswordService (economics)ImplementationUser interfaceStatistical hypothesis testingDirect numerical simulationReverse engineeringEmailCommunications protocolPoint (geometry)Address spaceFilter <Stochastik>AdditionDifferent (Kate Ryan album)Data conversionInformationSystem administratorHoaxIP addressMereologyServer (computing)Graph (mathematics)Latent heatStandard deviationNeuroinformatikSelf-organizationChemical equationPower (physics)BitQuicksortMessage passingPerspective (visual)Goodness of fit2 (number)Physical systemDiagramType theorySubsetRandomizationDirect numerical simulationExistenceMassRow (database)PasswordSoftware testingRight angleCASE <Informatik>CountingSound effectVirtual machineDirection (geometry)Cue sportsDependent and independent variablesLine (geometry)Statistical hypothesis testingFlow separationMoment (mathematics)Computer animation
Limit (category theory)Server (computing)Inclusion mapDomain namePoint (geometry)MereologyDirect numerical simulationMechanism designMountain passSample (statistics)GoogolDirected graphWebsiteDirect numerical simulationServer (computing)Maxima and minimaField (computer science)Different (Kate Ryan album)SubsetPhysical systemAdditionMessage passingSpeicheradresseHoaxEmailIP addressCommunications protocolRow (database)Computer configurationSoftware testingDomain nameGoogolNumberBitDrop (liquid)Asynchronous Transfer ModeAddress spaceFrequencyCASE <Informatik>Covering spaceMetropolitan area networkParameter (computer programming)Condition numberBasis <Mathematik>MaizeDefault (computer science)Right angleForm (programming)Power (physics)Electronic mailing listSoftwareQuicksortComputer animation
GoogolInclusion mapMassSuite (music)Cellular automatonDefault (computer science)Mechanism designProcess (computing)ArchitectureServer (computing)SurfaceService (economics)Domain nameMountain passWebsitePurchasingClient (computing)EmailDirected graphEmailStatistical hypothesis testingDomain name2 (number)Point (geometry)Pointer (computer programming)Computer architectureMixed realityDifferent (Kate Ryan album)CASE <Informatik>Self-organizationRow (database)Flow separationServer (computing)Level (video gaming)IP addressRule of inferenceMessage passingMetropolitan area networkGame theoryMacro (computer science)Multiplication signWeightIdentifiabilitySpeicheradresseDrop (liquid)Arithmetic meanPhysical systemField (computer science)MaizeInstallation artObject (grammar)Set (mathematics)Web pageDataflowQuicksortWeb 2.0Address spaceDatabaseCondition numberSoftwareStandard deviationMultiplicationSystem administratorType theoryBinary codeVulnerability (computing)Filter <Stochastik>Resolvent formalismComputer animation
EmailMessage passingDomain nameDirect numerical simulationServer (computing)Formal verificationMereologyCorrespondence (mathematics)Row (database)Public-key cryptographyElectronic signatureDirect numerical simulationParameter (computer programming)QuicksortMessage passingCryptographyKey (cryptography)NeuroinformatikDomain nameSelf-organizationEmailSign (mathematics)BackupGreen's functionCombinational logicServer (computing)Reading (process)Point (geometry)Connectivity (graph theory)1 (number)Metropolitan area networkRight angleGame theoryFigurate numberPower (physics)CausalityGoogolIdentifiabilityComputer animation
Directed graphEmailMessage passingDomain nameElectronic signatureGraph (mathematics)Statistical hypothesis testingConfiguration spaceMereologyDomain nameIdentifiabilityMessage passingGame theoryComputer animation
Directed graphDomain nameMereologyAddress spaceCellular automatonEmailParameter (computer programming)Block (periodic table)Content (media)Electronic visual displayMessage passingGreatest elementEmailMessage passingMereologyGreen's functionSelf-organizationCASE <Informatik>SoftwareDomain nameSign (mathematics)CodeGoogolValidity (statistics)OvalSinc functionFunctional (mathematics)Type theorySuite (music)Electronic visual displayGreatest elementGame theoryHash functionDefault (computer science)Group actionClient (computing)Presentation of a groupOnline helpProcess (computing)PlanningCondition numberFood energyConfiguration spacePhysical systemMobile appMusical ensembleParameter (computer programming)Multiplication signElectronic mailing listMetropolitan area networkPower (physics)Computer animation
Directed graphTraffic reportingAuthenticationDomain nameMessage passingEmailSample (statistics)Green's functionWordCASE <Informatik>Traffic reportingConfiguration spaceQuicksortDataflowFunctional (mathematics)Conformal mapMultiplication signSoftware testingEmailSemantics (computer science)Domain nameDirect numerical simulationSystem administratorKey (cryptography)Row (database)Maxima and minimaAsynchronous Transfer ModeMereologyStatistical hypothesis testingMessage passingLetterpress printingGame theoryMathematicsRevision controlMeasurementCountingPeer-to-peerOpen setMetropolitan area networkComputer animation
EmailCombinational logicSource codeCondition numberMultiplication signGame theoryLink (knot theory)Graph (mathematics)Physical systemForm (programming)TheoryBoss CorporationVector potentialMathematicsSoftwareImage resolutionEmailDefault (computer science)Validity (statistics)Statistical hypothesis testingGoodness of fitFunctional (mathematics)Computer animation
SoftwareImplementationNP-hardLimit (category theory)EmailData managementKey (cryptography)Direct numerical simulationDirected graphSoftware testingServer (computing)EmailSoftware testingMultiplication signSelf-organizationSoftwareFocus (optics)Row (database)Direct numerical simulationPhysical systemTheory of relativityAreaComputer animation
Directed graphComputer configurationTerm (mathematics)EmailLink (knot theory)Web 2.0Server (computing)ArchitectureProjective planeInternetworkingSystem administratorNumberTraffic reportingMultiplication signOpen sourceImplementationComputer animation
Open sourceRow (database)System administratorNumberDirect numerical simulationReverse engineeringCASE <Informatik>Self-organizationError messageInclusion mapLecture/ConferenceMeeting/Interview
NumberStandard deviationDrop (liquid)Row (database)1 (number)Self-organizationEmailGoogolServer (computing)Lecture/Conference
Domain nameSystem administratorEmailWebsiteError messageBasis <Mathematik>Process (computing)CASE <Informatik>Parameter (computer programming)InternetworkingFigurate numberAddress spaceGoogolLecture/Conference
InternetworkingMultiplication signEmailRight angleAddress spaceProcess (computing)Message passingSpeicheradresseCASE <Informatik>Physical systemLecture/Conference
Image resolutionNumberEmailCASE <Informatik>Special unitary groupRow (database)Physical lawGoodness of fitSelf-organizationArithmetic meanSoftware bugMereologyReal numberComputer configurationConfiguration spaceData storage deviceRight angleLecture/Conference
Multiplication signLine (geometry)Lecture/ConferenceComputer animation
Transcript: English(auto-generated)
Have you ever wondered how to almost perfectly fake an email? Then you might be actually in the right talk here. We have our next speaker, Andrew, who is currently working for the National Cert of Latvia as a security researcher. And he's going to talk about email counterfeiting
and strategies for modern anti-spoofing. The stage is yours.
So, greetings. I am Andrew, and I work for Latvia National Cert. One of our current goals is improving the state of email security in our country,
which we mostly do through raising awareness about this issue and communicating best practices. And, of course, we are not the only organization that is doing that. There are many more sets in other countries, and there are various non-governmental organizations that are doing the same, and commercial entities. However, so far, frankly speaking,
our collective progress has been quite underwhelming. So, for example, here is one stat, which is usage of one specific technology, DMARC, which, as you will learn in this talk, is quite important, and I hope that everyone will start using it.
So, on the left, there are 20,000 domains across all the world, which are important domains for important organizations that really should know better. And on the right side, we see top 500 EU retailer domains.
And across both of these groups, two-thirds haven't even configured DMARC yet, and out of those that have configured, majority hasn't enabled strict policy yet. So, if there is just one key takeaway from this talk, I hope that it will be that everyone should start using DMARC.
It is important to use it even for domains that are not supposed to send email. So, one explanation for this low adoption rate, I think, is that there are seemingly too many competing technologies. This is contents for my talk,
and I really tried to do my best to trim it down, but, as you can see, there are three abbreviations, well, an SMTP, and out of this SPF, DKIM and DMARC, actually, two are... I don't even remember the whole name for them,
but still, they are all important. And, of course, this problem that there are too many buzzwords, too many technologies, and it's not clear which ones we should use, it's not specific to email. And we have this across the industry, and as a security industry, I think, by now, we have found at least one way to solve it,
and it is penetration testing. So, when the penetration test has been run properly and the results have been published, then we can start talking, we can shift the conversation from talking about whether your organization prefers technology A or technology B.
We can instead start talking about the questions that really matter, such like, is it possible for someone, for some third party, to spoof your organization's emails, and to send such emails to your, for example, customers, or your partners, or to media organizations,
in such a way that they will think that the emails really came from within your organization. So, that's why penetration testers are the key audience for this talk. However, I hope that any blue teamers in the audience also will find this talk interesting. I'm sure that you already know all the basics about the email and about these technologies,
but looking at a problem from the different side, from an attacker's perspective, sometimes can really put things into perspective. It can help you understand what you should focus on when protecting your environment. And finally, the SMTP protocol, the technology that runs on the knees,
or email conversations, is actually relatively easy to understand, and also the lessons learned from all of this journey from SMTP, how it became, and how it's possible to spoof it, and all the technologies that are trying to prevent spoofing.
I think it's an interesting case study, and it should be interesting to follow, even for people who are new to email. And finally, threat landscape. So, email security is quite a wide topic, and so today I will only focus on one small part of it,
which is successful spoofing of emails, tampering attacks. And I know that many penetration testers already incorporate some part of phishing or spear phishing emulation into their engagements, but as far as I know,
they mostly do it from the social engineering perspective, using such tools as social engineering toolkit, for example. And I don't want to argue that it's important to do that and to demonstrate to the customer what risks are in regards with social engineering.
However, I think you're doing the service to the customer if that's the only thing that you are testing from the email perspective, because from the customers, from managers, for example, perspective that are reading your reports, if they only mention social engineering attacks, then the logical conclusion is that
the best way to mitigate these threats is by educating your personnel, especially those that are least technical. As you will see in this talk, there are quite a lot of attacks, and many organizations are susceptible to them, which are much better than that, and no amount of user education will help here,
because we can't expect users to check headers, for example, manually. So we actually need to improve our email infrastructure. No way around it. And finally, before we move on to actual technical stuff, there's a little secret, which I think might help people
that are not working in the email industry understand why we have such problems. And it's that for email admins, historically, they value availability of their system and reliability much more than security, and that's because that's not an ideological decision.
It's a very pragmatic one. So, for example, if you are an email admin in an organization and some of your customers stop receiving invoices, your management will find you and will inform you about it and will ask you really nicely to fix it as soon as possible,
even if it's not your fault, if it might happen that the problem is on the other side of email, not on your server. And, for example, if some of your employees can't receive email soon enough, for example,
to restore the password or to verify the email or to use a multi-factor authentication token, and they can't log into some important systems, again, they will find you, and you will need to solve that. But if your system has some security vulnerabilities, if it's susceptible to spoofing attacks and so on,
then not users nor management will normally notice it. You might not notice it, but you have this vulnerability. So that's why, obviously, penetration testers are important. Okay, now we can finally start talking about the technical stuff.
And we will start with a short introduction to SMTP protocol. SMTP is the protocol that underlies all email communications, and it's actually pretty easy to follow. So here's a data flow of what's happening when one person sends email to another person. For example, Alice is sending to Bob, and they are working for different companies.
They use different domains. So what's happening here is that both of them, let's say, use email clients, such as Outlook or Thunderbird. And Alice is sending email. It's going through this protocol, SMTP, to Alice's mail server. But important to note is that this is an outgoing email server.
Usual organizations will have two types of servers, one for incoming transactions and one for outgoing. And for smaller organizations, it might be one server, but again, it's important for a penetration tester to think of these as different systems because even if it's physically one machine, it will have different configuration for outgoing mail and for incoming mail. So as a penetration tester, you need to check both of them.
Okay, now when Alice's server tries to send email to Bob's server, there's sort of a problem in that the server needs to somehow automatically find what is the right server to send email. And it is done through this blue box, MX, which is a specific DNS record type, MX.
So that's something that is maintained by Bob's organization. So Bob's organization, if they want to receive email, they create this DNS record, and they say that, okay, if you want to send email to us, please use this particular server. So it should point to Bob's server. And now Alice's outgoing server, knowing Bob's incoming server address
can communicate to that, and then later Bob will receive its email. So the part that we as penetration testers will be trying to breach is actually between Alice's server and between Bob's server. And then we need to think about the second example,
which is the opposite way. And you might think that it's a pointless example because we are just basically changing the direction of traffic. But the important part here is for us as penetration testers to understand that our client only controls part of this transaction. If our client, let's say, for the rest of this presentation, is Alice or Alice's organization, then in the second example,
when we are sending mail from Bob to Alice, then we'll be sending emails only basically part of this transaction will go through Alice's servers. In the first example, if we were sending email from Alice to Bob,
it wouldn't be so. So if it's a bit confusing, that's okay. We will return to that a bit later. And finally, there's a third example which looks similar, but not quite. And that's if Alice is communicating. Alice is our customer, and if she is communicating with her coworkers, which are using the same organization, same email server, same domain.
In that example, again, there will be two, at least logically, two email servers, outgoing server and incoming server. But both of them will belong to our customer. So right now, if you are not familiar with email, it's just interesting to try to think which of these scenarios,
three scenarios, which of them are easier to protect. And a bit later we will see how it's actually happening. And then we need to look at what actually is being sent when email is being sent. So again, it's using SMTP protocol, and it's a really nice protocol.
As you can see, it's just text, so it's plain text protocol. And it's very easy to play around because you can just open a telnet connection to the right server, and you can try writing down the commands just with your hands. So you can try mangling something or modifying or trying different types
and see in real time how it's going on. So on the left side we see here two parts which are defined by SMTP. So first of all, there comes SMTP envelope, which basically you connect the server, say hello, then you specify the sender of email and recipient.
Mail from is sender, recipient is Bob, for example. And then the second part starts with data and ends with quit, and that's the part which is called content message. So just if you want to play around with it a bit more, this is defined by a different standard, which is not that important for penetration testers, but if you want to look in the details, then it might be important.
And this internal message, which is called either content or SMTP message, it again, it contains two parts. One is headers and another is body. And I think some people might not be familiar with email, but probably everyone is familiar in this audience with HTTP, and this looks quite the same, so easy to understand.
But the interesting part here is that you might have noticed that we have Alice's and Bob's addresses twice, right? For example, Alice is specified on the second line, mail from, and then we have the same address, Alice at her organization, from header, the red ones are the headers.
And the same goes for Bob. So why is that? Well, it comes down to how we see email. I, as a, I think, normal, regular person who has used email in the past quite a lot, I usually see them as described on the left side, which is sort of postcard.
So on the postcard, there is someone who has sent it, the sender, there is a recipient, that's usually me I'm receiving, and then there is some message. So at least that's how I perceived it, before I learned a bit more about it. But email admins and the standard bodies, they see this situation as the one which is shown on the right,
which is, there is an envelope, and inside the envelope, then there is this message or a postcard maybe. So you have two addresses in this scenario. You specify the address from and to whom you are sending the envelope, which is the part that post office, for example, will look.
But post office won't look generally inside your envelope. And inside the envelope, there is another message, and that internal message is actually meant for recipient. So actually, you could do even more, and you could even put the whole envelope with the message or with the postcard inside another envelope. And this sounds crazy to me, as a regular person,
but actually, email allows that. And in the RFC, in the standard document, there are some examples why that would be necessary, why such things are allowed, but they are confusing. And so as a result, here in this first example, we see that we generally, we are specifying the same address twice.
But as a penetration tester, the question that we should be asking is, so is that required, actually? Is that always true, or is it just like wishful thinking? And it's actually wishful thinking. So standards specifically do not say that you should be specifying the same address for recipient
or from the sender on envelope and inside the message. So you could actually tweak them and send different stuff. So actually, there are much more headers than what I showed. The ones I showed, I think, are just the ones that we all have experienced,
because if you are just using email, that's usually the stuff that you see. You see the date, you see the subject, you see who sent you something and to whom it was sent, usually yourself. And there might be, of course, more recipients. Oh, yeah, and another question is, which one is actually, if we have specified for some reason,
by accident especially, if we have specified different addresses in this envelope in the message, which one the user will see, the recipient? And it's actually the headers inside. The message is the one which is intended for a user. Okay, so as I was saying, there are actually standards allow a bit more headers,
and there are actually three headers, from, send and reply to, which are semantically really close. And in the standard, it actually explains when you should be using which one, and the funny thing for me is that, for example, from header, which is usually the one that we see, it might contain, by reading the RFC, you will see that you shouldn't have more than one such header,
but the header itself might contain multiple addresses. Personally, I've never received an email which would come from different people, but that's allowed. But the important thing to understand here, again, is the backwards compatibility that I mentioned before. So even those standards explain how you should use each header
and that you shouldn't have more than one of each of these headers. In practice, actually, you can send malformed email. You could send email with multiple headers, the same header from header multiple times, or you could send header which does not contain from but contains sender. According to RFC, that's incorrect, but in practice, it will work.
Most organizations, most email servers will try their best to parse your completely malformed email because they really are concerned about lowering the support costs. So if something does not work, then you will come to them. So it's better to make that everything is working most of the time.
Of course, for penetration testers, that means that you can play around with this because there are different implementations, and exactly which header, for example, if you have two headers, will be shown or will be used for some algorithm. It depends on the particular implementation. So because there are so many implementations that are interconnected in different ways,
you could and you should, as a penetration tester, try various things. For example, add the same header multiple times. Okay, now that we have covered these basics, let's actually look into how you would try to spoof an email. For example, here again we are coming back to these diagrams that we've seen before.
For example, in the first example, Alice is sending email to Bob. Let's say we are Chuck, so we are the third party. We are penetration tester, licensed. We have an arrangement that we are allowed to do this, and we are trying to send spoofed email to Bob. And in this example, we are trying to spoof Alice's message.
So our intention is that once Bob receives email, it should look to them, to Bob, that email was sent by Alice. So risk for this, okay, I will not cover the risk. I think you can imagine that, for example, fake news is one of the problems that we have seen in Latvia.
This was used against government bodies, and when someone sent fake news email to other people, organizations, and so on, and were trying to impersonate some government person. And, of course, you could imagine yourself how it's not a good thing, if it's possible.
But the interesting thing here is that even though Chuck is doing attack, it depends on your perspective. It might look like attack on Alice or on Bob, but in this case, email won't go through Alice's systems. As you can see, Chuck is sending email directly to Bob's incoming server. Now there's a second type of attack that we looked at.
If we are sending email in another direction, from Bob to Alice, and our customer is Alice, so we are testing Alice's server. And in this case, we are trying, again, we are Chuck, we are sending email. In this case, email will go through Alice's systems. So interesting question is, which is easier to protect?
It might seem that since in the second example, email is actually going through Alice's systems, it means that Alice has more power to do something, to do some additional checks and balances and so on. But actually, as you will see in the future, it's easier to protect the first example. So even though our customer is Alice, we are trying to protect Alice,
but it's easier to protect and practice this example where someone is sending email, trying to impersonate Alice. Okay, oh yeah, and there's a third example, which is if Alice is communicating with her colleagues inside the same organization, again, we are Chuck. In this case, again, we will only send email
to Alice's incoming server, not to outgoing server, right? So important thing to note, and again, in principle, this third example is the easiest to notice because Alice's organization presumably knows that her emails always should come from this particular outgoing server, right?
Like if we are sending email from Alice's colleague, then incoming server in principle should have all the power even without any standards and stuff like that. But in practice, sometimes, actually quite often, there will be a specific whitelist for Alice's own organization.
So some checks won't happen if incoming server for Alice is receiving email, which is coming from, again, Alice. And by the way, there's this example. We've seen that for the past few years. I think it's not specific to Latvia. So here, for example, it's Canadian address, if you can see.
And these are these emails, which are fake, like ransomware stuff. Basically, they are telling you that they have hacked your computer or your email in this case, and they have arranged all sorts of activity or have some blackmail on you, and please send them your money in bitcoins to their address.
So these emails, interesting part about these emails is that they are usually, in order to prove to you that they have access to your email account, they are sending email from your address to your address. And for many people, that works. So they see that someone has hacked their account, obviously,
because they received email from themselves. So as you will see a bit later, it's actually easy to spoof such emails if they haven't been, any protections haven't been put in place. So the important thing, I hope that no one in this audience is falling for such scam, but if you have some friends or colleagues
that have contacted you and told you about such emails that they have received, that one of the things besides checking the passwords, starting using and so on, is just maybe you could tell them that they should contact their email administrators or IT team and ask them about anti-spoofing protection,
because obviously if they are able to receive such email and it's not filtered, something is wrong. Okay, and now let's see a spoofed SMTP conversation. So that's an example similar to the previous one, but in this, now we are actually Chuck.
So this is sent by Chuck to Bob, but we are pretending to be Alice. The question is, can you see the difference, how this is different from the previous one? And it's hard to see the difference because there is no difference. That's the same conversation. So the point here is that SMTP protocol by itself, it actually doesn't have any protection. So yeah, you could just, for example,
if you are that guy that is sending the fake ransomware letters, you can just write down this text and just dump it to Telnet and it will work for many organizations, not for all. And of course, email admins know this stuff, know that SMTP is not very reliable in this regard,
that it's easy to spoof and so on. And there have been many attempts to add some protection just like ad hoc way. So no standards, just run some, add some additional filters and stuff into your own mail. And some of these protections actually break RFC if you read it, but who cares? Like, RFC is not a sacred text,
so I absolutely approve this, for example. So yeah, go on. But the problem is that there is not enough information. So if you think back here, if we are Bob and we are trying to protect our systems, so we are Bob's system administrator probably, or Bob is a system admin, and we are trying to add some additional rules and stuff,
then what actually can we do? So one example that I listed here is doing this SMTP callback, and that means that we are just, when we receive email from Alice, we actually check, does that email exist at all? Because many spammers, what they will do, they will just send email from non-existing emails,
and it will work if you are just running your own SMTP server. So SMTP callback is basically when you are receiving email from, for example, Alice, you are running, spawning a separate process, which will try to connect back to Alice's server, and it will try to send email here.
If a server says that, yeah, that's okay, such email exists and so on, you actually stop the conversation. You don't continue with sending email, but then your system can automatically find that actually this email really exists. So another way to do this is through checking this hello, and this is the first line, and the first line,
normally it should tell you the host name of the server that is sending email. Interesting part, so according to RFC, again, you shouldn't check it, you shouldn't verify, and if it doesn't, if it's random string, you should accept email still. But what many servers will do is they will try to verify that,
first of all, this host name, which you are telling that you are this host name, first of all, that it really points to the same IP address, and then they do the opposite. So they will take IP address and try to run reverse DNS PTR query, and they will try to find whether that IP address
really responds to this host name. So, again, as a penetration tester, we should be aware of these protections, ad hoc protections, because they are, if you don't know about them, you will try running something and it won't work for you. But they are easy if you are aware of them and if you have identified that this organization uses them. They are easy to bypass, so they don't offer good protection.
They are meant to protect from mass abuse, from spam. Okay, so SMTP, as we have seen, by itself does not offer any protection. So which additions to the protocol actually can we use to protect ourselves? One of such protocols is SPF,
and what SPF does is it's trying to be like mirror MX system. MX system is the one which basically Alice can use, Alice's server can use to automatically find the server that belongs to Bob and its incoming server.
So SPF is the opposite of that. So that idea is here to run the system automatically on the Bob's incoming server. And now when Bob receives the email, they can run, again, DNS query, and they can find what IP addresses actually should belong to Alice's outgoing server.
Right, so I think it's easy to understand. It's actually a meaningful way. It sounds meaningful addition. And the one field that is checked in this example is this envelope sender. Okay, and here's an example of minimal SPF syntax. And as we can see, I think it's easy to understand.
Even if you don't know the syntax, it lists IP address, which should be IP address of outgoing server, legitimate outgoing server. And then it says there's minus O, which again is easy to understand. In this case, it means that that's the only one. So if you receive a message, message comes from this IP address, that's cool.
Accept it. If it's something else, then just drop it. And there are multiple ways to specify the IP address. You could just specify the IP address. You could specify IP subnet. You could specify DNS hostname. So it's just for admins, basically for penetration testers, it doesn't do much difference. For admins, it's just easier to maintain these systems.
And then there are these qualifiers. Qualifiers is something which you put before the methods. For example, here in this example, IPv4 doesn't have any qualifier. There's no plus or minus or something. That's because plus is assumed by default.
So by default, everything that is listed in SPF record should match some legitimate SMTP server, outgoing server. However, there are other options. You could use minus, which is fail. And that means if something matches this record, for example, minus O is the one which is the most often used.
It means if it matches this one, so that's usually the last one. Then please drop the mail. It's not real. It's fake mail. And then there's this third option, which is software. And that's meant for testing, period. So when you are just starting to implement SPF, there might be some...
So the problem is that you might forget, for example, to add some SMTP servers. So because you haven't done that before, maybe you think you have only one SMTP, actually outgoing server, but in fact, you have multiple of them, multiple ways to send email. So in that case, if you were to set the SPF record
with fail, strong policy, then your users won't be able to send a message anymore. So that's why testing is good. However, here are some other examples, a bit more complicated. One of them is with include. So instead of defining the policy yourself, because you are using third-party, for example, Google in this example, then you will just include whatever Google has published.
And the interesting thing is this usage of SPF. If we just look at the amount of domains that have defined some sort of policy, then the number looks pretty okay, I guess. For example, for most popular domains, that's around 70%.
But the problem is that the majority of them are either poorly configured, or they just use the soft-fail option. And what soft-fail practically does is nothing. You still can, even if there's policy with soft-fail, you can, in most cases, you can spoof your email, and it will still go, because recipient side will think that it's just in the testing mode.
You shouldn't drop email automatically. Yeah, so actually the percentage isn't that great. However, the most important thing for us as penetration testers is to understand, so what do we do when we see this SPF? That means that now we can't spoof mail.
And no, it's not that it's game over for us. We can do some stuff. So first of all is this software that I mentioned. And that's basically, you have some rules, rules, rules, and then in the end you are putting typically just this software all. So if we as penetration testers will try spoofing
from some unknown IP address that hasn't been listed in the previous rules, then do nothing. Do nothing, I mean, don't drop email. That's good for us, right? That means that we can actually spoof just in the same old way, and it will mostly go. So the one note here is that some systems are not using
just this binary classification, whether something is good or bad, but they are trying to run some scoring. And then it might be that even if you have this soft-fail, they won't automatically drop your email, but maybe they will add some suspicious level to it. But the important thing is that it's not automatically game over.
Another thing is this include. So include is very convenient when you're using third parties. But the problem is that it's not what it sounds to some people, at least even in the standard. In the standard, it mentions that it was poorly chosen name.
And the reason for that is that it's not a macro. So to understand what's happening when this include is, you shouldn't just copy-paste everything from inside recursively to the top level. It's not how it works. It will try running all the checks inside this include, but then if it fails, it won't automatically drop the message.
It will go to the one level top, and it will try running the other rules. So the problem with that is that two cases that are the most common is either if you just forget to add this minus all to your system administrator or have forgotten to do that.
In that case, even if the include has minus all, it won't work, because when the recipient will be checking it, minus all inside include does not mean the same as it does on the top level. And the second would be if they have added all, but it's soft fail all.
And some admins might think, but that's okay, because I'm including Gmail, and Gmail has this hard fail. It doesn't work that way. And then one which actually is, I think, maybe the most common case is that often you actually see these type of SPF records where there is lots of stuff inside.
There's IP addresses, there are these air records, there's a mix, there's a pointer, basically everything that admins could think of. And the reason is that most commonly they are just not sure how it works. They are not sure what they should put inside. So, for example, one thing that points that out is if there's a mix record inside the SPF,
most commonly most organizations, unless they are very small and just have one server, they will have different servers, different IP addresses for outgoing mail and for incoming mail. That means there is no practical, for these organizations, there is no practical reason to include a mix into SPF because no mail should go out through their incoming mail server.
And another case might be that the admins understand how it works, but it's really truly their architecture is really messy and they are sending mails from many, many different points, which is good for penetration testers. That means that they are not well organized.
Okay. And then there is another flaw, which is that granularity isn't very well suited. So the only thing you can, there are multiple record types, but all they do basically are resolve to IP address. But as you can imagine, in many cases, IP is not linked just to mail server.
So on one IP there might be mail server and web server or database or something else. And that means that as a penetration tester, you can exploit this something else, not mail server itself, because mail server usually is pretty low key. There's not many vulnerabilities there. You just patch them and that's it. But those other systems, for example, web, it's easy to exploit in most cases.
So then you can elevate, in some sort of privileges, by gaining access through some other server on that IP address or IP range. You can start sending mails and they will pass all SPF filters. Okay, so one example is shared hosting, which is the very common case.
And the problem with shared hosting is that in this case, okay, you have IP address of SMTP server. Maybe that's server only used for sending mails. But the server itself works not just for you. It works for many domains, maybe hundreds or thousands of domains. That means as an attacker, again, you can exploit at least one of them,
or for shared hosting you can just buy, you can become a customer of that shared hosting. You don't even need to exploit anything. And then you can potentially start sending mail, which will look, as far as SPF is concerned, just like the real one. And another one is this checking run identifier. And this is probably the worst problem with SPF.
It is that, as I mentioned before, there are at least two identifiers, typically envelope sender, the outer one, which lists the sender, and then there's the internal one, which is usually from header. But out of those two, SPF only checks, if SPF is the only technology that you are using,
SPF only checks the first one, envelope sender. And as I mentioned, in most cases, actual users that will receive the mail, they won't see envelope sender. They will see this other one, from, for example, or one of the other headers that I mentioned. So this behavior is fixed actually by DMARC, which is the technology that I mentioned.
But the majority of SPF installations, domains that are using SPF do not have DMARC, so they are not protected by this. So even if the SPF is completely great, for attacker, it means that you only need to, what you need to do to pass SPF is to set envelope sender to something else. For example, your own controlled address,
which will pass all SPF checks. But then inside the from, you can show the header that will match this organization that you want to pretend to be. Okay, so then there is another technology which is supposed to fix this, and it's DKIM.
As we have seen, SPF is not enough. So DKIM, sorry, the wrong letters, but yeah, domain keys are not defined. That's the DKIM, and you don't need to remember the long name, just the short name. And what it does, basically it uses cryptography, which is nice, right? It's math, it's hard to break for attackers,
and what it does is it signs every mail. So every mail that is going out through the DKIM-enabled server will get signature, which you can, as a recipient, you can cryptographically verify. So as you can see, how it looks is actually pretty hard to see because it's not meant to be processed by humans.
It's cryptography. It's meant to be processed by computers. But the important part here is basically the yellow stuff is this cryptographic signature, but the green part is what's called domain identifier, and the red part is what's called, I don't remember how it's called,
but basically its idea is that you can have multiple keys for your organization. For example, your organization might be sending mails from your original SMTP server, then you might have a backup one, or you might be sending some messages from Google or some marketing campaign and so on.
And then each of them might have different red, this parameter. The problem is, and then recipient will need to run DNS query, which is the second example, using this combination of green and red one, and then they will get the public key, and they can use this public key to verify the signature. So it sounds really nice. The problem here is, no, not a problem yet.
So how to use it? I think it's easy if you understand the public cryptography. So on the sender side, you need to first generate public and private key pair, then you publish the public part in the DNS, then you use private key to sign each message. Now recipient does sort of the opposite. Once they receive the email,
they figure out through this red and green part, they figure out the correct DNS record to run, run it, get the public key, and then compare whether this public key corresponds to the signature. So it sounds really nice, right? What's the problem? So custom selectors, that's the name. So the problem with that is that the selectors,
there might be multiple selectors. As a DKIM, when you are doing configuration, you can select as many custom selectors as you want. And the recipient doesn't know whether you actually should have used selector and what selector you should have used. So the problem is that while,
if we are talking just about the vanilla DKIM, modifying existing signature is hard for penetration testers or for an attacker. But it's easy to just remove it because if you have removed DKIM at all, the recipient doesn't know that it should have been there because in order to check, they need to, so here for example,
in order to check the signature, I need to know this green part and this domain identifier and the selector, which are part of this header, right? So that's a huge problem. And that means that we can actually, while we can spoof DKIM itself,
we can just trim DKIM, send the message without it, and if the DKIM was the only thing which protected this system, it will work. So it might not get the green check mark or whatever, but it will get to the recipient. And another thing is this domain selector. Why do we even need to set that?
Because the best practice, of course, is that you have envelope sender equal to from header equal to this DKIM domain selector, right? So if I'm sending from Alice, then all three should be Alice.oracle or whatever. The problem is that it's not mentioned in RFC, but that should be the case.
So what exactly happens when it's not that way? For example, on the right side, there's some real domain which was using Gmail, Google Apps, Google Suite. And in that case, by default, Google Suite will sign all messages. But if you do not do your own configuration,
it will sign them with domain it controls, which is this gap SMTP. And what it means is that although technically something has been signed with DKIM, it wasn't signed in the way that you can trace back to your organization. It's something completely else. What exactly recipients should do in that case? Should they just ignore it? Should they reject the message or something?
So the correct way would be not to reject it, but to just consider it not valid, at least not a valid DKIM. But it actually depends. So some validators will just see any DKIM, will validate it, and will say that's cool. That matches RFC. So but now the interesting part,
modifying DKIM, which I don't have time for, but the idea is that in some cases, this is not always, but sometimes you actually can modify. The easiest part to modify in the messages are headers because DKIM, since it's placed in headers itself, it does not automatically sign all the headers. It's like a chicken and egg problem.
So by default, it only signs one or two headers. And you can specify more headers that need to be signed, but it doesn't happen automatically. So the easy part for an attacker is to add another header. If that somehow helps you in your plan, then that's easy to do. You just add another header. An interesting part is,
although the RFC, as I mentioned before, mentions that some headers such as subject or from should only be present in one copy, actually you could add more than one, for example, from header. And what happens in that case is pretty interesting. DKIM will match if you have told to DKIM that from header should be, for example, signed.
Then it will match and sign first from header from the bottom. But quite a lot of software, email clients, will actually only display to the user first from the other side, from the app side. So what it means is that the attacker can mangle or overwrite headers by just adding new headers to the top.
And this actually problem is mentioned in the DKIM RFC, and the protection that they propose is this code over signing, so you can go and read the RFC, but not everyone is doing that actually. However, that only goes to the headers. So sometimes that is good, sometimes that's not good.
Modifying message body is actually much harder to do. Basically the naive way we do through the cryptography, which we don't want to do. And another way is through this one parameter, which is body length, and that's actually like questionable functionality that DKIM has. Sometimes you can specify that the hash,
like for signing purposes, we shouldn't consider the whole body, but only first something bytes. So that's actually useful in some cases regarding with mailing list, but for the most part, that's not useful. And in practice, most email software does not do this. If it does, then it is susceptible to, potentially,
to this overwriting body as well. You could add another MIME type and then modify headers to show that different MIME type, and it will pass DKIM. So in this case, it actually will show, for example, the green button or whatever, because DKIM will be valid. So now there's the third technology, which is called DMARC.
And again, there's the full name, which is long, but in this case, actually, it means something. There are two key words, reporting and conformance. Reporting is the one which most admins are familiar with, because that's how DMARC, I think, often is being sold to them. Reporting means that when you have some problems,
in this case, you actually get to tell other side what to do in that case. So basically, you tell them to send you reports, either once per day or every time and so on. So for penetration testers, it's not that useful. Potentially, we could use that to understand what sort of configuration is running on the other side,
but currently, this functionality actually is not that widely implemented. However, the other part, conformance, it's actually really, really, really powerful. What it does is it corrects these major flows that I mentioned in SPF and DKIM. So first of all, DKIM had this massive problem that if you just strip down the header,
then the recipient has no way of knowing whether there should have been DKIM in the first place. If you are using DKIM alongside with DMARC, that fixes the problem, because DMARC specifies just that you have DMARC itself. It means that automatically at least one of the SPF or DKIM should pass. So automatically, DKIM's major problem is solved.
The other thing that it changes is it changes the semantics for SPF. Now SPF, if you have both, SPF and DMARC, it means that SPF should be checked against from header. And as I mentioned, that was the major flow with SPF, because if you are using SPF itself, even with the hard fail mode and so on, it means that attackers can modify from header still,
and the recipient won't know any better. So minimal example of DMARC is really, really small, and I think it's easy to understand. You have just, it's a DMARC, it's reject. You need to find out the right place to specify it, but it's easy. And all you have to do is create this one DNS record. And the benefit for that is even if you don't have DKIM and DMARC,
if you have created, sorry, if you don't have SPF and DKIM, but you have created DMARC, effectively what it means is that this domain should not send any mail, because for recipient to consider mail valid, at least SPF or DKIM should be valid as well.
If they are not, then they can be valid. So in fact, what it means is that most domains out there should consider enabling DMARC. That's just the right thing to do. Okay, so there are more tags. So in the wild, these DMARC records might be much longer, but it's not of much use to penetration testers.
So important part here again is this policy, which can be three values, none, quarantine, and reject. And if it is a quarantine, that means if there's a failure, the message should go to the spam folder. If it's reject, it should be rejected outright.
And if it's none, it means it's in testing mode. So and this is the picture that I showed in before, which shows that actually even though DMARC is really like the best technology out of these three, it's not really widely used. Unfortunately for defenders, quite a nice fact for all penetration testers out there.
That means that you can in fact spoof most of the mails out there. Okay, so how do we work around it? Sorry. So what happens if actually someone has implemented DMARC? Does that mean that now penetration testers can do anything?
You don't even need to do any research. No, it doesn't. So in practice, if someone has implemented both DKIM and DMARC, but not SPF, so they have only two of them, that's a really cool combination. DKIM is pretty powerful, and the major flaw that it had, DMARC solves. So this combination is really cool in theory.
In practice, the problem is that in order to protect your own mails, the recipient side should validate both DKIM and DMARC. And unfortunately, quite a lot of software still does not do that. One such software is Microsoft Exchange, and even if you are not running Microsoft Exchange, changes are good that some of the partners that you are communicating with are running Microsoft Exchange.
And by default, it doesn't have any functionality to parse DKIM. So in fact, most systems still need to enable SPF just for practical purposes, which is good for penetration testers, because if SPF and DMARC are enabled by default together, then again, that fixes one of the major problems with SPF,
but it does not automatically fix other problems. There's not enough granularity and potential for misconfiguration. And the interesting fact is that DMARC only requires that one of the other technologies, SPF or DKIM, is passed in order to consider email valid.
There's no way in DMARC, even though there are many other selectors, there's no way to specify that both of them should be valid or that DKIM should be preferred to SPF. In practice, what it means is that for most systems that enable all three of them, which is a good practical solution, from penetration tester side, we can just ignore DKIM outright and just focus on SPF, because SPF is the weakest link in this situation.
Okay, so just a minute for recap. I'm not sure if I have any more time. Not many time I have. Okay, so, sorry. Yeah, and so one really important note is when you are testing the systems, consider both scenarios. So, don't focus just on, if you are, for example, testing Alice,
Alice is the organization that is your customer, don't just focus on testing email sent impersonating Alice, but also test the other side. Because in this, here you can see that it's easy to implement, for example, SPF and DMARC, because for both of them, you only need the ice configuration, just one record per each.
However, actually testing them, like validating them properly, is harder. For the first, you need software support, and you need to configure it correctly as well. So, in practice, it might be that many organizations that have enabled DMARC or SPF on the DNS side for outgoing mails,
they are not actually properly validating it. Yeah, okay. Sorry, I don't have time for that. So, probably that's it. So, maybe some questions.
Thanks, Andrew, for this nice talk. Sure, we have time for a couple of questions. So, there I already see one person, microphone number two. Hi, thanks a lot. Do you know some good tools to monitor DMARC reports that I get sent by my recipients?
Yeah, so this is a really good question. We, as I said, we are really suggesting everyone to enable this tool. Unfortunately, as far as I know, all the tools that are popular on the Internet, they are collecting some data on you. So, they are using it for marketing purposes.
So, they are not very good for privacy, if you are concerned about that. So, you need to implement something yourself, or you need to start some open source project, maybe. Okay, microphone number one, please. Thank you for the good talk. Me, myself, I would consider myself a mail administrator. I sometimes get advice to shorten your SPF record,
because if it's too long, it gets dropped anyway. For that, I sometimes get advice to drop the PTR record. But, in your talk, you say the PTR record is useful for reverse DNS checking, which I find very useful as well. How are you about shortening your SPF,
and how are you about the PTR record in general? Well, it really depends on your particular use case. So, it might be the case that some organizations really need this long SPF, and there's no way around it. What you could do is use includes, because they are not macros, so they won't get expanded.
Your record doesn't become longer if you use many includes. But the problem, which I would suggest to you, is actually reconsider whether you really need that many records, if it's too long, because the very common problem is that, unless you are Google or something like that, you don't really need that long SPF.
It's probably an error for most organizations. Just briefly, number one. On the PTR record, I heard that it's dropped,
not dropped on the standards, but it's not in the standards. It is in the standards, yeah. No, PTR record by itself, if it's really your use case, I'm not aware that it would automatically drop somewhere. By itself, it shouldn't be a problem. Okay, we have a couple of more questions here. So, number six in the very, very back. Yeah, thank you for your talk.
It's not directly related, but even on the other side, it should be related. If mail server accepts, because DKIM, DMARC, and SPF, everything is fine, but especially Google, for a lot of organizations, the mail is delivered, but classified as spam, it means on the inbox of the recipient,
it's not displayed. Have you a solution to solve this problem against Google? Yeah, okay, so I have differing opinions about that, because one thing which actually enables, which we actually should be telling, thank you Google, is that they are so strict, because that's the only reason
that we even have this high percentage of even improperly configured SPF. The only reason there are 70% websites are using SPF is because they need to communicate with Google, and Google won't accept your mail if it doesn't have even SPF on the baseline. So, I actually enjoy, as a job that I do,
I would prefer that Google does what it does, but I understand the real admins which have this problem. Google has the tool, you probably know about it, where you can check what it considers about your domain. So, you need to consider this problem on case-by-case basis. Quite often what happens is that even though you have this DKIM, DMARC, and so on,
it's not configured correctly. So, that's what the talk was about. So, you have it, you probably think that you have configured it correctly, but there are some errors. Okay, let's give priority to the Internet. We have one question from the Internet. When attempting to verify an address, how to handle no-reply email addresses?
No-reply, sorry? Can you read it again, please? When attempting to verify an address, how to handle no-reply email addresses?
Maybe it was about the no-reply header? No, not existing? How to handle no-reply email addresses? Okay, so I will try to get the answer how I understand it. So, what often happens is that email will be sent from non-existing addresses.
So, maybe that's what the question was. For example, there is no-reply, and it's not the problem itself, no-reply. The problem is that it's not a real address, there's no such address, right? And so, I don't have an answer for that. Because according to RFC, you should still accept it. Practically, as I said, lots of mail systems already are dropping these addresses.
If you're sending from non-existing, unless you are Google something large, so you have been put into whitelist, you just won't be able to do that. You won't be able to send email from non-existing address. So, if that's your situation, create the address. Make it like, remove all the email that comes there,
but create the real address so that you are acceptable. If you are on the other side, so you are receiving this email, it depends on this particular use case. So, just check what's going on. If you can contact them, contact them. If you can't contact them, then you should decide what is the risk if you are dropping these addresses. Are they important for you? So, according to RFC, you should receive and process these addresses.
Okay, microphone number four, please. Hey, thank you for this talk. Do you know about effort to solve problems with big email senders like online book sellers, which are very great,
because they don't seem to have their own SPF records, for example, in control? Yeah, so, in many cases, you can't just contact them. So, it's just the question that they haven't thought about it, or maybe no one told them what to do, or maybe they don't know how to...
do better, right? So that's one of the part that we, as a cert, we are doing. If you have this problem with some large company in particular country, I would suggest to contact cert, even if it's not a governmental organization. For example, in Latvia, if that would be a Latvian company, we would do the triage. We would try to talk to them, explain to them why they need to change and so on.
So that's maybe one option for you. But the practice is that if something looks to you as a third party, as a wrong configuration, and this is one I couldn't mention in this talk, if something isn't perfectly secure, it doesn't mean that it's wrong. There might be actually business case why it should be this way, right? Because for example, if it's a large, I don't know,
Amazon or something like that, and if they have tested and they know that when they enable very strict configuration, some percentage of their emails just doesn't come. Not because of their problem, because of someone else's problem, right? But then there's actually a real business case that they are not, it would be stupid for them
to enable this too strict configuration, knowing that it will damage their business. That makes sense, right? Okay, we are unfortunately running out of time. For those who are on the microphones, please just line up with the speaker next to the desk. He's gonna talk to you perfectly sure.