Domain Fronting is Dead, Long Live Domain Fronting
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Title of Series | ||
| Number of Parts | 374 | |
| Author | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/49787 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
DEF CON Safe Mode101 / 374
11
35
63
70
74
86
90
98
102
103
104
105
106
107
109
110
113
114
117
119
121
122
123
124
126
127
128
130
136
137
138
142
146
151
152
153
154
159
160
161
163
165
166
167
168
169
171
177
189
214
226
231
232
239
240
246
247
250
255
256
265
267
268
269
270
271
272
274
279
280
283
289
290
336
337
344
360
362
363
364
365
367
00:00
Public domainAsynchronous Transfer ModeComputer networkTransport Layer SecurityProxy serverRoutingComputer fontDirect numerical simulationEncryptionCommunications protocolSoftwareEmulatorPublic domainTransport Layer SecurityProxy serverCybersexSpectrum (functional analysis)Information securityEnterprise architectureIntegrated development environmentGroup actionSharewarePrice indexComputer animation
00:51
Public domainTransport Layer SecurityServer (computing)Public key certificateSystem identificationDirect numerical simulationAsynchronous Transfer ModeUDP <Protokoll>Address spaceDependent and independent variablesEmailHost Identity ProtocolGamma functionEntire functionProcess (computing)GoogolCloud computingContent delivery networkWebsiteClient (computing)Computer configurationInternet service providerBand matrixBefehlsprozessorIdentity managementPlastikkarteCommunications protocolRevision controlSuite (music)Internet service providerTerm (mathematics)Message passingWebsiteVector potentialService (economics)Punched cardSystem callMechanism designIP addressEncryptionPublic domainDirect numerical simulationSoftwareAddress spaceEmailContent (media)Web 2.0Client (computing)Level (video gaming)Inheritance (object-oriented programming)MultilaterationFilter <Stochastik>NumberGame theoryComputing platformLeakProxy serverTransport Layer SecurityProcess (computing)Self-organizationPressureEnvelope (mathematics)Public key certificateServer (computing)Dependent and independent variablesExtension (kinesiology)Connected spaceInternetworkingKey (cryptography)NeuroinformatikContent delivery networkCloud computingCASE <Informatik>Computer virusMobile appSoftware testingPhysical systemComputer animation
06:14
ComputerAsynchronous Transfer ModePublic domainDirect numerical simulationTransport Layer SecurityGUI widgetFirst-person shooterClient (computing)Extension (kinesiology)Local GroupEncryptionLengthServer (computing)Address spaceSanitary sewerPublic key certificateDependent and independent variablesCache (computing)Web 2.0Hacker (term)Key (cryptography)Content delivery networkComputer networkInternetworkingPoint (geometry)EmailHost Identity ProtocolSoftware testingCryptographyStandard deviationDrop (liquid)Computer configurationDefault (computer science)Physical systemComputing platformNetwork socketDirect numerical simulationStandard deviationConnected spaceMereologyEncryptionServer (computing)Projective planeEmailOrder (biology)HexagonInformation securityInternet service providerKey (cryptography)Row (database)Public-key cryptographyPublic key certificateEnterprise architectureClient (computing)Public domainElectronic mailing listPoint (geometry)Content delivery networkQuery languageIP addressRoutingValidity (statistics)Scripting languageAdditionBlock (periodic table)Vector potentialSoftwareTransport Layer SecuritySharewareConfiguration spaceBinary codeCross-platformInternetworkingRevision controlCASE <Informatik>Buffer solutionCache (computing)Different (Kate Ryan album)Electric generatorShared memorySet (mathematics)Resolvent formalismArithmetic meanOcean currentChainLibrary (computing)Metropolitan area networkWebsiteRotationString (computer science)Computer animation
11:37
Public domainAsynchronous Transfer ModeContent delivery networkNormal (geometry)Transport Layer SecurityCloud computingPointer (computer programming)Quantum stateComputerSurjective functionComputer iconHoaxConnected spaceEmailServer (computing)Green's functionPrice indexStandard deviationPublic domainAdditionMathematicsMultiplication signFunction (mathematics)Transport Layer SecuritySource code
13:30
Public domainDirect numerical simulationSign (mathematics)PasswordAsynchronous Transfer ModeBlackboard systemElectronic mailing listOpen sourceServer (computing)HTTP cookieContent delivery networkPublic domainDirect numerical simulationService (economics)CryptographyWebsiteElectronic mailing listInformation securityBitComputer animation
14:33
Content delivery networkPublic domainAsynchronous Transfer ModeSingle-precision floating-point formatSign (mathematics)ComputerLine (geometry)Chinese remainder theoremTransport Layer SecurityTerm (mathematics)Hill differential equationServer (computing)Internet service providerIdentity managementSoftwarePublic domainExtension (kinesiology)Exception handlingStandard deviationVariety (linguistics)Single sign-onSource code
14:56
Content delivery networkAsynchronous Transfer ModeComputerWebsiteAbelian categoryBlock (periodic table)Digital filterProduct (business)Installation artData typePublic domainTransport Layer SecurityFirewall (computing)PiQuery languageWeb 2.0Set (mathematics)Firewall (computing)Decision theoryComputer animationSource code
15:31
Firewall (computing)Public domainAsynchronous Transfer ModeContent delivery networkEnterprise architectureIntegrated development environmentMathematical analysisTransport Layer SecurityInstallation artPublic key certificateRootMountain passCryptographyEncryptionDefault (computer science)Error messageUsabilityClient (computing)Revision controlDefault (computer science)Public domainFirewall (computing)Revision controlSoftwareEnterprise architectureTransport Layer SecurityIntegrated development environmentCryptographyUser profileFilter <Stochastik>Error messageMultiplication signSoftware testingRootPublic key certificateMetropolitan area networkComputer animation
16:42
Content delivery networkAsynchronous Transfer ModePublic domainPhysical lawProxy serverCryptographyFirewall (computing)WebsiteHash functionScripting languageWindowDefault (computer science)Public domainWeb browserElectronic mailing listFingerprintKey (cryptography)Public key certificateLoginRule of inferenceEnterprise architectureResultantIP addressGoogolSoftware testingConnected spaceDependent and independent variablesEmailServer (computing)Information privacyDirect numerical simulationCategory of beingVariety (linguistics)User profileTransport Layer SecurityPersonal identification numberInterface (computing)Computer animation
18:53
Communications protocolUDP <Protokoll>Asynchronous Transfer ModePublic domainContent delivery networkTransport Layer SecurityComputerStreaming mediaServer (computing)Query languageFirewall (computing)Proxy serverConnected spaceSoftware testingCommunications protocolSingle-precision floating-point formatServer (computing)Duplex (telecommunications)Computer programWrapper (data mining)Client (computing)TelecommunicationMultiplication signWebsiteService (economics)Regular graphLevel (video gaming)Computer animation
19:52
Public domainContent delivery networkAsynchronous Transfer ModeConnected spaceSoftwareResultantFirewall (computing)Projective planeComputer configurationCalculusSystem callLocal ringProxy serverIP addressFile viewerSocket-SchnittstelleClient (computing)Server (computing)Public key certificateStandard deviationSoftware testingNormal (geometry)Direct numerical simulationCryptographyMehrplatzsystemTDMAWeb 2.0WindowGame controllerPublic domainInternetworkingWebsiteVery-high-bit-rate digital subscriber lineWeb browserAcoustic shadowAdditionGoodness of fitExpected valueComputer animation
23:37
Asynchronous Transfer ModeContent delivery networkPublic domainSoftware frameworkSource codeInstance (computer science)Computer animation
24:11
Content delivery networkPublic domainAsynchronous Transfer ModeBit rateMereologyFile formatString (computer science)Pivot elementCorrelation and dependenceExecution unitCodeMathematicsConfiguration spaceKey (cryptography)ParsingLine (geometry)Public domainServer (computing)Computer configurationTransport Layer SecurityWeb browserComputer animation
25:20
Asynchronous Transfer ModeContent delivery networkPublic domainIRIS-TWindowClient (computing)Computer virusDoubling the cubeServer (computing)Hacker (term)EncryptionComputer animationSource code
25:48
Public domainContent delivery networkAsynchronous Transfer ModeRange (statistics)IP addressFunctional (mathematics)Expected valuePanel paintingSource code
26:43
Content delivery networkAsynchronous Transfer ModePublic domainTransport Layer SecurityBlock (periodic table)Extension (kinesiology)FlagRule of inferenceComputer networkWebsiteFingerprintServer (computing)CodeSystem administratorTelecommunicationSoftware testingSoftware developerDisk read-and-write headCryptographyClient (computing)GodPublic domainWindowOpen sourceCodeServer (computing)Transport Layer SecurityForm (programming)MathematicsSoftwareRule of inferenceLibrary (computing)Computer configurationClient (computing)CAN busReal numberInternet service providerGoodness of fitStandard deviationBlock (periodic table)Pattern languageContent (media)BitConnected spaceWeb browserFingerprintVirtual machineExtension (kinesiology)EncryptionDefault (computer science)Projective planeWeb 2.0Dependent and independent variablesComputer virusType theoryContext awarenessSingle sign-onInformation securityUsabilityComputer animation
30:39
TwitterPublic domainAsynchronous Transfer ModeComputer animation
Transcript: English(auto-generated)
00:02
Hello everyone, and welcome to Domain Fronting is Dead, Long Live Domain Fronting, using TLS 1.3 to evade sensors, bypass network defenses, and blend in with the noise. My name is Eric Hunstead, and I'm the CTO and Adversary Emulation Lead at 6Gen, a full-spectrum cybersecurity company based in Annapolis, Maryland. Today, I'm going to go over what Domain Fronting is, including HTTP and HTTPS basics.
00:25
I'll talk about TLS 1.3 and encrypted server name indication, and how this can be used for domain hiding. I'll show some demos of this in action, from basic bypasses, to bypassing an enterprise TLS decrypting firewall, and finally, I'll talk about some ideas for defenders who may want to detect this in their environment.
00:43
Domain Fronting 101 If we're going to discuss a new method for Domain Fronting, let's all get up to speed on what Domain Fronting is, why it works, and what happened to it. Like many things in security, to understand the cool stuff, you have to dig into the basics. To understand Domain Fronting, we really need to understand HTTP and HTTPS.
01:01
Specifically, we'll be focused on server name indication, which is a special extension added to TLS that allows multiple sites to be hosted on the same server. With TLS 1.3, this extension can be encrypted, and combined with encrypted secure DNS, we can do Domain Fronting 2.0, or domain hiding. Okay, let's start with HTTP.
01:22
I'll skip over the super low-level stuff as it's not relevant here. The first external request involved in an HTTP connection is a DNS request. This DNS request gets sent out unencrypted via UDP to port 53. The DNS server responds, also unencrypted via UDP, with an answer that contains the
01:40
IP address of the domain from the request. Now that the user's computer knows where the domain is hosted, it can issue an HTTP request to that IP address, with the host header indicating the domain the user wishes to browse. This request uses TCP, but it's unencrypted. The web server responds with content, in this case some simple HTML.
02:02
This was fine for the early days of the internet, when it was just a few research institutions, but as soon as people wanted to conduct business online, the obvious issues with completely unencrypted traffic had to be fixed. Plus, without encryption, all your data is free game for any number of actors that are positioned between you and the web server. HTTPS was the answer to this problem, and it starts off the same way, with a DNS request.
02:25
The DNS request and response are still unencrypted when using HTTPS, unless you use a different system which we'll talk about later on. Here's where HTTPS is different. The connection to the server starts with a TLS connection, specifically a client hello. In this packet, the server name extension is used to tell the web server which website
02:44
the user is requesting. This allows the web server to return the proper certificate for that site. The server returns the certificate in a server hello packet, and then the client and server can agree on which encryption algorithms to use and exchange session keys. All traffic after this is encrypted, but both the client hello and server hello are
03:02
unencrypted, which leaks the domain and certificate to anyone between the client and the web server. And we know there are plenty of organizations interested in that data. With the basics covered, let's talk about domain fronting. Domain fronting is a technique to get around network defenses or sensors by connecting to an approved server, while hiding the true destination of the HTTPS request.
03:25
One of the biggest restrictions of domain fronting is that the fronted domain and the true destination domain must be hosted by the same service provider. Typically this is a content delivery network. So how does domain fronting work in practice? Well, it starts with the same DNS lookup as before, and the TLS handshake is the same
03:42
as well. However, in the handshake, the client connects to the front domain, not the true destination domain. Once the TLS handshake is complete with the front domain, the fronting begins. The client sends an HTTP request to the front domain wrapped in TLS, but with a host header of the true destination domain.
04:01
When the content delivery provider receives this request, it inspects the host header, determines that it's for a different domain hosted on the same provider, and dutifully forwards the request to the true destination domain. You can think of domain fronting like a postcard inside an envelope. On the outside of the envelope, the client writes the address of the CDN, but on the
04:20
inside, the true destination domain is on the postcard. The network filters or sensors are like mailmen, who can see the outside of the envelope, and they deliver it, since the CDN is an approved address. However, when the CDN opens the letter, they deliver the postcard to the true destination internally. This was working well on all major CDNs until April of 2018, when the Russian government
04:42
put pressure on cloud providers to stop it, since the popular messaging app Telegram was using both Google and AWS for domain fronting. Both Google and Amazon stated that the reasons were technical and domain fronting was either never supported or a violation of terms of service, and other providers followed suit to include Cloudflare.
05:00
Microsoft's Azure platform still allows domain fronting for now, as do a few other smaller CDNs. Domain fronting is a great censorship or network bypass technique, but it has some major problems. For any network censorship bypass to be effective, it has to be really painful to block, and since the largest provider shut it down, it lacks the punch that it once had.
05:21
Also, the true destination domain has to be hosted on the same service as the front domain, which limits the potential front sites and imposes costs on anyone wishing to set up fronting. With the background out of the way, we can talk about TLS 1.3, encrypted server name indication, and what can be called domain hiding. First, let's talk about TLS 1.3.
05:41
It's seeing steady growth on the internet today. This is data from Quali's SSL labs, which tests the most popular 150,000 sites. As you can see, TLS 1.3 is on the rise, and is supported by 31.7% of those sites tested last month. Cloudflare is seeing 59% of all traffic using TLS 1.3, again showing a steady increase.
06:04
Before we dig into the mechanics of a TLS 1.3 handshake with ESNI, we first have to secure the DNS process. If a network defender or sensor can see our DNS requests, they can easily be blocked. Also, secure DNS is a cornerstone of ESNI, which we'll see in a moment.
06:21
The first solution is to simply wrap a DNS query in a TLS connection. To make things even more difficult for network defenders or sensors, DNS via HTTPS is a newer standard that is seeing more widespread adoption. Once we have an encrypted way to perform DNS queries, we can use this to retrieve a public key for a domain which we can use to encrypt the server name in the client
06:42
hello packet. In this screenshot from Wireshark, you can see the encrypted underscore server underscore name TLS extension, and the encrypted SNI value is just an encrypted hex string. No longer can defenders use the server name value to block traffic based on a plain text value.
07:00
A TLS 1.3 connection with ESNI relies on having the server's public key in order to encrypt the server name in the client hello, and the current standard for how to get this key is to use DNS and query the underscore ESNI text record for that domain. This request returns a base64 encoded public key the client can use to derive a session
07:21
key that encrypts the server name. Providers that support this should rotate this key to provide some forward secrecy. Cloudflare for example rotates this key every hour, but allows a few hours of buffer in case a client has a slightly stale key from a previous request or from caching. Armed with the server's ESNI public key, the client derives a session key and uses that
07:44
to encrypt the server name and sends the client hello. The server has the corresponding private key and can drive the same session key which it uses to decrypt the server name and performs the same steps as standard TLS. This session key generation ensures that the key is tied to the session that generated
08:00
it and prevents replay attacks. Another difference between TLS 1.3 and previous versions is that the certificate returned by the server is encrypted. This is possible because in TLS 1.3 the client hello includes a list of supported ciphers and the client also makes a guess as to which key agreement algorithm the server
08:21
is going to choose and it preemptively sends a key share for that algorithm. So all the plaintext portions of HTTPS we saw before have been mitigated. However there's still weak spots where sensors or network defenders could stop the connection. Even if the DNS request itself is encrypted inside either TLS or HTTPS, the data returned
08:42
by a resolver may be poisoned. Imagine a scenario where a country or enterprise returns their own key for all underscore ESNI requests no matter the domain to enable man in the middle. DNSSEC solves this problem by signing the record set and allowing the chain of trust to be checked. The details of DNSSEC are outside the scope of this talk but I encourage you to check
09:02
it out. But what if encrypted DNS is completely blocked? The solution here is to bootstrap the connection by pre-loading the current ESNI keys by some other means. Maybe encrypted DNS is blocked but what if I have a script that updates a github gist with the current keys every hour? Or you could just bake the keys directly into your binary if you know it's going
09:22
to be executed in the next hour or two. Okay let's say we get a valid ESNI key but the IP address of the server we want to connect to is blocked. The IP may be shared with other sites but it doesn't have to be. Is there a way to route from arbitrary domains and IPs to our true destination domain and IP?
09:40
The answer is no but let's see how close we can get. So far I haven't introduced anything new but we have covered a lot of ground and identified potential issues and solutions around domain hiding with TLS 1.3 and ESNI. Let's tackle that last point and get around IP or domain blocks. We'll do this by leveraging the world's largest CDN.
10:03
Cloudflare was founded in 2009 and has had explosive growth. It's now the world's largest CDN with the most internet exchange points as well as being an authoritative DNS server hosting over 26 million domains. Cloudflare is on the forefront of internet technology and supports TLS 1.3, ESNI, WebSockets,
10:23
QUIC and DNSSEC. This new technique, which I'm calling domain hiding, accomplishes the same goals as domain fronting but uses different technologies. A TLS 1.3 connection with an ESNI of the true destination is made to any Cloudflare IP and the underlying HTTP request also has a host header of the true destination.
10:44
This enables any Cloudflare-owned IP to act as a front for any domain hosted by Cloudflare DNS. This technique was first shown to be possible by Robin Wood, so props to him for the discovery. Today I'm releasing Noctilucent, a project that enables ESNI in Go's crypto-slash-TLS
11:03
library as well as exposes two additional configuration options, ESNI Server Name and Preserve SNI. It can be used by any project currently using the standard crypto-slash-TLS library as it's backwards compatible. It comes with a demo client application that allows you to control just about every part
11:21
of the TLS connection as well as the HTTP GET request so you can play with what's possible using this technique. It also supports WebSockets and through the magic of Go, it compiles cross-platform to a single binary with no dependencies. Here's what a standard TLS 1.3 connection looks like with Noctilucent.
11:42
We've specified the TLS host in red, the server name indication in yellow, and the HTTP host header in green. The output shows what we expect. There's no ESNI, the SNI is set to Cloudflare.com, we're connecting to Cloudflare.com on port 443, and our GET request to Cloudflare.com returns the expected 301 response.
12:04
Nothing fancy yet. Let's enable ESNI and set the ESNI server name. Everything is the same as the normal TLS 1.3 connection we looked at on the last slide, except this time the ESNI value is set to Cloudflare.com and there is no standard
12:21
SNI. Progress, but no fronting or domain hiding quite yet. Now I'll change the ESNI server name and HTTP host header to a domain we control, in this case, defcon28.acthis.computer, but leave the TLS host as Cloudflare.com. Notice that this connection is still going to Cloudflare.com, but the HTTP request successfully
12:45
goes to defcon28.acthis.computer, which returns hello defcon. To anyone between this host and Cloudflare, it appears to be a TLS connection to Cloudflare.com. We've successfully achieved domain hiding. Now the fun begins.
13:01
Here we send an ESNI and standard SNI, where the standard SNI is Cloudflare.com. Because Cloudflare ignores the standard SNI when an encrypted SNI is used, we can set it to anything. The HTTP connection works as before, and this time, anyone between this host and Cloudflare
13:20
looking at SNI data thinks this connection is going to Cloudflare.com, in addition to the underlying TLS connection actually connecting to a Cloudflare.com IP. What I've demonstrated is the ability to arbitrarily front or hide any IP behind a domain that is using DNS provided by Cloudflare. The true destination IP doesn't have to be a Cloudflare worker or other Cloudflare
13:43
service. In fact, all the examples today are IPs hosted by DigitalOcean. Best of all, the requirements to sign up for Cloudflare DNS are minimal, and it's free. What domains can we use for this hiding? It turns out a lot. Over 21% of the top 100,000 sites are behind Cloudflare and allow this to work.
14:06
Some notable examples are shown below, including three of 11 domains, which are both on the Palo Alto decryption whitelist and behind Cloudflare. I'll demonstrate why this is important in a minute. There's a little bit of everything on this list, from security-related sites, banks, sports,
14:25
higher education, streaming services, and even government sites. And porn. So much porn. Here's a good example of how the variety of available domains to hide behind can be useful. This is a request to www.octa.com, a single sign-on and identity provider used by many
14:44
enterprises, but the actual request is going to a server I control. To a sensor or network defender, this looks like standard Octa traffic, with the exception of the ESNI extension being used. The logical next step is to try out Noctilucent with some web filtering and see how it fares.
15:02
It turns out most web filtering is done by inspecting the SNI and making a filtering decision based on that. Here I set up the Untangle firewall with federal government settings and explicitly blocked defcon28.hackthis.computer. On a host behind the Untangle firewall, I made this request, which succeeded.
15:22
I set the unencrypted SNI to bypass-untangle.com to see if it appears in the logs anywhere. After repeating the request 30 times or so, back on the firewall, we can see that the top domains include the decoy domain that we sent with the request.
15:41
Simple SNI filters were easy to defeat, but what about HTTPS decrypting firewalls? These are often seen in enterprise environments and work by breaking and then re-encrypting HTTPS using a root certificate placed on endpoints behind the firewall. You can think of it as a corporate man in the middle that allows the network defenders
16:00
to inspect full, encrypted packet data. Kazakhstan actually attempted to implement this nationwide for a few weeks back in July of 2019 before major international backlash forced them to abandon what they called a quote, test. I headed to AWS and spun up the latest and greatest in enterprise firewalls, a Palo
16:21
PAVM version 10.0.0. Released in June, this version touts the ability to decrypt TLS 1.3 with a two-time speed boost. Imagine my surprise when it was all set up and the default decryption profile didn't include TLS 1.3, instead it let it pass through while logging an error about a version mismatch.
16:42
Once set up with a decryption profile that enabled decryption of TLS 1.3, I pulled the list of default domains that are allowed to bypass decryption for a variety of reasons, such as having a PIN certificate. Here, the firewall interface is on the left and a Windows VM behind the firewall is on the right. We can see that decryption is enabled and we have no other decryption rules that would
17:03
allow traffic to bypass decryption. Normally, in an enterprise environment, categories like banking or healthcare bypass decryption due to privacy laws, but this setup is as strict as possible, no explicit exemptions. On the Windows VM, I'll run a PowerShell script that checks the certificate hashes of major sites against hard-coded known fingerprints.
17:24
We see that it fails, indicating that all tested sites are being man-in-the-middled. Additionally, opening a browser and going to google.com shows a lock icon, but inspecting the certificate shows that it's the firewall certificate. Switching to the command line and using the Noctilucent test client, I'll set the
17:40
TLS host at mozilla.org, the host header to defcon28.hackthis.computer, and the unencrypted SNI to mozilla.org, which is on the decryption exemption list. The SNI is preserved and set to mozilla.org. A connection is made to mozilla.org on port 443 and we see the hello defcon28 response we expect from the test server.
18:03
We'll run a nslookup of mozilla.org so we can look for the connection in the logs. Looking at the firewall traffic logs shows a connection to one of the IP addresses from mozilla.org.
18:22
Moving to the decryption logs, we see DNS over HTTPS, which is the ESNI key lookup that is performed before each TLS connection, but no traffic to mozilla.org's IP. To make sure we didn't miss it, I'll add a filter for the two possible IPs.
18:42
No results, because mozilla.org is exempted from decryption by default. We have successfully defeated an HTTPS decrypting firewall by using built-in default exemption and domain hiding. This bypass is great, but every time we want to send data, we'll add an entry to the traffic log, which could stand out. What if there was a better way to hide?
19:02
This technique works at the TLS level, so any protocol that Cloudflare supports underneath would also work. We saw that regular HTTP requests work, but create a new connection for each request. WebSockets are a technology used by many streaming services that enable full-duplex communication channels over a single TCP connection, and we can take this a step further by using
19:24
one or more of these WebSocket connections to tunnel arbitrary TCP or UDP packets to a proxy beyond the firewall. Here's the Noctilucent test client sending and receiving via WebSockets to our test server while hiding behind www.octa.com.
19:41
WebSockets are the perfect candidate to use as a wrapper for a proxy protocol that allows us to send arbitrary TCP or UDP traffic using a helper program while only making a single TCP connection. The Noctilucent project includes a fork of the amazing Cloak project by Andy Wang that enables two new options in the client configuration, ESNI server name and PreserveSNI,
20:04
which operate just like they did on the test client. Cloak is a universal pluggable transport that cryptographically obfuscates proxy traffic as legitimate HTTPS traffic, disguises the proxy server as a normal web server, and multiplexes traffic through a fixed amount of TCP connection while providing multi-user
20:22
usage control. With the additions from Noctilucent, it can now do domain hiding as well. Here's the same Palo Alto and Windows VMs from before, with a public IP of 3.14.44.10. This time, the Windows VM is running a Noctilucent Cloak client and a local ShadowSocks client.
20:43
The Cloak client is connecting to a VPS running the standard Cloak server and a ShadowSocks server. Firefox is configured to use the local Socks proxy provided by the ShadowSocks client running locally and I can browse the internet as I would normally. The certificates of websites in this proxy browser are legitimate as they're not being
21:02
intercepted by the Palo Alto firewall, and all traffic is appearing to come from the DigitalOcean VPS. The speed is surprisingly good. During testing, I was getting speeds up to 100 megabits per second using four TCP connections
21:22
with Cloak. If we look at the firewall logs, we only see the four connections to Mozilla.org, which are not decrypted, while the client has been browsing freely, with all traffic remaining encrypted and uncensored. Switching to the decryption logs, we see the DNS over HTTPS requests, but no requests
21:44
to Mozilla.org. I'll filter for the Mozilla.org IPs, but we'll get no results. What we see here is arbitrary traffic at high speeds using only four TCP connections to what Palo Alto thinks is Mozilla.org.
22:04
Can we use this technique for our pre-existing red team tools? If they support proxying, you bet. Here's a Cobalt Strike listener with a SOX proxy setting for localhost port 9999. For fun, we'll set Cloak to use bitdefender.com as the front domain, which has the IP address
22:22
shown here. After starting the Noctilucent Cloak client and local shadow SOX server in SOX4 mode, we'll double-click on an innocent-looking EXE. We get the callback immediately, and it works as expected. And of course, the first thing you do when you get access is pop a calc.
22:48
Switching to the firewall and looking at the decryption logs, we only see entries for CloudFlareDNS.com. I'll paste in a filter for the two possible IPs for bitdefender.com, and we get no results.
23:09
Looking at the traffic logs, we see four connections to the IP of bitdefender.com, as well as lots of DNS over HTTPS.
23:21
Our Cobalt Strike beacon is still functioning as normal, and any network defender is none the wiser. Packaging up Cobalt Strike, Cloak, and Shadow SOX into a clean package, possibly communicating over name pipes instead of sockets, is an exercise left up to the viewer. Red teamers are no doubt wondering how much work it would be to integrate this with
23:42
their existing tooling, and I can say if you're using Go, it's almost no work at all. Here's a new C2 framework called DeimosC2. It was released just five days ago, and is built using Go and Vue.js. I've set up an instance here with an HTTPS listener with a public IP on port 443.
24:04
Going into the listener settings, we can view the source code for an HTTPS agent. It has the C2 host and port in the code, and the agent will connect directly to this host. Switching to an editor, I've made a few modifications to get this to work with Noctilucent.
24:23
First, I set the host to bitdefender.com, and I've added two new variables, front domain and actual domain, which are set to bitdefender.com and deimos.hackthis.computer. The actual code changes needed to get everything working is just 30 lines. First, we have to query for the ESNI keys and parse them.
24:44
If we wanted to, we could bake in the current keys here to avoid the network traffic related to querying for them. Then I define a TLS config, and set the minimum value to TLS 1.3. I'll define the two new options from Noctilucent, ESNI server name is set to the actual domain, and preserveSNI is set to true.
25:04
I define an HTTP client by hand, so that I can tell it to use this TLS config when dialing the host. Besides that, the only change to the agent is having the HTTP client use the actual host when it makes its POST requests.
25:21
Under Windows VM, I'll set a filter in Wireshark for the IP of bitdefender, so we can inspect the client hello. I'll double click this new innocent exe and switch back to Wireshark. In the client hello, we see the unencrypted server name is bitdefender.com, and the encrypted server name is just a bunch of hacks.
25:49
Back on the C2 server, we have an agent calling back. Notice the external IP. When using this technique, the reported external IP will be from one of Cloudflare's ranges.
26:11
Just like with Cobalt Strike, everything works as normal. We can run whoami, pop a calc, and check the public IP.
26:28
Besides the external IP appearing to be from a Cloudflare range, the agent functions as expected.
26:44
Back on the Windows VM, I'll search for traffic to the C2's server IP. No packets. There's no trickery here, but just a small modification, this Go-based agent is using domain hiding. So what is the blue team to do, given that we've just seemingly bypassed the most sophisticated
27:02
network defenses? The easiest thing to do would be to block TLS 1.3, but as that's likely anywhere from 25 to 50% of the TLS traffic leaving your network, it might make a few users unhappy. You could block Cloudflare, but dropping 21% of the most popular 100,000 domains probably
27:21
is not a good idea either. What about just targeting client hellos with ESNI's? It's possible, and one vendor has had the ability to do that for six months now. However, since you can't determine the destination site, it's all or none blocking. There's no way to only block selective domains if you block TLS connections that have an ESNI.
27:41
I was surprised by the lack of support for this across major vendors, but perhaps it's just not an issue yet. The most sensible way to detect this specific technique is to alert on packets that contain both a server name and an encrypted server name TLS extension. There are no default helpers for TLS in Snort, besides SSL underscore state, which can help
28:02
you get the client hello packet, and TLS dot version, which can get you TLS 1.3, so you'll have to get creative with the content parser and look for the extension types. SecureCata has a bit more horsepower than Snort, but only has a selector for TLS dot SNI and not TLS dot ESNI, so custom rules will be required here as well.
28:23
It's going to come down to tried and true techniques that work regardless of traffic content. Things like beaconing detection and anomaly detection can help show clients that all of a sudden start sending regular traffic to destinations that perhaps don't make sense. There are even some open source tools that do just this.
28:40
A network defender should be asking if a traffic pattern to a given endpoint makes sense in context. Perhaps that Windows machine in accounting shouldn't be sending gigs of data to your single sign-on provider. TLS fingerprinting with JAW3 and JAW3S is another technique that could be effective. JAW3 looks at the specific options used in a client hello that can sometimes uniquely
29:02
identify a client, and JAW3S does the same for servers. This will work until attackers tune their tools to be identical to browsers and common web servers, and work is already underway to do this in projects like Cloak and UTLS. As always, a layered approach that uses strong, instrumented and managed endpoint detection
29:21
and response and other techniques is the best way to prevent compromise. Or as Microsoft so helpfully says, just don't let untrusted code ever run on your server. To wrap up today, I've discussed the basics of HTTP, HTTPS, domain fronting, and a new form of censorship-resistant communication, domain hiding.
29:43
This technique is usable today with Noctilucent either as a drop-in replacement for the Go TLS library or as a proxy. The ESNI standard is still very much a draft, and this technique will continue to evolve as it changes and adoption grows. In fact, it's not even called ESNI anymore.
30:01
Just as with standard domain fronting, this relies on the CDN permitting it. However, in this case, besides preserving the standard SNI header, we aren't really doing anything terribly out of spec. I look forward to seeing the SNORT and secure ACADA rules to detect and block this technique, and like every good red-teamer, I'll push my customers with the latest techniques, hopefully
30:20
before the real adversary can. I want to give special thanks to Robin Wood, who first showed this technique was possible with a rough POC using a fork of OpenSSL, Andy Wang, the creator of Cloak, and Nick Sullivan from Cloudflare, who helped get me data on TLS 1.3 adoption. Noctilucent is available now on GitHub.
30:41
If you have any questions or comments, feel free to hit me up on Twitter, where my personal handle is at badsectorlabs. Thanks for watching.