Packet Hacking Village - Hunting Certificates and Servers
This is a modal window.
Das Video konnte nicht geladen werden, da entweder ein Server- oder Netzwerkfehler auftrat oder das Format nicht unterstützt wird.
Formale Metadaten
| Titel |
| |
| Serientitel | ||
| Anzahl der Teile | 335 | |
| Autor | ||
| Lizenz | CC-Namensnennung 3.0 Unported: Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen. | |
| Identifikatoren | 10.5446/48742 (DOI) | |
| Herausgeber | ||
| Erscheinungsjahr | ||
| Sprache |
Inhaltliche Metadaten
| Fachgebiet | ||
| Genre | ||
| Abstract |
|
00:00
ServerHackerServerBitComputeranimation
00:35
SoftwareSoftware EngineeringComputeranimation
00:50
MereologiePortscannerInternetworkingZeichenketteCodierung <Programmierung>Domain-NameCodierungMereologieÄhnlichkeitsgeometrieDomain-NameStrömungsrichtungDatensatzMultiplikationsoperatorRegistrierung <Bildverarbeitung>InternetworkingDivergente ReiheBelegleserBildschirmmaskeCodeWeb SiteComputeranimation
02:22
ProgrammierumgebungInternetworkingRegistrierung <Bildverarbeitung>Computeranimation
02:37
Gerichtete MengeCodierung <Programmierung>Message-PassingInternetworkingPortscannerGray-CodeMessage-PassingEinsTwitter <Softwareplattform>Computeranimation
03:04
InternetworkingTLSInternetworkingExpertensystemTLSComputeranimation
03:37
ServerZufallszahlenVorzeichen <Mathematik>SpezialrechnerTLSRadikal <Mathematik>ServerClientMAPKryptologieIndexberechnungEinfach zusammenhängender RaumBefehlsprozessorEinfache GenauigkeitZentrische StreckungPunktBrowserTLSSpeicherabzugComputeranimation
05:07
GoogolDirekte numerische SimulationHumanoider RoboterSpeicherbereichsnetzwerkMarketinginformationssystemQuellcodeMaßerweiterungHydrostatikTLSGoogolMailing-ListePlastikkarteDomain-NameMultiplikationsoperatorComputeranimation
05:22
InternetworkingTLSPortscannerKeller <Informatik>DatenstrukturClientServerTelekommunikationAnalysisSyntaktische AnalyseSchätzungRuhmasseDiskrepanzKonfigurationsraumQuick-SortServerParserInternetworkingNetzadresseProgrammierungWeb logClientDigitales ZertifikatSchaltnetzTLSMinkowski-MetrikSyntaktische AnalysePortscannerMereologieEinfügungsdämpfungVerdeckungsrechnungOffene MengeCodeComputeranimation
07:42
TLSKanal <Bildverarbeitung>InternetworkingServerNetzadresseService providerGrundraumEinfach zusammenhängender RaumPortscannerTLSComputeranimation
08:38
MereologieDirekte numerische SimulationMereologieBildschirmmaskeDirekte numerische SimulationSchaltnetzNetzadresseComputeranimation
08:58
Direkte numerische SimulationFächer <Mathematik>BinärdatenDatenstrukturTopologieMultiplikationsoperatorWeb logMinkowski-MetrikSchnittmengeCASE <Informatik>DatenstrukturOrdnung <Mathematik>CodeDatenkompressionReverse EngineeringDirekte numerische SimulationGrößenordnungElektronische PublikationSkriptspracheWurzel <Mathematik>ZeichenketteMini-DiscPunktCOMComputeranimation
11:02
Direkte numerische SimulationPuffer <Netzplantechnik>LaufzeitfehlerDirekte numerische SimulationCodeServerBenutzerbeteiligungBruchrechnungDateiformatPufferüberlaufSchnittmengeComputeranimation
11:44
QuellcodeWeb logMetropolitan area networkWeb SiteRuhmasseQuellcodeSystemprogrammServerComputeranimation
12:16
QuellcodeArithmetisches MittelSchnittmengeGüte der AnpassungComputeranimation
12:33
AbfrageDirekte numerische SimulationTLSQuellcodePufferspeicherServerIdentitätsverwaltungHackerDemo <Programm>Explosion <Stochastik>Zentrische StreckungSchaltnetzProgrammierungSchnittmengeMailing-ListeResultanteWeb logServerSoftwareschwachstellePunktPufferüberlaufDirekte numerische SimulationDatensatzGüte der AnpassungPortscannerVerschlingungInternetworkingMereologieDemo <Programm>TLSComputeranimation
14:35
Demo <Programm>TVD-VerfahrenMultiplikationsoperatorZweiBelegleserAppletRPCWeb logResultanteCodeComputeranimation
15:26
Konvexe HülleDemo <Programm>Reverse EngineeringComputersicherheitWeb-SeiteDirekte numerische SimulationFormation <Mathematik>ZeitzoneE-MailSchlussregelElektronische PublikationComputeranimation
Transkript: Englisch(automatisch erzeugt)
00:00
Well, so it's it's gonna be just 229 right now and Originally, we're gonna we were gonna have another person introduce Introduce Sam, but he's a little bit camera shy. So I guess I got to do it. So I guess the best way I can introduce our next speaker is
00:22
Council nine black badge winner Sam Thank you Thank you Hi, my name is Sam. Today I'll be presenting on hunting certificates and servers. And so let's get started Our team council of nine has won the black two black badges through the badge challenge
00:44
As I mentioned, I'm also a software engineer at Akamai So one thing I have to be very clear here the pain is expressed here on my own no one else's and Be careful when you connect to any host online and always seek explicit permission before attempting to
01:03
exploit etc so There's three parts of this so part one is getting the internet. So this is actually a Defcon story At the as part of the badge challenge We noticed the person who would run the badge challenge every year One zero five seven would use the same TLDs over and over again in particular
01:23
He was very fond of the dot codes TLD Unfortunately, there are not a lot of dot codes Domain names so he picked up on this and started monitoring for any new dot codes registration online and as well as scanning the internet for domains hosted that contain a dot code
01:42
To domain we'd also then search for relevant who has records as well And this was a pretty closely held secret by our team There are about ten people or so that knew this at the time and I'm really only sharing it now simply because it's the contest is largely over in its current form anyways, and
02:01
So our scanners actually worked they picked up one domain We got a slack notification that gray dot codes was registered and it contained lost handle Therefore it must be him obviously So it contained a series of puzzles. So we visited the website We solved we spend about 72 hours not really sleeping solving the puzzles thinking it was a registration for a contest or something similar and
02:23
We sent through the solution on the final step this was November 2nd 2016 and Lost replied back. I don't know what you're talking about essentially and it appears you controlled So I sent him I sent gray codes a message on Twitter asking who he was and what he's doing and
02:45
It turns out we weren't the only ones who got trolled all of the other teams who are competing against us At Def Con that year also got trolled and Yeah, it actually turned out it was one of the ten people that we have told about this trolled the rest of it us using our own scanner
03:03
So that was kind of my first introduction to scanning the internet and Doing large-scale reconnaissance in the IPv4 space and that really got me thinking of what else I could find and search for on public internet and One thing I'm very familiar with is
03:21
TLS certificates and I as you might know TLS certificates contain host names and So before I talk about scanning the internet for TLS certificates I have to talk about TLS and this is going to be really basic. I'm sorry if you're a TLS expert TLS handshake you send over in the
03:42
Client hello you send over what you're connecting to in the server name indicator or who you're asking for Then the server will return back a certificate to you and then there's a handshake that occurs that drives the secret that's used to communicate and Yeah, so that's that's kind of like a high level of how TLS works
04:02
So if you go to HTTPS any website today, this is what your browser will be doing So I want the host names that are found in that second step there and so I wanted an efficient way to Find to get these host names at scale
04:20
Ideally from as small server as possible. I'm not operating with any budget here. I'm running VPS with a single CPU core so I cut off the rest of the connection and So when I get a certificate back from the Host name sorry when I get a certificate back from the server I would just immediately kill the connection and not continue along with the heavy crypto that's involved in doing the rest of the handshake
04:46
This isn't really anything Crazy it's entirely by the spec you're allowed to exit out of the handshake at any point if something goes wrong So yeah, so by doing this I'm able to
05:01
save CPU costs on my server and connect to many more servers for as cheap as possible So what's the next time I certificate just to give you an example? This is one for google.com and as you can see it contains many many host names And this is fairly common. You'll see either wild card or just a list of domain names
05:23
so as I kind of mentioned previously I Scan the internet for these so for every so I set up a scanner so for every IPv4 host I did two things I ran mass scan over it which just checks whether or not port 443 is open I do that because nothing I write will be faster than mass scan
05:42
So this is a good first filter to prove that 443 is open I Then wrote a Golang program that sends the TLS client hello And then gets the server certificate back and then just immediately disconnects and so the pseudocode roughly looks like this there's a few parsing steps in between, but this is essentially all that I run and
06:03
I had to modify the Golang TLS stack to get this early return because it's not really a standard thing to do it was Actually surprisingly easy. I really love working with the Golang TLS stack It's much much easier than working with OpenSSL if you're familiar with OpenSSL so
06:21
This is what I ran or set up and ran and it took about 72 hours or so so I got about 12 gigabytes of data back which was host name IP address combinations and I then asked myself some really basic questions like am I finding every host and this scanner identified 51 million hosts online in the IPv4 space a
06:42
2015 paper that I found identified 42 million one really interesting thing is Shodan only finds 42 million today I don't understand the discrepancy between my scanner and Shodan I suspect that Shodan has been blocked from parts of the internet But I don't have any data to back that up And I have to ask myself am I finding every certificate online and the answer really is no
07:03
It's common server configuration In that client hello I could specify an SNI. It's a client server configuration that the TLS client hello is used to differentiate clients and Based on what you pass in there, they will turn a different Certificate back to you. And so this scanner will miss all of those
07:21
Another thing I have to point out is that the Golang X.509 parser is very strict so if your server was doing something weird with certificates or Really just had any sort of non-standard X.509 cert it likely will not pass through Golang's X.509 parser and there's actually a blog post on how to parse malformed certificates there if you're interested
07:43
So I also have to ask who else is doing this because none of this is really New or revolutionary in my opinion, so I set up a TLS server, and I just captured traffic and I set up just got the pcap back and I actually found three servers that did the exact same thing that I'm doing two of them were from universities
08:05
Which were scanning the internet for their own research purposes and one of them was from a hosting provider in Germany as well So clearly there's somebody in Germany who's doing the exact same thing and simply not publishing it in any paper that I found I believe that they're doing the same thing as I am because
08:23
Of the way that they closed the connections as well as there was no SNI sent in the connection They made to me so clearly they weren't trying to connect to a host They were just simply trying to scan IP range or connect to my IP address You just get back the certificate that I would return So I now have 12 gigabytes of hostname
08:44
IP address Combinations which is a lot of data, and it's like not the most friendly form to work with 12 gigabytes of text so This is part two which is how to search large parts of large DNS datasets And so this actually came about because I want to use the rapid7 datasets so in the rapid7
09:06
DNS they have DNS listeners online for fdns and rdns requests and Forward and reverse DNS requests and they You can just go to their website download this it's a great resource. I love it
09:20
It's just really hard to work with because it's 10 gigabytes of compressed text which expands out to about 100 gigabytes of uncompressed text and So this always took the long time to search every time I wanted to search through this it took about 20 minutes Which is a little bit insane so I found myself trying to write better Or trying to script. I should say better or more fast ways to decompress and grep or
09:46
Use more disk space to grep faster, and this always just took a long time, and I actually wrote a blog post about this Which is linked to there back in February and so in order to
10:01
Sort the data I took advantage of the DNS structure so DNS is structured such that when you make a DNS request for in this case blog.rbsam.com you actually first go to the root which is dot roughly com and then you go to com and then you go to com.rbsam and then you go to com.rbsam.blog and
10:21
So you can take advantage of this in order to sort the data, so I reversed every string roughly and Sorted it at this point because the data is sorted in order to find a Hostname that I'm looking for in this data set. I simply have to binary search it which is in order of magnitude faster than
10:45
grepping through the files If you're familiar with how binary searches work I talked about this in much more detail in my blog if you're interested in the technical details I also have code online on how to do this and This is actually something that's what's online as well
11:03
So I put this online using a go along web server. It's available today on DNS stop buffer over run buffer over dot run Slash DNS, and you get back data from the rapid seven datasets This I use this myself It's a great way to just quickly grab through them if you're searching for something and I also posted the runtime on there
11:23
The runtime is usually a fraction of a second to binary search of this data And I also linked to my github account here, which has the data or has the Code to generate this server in it or generate. Sorry convert the data into us
11:41
Searchable format that's usable by the server So one day I woke up and I checked the traffic for this website And I saw this and I thought man, I'm a really good blogger But what had actually happened was I got pulled into something called a mass which is a
12:00
utility for host Hostname reconnaissance death published by OASP and this was completely unknown to me. Luckily the server held up I'm a little bit proud of that personally, but that was the That was the source of all this data and The fact that it's so heavily cached means that everyone searching for the same host names or they're repeatedly searching for the same host names
12:23
Over and over again, which I found really interesting So I actually don't collect data on this I don't actually know what everyone's searching for and I even frankly don't really want to so now that I have a good data set and I have a good way to search it I want to put it all together and So similar to the DNS records with rapid7
12:43
I hosted this online at TLS buffer overrun over dot run slash DNS And this is actually online today, and you're welcome to try it out It's literally just a combination of the Dataset that my scale scanner picked up as well as
13:02
the server behind the rapid7 dataset and This is actually fully automated at this point. So this will refresh once a week So I then have to compare myself to what else is out there I'm mostly just curious to see if this is even necessary and
13:21
So shonan.io should contain similar results. It's not free. I found that My TLS scanner tends to pick up some more results than shonan Which I found interesting and that comes back to my previous point that I think shonan is being blocked by large parts of the internet But I don't have any data to really prove that certificate transparency monitors are awesome and but they only contain publicly trusted
13:42
certs and they don't link back to where the cert came from and rapid7 actually has a TLS dataset, but It's only the new certificates they encounter in their scans every week They don't have historical data unless you have an account with them And there are many others the OAuth AMS tool has a great list of existing resources online
14:03
so Just give a demo here. Obviously hack yourself first plug in your company's name. See what you find. I find that really interesting there's also when I build these tools one of the first things I do is I run them against .mil and I report whatever I find back to the DoD vulnerability disclosure program. So if you're interested in finding vulnerabilities
14:25
Go look at the 473,000 results here find a hostname try and exploit it report what you find So I did that and I found a WebLogic remote code execution that was still online and from
14:43
2017 exploit and I was able to Exploit and report it in and I just do this simply to test my tools, but at the same time it's because of the Because of the military has such a variation of Technology online because it's all subcontracted out. It's not all PHP. It's not all Java
15:04
your tools will likely pick up something interesting and So actually one interesting thing about this is that they actually blocked outbound traffic which made Which would likely make automated scanners fail here So I was able to demonstrate this by injecting a sleep of 12 seconds
15:21
And then the results of 12 seconds to return back to me demonstrating RC So, yeah pack yourself first hack your military first Thank you, I guess questions I
15:45
use Linode with Linode if you are a Security researcher or sorry if you are doing something for security research You can get a security researcher designation on your account Which means that as long as you follow their rules and you have like a
16:03
Page that links to what your tool is doing connecting to other hosts. They will Automated they'll have an automated reply to anybody filing abuse complaints against you. No, it's only
16:23
It's it it matches based on the reverse of the DNS name So you could do like dot mill army dot mill and it would return you everything underneath that zone Any other questions, thank you. I hope this is interesting