He then came back to Recon Village last year to tell people how not to do that because of what he did the first time. And now he's back again. So he kind of loves coming back to our village, and we're really pleased to have him. And without further ado, I'm gonna hand over, and enjoy. Perfect, thanks.

Thank you guys, thank you everybody. And I just wanna say, you know, speaking at a smaller village, not that this is a small village, but speaking at a smaller village, it's really intimate, it's really awesome, and I like to see my audience, even though I brought like half of you guys here. So I just wanna say thank you guys very much.

You can follow me on Twitter there at the handle that you see there. That's also like all of my social media, pretty much. So there you go for doxxing myself. All right, and actually, here we go, starting with who I am or who am I. I know that's a funny little tagline there. But this is actually my favorite slide because it reminds me of Mushu the dragon from Mulan.

Right, who am I? Who am I? I am the Gray Noise co-founder and co-host, well, which is a podcast if you don't know. I'm a B-side speaker, I'm a Defcon speaker, I'm a Recon Village speaker, I'm a Telefreaker, I'm a 2600 writer, and I'm a board member of the Synchop locally here in Las Vegas.

I try, I try to be a comedian, maybe it's good, maybe it's not, I don't know. Who was there? Was anybody there? You don't count. I'm kidding, I'm kidding, Muffins. Okay, so I wanna know who you guys are, and that's probably for my own own synth. But any first-time Defconers here?

Okay, cool, any old-timers here? I'll use the same joke. Any egg-timers? No, okay, kitchen-timers. No, okay. All right, do we have any people in the audience who are like private investigators or just people who do this professionally, OSINT, these types of investigation? Okay, awesome. Awesome, I got people to prove myself to, all right.

Any data scientists out there? Okay, awesome. I'm recently getting into that space, that's cool. Any reporters besides myself as press for Defcon this year? No? No reporters? Okay, awesome. All right, so these are standard disclaimers. I don't wanna hurt anybody in my research. This is all open source.

Try not to be an asshole if you're gonna replicate my work. I'm doing this for educational purposes only, right? And here they are, actually. So I anal, I anus. That's been a joke for the past couple years. I am not a lawyer, I am not a stalker. I anal, I anus. Sounds better that way, I guess. So again, I'm not here to make life difficult

for anybody, so use whatever you learn today wisely. And of course, again, this is for informational purposes only. All right, so some caveats before we get started with the actual meat and potatoes. I went into this research not knowing what I would find. I wasn't like looking for something specific. I just kind of aimed my code cannons at it and I wanted to see what was gonna happen.

So yeah, there was no particular target. I'm not trying to target anybody specific, no stalking this year. And this is just, there is much more to do in this space. This is just scratching the surface. So hopefully with the help of you guys, we get something done. But after the research, what can I tell you that we can expect here?

Well, there is porn. There's porn, obviously. There is username enumeration, password enumeration, and we'll get into that a little bit, sensitive docs and so on and so forth. And thousands and thousands of various media files. That's kind of the fun one. That's just fun to watch. Okay, so let's define a URL shortener.

So a URL shortener, of course, you're taking a very big, long, descriptive URL and you're shortening it into something that is shareable so you can share it among, let's say, Twitter or text messages or it's just something very quick and easy, something that's memorable or at least easier to remember. But all it really is is a 301 redirect for your HTTP codes. These are just a 301 redirect to a different URL.

So knowing that, we can follow it back. We can follow it back to where the original link went to. Now, I chose isgood as my target. That's is.gd as the URL shortener for a couple of reasons. There's no membership required in order to shorten a URL.

There's a consistent slug length. So what I mean by that is a lot of URL shorteners, maybe their slug length varies in size. So you have a URL shortener that only used five characters, maybe some that used eight. isgood only uses six, no more, no less. I feel like there's a Monty Python joke in there somewhere. I won't attempt it, but.

So a six-character slug length. It's always gonna be six characters. And so there's also advanced shortening features which makes this easy and low-hanging fruit. So things to consider, of course, rate limits. I did check the terms of service for isgood. And you can scrape all you want

as long as it's only one link per second per machine. So yes, every hour then you're basically able to scrape 3,600 links and so on and so forth. So that's, what is that, 8,400 per day? Something like that. But yes, 3,600 an hour.

So terms of service, of course, if you're going to use isgood as a URL shortening service, they have some things that you'd want to mind, of course. No spam, no child pornography, of course, thankfully. No malicious content, so you can't URL-shorten malware. If they detect it, they'll disable it, et cetera, et cetera.

So basically, don't be a dick. All right, the maths, because there's always maths. Now, no, I'm not British, but I do love the area. I've never been there, but maybe I love the culture. So okay, like I said earlier, all of the slugs are six characters in length, which means if you, and they don't use special characters either, so no plus sign, no parentheses, nothing of that nature.

So you're looking at capital letters, lowercase letters, and then zero through nine as your numbers. So if you take a look at all that, that's 62 character possibilities in each space. So it's 62 to the sixth power, which gives you about 56 billion links, closer to 57 billion links, actually.

Now, no, not all of these URL-shortened links are actually being used, and we'll get into that in a second, but this is the entire space. This is exactly what can be shortened. And of course, if you have a lower pronounced, or a lowercase word, like zero, A through Z, zero through nine, it shortens that to about two billion, okay?

So we're kind of whittling away the possibilities, or we're trying to whittle away from population size to a sample size. Now, isGood actually has some advanced, advanced URL-shortening features, such as lowercase word pronounceable, okay? What exactly does that mean?

Lowercase word pronounceable is just consonant, vowel, consonant, vowel, consonant, vowel. It's not actually a word, it's just what they consider word pronounceable. So, bah bah bah bah bah bah bah bah bah bah bah, bah bah bah bah bah, what about you? Sounds like a, sold that hog for a $35 bid. I wish I could do auctioneering better, but it made the joke better.

So the word, so the lowercase word pronounceable character set is, either you're starting with a vowel, or you're starting with a consonant, and going from there. So it's either ABABAB, BABABA. And so with that, this actually whittles down our space to about 2.3 million links, something that's just a little bit more manageable.

Now, so how do we do that? On the right side of the screen, you're gonna see a list of the slugs that I generated, and then I'm just gonna plug that back into a cURL request to grab that data and see where that link is going to. So the top code snippet is just how I generated the links. The bottom code snippet is just taking that,

putting it into a cURL request, and then seeing what comes back. That's very simple, that's all it is. The cool thing about this is that it's all native Linux tools, a little bit of Python, but it's nothing that's not on a computer already. So here's an example of the scrape as it's going on. I just took a screenshot. I let this go on for about a month and a half

on three computers. Could I have done it quicker? Yes, but it was kind of cool to watch it as it computed, right? The cool thing is like you go to a coffee, you come back, oh, I found something else. You go take a nap for about eight hours, you come back and it's still running. So for a month and a half, it was better than watching a screensaver at least, right?

So what you'll see here is that if the link is resolvable, it'll show you the new URL, the URL that it actually points to. If it is not resolvable, it'll just kick back your original request, which is the HTTPS, you know, is.gd slash whatever the slug is. And so that's exactly how that works.

So there were a couple of errors along the way. So bad requests, hosts that no longer exist, you would get for like a 404 error. Also, things that would hold up the script would be something like streaming media, right? Because the curl request just constantly grabs data and it actually hung up my server a little bit

or the script. So I would have to stop it, move that URL, and then continue on with the scrape. Because imagine like a URL being shortened to like SOMA FM or whatever, the curl request is just constantly pulling for that next media, that next piece of information. So that was a problem.