RECON VILLAGE - Emergent Recon - fresh methodology and tools for hackers in 2018

Video thumbnail (Frame 0) Video thumbnail (Frame 8433) Video thumbnail (Frame 10928) Video thumbnail (Frame 18653) Video thumbnail (Frame 26233) Video thumbnail (Frame 39041) Video thumbnail (Frame 51848) Video thumbnail (Frame 64655) Video thumbnail (Frame 77461) Video thumbnail (Frame 81180) Video thumbnail (Frame 90892) Video thumbnail (Frame 96536) Video thumbnail (Frame 98437)
Video in TIB AV-Portal: RECON VILLAGE - Emergent Recon - fresh methodology and tools for hackers in 2018

Formal Metadata

RECON VILLAGE - Emergent Recon - fresh methodology and tools for hackers in 2018
Title of Series
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Release Date

Content Metadata

Subject Area
Recon is an art AND an science. The landscape for methods of finding hosts to attack is constantly changing. Whether you call it “Asset Discovery” or something else, it remains a core part of bounty hunter and red teaming life. Join Jason as he expands on his ever changing recon methodology. This talk will focus on what tools to incorporate (and which tools not to). It will outline new methods coined in 2018, plus frameworks to automate and document your workflow. Topics include: brand/TLD discovery, host enumeration, application threat modeling, and more!
Axiom of choice Web crawler Group action Parsing Range (statistics) Sheaf (mathematics) Function (mathematics) Stack (abstract data type) Public key certificate Software bug Neuroinformatik Web service Different (Kate Ryan album) Cuboid Descriptive statistics Physical system Scripting language Mapping Electronic mailing list Bit Repeating decimal Type theory Data mining Process (computing) Search algorithm Self-organization Website Right angle Resultant Spacetime Windows Registry Autonomous System (Internet) Slide rule Revision control Spreadsheet Goodness of fit Latent heat Internetworking Hacker (term) Profil (magazine) Software testing Domain name Matching (graph theory) Information Weight Autonomous system (mathematics) Database Subject indexing Software Network topology Hydraulic motor Game theory
Mobile app Dynamical system Greatest element Sequel Multiplication sign Function (mathematics) Parameter (computer programming) Web 2.0 Hacker (term) Green's function Touch typing Software testing Vulnerability (computing) Domain name Injektivität Default (computer science) Information Content (media) Electronic mailing list Cross-site scripting Message passing Arithmetic mean Password Website Right angle Arithmetic progression
Suite (music) Functional (mathematics) Web crawler Server (computing) Game controller Thread (computing) Link (knot theory) Real number Multiplication sign View (database) Tap (transformer) Set (mathematics) Parameter (computer programming) Web browser Computer programming Number Direct numerical simulation Different (Kate Ryan album) Profil (magazine) Cuboid Software testing Recursion Booting God Vulnerability (computing) Form (programming) Exception handling Domain name Demo (music) Information Inheritance (object-oriented programming) Block (periodic table) Projective plane Mathematical analysis Analytic set Bit Type theory Uniform resource locator Website Point cloud Self-organization Right angle Spacetime
Web page Functional (mathematics) Web crawler Mobile app Link (knot theory) View (database) Online help Parameter (computer programming) Number Twitter Software bug Revision control Web 2.0 Pointer (computer programming) Goodness of fit Software testing Traffic reporting Absolute value Physical system Domain name Theory of relativity Information Key (cryptography) Mapping Demo (music) Electronic mailing list Content (media) Database Uniform resource locator Query language Blog Self-organization Website Right angle Freeware Probability density function Reverse engineering Spacetime
Web crawler Presentation of a group Parsing Beta function Code System administrator Multiplication sign Source code 1 (number) Set (mathematics) Function (mathematics) Image registration Coma Berenices Client (computing) Mereology Information privacy Public key certificate Computer programming Permutation Software bug Direct numerical simulation Spherical cap Different (Kate Ryan album) Computer configuration Hypermedia Software framework Extension (kinesiology) Information security Scripting language Beta function Web portal Theory of relativity Mapping Block (periodic table) File format Data storage device Electronic mailing list Menu (computing) Price index Instance (computer science) Port scanner Process (computing) Numeral (linguistics) Googol Repository (publishing) Website Right angle Video game console Resultant Reverse engineering Asynchronous Transfer Mode Web page Server (computing) Functional (mathematics) Service (economics) Computer file Link (knot theory) Computer-generated imagery Tape drive Virtual machine Web browser Mass Automatic differentiation Field (computer science) Product (business) Internetworking Profil (magazine) Hacker (term) Authorization Energy level Software testing Proxy server Domain name Authentication Default (computer science) Multiplication Graph (mathematics) Information Key (cryptography) Mathematical analysis Analytic set Database Line (geometry) Limit (category theory) Cartesian coordinate system Cross-site scripting Graphical user interface Uniform resource locator Word Enumerated type Search engine (computing) Password Network topology Video game Musical ensemble Table (information)
Group action Presentation of a group Randomization System administrator Source code Range (statistics) Execution unit Water vapor Open set Computer programming Software bug Telephone number mapping Direct numerical simulation Medical imaging Different (Kate Ryan album) Hypermedia Core dump Negative number Error message God Touchscreen Remote administration Mapping Digitizing Multitier architecture Electronic mailing list Arithmetic mean Chain Phase transition Permian Reading (process) Web page Point (geometry) Computer file Computer-generated imagery Online help Mass Latent heat Term (mathematics) Hacker (term) Authorization Energy level Data structure Traffic reporting MIDI Default (computer science) Key (cryptography) Expert system Content (media) Line (geometry) Cartesian coordinate system Timestamp System call File Transfer Protocol Graphical user interface Uniform resource locator Word Software Visualization (computer graphics) Enumerated type Query language Personal digital assistant Table (information) Resolvent formalism NP-hard Multiplication sign 1 (number) Port scanner Set (mathematics) Function (mathematics) Stack (abstract data type) Mereology Formal language Web 2.0 Mathematics Spherical cap Cuboid Flag Software framework Position operator Thumbnail Scripting language Area 3 (number) Term (mathematics) Measurement Type theory Data mining Googol Auditory masking Website Self-organization System identification Configuration space Hill differential equation Right angle Resultant Purchasing Functional (mathematics) Server (computing) Service (economics) Link (knot theory) Image resolution Virtual machine Heat transfer Web browser Login 2 (number) Revision control Programmschleife Internetworking Software testing Utility software Proxy server Domain name Time zone Multiplication Forcing (mathematics) Projective plane Mathematical analysis Analytic set Planning Database Password File archiver Point cloud Routing Communications protocol Buffer overflow
Point (geometry) Server (computing) Clique-width Computer file Length Code Multiplication sign Modal logic Traverse (surveying) Twitter Software bug Web 2.0 Revision control Profil (magazine) Software framework Computing platform Plug-in (computing) Vulnerability (computing) Scripting language Injektivität Email Mapping Information Validity (statistics) Electronic mailing list Variance Price index Cartesian coordinate system Exploit (computer security) Cross-site scripting Application service provider Graphical user interface Website System identification Hill differential equation Right angle Library (computing)
Web crawler Dynamical system Group action Parsing Multiplication sign System administrator Source code 1 (number) Parameter (computer programming) Web 2.0 Bus (computing) Software framework Vulnerability (computing) Scripting language Graphics tablet Computer icon Mapping Block (periodic table) Software developer Image warping Electronic mailing list Staff (military) Parsing Demoscene Website Configuration space Hill differential equation Right angle Web page Functional (mathematics) Game controller Service (economics) Link (knot theory) Computer file Limit (category theory) Black box Web browser Heat transfer Login Robotics Hacker (term) Internetworking Newton's law of universal gravitation Information Forcing (mathematics) Content (media) Mathematical analysis Directory service Cartesian coordinate system Uniform resource locator
Web page Computer file Link (knot theory) Code Multiplication sign Robot Tesselation Set (mathematics) Port scanner Coma Berenices Web browser Mass Open set Direct numerical simulation Sign (mathematics) Goodness of fit Cross-correlation Software testing Data structure Computer-assisted translation Scripting language Domain name Mapping Forcing (mathematics) Projective plane Mathematical analysis Electronic mailing list Price index Directory service Public-key cryptography Data mining Word Googol Software repository Personal digital assistant Password Point cloud Configuration space Self-organization Website Convex hull Right angle Table (information) Resultant
Web page Computer file Link (knot theory) Robot View (database) Source code Function (mathematics) Web browser Hand fan Web 2.0 Goodness of fit Internetworking Different (Kate Ryan album) Scripting language Domain name Multiplication Matching (graph theory) Information Forcing (mathematics) Projective plane Electronic mailing list Mathematical analysis Process (computing) Personal digital assistant Password Website Configuration space
so this is my testing box so before we get started with the talk all of these ideas are a methodology on how to do recon now I have with basically bubblegum and popsicle sticks stitch this together into some automation just bash scripting write a lot of this stuff can be done automated Lee some of it takes like kind of contextual knowledge but most of it can be automated so we're gonna choose one of two targets bolt up bug bounties and bugcrowd I'm not doing anything illegal that to enumerate while we're doing the talk so you have your choice of twitch TV or our Tesla who wants twitch bitch okay who wants Tesla okay twitch is going it alright so let me start this up and this takes a while to run so alright so this is literally my script I use when I'm red teaming or buh-buh-buh ground hunting and so we're gonna go through the talk and the output but this takes a little while to run hopefully it'll be done by the end of the talk okay emergent recon already did the intro the only thing I will say additionally is that I'm a dad I love my kids and that's my girl winning well not winning but solving her first CTF challenges at a wot CTF in Santa Barbara so really proud of her there I'm also a huge gamer I play a bunch of games so you probably socially enumerate my battle tag or something like that and add me on Steam or Wow or whatever I play so alright the first section discovering IP space so one of the methods I use to discover IP space is keyword searching by organization now these slides so Tesla but the tool that we're running we're gonna do twitch so the best place to do this searching for what's called an autonomous system number is this site called DG phe net the reason the reason you want to find a ton of the system is I'm right if you're a large enough organization and you run your own network really the the Internet's not globally connected computers it's globally connected little networks well not little at all they're very large they're called autonomous systems when you have a large enough system or network you have to register it and so when you register it you have to register a description or a name of your company this is one of the only sites that allows you to search keywords to match company names to as autonomous system numbers so here you have Tesla Motors and you can see they're starting registered IP space on the right hand side there that 209 133 is 709 Oh 2 for Tesla Motors England now what's awesome about keyword searching for this type of data is basically they might have registered another autonomous system number under a different entity name and so here you can also see on the left hand side that searching for just Tesla also came back with Tesla engineering group so they might have a different autonomous system that I want to enumerate for a wide scope bound so then there's always your your verbatim registries right Erin and right these have Whois data reverse to his data on anybody they both have web services that will allow you to do keyword searches on things like twitch or Tesla or whoever this will start giving you back IP space that you can start adding to basic your list of things that you're gonna hack all of this goes around building a list of things you want to hack in your red teaming engagement or your bug bounty hunting so someone said showed on earlier show done also allows you to search by the organization tag so here you can say org : tesla + motors you can also search just by keyword Tesla but you'll get a lot more more results this will start giving you everything that showed on asking you know what showed on is it's an internet spider that goes a little bit deeper and has indexes and is available to hackers it'll basically profile technology stack IP information certificate information a little bit deeper than any regular spider goes and it keeps it in this large database that you can query for almost free and even the paid version isn't really not that expensive for for showed on so here you can search Tesla Motors in this instance and find out that there's a whole bunch of systems already out there you can also search for title this parses of title which can be really useful because if you're looking for a specific type of device or technology that they might have on their network that's vulnerable you'll see it right away in the showdown research results so this is a little stopping place and I actually don't really show this very much but I because it's not really hacking but it's super useful for me when I'm doing this large base recon on a site like twitch so I organize all my testing inside of mine maps and you could do this inside of an Excel spreadsheet or OneNote or something like that but I use X mind and I thought I'd just kind of show you how this works out so here I've started a campaign or a bounty hunt against twitch right so the top-level node in my mind map just says that it's twitch the company right because actually I know that twitch has a lot of domains and they're not all named twitch TV right I'm gonna discover that in the tool that we just looked at and some other the tools we're gonna look at so I'm just gonna start filling this out as we're going along right so if I go if I go look for you know twitches IP space I'll add a tree here and I'll just say IP space here and then I'll keep that's not spelled right good job and I'll keep that range here I can go look for it in a second and then I'll start adding domains in the top-level node now this doesn't look like much when you start it but by the end of this you will have a lot of data that you can start working with in this tree what this ends up looking like is something like this there is what it
ends up is looking like something like this where I have nodes of a top-level domain that's in scope twitchtv in the upper right hand side and now i have lists of subdomains and outputs of tools and this is how I organize my information and basically how do you eat an elephant with all of these sites that you're finding all these subdomains well one bite at a time right like I test each site individually one site at a time once I find them I do an abbreviated methodology for web hacking on each one of these sites once I know that they're real and they're owned by twitch and then I just go through things like content discovery like they're dynamic parameter discovery fuzzing basic cross site scripting checks basic sequel injection checks default passwords all that kind of stuff and then I mark them by progress so if it's green I'm done with it I've made a first pass over that domain and if or actually if it's green it didn't have a vulnerability a check mark is based if I finish the testing on the site or not so here we have a red entry on the bottom that has no check mark in his red means I found a vulnerability I'm probably still testing on it orange is like I put it off till later something about that site has caused me to say I don't want to do this right now I'm going to put off this work till later because maybe this subdomain has some special technology that I just don't want to dive into right at this second and I do this for every top-level domain so down here you can see that twitch actually owns Justin touch TV curse Forge they also own own curse twitch app and curse app those are all top-level domains they own and I'm gonna do this same recon methodology for each one of those top-level domains alright so
discovering new targets let's see here we go yeah so showdown was last thing so
now now that I have some IP space right showdown is giving me some information I have maybe the main IP space stuff from just which I know is their main site and then you know I have some other information maybe I got other places now I want to see if maybe they have some different kinds of brands not just and maybe not just having their IP ranges I want to find out if twitch has acquired anyone really recently I want to see where they're linking to off their main site so there's a lot of good tools to do this and I want to do some tracker analysis of their ad and analytics because this will reveal other places where they're using those ad analytics and I can add those because they're probably related to twitch so acquisitions is pretty simple this hasn't changed much in the last year CrunchBase is still the number one place to go to find acquisition data a lot of day traders use CrunchBase to figure out information on trading for or investing in companies startups whatever it also has a lot of news but the subsection that says acquisitions you can drill down into any company and see pretty concretely who they've acquired and it's updated really really frequently so here you can see in the last few years Tesla has acquired grommet engineering Solar City and Riviera tool now when you take over an organization like this if you're a big parent organization you probably decommission all of their IP space and probably their servers you dead link all of their DNS entries or remove them all together you new call their sea names but really that does actually happened 100% they probably still have some cloud infrastructure out there that they forgot was up and running they have probably customer data leaked some places this happens all the time so if you're a wide scope program like Tesla who says we care about all security vulnerabilities on our bounty I look at these two I look at solar city I look at you know Riviera tool to see if these domains when the use when the websites used to exist you know reveals some type of sites that maybe have vulnerabilities in them so link discovery is this idea finding out what the main site is linking to and there's often a lot of links that are outgoing from a website or even incoming and you can do this recursive link discovery in a tool called burp suite how many of you use burp suite before very good all right so we're gonna do this real quick and we'll try it on twitch see if it works if the demo gods will love me when I boot I'm gonna boot up my burp testing profile here and put its burp on the right when I started this morning my license had expired I had to buy a new my CSUN I was like oh this is painful all right so now we have burp and our browser on the Left unless just let's do some things before we start so the first thing we want to do is make sure that spider in burp is not passively spidering as I'm browsing I don't want to do that I just want to instrument a certain thing right now so I'm going to disable pass the spider as you browse I'm gonna say max link depth for this exercise is one in burp and Max parameters that I want to crawl is 25 and then I'm going to say when it sees a form a log in form usually Bert prompts for guidance this is super annoying if you've ever used before so you can either say don't submit log in forms or you can say automatically submit these credentials I usually for this exercise choose don't submit log in forms number of threads is fine and then forms in denim in general I'm just not going to submit forms for this spider so I've set up my burp spider settings here and now I'm going to set up some scope so if I go to the target tap and I go to oh really what I want to check out first is is adding anything that I know says twitch now burp has a little bit of different functionality in last year that they've launched so you have the verbatim ability to add a real domain or URL here to add in scope and normally when you right click on a site not something to scope that's what it's doing I don't actually care about that because I don't have I don't know any domains except for the main one yet that I want in scope what I actually want to do is basically say anything that has the keyword twitch is going to be in scope for this project so if you click this block with this box right here that says views advanced scope control you can say add and you don't have to do any more supply a fully qualified URL so here I can just now say twitch and just say okay and so now that becomes my scope for this project I'll say yes here do I want to limit history for just that thing okay so now let's go to twitch TV alright make sure burps on oh that was the dude streaming my bad that really confused me all right alright so we're now proxying twitch through burp you can see that interceptor has or intercept it said yo
there's traffic coming through here so we're gonna intercept off and just let everything go through and then go back to our site map alright so already just by visiting the main page we have some sub domains that are hot linked off of the main page and the idea here of this link discovery idea is we're gonna iteratively now spider everything we find to find more subdomain so first let's just choose GTL that swift which will this to or will a basically spider this well there we go i twitter on TV well we'll just we'll just spider everything if possible yeah right click spider here alright already the spider has started to return a whole bunch of content for link stuff not just on twitch TV now but also the things we just chose a second ago so now we start to get a pretty good map of of stuff that twitches is related to now because I am parsing on a keyword we're also getting the benefit of seeing what vendors they integrate here right some of these are not domains they're ad systems on or like other things that have twitch in the URL or the domain which can be pretty useful for us to know to gather information on on the target so the idea here is is recursively I would start selecting this and keep on spidering until I built out a huge map of domains and those would go on my list of things to test I need to test every single one of these things which seems like a daunting task but if you abbreviate your web testing methodology you can absolutely do it I had a bounty the other day that was upwards of I think 5,000 found live hosts and I spent well this is on the other day I was other month I spent about a month on it and I think I made twenty grand in a month so it's not bad okay so you would then select all these spider these how do you take this information add it to your mind map or just make it in a list so you can feed it to other tools but if actually doesn't have a copy all targets function which I hope they they do pretty soon but the way you have to do is actually you have to have the pro version and you select everything here and you right-click and you say engagement tools analyze target and analyze target builds you this report PDF report of all the domains and information about what dynamic parameters they have and this is you know burp usage but if you save the report here I won't do it right now but if you save the report it gives you a PDF report and this is the most effective way to just copy and paste the list of targets that you hadn't left hand side right there it's just there just doesn't this to function to copy all targets on the left-hand side of birth so save this as a PDF open the PDF copy the table and dump it into something else any questions alright quiet a few hosts for for twitch what yes yeah to do that you need to have the professional version of burp unfortunately yeah that doesn't exist in free alright so yes you use what sound for what Oh zap not zap yeah um good question I don't think that they have that function either but you can use that for everything in here the same same stuff happens ABS app has a spider in fact that spider right now might be arguably better than zero burps because it handles JavaScript really well although burp just released a BA a blog really recently that they're updating their whole spider engine to just be awesome I'm so excited about that so maybe that will change in you know a couple weeks but zap elapsed absolutely work for this workflow as well as well as like Charles and some of the other intersection crosses all right maybe well new piece alright so that was a demo for a link discovery oh not sure we won't present cool all right so someone said who is data or reverse who has data is a method where we could start to find related domains or IP space of some of our targets now this is a tool that's relatively new called bomb link it's written by a guy named into view he does this really cool Twitter thing called Red Team tips he's as over like 200 Red Team tips that he tweets out every other day or something like that and there's some really useful nuggets in there if you didn't red teaming or just bug bounty in general but he created this tool that's based off of this site called boxxy boxxy is a website that offers an API that's really cheap to access for and it has a free version as a free number of queries you can do it do to it for reverse Whois data so he created a tool that will recursively basically look at a couple of things for an organization mostly the most useful one is organization name so I'm gonna I'm gonna show this real quick and what this looks like so this will query this walk see database this tool will with an API key that you sign up for it you can do I think 150 queries before your free version is gone and it'll help you basically search for any that has the registered company name of
whoever registered your main target so let's see if this works for for twitch I'll make this bigger in a second [Music] [Music] caps lock was on uber hacker up here I can't get in with caps lock okay alright so my test block so let's see the tools and I think Dom link is where I'm putting all this stuff okay so - Dom link let's see how twitch has their Whois information set up twitch dot TV - see ok let's see if this works ok so there registrant name is twitch Interactive Inc which i think is actually correct right so this tool is gonna ask me do you want me to check the whole database for twitch Interactive Inc and give you back what domains those are so I'm gonna say yes now this can get iteratively recursive because they might have actually registered under twitch or twitch Interactive or twitch engineering like people register stuff very weirdly there's no there's a lot of companies it's not like a standard way that you do this unless you're very patent down with how your registration works so a lot of the times it will it will actually alert me on a couple of these it'll say I found five with the keyword twitch in them maybe we should check all of these out before I'm just gonna say yes and here we go so we have a ton of host data for other sites that are twitchy twitchcon obviously their conference so they host the domain for that that's in scope why twitchcon twitch with the three something they might be parking some of these but that's okay i still want to check him out once i when i did this i used a key word for a large manufacturer and what I ended up finding was a domain that looked nothing like any of the other domains I had seen or any other brands but they had indeed registered it with the same company name turned out that the company was hosting basically a whole bunch of new school well new school for them portals for code repositories basically JIRA Jenkins all kinds of CI CD stuff and they thought that the protection to this was that they never told anyone about the URL like it was only internal rights security by obscurity so they thought that they didn't need to put authentication on it so i walked in stole all their source code passwords routed the Jenkins server through script console and that was really just right out of this tool so this can be really useful to find targets twitch Amazon highways yeah so so now you have this output now what do you do with this output I mean basically it's as simple as copying and pasting it into the line map so if I go back here this is where the fun starts so each one of these should get a node if copy and paste works yeah okay well you get the idea each one of these should get a note which GE twitchcon everyone should get a note alright just go back to presentation it's okay so far useful yeah okay cool questions question yeah it's a different database it's where it's a reverse who is focused database and their parsing more fields in the Whois information then that site is I've used both I find this one to be way more effective yeah okay so now we also want to find maybe we're still on this track of trying to find top-level domains acquisitions other sites that are related to twitch because they will be in scope for a large scope down now every company use ads and analytics tags right yes yeah yes you can yeah yeah through you can do it through that same one boxy or that 12 Dom link it has multiple options you can specify you want to search by company keyword you can specify register name you can specify a whole bunch of stuff so you had to check into the like the stuff of that one yeah so built with basically is this company that does technology profiling and add an analytics analytics basically they check out every site on the internet they spider spider it by looking at their left over source snippets or text files or just default configurations of some frameworks they know that your site is running fast ly and you know is hosted you know with sis X server technology and is using these JavaScript frameworks and they also know what and that'll add analytics you're using by the format of the key that you embed in the web page now we can use this to our advantage because if you're a company like twitch you have a New Relic key and you have a Google Analytics key or some of these other keys now luckily for us they allow us to look at these relationships between sites so here I've drilled down into twitches and those are their analytics keys on the middle pane right there I can click on any of those and see where else their analytics keys showed up on the internet what other sites are in the built with database so this allows me to find brand new stuff that I might not have seen before like Binney TV and you know booth Saban these are all streamers that actually twitch is promoted so much that they now have their own twitch hosted sites and so these might be in scope for the bounty they might be custom code you never know so so this is really this is really advanced so basically to do this you use the built with you can go to build a site and just do a search on their on their site or you can use the chrome extension which I can show you now so in my browser up here if I go to twitch again or actually doing my testing profile here don't that's not my testing profile they close it close it there it is okay so now that I'm at twitch built with the extension is installed it's just in the chrome store and click it maybe burps messing it up real quick oh there okay cool just takes a little longer it gets in proxy so here you can see that I get some tech I get some tech information and this is useful in hacking any way to find out the stack they're using or JavaScript frameworks they're using and you can drill down into this back here sorry this is also very useful in other parts of the methodology is knowing what they run right because you're gonna look for o days and frameworks they use or whatever so this is useful but then if you go to the second tab up here yeah question appetizer doesn't do any of the admin analytics tracking it just does a technology profiling so it'll do like it'll do just as good of a job if not if not a better job then built with but it's yeah it's limited on the other functions so let's try on the website yeah another question yeah so that's the problem right I would have scaled it if I could the problem with built with is that it's it's a paid tool they really want you to pay for it so actually it looks like they just updated it right when I was doing this presentation so I used to work right inside the bookmarklet now it looks like I have to login with a free account to get the relationship information or maybe use the search on the website they have an API it's expensive it's really expensive to use the API but I don't know of anyone else doing this analytics tracking the analytics code tracking across multiple domains so kind of stuck
with it right now if I want to use this method anybody else knows of anything cool like that I would love to know so let's write let's try the insight that's all right all right here we go all right so relationship twitch TV I can see that here's there you a code right here there Google Analytics code if I click on that now I get the domain information let me make this a little bigger so here in the left hand side I can see related domains I also get a heat map over here of like how much they're related although I'll be honest I don't really know how to read this graph really well yet but I use the data from the table most often so start the tree you know this is kind of same stuff we were looking at before some guild sites faceless TV this happens a lot actually is what happened with this kind of linked analysis if they're gonna do a beta product and they haven't even let anybody know about it a lot of times I can look at this data and know that they're doing that data product like I knew way before some video games that even launched or gone into beta that twitch with already partnered with them and got sites ready for them and already integrated the analytics code into the page and I end up like knowing beforehand not really useful for a hunter but exciting for a gamer so yeah alright yeah you don't you have to visit them the indicators I wouldn't say a compromise but the indicators of ownership are usually the site has a privacy policy that links back to twitch or a trademark that links back to twitch in the footer that's how I usually know that these are related yeah yes yeah absolutely yep you can use gns registration information you could go back to the Whois information - and just verify that these ones match you know who is look up and stuff like this yeah absolutely sometimes I'm reckless enough that I don't do that but you know it just depends yeah yeah yeah all right presentation mode all right so that's tracker analysis basically so other things that you can just do are the things we just talked about so trademarks exist across many sites right you have to embed your trademark in policy and privacy policy and any site that you launch for a business nowadays they protect you legally so searching for things like Tesla C 2016 Tesla C 2015 Tesla C 2017 and then in URL : Tesla is a quick google dork to try to find some sites that are related to Tesla yes I I don't look at that honestly when I'm doing bounty hunter I county hunting stuff I don't I don't care as much so that's a horrible answer but yeah I just like yeah yeah I would I would report it like I mean I have that instances before we're like I found a site it has the privacy policy but I can see it's obviously not managed by the company it's a third party it's vulnerable to something like I I picked up right away it has like some kind of search functionality or reflected like text and it turns out to be vulnerable cross-site scripting you know bug bounding situation I just ping the customer and I say I found this site it should be in scope because you have a wide scope program but also might be in scope cuz I don't know if you own it they have your trademark on it I just say those words a lot of times they'll be like thank you we'll contact the owner of the service we think we knew who it is and they'll usually award me a bounty for just having found something that they had no idea was up or have lost track of okay so now we've done IPS and brands and like related sites so now we're gonna get into discovering subdomains so we talked about subdomain numeration right like finding now that we have Tesla comm or twitch TV those are top level domains now we want to start finding subdomains of those sites and those can individually map to IPs and be their own applications which all are in scope for bounty-hunting or red teaming so really there's two methods to enumerate subdomains one is subdomain scraping and the other is subdomain brute forcing now subdomain scraping is the idea of taking search engines databases census Baidu DNS databases SSL certificate databases even virustotal wayback machine there's about 65 total sites around the internet that harvest large sets of data about domains or maybe they're not even really made for that they're made for other things but allow us to do searches to find references to domains and they all in some way or another offer access to an API or can be scraped pretty easily by some Python to return so this is relatively new actually we weren't doing a ton of this in pentesting until like the last couple years actually nobody was really looking at scraped information off the internet to identify subdomains and maybe assets of a company and a Red Team engagement but now it's all the rage this is actually one of the best methods to find subdomains and secret sauce of your clients or your bug bounty targets so there's a ton of sources what happens is really there's been two advents one was a one was a tool called sub Lister which was one that everybody used for a really long time and it was it was maintained for a while then kind of fell off wasn't really maintained and then two authors really recently released two tools that are epically good and so they each have different sources and they each have different features and functions so I can't decide which one I want to use in my methodology if I would if I would be asked to use one tool so I just script them both together when I started that scan at the beginning of the at the beginning the presentation that's just some bash in the background running concurrently this tool and the next one I'm going to talk about sub finder and that's the reason the tools to take so long is because it's scraping all these sites right now so this is a run of a mass against Netflix who also has a pounding public bounty and what it does it goes out to I think between a mass and sub finders 65 sources I think and it also offers some some cool stuff like permutation scanning so first thing it'll do is they'll scrape all those sources for references to Netflix com then it'll tell you on this page I found media died Netflix calm and geode on Netflix calm and it's out 104 dot Netflix calm I have no idea what that is and so it'll give you back a list of now targets and you can see in this methodology for a big company you're starting to gather hundreds of targets that you can go after which for a bug bounty hunter is good right you can hack the main site or you can focus on these other sites there's really no limit to the kind of stuff you can find a lot of my buddies are on the Walmart Red Team and they use the same methodology this recon methodology to find stuff that's just been left out there especially when they acquire a new company they do this same stuff same enumeration so the other helper is that this tool has is it includes some reverse DNS stuff but it also has what's called permutation scanning so you see here the second result here was media done Netflix comm on the right hand side from this tool and that's great so what it'll do is it'll add common prefixes to that subdomain like one - Mediacom or user or dev or prod or whatever that Mediacom and then it'll try - and it will append keywords and a dash and a dot to try to find additional hosts that are related to that subdomain and see if they resolve and if it does resolve add it to the list and so this is called permutation scanning in subdomain enumeration and so a mass includes a function to do this so the other one is sub binder written by Iceman awesome hacker name I love it and this one has a multi resolver brute-force ur it built in so if you want to do brute forcing hand so domain scraping via the same tool you can use sub finder and it's pretty efficient it also can output in JSON so you can feed some of this stuff other tools especially if you're using something like aqua toned sub finder supports now aqua tone output so if you wanted to use sub finder for the discovery instead of aqua tone which is
a framework for OSINT that finds the same kind of stuff you can basically take the output of this put it into aqua tone and then use aqua tone for the later phases of what he calls the domain flyover but really it's just subdomain analysis so I used to have a fancy table saying like what was better about each tool when it was sub lister and some other ones like enum all which is at Y route I've completely deprecated my own tool I just want to use what works aqua tone sub lister and anything else for scraping but it doesn't matter anymore stuff finder a mess or the two tools you want to use or the best and breed right now and they will be for quite a while they handle the most sources and they're the most effective okay so that's subdomain scraping now you have subdomain brute-forcing how are you on time I have five minutes Oh God okay so do forcing is the idea that you just try to resolve a whole bunch of random stuff like admin comm or twitch TV and media dot and if you get it was a resolve it means that site exists or some kind of DNS redirect right and this is time consuming brute forcing anything passwords or DNS entries is this time consuming in pentesting used to take a long time there are newer tools nowadays and the one we're going to talk about is mass can mass can or mass DNS or and mass DNS isn't one we're going to talk about mass DNS used to take what used to take are well used to take a day to do or maybe a week to do with a large list of words to try to brute force subdomains I just cut that time down into a minute and 24 seconds and how it did this is it's written in C first of all very fast and then it uses what's called multi resolvers basically instead of using one DNS resolver to do that resolution it cuts it into groups of ten resolvers and then distributes your word list across ten resolvers and then brings it all back to you in the same tool so it drastically cuts down the time to do this type of group for a one million line subdomain canary runs in 1 minute 24 seconds that's unheard of so this is kind of what everyone's using right now so what about those word lists well this is every on the right every tool that I've ever known in my pen testing career about hunting career that ever existed for subdomain brute-forcing Fierce a lot of people use Spears for a long time knock like a whole bunch of tools came out for subdomain brute-forcing over the last 10 years they all a different word list and then individual projects came out with word lists to try to do subdomain brute-forcing I basically chatted and unique them into this one list called all dot txt and this is what I use with mass DNS this is what I feed mestia DNS well ok I'm fine this that I could stay a little long ok so there's some other newer kind of tools or kind of newer projects out there when it's called common speak what common speak is is they used bigquery on a whole bunch of sites to parse out their subdomain structure and their URL structure now the subdomain data is awesome like it actually really gives you keywords in terms to look up subdomain data that is pretty new right they they went out in they analyzed hacker news and HTTP archive and stack overflow overflow and basically if you think about it they're just spidering these sites and every time they see a URL mentioned or a domain mentioned they capture the subdomain and add it to a list and then do some analytics on it and they say all right new school companies are probably using these names for their subdomains so if it totally became fashionable to name your subdomains after characters in Harry Potter if Harry Potter made a reverse resurgence doing a big query result against one of these sites might tell you that that's become fashionable again is how people name their subdomains which is almost like language analysis added to sublimate some domain enumeration they also try to do the same thing with URL data and how urls are structured like if they're URL path this has been less useful for me but I'm still waiting for the author to like sell me his dream I'm like why this is super useful because what I looked at it it's very application specific right everybody has a different URL path there's not many common occurrences I see unless you're using a purchase software or a licensed software or something like that so but the subdomain data is awesome I've integrated into the all about text file so it's in there now so you really you could just use the all that text file any questions so other ways to find subdomain data so if you if you have question no he's taking a picture all right so if you have if you have DNS or sorry if you have DNS SEC enabled there's this idea of how DNS SEC links to the next sub domain set up for your organization right I'll be honest I don't know exactly how this works I'm not a DNS SEC expert maybe someone in this room knows this better but I know that they do link in a reference to each other when you setup the NSX so there's this idea of what's called insec walking and there's three tools for this l DNS utils and sec walker and n sec map and what they do is they basically iteratively go through every domain that you hand it the first one and look for the next reference one in the insect chain and basically what happens when you run one of these tools did you get kind of an old-school looking DNS zone transfer which is amazing because nobody lets you do that anymore and if they let you do that a lot of this would be you know loops like fightin zone transfer great like gimme me all that so if they have DNS SEC enabled they can that you can use one of these tools there's a whole presentation about it that bharatham are he did at the bugcrowd conference level up two years ago I'm reading he walked through doing all of this it's called esoteric subdomain enumeration techniques which was excellent I really loved it other ways that you can search for domains is searching github or get lab or you know sources you know like that and you can also just do some some google talking so all right so now we've got a giant list of targets in our scope or our No campaign for red team or bug bounty you know thing so do we do now well it's very general like in pen testing you do port scanning except for for a long time we were using n map which nothing gets in that but it's slow it's really slow and it's a great tool and I use it in the methodology but just in a different place so things like Z map and masscan are infinitely faster to just do a general port scan so here masscan to do a full port scan of a large target ASN takes about 11 minutes to finish and it's on 65,000 hosts right that's super fast four reports can you know I'm running this on a mid tier digital ocean box so like also not the most bandwidth that I'm working with but ten map would have taken a week right I remember when I was doing pen testing full time when I was like just a scrub we would kick off this really ugly Python script and and it would have to kick off and map up the meaning of the script for a large company's domain and then we'd come back four days later and it would have finally completed now and that has gotten a lot of tuning to to get faster there has been a lot of tuning but maths can still blows it out of the water so what I do is I do the initial port scanning with masscan the fast port scanning of the whole IP or of ARP with every port right the full range of 1 to 65535 and then if that tells me a port is open then I feed that to nmap and nmap only gets the port so I know I'm open and then I use nmap because it's stronger in other areas it allows you to do version scanning it allows you to add and map scripting engine checks and so I'll feed the mask and output to end map and as you can see this is a methodology that can obviously be automated right you can script all this together yes yes absolutely yeah yeah that's actually a typo so yeah thank you yes yes sometimes it does come backs with false positives that are asynchronous because it's asynchronous really I don't have a great way around that right now so like when I get back that data of port data what it'll look like in my automation is is that it looks like every port is open on a box and with a mask in and it actually like since I'm logging the port scan to a flat file that flat file ends up being like a like a huge amount of megabytes and it slows down my tool I'm still
trying to figure out a way to like not do that so it's in my issues list at home but yeah I mean it happens less often nowadays like I think that if you tune masks and really well you can get around some of it so yeah yes yes this is true okay so the question was a lot of places blacklist you when you start doing scanning right this is a running joke I don't know you've heard it before like what happens here is a lot of cloud based laughs Akamai and CloudFlare and some other ones when you start getting into port scanning or when you start requesting a lot of web requests against a host or you even send one iota of attack traffic to the site they'll put you on this global blacklist which will blacklist your whole house and then your wife my wife will come to me and be like why can't I get to Amazon why can't I get to United I'm literally she was trying to go to a funeral with for her mom's mom and she couldn't register a plane ticket because I'd blacklisted our house or IP on this global blacklist so this is this is a running thing I test over a VPN all the time now and I make sure to have a VPN that has a quick proxy switch fiction or function so I use ipvanish because ipvanish has a whole bunch of functions that I like like the checkbox that won't let you connect to the internet unless you have the the VPN enabled and it also has a button that just says quickly change IP so once I get blacklisted from one I'll just switch to another but I don't test over my home network anymore so VPN questions okay all right so now you've got back some port scan data immediately you're going to start to notice things that are interesting here obviously you're gonna notice the four four threes in the 80s which are what we're gonna test for web testing but you're also gonna start to notice remote administration remote administration protocols and database servers and things like that all of these can be brute forced with password lists and so this is the part that gets you know you've got to make sure you have permission at this point so when you do the masscan and you get the output back then use then you serve it to mmm an nmap with nmap you do the /og flag to get back the grep level output where you can do with the grep herbal output of a service scan from nmap is feed it to this tool called brute spray and Bruce spray is pretty cool it takes the nmap G map file or GN map file and it will parse it for all remote administer protocols and using I think it's Medusa's the core technology choosing under it will brute-force everything in that GM met file for credentials and it will do it concurrently which is kind of useful so here I it arrived specified a password list it comes with a default user and password list this is honestly enough for me I'm not trying too hard core brute-force into stuff like that's usually not even scope for the bug bounty in red to me it is so you know maybe you want to use a more advanced user name and password file list here a lot exist out there SEC list is probably one of the best ones that you could find for password and username lists but I'll just use the default one here and I'll run this after the mask an and and that finishes it is this so it locates the file and you can see that inside of my GN map file it identified nine FTP services eight SMTP I'm actually nine SMTP eight SSH hosts once helmet and one one MySQL in or delivery force all of these for common usernames and passwords and alert me when it finds it so then I have now started to look at remote administration program calls but now I have all of this data for websites for for three and eighty these are actually what I spend a majority my time testing on a bug bounty has websites but I have so many now I have hundreds of domains for a large scope company like if what's the largest company you could ever think of it's an Accenture Microsoft right you do this analysis for Accenture and Microsoft you're getting back like I would say well I know for Microsoft it's in the it's in the like thousands of live hosts that you're looking at right and I don't know how many of you have ever done a campaign against a thousand hosts it's a lot of work and to really know where to start you could start at the top of the list looking at those hosts but also you could do this measure this method of kind of visual identification so what you do is you use a tool called eyewitness there's other tools that do this aquatune does this as well there's another tool called HTTP screenshot and there's another tool that I just identified that somebody told me outside in the Recon village tables about that's even better for this but I haven't tried it yet so it's at the end of the presentation the idea is you find some tool that visits all your URLs or domains and takes a screenshot that's all it does takes a screenshot dumps it in a folder now you can open up that folder with large thumbnails enabled or whatever and just look through and kind of eyeball like yeah that's that redirected to the main site I don't care about that this redirected to a help site kind of don't care about this oh this looks like an admin back-end login really care about that right and so you visually I start identifying things that you care about out of this large list and that helps you prioritize what you test first so you get kind of return on your time what I witness did that I liked a lot is you can feed it just a list of domains and not whether it's HTTP or HTTPS and so this tool I witness in specific would try both for me I didn't have to recreate multiple lists with HTTP and HPS and then feed it to the tool the tool did it for me but it's also kind of slow these sometimes are prone to error these tools taking screenshots because they're using things like phantom j/s which is just kind of a garbage fire sometimes so yeah you just have to take it with a grain of salt that maybe you'll get some false negatives in this idea I probably still recommend like when I'm doing this against a smaller target if I have less than a hundred hosts that I found I'm just loading all that in a browser and just visiting the pages myself using some chrome plugins it's over a hundred hosts then I'll do something like this and go try to automatically do the screen shot at it yes yes it is an iWitness they have that function where you can dynamically add a set of ports to the end of the domain yeah which I actually don't think a lot of the other tools that are trying to do this have supported yet so yeah like if they were running an SSL so HTTP service on another port right yeah okay okay so this one this one is kind of new and is actually like really simple but who here has used wayback machine yes it's one of my favorite sites and wayback machine are awesome if you've never used them before what they are as a website that takes periodic snapshots of most sites on the internet that they can find and then they will take these snapshots and show you the front page or maybe several pages on that site with a date and timestamp and they keep the image up there and so what you can do is when you start going to these sites what you're gonna notice is that a lot of them are like infrastructure that's not serving a real webpage there's no application it's it's really either an API or some kind of infrastructure that just need to be hosted but it's hosting other ports and a web server for no apparent reason so you'll notice that when you're going through your testing you'll get like a lot of blank pages or like stuff that looks like it's been nuked but once was there like a lot of content so what you can do is you can go and look in the web archive history and see was there one sensitive content there this is actually how a buddy of mine Brett like I had done this and been successful in a couple cases but a buddy of mine Brett who's a bug bounty hunter was like it was kind of a nice to see that he's found some really high-impact owner abilities using this methodology so you know when you do this how do you automate it right well there's some tools to do it there's one called BAC unifier and another one called recon cap that supports scraping the links to the image of the history of the site that you're looking way back into and so these kind of get integrated into your methodology when you see sites that have little content on them or look like they used to be sensitive but have since been fixed you go to their way back enumeration and then bingo that's like a configuration page that has like a private API key that's still being used and they thought that removing the web page was the solution but actually the content was cached on the Internet all right so we
did way back in New Marais ssin we did a visual identification all right so now for each one of these sites right I probably have a node in my mind map for all of them and I just start going down the list and this is what I do the first thing is identify the platform with Bill width right the chrome plug-in I have a Python script that I'm gonna release I just have one bug to fix because they change the API low time and I need to now change it to do use a free account but I have a price on script I built that will scrape built with and give you the technology profile back for your site so you can integrate it into your own tools I'll release that afterwards and maybe tweet about it on my Twitter or you can use WAP eliezer which is somebody mentioned there's another one called what web which is also a technology profiling script there's a whole bunch of them so they're all pretty good at this point I want to see what my target run is my target application what does it run like are we looking at PHP what JavaScript frameworks like I want to start getting that information so I know well has request validation so cross-site scripting not really going to be super successful so super successful for many many times unless I'm looking at a custom piece of code or something like that PHP you know very prone to path traversal tax or command injection and stuff like that old-school loner abilities so I got to put like my 1990s hat on and be like all right ready to attack PHP so so this gives me that information the built with and then also there's another one called retired GS which is super cool which will give you the full version length of all their JavaScript libraries which is easy easily cross-reference of all with has there been any vulnerability is since the version or since the version they're using so immediately I know if they're using outdated JavaScript libraries and if there's cross-site scripting vulnerabilities nose or anything else there's also a newer ish plugin which is called Vohland scanner verb owner scanner and you loaded into burp and it basically does the same thing that retired @j s does but it's up for the server stack so you think like n map gives you VAR n map not if you think like necess when you do an S to scan it comes back and it says this server software is this version because I know by the header or identified that there was still an install file left or something I know it's this version and that version had a vulnerability in it this does this through burp so you set this up here and every time you visit a new site this looks at those key indicators and says yeah this is old it probably has a CVE associated to it you should pro go check to see if that's exploitable
all right so I'm on a main site I technology profiled it you know I kind of know what I'm going up against I've maybe found some CVS that I have to
identify the next big hurdle is parsing JavaScript so sites that are you know well every site nowadays is using heavy JavaScript right like and dynamic spiders and even burp in its current incarnation is just not good at traversing JavaScript it's no technology is really great at it so to be great at this I have to add separate tools to my methodology so Zapp is actually really good for this apps Ajax fighter is like a headless browser that will execute a whole bunch of functions on a page and basically return you know how you instrument an application that's heavy JavaScript and allow you to find parameters and even things like Dom based cross-site scripting very easily so Ajax fighters is pretty cool the other one is called link finder which is a standalone tool that you feed a URL or a list of urls and it will go in and pull down anything that sees in the source code of all the pages of a spider for full URLs absolute referenced URLs or dotted URLs relative URLs with at least one slash or just references to files and it'll go through all of the JavaScript files on a site and parse these out for you and build them into links so that you can visit them inside of either your browser or just directly through burp repeater or something like that this has been really successful with things like API functions that are maybe loaded on a page in a large piece of JavaScript on how to implement the API but since you're there just using one function you're only executing one one-hundredth of what the API could do well now you get the pads for all of the API and how you instrument them and if they haven't basically set up access control having this mapping and knowing how to work that API without a document is absolutely glorious you can find vulnerabilities even these things reference configuration files sometimes like a lot of times we're missing out on a lot of this information in JavaScript files now we're not as much anymore because we have helper tools like link finder very similar is J's parser actually written by Ben who was in the room earlier I don't know if he ever got in it just said but he was here or help he helped write this does the same thing parses out paths that are referenced in JavaScript so how do you feed these tools those pages it's pretty simple you go to your top-level target in burps sitemap you go to engagement tools again you have to have the pro engagement tools you say fine scripts and then you copy the selected URLs that have scripts on them and then you feed J's parser or link finder that list and it will automatically go and do its magic and find the URLs that you need to add all right so now we parse JavaScript we have a good map of the application what we're doing etc okay so now we want to do content discovery the idea of content discovery or directory first content discovery a directory brute-forcing who has done this before anybody directory brute force and no one's used der Buster before or anything ok there we go all right I was worried there for a second but the idea here is you have twitchtv right and you've spider and use it as a user and you know all these paths and functions that it's executing but that's not the whole story absolutely there are back-end URLs that are used by service staff like admins there are usually configuration pages installed by frameworks or login pages installed by frameworks there's just a whole bunch of stuff behind the scenes now how do I know about that when I look at a site I do directory brute-forcing with something like der bus turnout der Buster is super old-school nobody uses der bus tour anymore we have now moved on to some command-line tools that have instrumented the same thing that doorbuster does but much faster like go Buster go Buster is one of my go-to is another one is der search which I've been using more lately and the reason you would use one of these tools over the other is the amount of information that gives you back and control of the directory brute forcing five minutes okay got it and so these tools will allow you to go through a large list of directories the best one right now is robots disallowed written by Daniel over here he went out and spidered all of the robots.txt files in the large Lexx illustrate or the top alexa whatever and then built it into a list for us to use because if you think about a robots.txt file it's what developers don't want you to find right so now we go to every place they don't want to find they want us to look at stuff and we look at stuff and that's what a hacker does so so go Buster and robots disallowed pretty one so this is another list that's pretty good for this I took robots disallowed and again every tool that I ever found to do directory forcing and I catted it into one list it's pretty but it's still pretty good it works for me it's a large list so this will take a long time to run on a target the other idea is that now we have a whole bunch of functions maybe that are linked in JavaScript er we got from other places but we don't know what parameters they take to actually action the function so you can brute force parameter names as well once you find a function that seems juicy and I only ever do this when a parameter seems really juicy like you know like admin equals whatever you know transfer data you know depreciate user like whatever the keyword seems juicy I'll do this kind of analysis otherwise this is another time consuming step brute forcing anything is time-consuming so there's also a list of the most common most common parameters on the internet that's integrated into a tool inside of Berk called backslash powered scanner it's the top parameters that appeared on websites from the Alexa Alexa list and you can feed this to a tool called pram F which will try to first of all to try to find frantic commonly known parameters on a URL or on a script or on a rest path and then it'll feed at this list and try to group for some if it can't elicit what they are verbatim so this also works really well this list is backed out of this backslash powered scanner list called params it's in the folder on port speakers github it also is really useful for API fuzzing so if you're up against the rest path like you find an API and it's a rest-based API this list is really good for trying to find rest-based API functions when you're doing a block so a black box web of it alright we're almost done I swear
alright so the other one is this idea of auxilary testing like just some random if I wanted to add in here a lot of people are committing bad stuff to github a lot of times they do it they're just on accident they commit passwords they commit config files private keys whole bunch of stuff so basically what I do in my script when I kicked off the one earlier is the first thing it does is it builds a set of links that are searches to github so here you can see I have a word here password which is actually the one you hit on the most with this kind of analysis and I build a link here I say github comm /search q and then my domain is this dollar sign one here right that's that's the domain I've so in this case it would probably be twitch com and then plus password and that's the search and if I click on this out of my console it'll take me to github I have to be logged in and it'll do a search on all github projects for any code that has been committed that has in it and password equals and I will find invariably a lot of these this seems dumb and simple it happens all the time and is worth a lot of money in bounties and is worth indicative of a lot of risk to your organization it just happens people go spin up custom code projects they forget they commit it to their own repo then they even remove it but the reference is still there in the history so this is a big thing that happens all the time so I build these links dynamically while the subdomain scraper and brute force are running in the background which take a long time I am manually going to each one of these github searches and trying to find out if they've committed stuff so I'm trying to stack my activities so that I'm never wasting time something new I've started to do is favicon analysis so the favicons that are associated to you know your little tab and it shows you like hey this is the Adobe favicon or whatever I pulled that down from the main website which I already know usually when starting one of these I hash it and then I pulled on everyone I see on every site that's in that port scan that's in the reference subdomains when I see if the favicon matches and then I know that for sure that is probably owned by that site and it might be something indicative that I need to test so favicon analysis and that methodology is pretty new I confirmed this with a friend of mine he's pretty good at recon this is actually worked out for him as well so found a couple of reference IP ranges in the cloud that he didn't know for sure were targets but them because he could verify via the favicon that they were was able to get permission to hack them when nobody else had ever seen them and found a couple bounties on that so super cool this was the last one I told you about that actually found out the table up there go grabber so the idea of HTTP screenshot or any of those screenshot tools this is a new one what I like here is that it's faster it's written and go probably so it's probably faster and I'm gonna try this when I get home I heard it's pretty good so I don't really have any data on this but if you want to go institute something quick you can try this when you get home so if we go back this is the total methodology wrapped up in kind of bubbles and a cat the tools that I use with it and this is iterating all the time like this is always changing every couple of months something happens where I change up my methodology I'll wait until all the cameras to do just yes what no what is that maybe okay yeah absolutely I don't have any experience with that I would love to add its methodology I let's talk absolutely yeah okay so that's most of it let's look at the automation that ran okay so here is a
holy crap all right that was a lot of stuff so this is my automation it's just glomming basket bash scripting it's nothing special right so I started on twitch TV at the top right first it says it's running a mass sub finder and mass DNS on twitch TV so it's gonna take a while then what it gives me is the CrunchBase links for acquisitions for twitch so I'll take this and paste it in my browser CrunchBase uses distill so I can't automatically pull this down into the command line I actually have to go visit the page as a human because this still is a really good bot protection so I just build the link here I go check it out I see who they've acquired I add them to my mind map and then I also want to see if there's any like kind of directory structure that twitch TV maybe uses or it's referenced very highly in Google search results so this is a high on google scraped our Google browser basically so Kember the name of it but I Institute this basically command-line browser to pull back Google queries and and then it gives me links that are referenced on Google for twitch TV so I start looking at these things and seeing if there's any strong correlations of sites I should test then it builds my github lists for me right so password so I'll just grab this and copy it or open in the opener browser so really hope
nothing shows up so somebody has a project here which has a JSON config file with maybe credentials in it here's a bot config for probably scraping twitch probably not run by one of their own employees here's a Khan file that doesn't actually specify a password here's somebody who's put a variable in for a password user and past that's really secure oh ah so this is going to use OAuth yeah so I'll look through multiple pages here when you're looking through the github output you can choose best match or recently indexed recently indexed will give you you know a good view of like if anybody's done something like really recently and maena they may have forgot to pull it out so I will use both I will look at both sorting views of this data and I will do this for all of these password ID RSA passwd there's a long list of these that I took from another tool I can't remember the name you remember Dan what the name of that tool is yeah some it's some other github like Dorking tool but it does all it does some of the analysis automatically but it was for a different use case so I took that and put it into building these links then the scraper and brute force source stuff finished processing and I want to make sure that these sites exists and are not just references or taken down right because it's scraping it's taking information off the internet you don't know if that sites may be taken down right so I built my own script to resolve everything not super fancy and it resolved IP so a lot of IP are a lot of domains were found a lot what what is it oh okay I gotta go so anyway you can do this at home this is not fancy scripting I get the I get the list of IP is to do and analyze I get the links for the sites I load them in the browser and I test you I test them doing web hunting that's it thank you very much [Applause]