RECON VILLAGE - Core OSINT: Keeping Track of and Reporting All the Things
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 322 | |
| Author | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/39959 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
00:00
Magnetic-core memoryTraffic reportingSlide ruleGoodness of fitInternetworkingMultiplication signLink (knot theory)BitTraffic reportingPresentation of a groupMashup <Internet>Table (information)XMLComputer animation
00:59
Software testingOpen sourceMagnetic-core memoryVideo trackingMUDStorage area networkProjective planeWeb 2.0Open sourceWeb applicationRight angleWebsiteSoftware testingComputer animationJSONXML
01:35
Magnetic-core memoryVideo trackingLine (geometry)Open sourceModal logicSoftware testingXMLUML
02:06
Magnetic-core memoryVideo trackingMathematical analysisTraffic reportingIP addressBitConnected spaceOpen sourceData acquisitionTraffic reportingXMLComputer animationDiagramProgram flowchart
02:53
Magnetic-core memoryVideo trackingPivot elementTraffic reportingMathematical analysisTraffic reportingOpen sourceXMLUMLProgram flowchart
03:23
Traffic reportingInjektivitätNumberOnline chatTraffic reportingCuboidBoom (sailing)SequelTwitterDatabaseWebsiteMereologyOpen sourceSoftware testingSocial classStorage area networkData conversionMotion captureFlagMultiplication signWeb 2.0MultiplicationXML
05:33
Type theoryCartesian coordinate systemBitMultiplication signProcess (computing)XMLUML
06:02
Magnetic-core memoryVideo trackingType theoryComputer-generated imageryMathematical analysisVideoconferencingUniform resource locatorWeb pageFunction (mathematics)Mathematical analysisTraffic reportingMultiplication signLine (geometry)Different (Kate Ryan album)Computer forensicsSoftware testingData typeXMLComputer animation
07:07
Magnetic-core memoryTraffic reportingFrame problemInstance (computer science)Address spaceRight angleShared memoryTraffic reportingSource codeInformationType theoryNumberRule of inferencePhysical systemPhysical lawConstraint (mathematics)NeuroinformatikComputing platformGraphics tabletSensitivity analysisWebsiteSingle-precision floating-point formatMultiplicationDifferent (Kate Ryan album)Computer animation
08:49
NumberEnterprise architecturePhysical systemMultiplication signTime zoneCybersexDifferent (Kate Ryan album)Type theoryMusical ensembleTouchscreenCASE <Informatik>Right angleMathematicsComputer animation
09:48
Magnetic-core memoryVideo trackingIntranetData storage devicePoint cloudLocal ringMultiplicationMobile appInformation securityPhysical systemPoint cloudCloud computingGoogolRight angleInformationServer (computing)XML
10:25
Magnetic-core memoryVideo trackingLevel (video gaming)Goodness of fitHecke operatorSelf-organizationOffice suitePoint (geometry)MultiplicationLaptopTraffic reportingDifferent (Kate Ryan album)InformationEnterprise architectureProduct (business)Process (computing)MehrplatzsystemArithmetic mean3 (number)Computer animation
12:13
Magnetic-core memoryVideo trackingStandard deviationTraffic reportingOrder (biology)Function (mathematics)Presentation of a groupComputer animation
12:50
Magnetic-core memoryTraffic reportingPivot elementVideo trackingTraffic reportingAngleProjective planeProfil (magazine)Open sourceFacebookDifferent (Kate Ryan album)Revision controlCycle (graph theory)DiagramComputer animationXML
13:57
Video trackingMagnetic-core memoryTraffic reportingProcess (computing)Different (Kate Ryan album)Phase transitionRevision controlType theoryDiagramMereologyMathematical analysisCycle (graph theory)Sheaf (mathematics)Right angleTraffic reportingDiagram
14:45
Video trackingMagnetic-core memoryBitRight angleCybersexSlide ruleSocial classJSONXMLComputer animation
15:24
Video trackingMagnetic-core memoryInformationPhysical systemUser interfaceVideoconferencingComputer-generated imageryRight angleNeuroinformatikCrash (computing)InformationNumberTwitter1 (number)Medical imagingAvatar (2009 film)Uniform resource locatorComputer animation
16:18
Physical systemMagnetic-core memoryVideo trackingTouchscreenStatisticsAvatar (2009 film)Type theoryPhysical systemTwitterComputer fileUniform resource locatorTraffic reportingGame theoryComputer animationXML
16:49
Magnetic-core memoryVideo trackingUser profileWeb pageRobotWeb pageTwitterInformationGeometryComputer animation
17:18
User profileWeb pageMagnetic-core memoryTwitterAvatar (2009 film)Computer-generated imageryComa BerenicesTraffic reportingVideo trackingVideoconferencingMUDMedical imagingRight angleReverse engineeringWebsiteProfil (magazine)Uniform resource locatorGame theoryComputer fileComputer animationXMLUML
18:06
Computer-generated imageryUniform resource locatorFunction (mathematics)Medical imagingUniform resource locatorFunction (mathematics)Density of statesPhysical systemType theoryBranch (computer science)XMLComputer animation
18:52
Video trackingMagnetic-core memoryWeb pageComputer-generated imageryDifferent (Kate Ryan album)WebsiteType theoryExistential quantificationNumberComputing platformProduct (business)Metropolitan area networkComputer animation
19:26
Magnetic-core memoryVideo trackingMusical ensembleNumberMusical ensemblePhysical systemSoftware testingProduct (business)Group actionType theoryEnterprise architectureComputer animation
19:57
Mobile appSet (mathematics)WordBefehlsprozessorExplosionObject (grammar)TrailVideo trackingMagnetic-core memoryTextsystemRight angleVisualization (computer graphics)Process (computing)Goodness of fitType theoryFamilyInformationDomain nameFacebookGroup actionSoftwareMultiplicationWordConnected spaceMobile appIP addressMultiplication signPresentation of a groupDomain nameComputer animation
21:24
Focus (optics)Video trackingMagnetic-core memoryVisual systemComputer-generated imageryMultiplicationHand fanBranch (computer science)MappingTwitterCartesian coordinate systemMobile appType theoryGame theoryBlogWeb 2.0XMLComputer animation
22:45
Magnetic-core memoryVideo trackingFreewareContent (media)Coma BerenicesStorage area networkCartesian coordinate systemLevel (video gaming)Social classSlide ruleFreewareStudent's t-testTexture mappingComputing platformData miningComputer fileWeb 2.0WebsiteUML
23:41
Process (computing)Magnetic-core memoryVideo trackingInclusion mapAddress spaceProcess (computing)Address space1 (number)WebsiteLevel (video gaming)AliasingNumberEmailType theoryDifferent (Kate Ryan album)Mathematical analysisFile formatIP addressInstance (computer science)Formal verificationDiagramProgram flowchart
24:44
Data storage deviceMagnetic-core memoryVideo trackingProfil (magazine)Type theoryInformationData storage deviceWebsiteWeb pageDoubling the cubeEntire functionContent (media)Point (geometry)Arrow of timeTwitterVideo gameBranch (computer science)
25:25
Magnetic-core memoryVideo trackingHost Identity ProtocolMehrplatzsystemProcess (computing)Type theoryUniform resource locatorComputer fileMultiplication signMappingData storage deviceMehrplatzsystemContent (media)Game theoryLevel (video gaming)NeuroinformatikFilm editingMusical ensembleAreaWebsiteRevision controlProbability density functionMultiplicationXMLComputer animation
27:12
Video trackingMagnetic-core memoryWeb browserComputer-generated imageryGoogle ChromeExtension (kinesiology)Computer fileWebsiteMultiplication signMotion captureTouchscreenDisk read-and-write headWeb pageGoogolGoogle ChromeGame theoryUniform resource locatorExtension (kinesiology)Computer animation
28:25
Video trackingWeb pageTraffic reportingMUDCASE <Informatik>Web pageData managementWordRevision controlComputer fileDigital photographyInternet forumThread (computing)Suite (music)Reading (process)XML
29:30
Video trackingMagnetic-core memoryWeb pageUniform resource locatorComputer-generated imageryHash functionData integrityHypermediaSide channel attackMetadataStreaming mediaMultiplication signRepository (publishing)GoogolSlide ruleWebsiteFunction (mathematics)Graphical user interfaceInternetworkingWeb crawlerComputer animation
30:38
Video trackingMagnetic-core memoryGoogle ChromeOpen setFunction (mathematics)TwitterService (economics)VideoconferencingIdeal (ethics)Web pageRepository (publishing)Computer fileWeb pageProfil (magazine)Module (mathematics)Content (media)ResultantGraphical user interfaceMultiplication signComputer animation
31:35
Magnetic-core memoryVideo trackingTraffic reportingGoogolWebsiteWeb pageMehrplatzsystemPosition operatorNegative numberTraffic reportingWordBitPlug-in (computing)Core dumpGoogle ChromeFunctional (mathematics)XMLComputer animation
32:57
Video trackingMagnetic-core memoryMobile appComputer-assisted translationLevel (video gaming)MappingShareware1 (number)Computer animation
34:03
User profileWeb pageVideo trackingMagnetic-core memoryTwitterComputer-generated imageryAvatar (2009 film)BitTraffic reportingVirtual machinePhysical lawOpen sourceDistribution (mathematics)Type theoryDiagramInformationAbstractionComputerLevel (video gaming)Link (knot theory)CASE <Informatik>2 (number)Mathematical analysisCuboid1 (number)Multiplication signGoodness of fitRevision controlVapor barrierDemosceneReal numberPresentation of a groupFreewareData conversionVisualization (computer graphics)MultiplicationInformation privacyFile formatHacker (term)WebsiteNeuroinformatikSoftwareNP-hardGroup actionComputer fileSineDisk read-and-write headComputer animationXMLUML
Transcript: English(auto-generated)
00:00
Okay, if everyone's ready, we'll get into the next session. Don't worry, I will be disappearing from this microphone very, very soon because I've got a pizza that's under that table that I need to eat. But I will just introduce Michael Halfman. I introduced him yesterday into the wrong name. So if I've done it wrong again, he's just going to ignore that. And we're swiftly going to move into his talk. And this talk is core OSINT, keeping track and reporting of all things.
00:21
And over to you. Thank you. Thanks, everybody. We're going to have a good time. When I first submitted this talk to the Recon Village, I was thinking, hey, this could actually be kind of a workshop. And then I thought, well, you know, maybe I'll do a little bit more of just a presentation. So this is actually one of those neat mashup talks. You're welcome to play along as I do certain things here.
00:41
All of this stuff. First off, my slides are on the Internet already. I'll give you a link at the end so you don't have to do the taking the pictures thing. You'll get the actual slides. And also, there's a lot of things that I'll be releasing during the talk. So you can again, you can grab that off the interwebs. So I am Michael Hoffman.
01:02
I go by Web Breacher on the interwebs. I have a nice website, WebBreacher.com, where I blog about a lot of these open source intelligence things that I find helpful, fun, entertaining, as well as web application stuff. I do pen testing, run a makerspace, open source projects, teach for SANS, wrote a course on open source intelligence for SANS.
01:24
That's a lot. But let's not talk about me. I have a question for you all. All right. This is actually a question where you're going to need to raise your hand. Yes. Exercise involved. Here we go. Ready? Raise your hand if you love to document. OK, wow, you guys are weird.
01:43
I wrote this line like it's going to be crickets. But no. All right. How many of you really believe? No. So, yeah, I absolutely don't like documenting. But I recognize that it's a necessity in penetration testing, as well as defense work, as well as open source intelligence.
02:02
We have to do it because if we don't, things go awry. Now, when I was younger, what I found about open source intelligence is I love to do the data acquisition. I love finding shit about people or things or IPs. I love analyzing the data and putting it together and finding those connections and finding how things actually appeared.
02:22
Whether it was using like Maltego or something else and finding those relationships. That was awesome. And then and then which of those things do I need to dive deeper in? Where do I need to pivot and go deep? I love doing all that. And then, of course, my my senior OSINT person or senior Pentaster would be like, yeah, Micah, you know that the reports due tomorrow.
02:41
I'm like, just a little bit more, dad or whatever his name was. Just a little bit more. You know, I've almost got more things I can find. And I always squeeze that little open source, the documentation to the end. Nowadays, adulting Micah, we I still like doing the same things, but what I'm finding is that I'm doing a lot more analysis.
03:04
And the reason why I'm doing more analysis is I have more data, more good data that I can go through. It's that data that you collect along the way is going to help determine where your OSINT investigation takes you or doesn't take you.
03:23
And it's very important because I've written a lot of open source intelligence reports. I've written a lot of pentest reports. I can't tell you the number of times I've gotten to the pentest report and like, yeah, we popped that box and I got sequel injection and I stole a copy of the database.
03:40
Who has the database? I know I popped in there like, yeah, you had it. Remember, you put it into our chat. Oh, and the chats, you know, now in slack, you know that that that part of the conversation is gone, so you can't actually grab it. And I don't have that stuff. That would be awesome sauce to my customer. Don't let that happen to you.
04:00
One of the things that I like talking about is documenting deeply as you go. When I do Web pen tests, I document if I'm doing sequel injection, boom, I'm tagging it. I'm looking at it. I'm recording it as I go. When I do something in open source intelligence, I have to record it as I go because that data might change in a minute.
04:20
You ever see somebody tweet something out and then it's gone. Wait, but you're like, well, that was cool. And you click on it's like tweet is unavailable. Yeah, stuff disappears. And if you're doing a multi day, multi week assessment, that stuff is not going to be there potentially when you get back to it. So document deeply. And, you know, what I found is that when you actually do document deeply, your reports get better because you have more data.
04:46
You don't have to start scrounging going, well, you know, I don't really have a screenshot of that. But if you imagine a Web site with a picture of the person on it, that's what no, you have the pictures as they showed up. And that's really the impactful part of it.
05:02
When we do open source intelligence, one of the things that we have to do is convey to our customers what we see and what it means. And that what it means is so very important for the impact. Over the years of teaching for SANS and teaching internally for my
05:20
companies, I've found that people are still using some older technologies to document. We do capture the flags in the SANS classes. Have you ever taken one? Usually on the last day, we do a capture the flag. And I see people pulling up these, oh, solid but old, tiny types of of applications to record their process.
05:43
And every time I see that, I'm like, oh, we can do better. And that's really what this course is about, of course. That's really what this talk is about, because if you're using these to capture your notes during an assessment, I'm guessing it's suboptimal. And I'll show you how in just a little bit.
06:02
But before we get to the tools, which I know everybody's like, all right, show me more tools, you know, release them. Before we get to that, let's talk about why you document, because that's really as important as what you document, because what you are collecting while you're actually doing the assessment, while you're doing the scanning, while you're doing the analysis is extremely important.
06:23
And we've got a lot of different data types. We have a lot of different pieces of data also that we need to collect to keep that timeline of evidence. If you're doing something for law enforcement or for forensics, you've got to keep dates and times. You've got to keep where you got something. Sometimes you've got to keep how you got there, too.
06:43
I've seen pen test reports and OSINT reports where it's like, hey, I found that picture of the of the person that was doing some kind of insurance fraud and and they are playing basketball in their driveway. And my customers like that is terrific. How did you get it? Where did you get it? What date was that?
07:02
And if you don't have that data, you're ultimately failing your customer. Now, when we actually do collect stuff, we have to think about it. We we may have a tendency to collect all the things, and that's great. Before we actually do an assessment, we have to think about what are the types of data, where might things go awry and what are my special considerations?
07:23
For instance, if you're doing some some gathering of sources, gathering of information, maybe on some dating sites or on some other more sensitive types of sites, maybe you find some classified data or proprietary data. Maybe you find some some data that is against the law to have on a computer or to be sharing.
07:43
How do you record that safely enough so you can include it in your report, but not violate any laws yourself by propagating that type of stuff? And there are rules for this. You have to think about it before you just collect. Some of the other things that I found is that when you are collecting information about people, sometimes people share an address, right?
08:03
Share a phone number. OK, if you're doing that text pad, notepad plus plus type of documentation, which is serial, right? It's like I got this person. I got that. Well, then, if you are putting that phone number on multiple people, now you have duplicate data in your report, right? It's that phone number here and that phone number up here, too, which is a little suboptimal instead of relating a single piece of information.
08:27
Another thing is some people are going to do an assessment in hours or minutes, and that's going to be really quick. Some people will do campaigns and look at their targets over months. Those are going to have different constraints for how, where and what you're storing that content on and your documentation system and platform.
08:48
Has to account for that. And another thing is some of you probably work in teams or maybe you share your data with the defenders or maybe you're working on cyber threat intelligence. You're CTI people and you're actually going to be writing this up to give to somebody else.
09:03
Well, when you actually collecting data in teams, it's a whole nother type of mess, right? If you've done this solo, you know, you can rely on yourself. You're like, oh, yeah, I'm doing this great. But when you're combining that data or getting somebody else access to it, it can be challenging. You've got to work that out first.
09:21
You've got to work that out before you dive into the assessment, especially if your team is around the world. I work with teams that are geographically dispersed across the United States. And hey, that time zone change, it matters because when I record something on my system and say, hey, I found this at this date and time. Well, is that UTC or is that East Coast time?
09:42
It makes a difference if you're going to court. It makes a difference in some cases. You have to think about these things. The other thing is, where are you going to store your stuff? Right. Cloud systems. There are some amazing cloud applications that are out there that make storing information so, so simple.
10:00
And yet there they can be a security risk, right? You're sharing your stuff with Google. Now, Google would never read your documents that you store in their servers, right? You're using HTTPS. It's encrypted, right? Yeah, you have to think about this. You have to consider it. And then also is your application that you're using made for multiple people to use it or just one?
10:26
Also, we have to think about, are we documenting this for our notes so I can remember my process, so I can create a report? Or are we documenting to hand off to somebody else so that they continue it? When I do some of my assessments, I do up until a certain point that I'm like, hey, I'm done.
10:43
You keep going, defenders. You go ahead and research the rest of those IPs or do it. And I'm doing that handoff. Well, there's a different level of documentation that you need there. I'd recently, we were doing some some work in a OneNote document. If you know about that Microsoft product, good tool, multi user, it works within an enterprise.
11:04
It's really good work sharing outside of an enterprise. And I was writing this, my notes for myself, you know, because the notebook wasn't shared out. Well, somebody else is like, hey, do you have notes on? I'm like, do I have notes on here? Let me just add you to this. And he looked at my notes and he was like, well, what the heck does this mean?
11:23
This is bad person. And what does this mean? Because I was documenting it for me. So we have to think about who our audience is and could be, because that and could be is that when you deliver your report or your notes, you never know where they're going to go. I've been in organizations where I deliver my notes to a customer and that customer takes that report.
11:44
And then six months later, I come back to somebody totally else in that organization. And that other person has said, hey, do you know Micah's quality of work? Do you have anything? And this person's like, yeah, take this really sensitive document and gives it to somebody else in their organization that has no need to know that information.
12:02
So my report, when I walk into that person's office is sitting right there on their desk. I'm like, well, I'm glad I didn't include sensitive stuff in there, but we have to think about that. We also have to think about how we're going to document. I always like to document more than I need because I can always scale it back.
12:21
I can summarize. I can redact or whatever it is that I need to in order to do my output, whether it's a presentation, whether it's to do something like a report. You can always summarize and redact. But if you don't have that data, it's hard to get it back. I've had people that that do an assessment and they're like, oh, it was there.
12:42
And they take a picture of the thing that says tweet unavailable and put that in the reports like that's that's not the way to do it. And then your angle, you know, where is this document going? Where are your notes going? Are you going to be continuing to work on this project? Is are you going to hand it off to somebody reported, et cetera?
13:02
Thinking about that is going to help you in understanding what to do with the data. Also, you don't know where the data is going to take you. I'm sure if I if I asked for a raise of hands, which I'm not going to, and asked you if you ever had a simple project that you were working on, maybe an OSINT project and you were just doing Facebook stuff.
13:22
And then the the the assessment maybe took a left turn or something, or maybe you're looking at some dating profiles or something like that and something weird popped up. I've done assessments where I'm just, you know, doing kind of a little background on a CEO or somebody that's a C-suite person. And then I find that there's a Tinder name with that same person that that that that person has been using for their Gmail.
13:46
It's like, huh, that's weird. And you look over there and it's a whole nother like lifestyle that this person is leading. You never know where your work is going to be taking you. Now, within open source intelligence, there's a lot of different versions of this diagram of this this cycle.
14:04
Essentially, we have requirements gathering in the upper right. Then that leads to retrieving data of some type, analyzing data and pivoting and reporting. And what I like to do is tag what types of things I'm looking for. What is my documentation? What is my reporting look like at each of these different phases?
14:21
Requirements gathering is is I'm going to be asking the questions of what am I doing, getting those requirements for my customers so that when I start gathering the data, I know what the hell I'm looking for. What are we finding? That's where you're collecting the data. Of course, if we also have, you know, the what is what are we missing parts in the analysis section and then when it gets to reporting of whatever kind, we're going to go ahead and make that now.
14:46
This might hit a little bit close to home for some of you, but stick with me here. Let's say that we have an example. Let's say we have a scenario. Maybe your son or daughter's teacher comes to you and says, listen, you know, I know you do that cyber stuff, and I'm wondering if you could help me find something.
15:04
There is this dude that has been cyberbullying a bunch of kids in the class, and he goes by the name Dread Pirate Roberts, written like it is on the slide there. Most of the kids that are playing on PlayStations are actually getting hit with this. So so I need you to go ahead and take a look at this.
15:22
You're like, cool. All right. Start me off with the username. I got that. So you start your documentation. Does anybody see anything wrong right off the bat? Yeah, we're in notepad. So notepad. What happens on notepad when you close notepad? Does it save it? It asks you if you want to save it, but then if you don't say yes, it's gone.
15:42
What happens if your computer crashes? It's gone. So let's say that you went ahead and did this. All right here, we're going to do Dread Pirate Roberts. The goal is this fine. So you go into DuckDuckGo and you do a search. Now the search here pulls up a Twitter account with the Dread Pirate Roberts, that actual username.
16:01
We've actually got a bunch of information here that we can pull from. We've got a Twitter account at number one. We've got an avatar. We can do some reverse image searching on. We've got an actual spelled out name, Dread Pirate Roberts. That's that number three and actually a location. Number four is County Claire. Cool. So we go back to our notepad document. We type it in.
16:20
All right, DuckDuckGo reports Dread Pirate Roberts found on Twitter. Here's a URL. Here's this. Um, there is a screenshot of the avatar and maybe you save it as TW avatar one. So now you have a separate file on your system. All right, cool. All right. So you look at some of the other things that are in that on that DuckDuckGo search. Oh, we've got Clash Royale.
16:41
Ooh, now we're getting to the gaming stuff. That was kind of the primary thing that we were looking for. There is a username, Dread Pirate Roberts. Cool. So we've got that. We're going to document that. And of course, now we've got the Twitter plus we've got this Clash Royale thing. So we've got to look up both of those profiles. So we're just going to put a to do on the page. Cool. All right, let's go back to that Twitter thing.
17:02
And so here on the Twitter page, we've got even more information coming across because again, we've got that avatar. We've got some tweets we need to look into. We've got some geo located tweets. Maybe we can get some information about where that person is or has been. And of course, we're going to go back to notepad. We've got to document this stuff, right?
17:21
We've got some more to do. We've got to do the reverse image search. We've got to do this and that. And then you do the reverse image search. And now you've got two hundred and thirteen sites to look at. And maybe you do ReconNG. There's this cool profiler module some dude wrote that goes through and takes a username and goes and looks at it across over one hundred fifty websites.
17:42
Well, you've got that. And so now this is a blow up of one of those. Now you've got four other sites, including adult friend finder that you need to go look at. You've got an Xbox. Well, you have to go look at it. You never know where your assessment's going to take you, right?
18:00
Sorry. Yeah. You've got Xbox gaming and stuff. All right. So we've got some URLs to go to. And in 10 minutes, we've got a hell of a lot of to do items. We've got images, we've got usernames, we've got URLs, we've got a lot of stuff that we have to collect. Plus, we have that ReconNG tool output, which we somehow have to include in our documentation.
18:25
And like I said, this was within 10 minutes. If you're doing this type of an assessment in hours, can you imagine how many of those branches you're going to have to actually deal with? How many to dos you're going to have? If we're going to do that, we need to have a documenting system that allows us to say, I need to go do that.
18:46
And when I do, I can collapse it or I can mark that that branch as checked off. We also have to have a documentation platform that allows us to collect, note, annotate different types of data.
19:01
Whether it's the pictures that we're coming across on our site or pictures of the websites or the data. And man, wouldn't it be nice if while we're just doing the work of OSINT, something's collecting all of that stuff for us. And some of you already know that Hunchly does that really, really well.
19:21
If you don't know about Hunchly, stay tuned. Now, when I made this talk, I thought about, well, there is a huge number of documentation products. And many of you that do like OSINT, CTI or even pen testing, you might have special systems or enterprise wide systems that you have access to.
19:40
I wanted to keep this talk geared towards that solo practitioner, that person that's doing that OSINT or that pen testing recon or whatever it is alone or in small groups of teams. So that's really my sweet spot here. And that's who I'm talking about. Now, one of the things that we have to think about is the type of data that we're going to be documenting.
20:01
I was here today for some of the other talks and earlier yesterday for a couple of talks, and I saw some great information on Maltego. If you saw Andrew present on Maltego and how it puts together data, we've got some data visualizer apps out there that are amazing for showing us connections between IPs, domain names, user accounts.
20:23
I had an assignment one time where I had to do I had to see if I could find information about a certain target. And so I went to her husband on Facebook and I found through his network a whole bunch of people with their same last names and then threw all of that right into Maltego.
20:43
And what Maltego did was I said, show this show this data. It showed me really good, tight packs of people that were interconnected, connected to another group of people that were that were connected very tightly. And it allowed me to see this is a family group and that's a family group and it's connected via these people.
21:02
So we have visualizers, Maltego, Gephi, Cytoscope, we also have word processing apps, word, LibreOffice, these types of things as well, and they're good for general documentation. But we also have apps that are made to make our jobs as o-centers a lot easier.
21:23
Let's talk about that. Now, I wanted to focus on one flexible apps that are going to make sense to most of you. It's not going to work for all of you. Some of you are mandated to do one type of thing or another. I also there are a lot of apps out there, so I don't want to actually say, hey, go ahead and use this.
21:41
And it just does one piece of the puzzle. I'm looking for those biggest bang for my buck because I don't want to focus on documentation. I want to focus on doing the o-cent and to have something document for me. Also, easy to use, always a requirement for me and ultimately decreases the work I have to do. Now, if you've looked at my blog WebReacher.com at all, you know, I'm a very big fan of mind maps.
22:05
Mind maps are amazing visual note taking free applications that are out there that allow you to organize data. Graphically, so you have one piece of data like a username like Dread Pirate Roberts. That Dread Pirate Roberts then breaks out into a Twitter account and a gaming account and the adult friend finder and stuff.
22:26
And you have nodes that that then branch out. When you're done working on those nodes, you can collapse it and say this node is done. I'm done checking out adult friend finder and now I need to go and look at other stuff. And that's really helpful for keeping track of where you are in your assessment and what you have left to do.
22:46
If you're interested in it again, this slide deck is online, so you don't have to take pictures. But on GitHub dot com slash WebReacher o-cent tools, I have a mind map that you can have for free. That's going to hopefully jumpstart your documentation.
23:01
Now, I created this mind map over a year ago. And when I was teaching my SEC 487 SANS class, I found that one of my students was doing a lot of note taking during class. I was like, what do you know? And he's like, well, I took all of the things that you said to do in class and I put it into that mind map. That's awesome. So when you get this file, it has a lot of the notes, a lot of the sites, a lot of the other things that are in some o-cent classes like mine.
23:27
Now, we've written it in the X mind application. X mind is a mind map software application and it's free for Windows, Mac, Linux. Again, easy to use on whatever platform you're using.
23:42
And what it has is it has the centralized process. It has about five different tabs or sheets, if you will. And each one's meant for different things. For instance, one of them is meant to discuss things about process. Like, hey, I have email address.
24:00
What do I do with it? If you were here for my yoga talk yesterday. Yeah, it's kind of the same thing, but in an easy to use mind map format. So here we have an email address. If you look there while you do email verification, you might look for that email on breach sites like we saw it will do earlier with have I been pwned. It shows you that process of what you can do.
24:21
But we also have a tab in there on data collection that says, hey, if you're doing research on a person, you might want to grab their name and address and phone number and date of birth and aliases. You actually type that into this document and it will organize it for you. We can also do the same thing for IP addresses, hashtags, sentiment analysis, whatever it is you're doing, you can put it in here.
24:44
And when you take that, that content that we had earlier, you remember our document, that notepad, and you start to fill it in into a mind map, the data comes to life and you can see how things are put together. I love it because instead of duplicating the information like, hey, that Dread Pirate Roberts name is here and here
25:05
and here, I can just use those double dashed arrows to say, oh, that that that username was found on Twitter. And then I can paste in there the username, the picture of what the site looks like on his profile page and put other data points in there. And when I'm done with investigating the Twitter, I can click on these things, collapse the entire branch.
25:26
You could store other types of data, other types of files just by putting it in there. Other things that we have to do, document the URLs, document the dates and times. Mind maps are manual. So you have to copy that URL and paste in there.
25:40
It's not perfect. However, organizing the data this way can save you a lot of time, a lot of effort and can be quite appealing to your customers. Now, it's not all fun and games, not all great. I will tell you this, that to be honest, there are some drawbacks with mind maps. First off, getting the data out is sometimes a pain in the ass because think about this for a simple
26:04
investigation like the Dread Pirate Roberts thing I was telling you earlier, we've got now 213 sites we have to visit. Yeah, we might use a tool like Eyewitness by Chris Truncher or Peeping Tom by Tim Tomes or something like that to scan all those sites. Cool. And we can shove that in there.
26:20
But what happens is this mind map keeps branching out and branching out and branching out. And when we're using it on the computer, we're just dragging over to this area or dragging to that area. Well, when it comes time to report, how do you take something this big and fit it down to A4 size or eight and a half by 11 size? Sometimes very challenging, so sometimes we'll cut it up or do other stuff with it.
26:43
And with the pro version of X mind, there's some better methods of exporting the content. Sometimes you can do it to a PDF. Multi user, not so much. So if you're a solar practitioner or you're handing documents off, a mind map might be something that you're that you could use.
27:01
And then everything is manual, which is something I don't like. So what I do is I'll use a mind map for the overall investigation. Where am I going? What do I need to do? But I'll also use Hunchly for that automated easy button approach to my investigation. And Hunchly is amazing and written by Justin Seitz and his team.
27:23
It is a great tool. It's a Google Chrome extension, and it makes our lives so easy because it does a bunch of things. Everything from going ahead and cataloging and keeping track of any files we download
27:40
or while we're browsing to recording screen captures of every Web page you visit. So when you're Googling or DuckDuckGoing or when you're tin eyeing something or when you're doing whatever on adult friend finder or whatever, that site keeps it sticking in my head for some reason. I'm sorry. When you're going to the PS4 gaming thing site, you know, those pictures are going to be captured automatically.
28:03
Along with the date and time that you visited them and the URL. So you don't have to do it manually anymore. You see how this is like I'm hearing angels singing right now. Now this cost about $130 Canadian per year. If you're in the United States, it's like $3 US or something like that.
28:26
So if you've never used Hunchly, this is Hunchly 2. A lot of people might be using Hunchly version 1. Justin and his team upgraded to version 2 and it's significantly awesomer, which is a word. Here what we have is a dashboard.
28:40
When you launch Hunchly, you can bring up the dashboard, which does overall case management and summary. It tells you how what case you're currently in, how many pages you visited, how many files, photos tagged, etc. And then one of the neat things about Hunchly is that sometimes you have these things that words you need to look for in pages.
29:01
If I'm looking for Dread Pirate Roberts, any page I visit, I want Dread Pirate Roberts to pop out at me. So I can set it up as a selector and then I tell Hunchly, hey, wherever you see this on a page, I want you to highlight it in yellow. So when I'm doing my DuckDuckGoing or when I do a forums thing and I'm looking at
29:21
some forum gaming forum and there's Dread Pirate Roberts, it pops right down on the page for me. Makes it easy for me to do my OSINT. Also, it keeps track of and I know this is a little bit small and all the slides are out there on the Internet, but this keeps track automagically of all the websites you visit in Google Chrome.
29:43
So if you're doing if you have this great like stream of consciousness, like, oh, this DuckDuckGo search showed me this and now I'm going to take this here and go there and go there. It's going to keep track of all that for you and the date and time you did it. It's amazing. And one of the neat things is that if you're visiting social media sites that allow
30:04
the metadata that's inside of some pictures, you know, the geolocation or what camera took that picture. If that data is still with the pictures, Hunchly will pull it out and highlight it for you automatically for you. Whoa, that's awesome.
30:21
Now, one of the problems we have is that sometimes we still need to run like Ill Will's tool. You might need to run some other tool like ReconNG or Spiderfoot or something like that and you have that output. What I like doing is doing some side channel loading of that data into Hunchly because I use Hunchly as like my repository of all my data.
30:46
That way, when I'm done with my work, I take all that Hunchly data and I export it and that's what I save. So what you can do is take that ReconNG data. Remember that profiler module I ran earlier? We can export that to a CSV or a text file and then visit it in a Web page.
31:05
Take a look at this. So I have exported to ReconNG results.txt. It's a CSV. And then I visit, I open that document in my Chrome. And now Hunchly has tagged at that date, at that time, that it has that content and it will highlight all of the Dread Pirate Roberts names in there.
31:24
And that way, when I'm looking through my data, when I'm searching through my Hunchly, I'm like, well, hey, where else do I need to go? Where else was this found? I can pull that data up and it's all in one place. I mentioned to you the EXIF data. Yep. If you're visiting Web pages that have images, it'll pull it out, such
31:42
as this beach shot that has actually the GPS latitude and longitude in there. Again, we need to go to another site to take it out and take a look at it to see where that is. Does it corroborate with the data? But it's one less step that you have to take. You don't have to run the EXIF tool or visit a secondary or tertiary site to pull that out.
32:03
Now, again, I like to present the positives and the negatives. Hunchly's got a couple of drawbacks, in my opinion. One, it's only Google Chrome. I like doing stuff in Firefox. A lot of my my best plug ins and stuff are on Firefox. I just like doing that. This is only Hunchly. Also, it's single user.
32:21
If I have two people that are doing the assessment, each of them is going to have their own Hunchly data. And as of right now, I don't know a way to easily combine that data, or even if I'd want to. But combining that data and creating a centralized report will be a little bit more challenging because you will have to combine that data. And the reporting, it does have a, hey, dump all of the Web pages I visited, dump all of the
32:45
pictures that I tagged. It has that, but it's not like Microsoft Word or it's not something of that caliber. So, again, this is something that we can use to do that automated documentation. And then we put it together. I know it's like, that's the one picture you chose, Micah?
33:02
Yes, it was free. So if you know why you're documenting and where you're going to be putting that stuff and who you're documenting with on your team and who you're documenting for, then what you need to do along with your team is find those tools that work for you and work for your customers and do things the
33:23
way you want to. I have actually mandated on my team that they use mind map to do their assessments. And I had one person and it just didn't work with the way that his brain works. That's fine. That's absolutely fine. I realized that mind maps work for people that learn visually.
33:41
It's just beautiful. But if you're not a visual learner, that's not going to work. Whatever. Make sure that you understand what the requirements are that you need to keep track of. And some people are not going to necessarily be able to use these tools. But you can check out these tools. And there's other ones out there, too. Many of them have free trials and they'll take it and try it out and see what you're doing.
34:04
Because if you find yourself or a colleague doing this instead of doing something like this, then they have a little bit of learning left to do. And with that, I will put my contact information. And again, my presentations are right up there on the OSINT Ninja Prezos.
34:23
And that includes yesterday's presentation and some other ones I've done as well. You're welcome to take them. And the link to our MySANS course as well. So I don't know if I have other I think I might have a second. Do you have any questions for me? Yes, sir.
34:48
So there are some some distributions like Buscador, Buscador and other things that are specifically made for open source intelligence gathering and in analysis. Do I have any recommendations? My recommendation is going to be a cop out because I'm going to say do what works for you.
35:04
I was having a conversation with the gentleman next door and he mentioned something something similar. I love Buscador, Buscador. I think it's a great tool for what it is. It simplifies a lot of those command line tools. I understand, you know, I come up as a hacker in this community and I know that command line tools are not hard for me.
35:24
Python, Ruby, Go, I get it. But for a lot of people, they focus more on law enforcement. They focus more on the open source intelligence or the CTI aspects. And that stuff's hard. And where Buscador excels, in my opinion, is it put it abstracts that whole you need to run this command on the this command on the command line.
35:43
And they give you a graphical just box type in the username and it does all that stuff behind the scenes. So that is a great distribution and it's 100 percent free. It's on Intel techniques dot com's website, Michael Bazzell's website. But some of the things that you could also do is create your own virtual machine.
36:01
Depending again on what your threat, what your work is asking you to do as far as who you're working on. Obviously, the more sensitive the data, the more nefarious the things that your place you're going to be going. You're going to want more of a barrier between your host computer.
36:22
Other questions, sir? OK, so the question is, are there any laws that would prevent a company from doing this type of reporting on their employees? And the answer is, I am not a lawyer.
36:42
Next question. No, the answer is yes. There are laws that companies need to be aware of. There's H.R. stuff. But mostly what I find is that many companies have privacy policies for their employees or some kind of policy that governs what type of information they will collect about their employees and about their employees social and how the social activities and how they will use it.
37:08
And that is usually what you see guiding this, except if you're in the EU, in which case you now have GDPR laws and stuff like that. So I hope that that's helpful. Yes, ma'am. You answer the question in your head.
37:23
Well, well done. Are there any other questions? Yes, ma'am. So the question is, is that in my assessment here, the example I used the Dread Pirate
37:43
Roberts user, a single person and then branched out is the mind map format good for multiple targets? Let's say you're you're investigating a gang or or something like that, a group of people. The answer is yes. I like it for documenting that stuff.
38:00
I don't like it for visualizing that I what I would do is I would document all the relationships in here on no taking wise and then probably make something pretty in Maltigo. Maltigo, there's a free version called Case File, which will take a a CSV and import it and it will make a beautiful diagram of this picture.
38:24
So for a report and for that, that finalized version, I'd probably use a real visualization tool instead of this. OK, all right. Well, I'm going to go ahead and say thank you very much for your time. If you have any questions, hit me up or come talk to me. Thank you for your time, everybody.