BLUE TEAM VILLAGE - Stop, Drop and Assess Your SOC: Sonducting and Using Att&ck Assessments

Video thumbnail (Frame 0) Video thumbnail (Frame 1412) Video thumbnail (Frame 2218) Video thumbnail (Frame 4037) Video thumbnail (Frame 5972) Video thumbnail (Frame 8035) Video thumbnail (Frame 11989) Video thumbnail (Frame 13096) Video thumbnail (Frame 14449) Video thumbnail (Frame 15757) Video thumbnail (Frame 16531) Video thumbnail (Frame 18433) Video thumbnail (Frame 19598) Video thumbnail (Frame 21502) Video thumbnail (Frame 22703) Video thumbnail (Frame 28292) Video thumbnail (Frame 31754) Video thumbnail (Frame 32716) Video thumbnail (Frame 34092) Video thumbnail (Frame 35312) Video thumbnail (Frame 36964) Video thumbnail (Frame 37936) Video thumbnail (Frame 39243) Video thumbnail (Frame 41005) Video thumbnail (Frame 42770) Video thumbnail (Frame 43543) Video thumbnail (Frame 45445) Video thumbnail (Frame 46680) Video thumbnail (Frame 48514) Video thumbnail (Frame 49983) Video thumbnail (Frame 51600) Video thumbnail (Frame 53021) Video thumbnail (Frame 53878) Video thumbnail (Frame 55116) Video thumbnail (Frame 55949) Video thumbnail (Frame 56787) Video thumbnail (Frame 59359) Video thumbnail (Frame 60260)
Video in TIB AV-Portal: BLUE TEAM VILLAGE - Stop, Drop and Assess Your SOC: Sonducting and Using Att&ck Assessments

Formal Metadata

BLUE TEAM VILLAGE - Stop, Drop and Assess Your SOC: Sonducting and Using Att&ck Assessments
Title of Series
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Release Date

Content Metadata

Subject Area
Traditionally SOCs look outward from their network perimeters, missing the adversaries already operating in their networks. As SOCs improve their capabilities by turning inwards, where should they start? What techniques should they be worried about? What tools will help them? Without knowing what your adversaries can do and what your current capabilities are, it’s hard to make improvements. This talk will describe how to use the MITRE ATT&CK framework as a “scorecard” within the SOC to understand and tune defensive capabilities, making it easier to answer these hard questions. We’ll describe key use cases for how SOCs can use ATT&CK, covering hunting, threat intelligence, red teaming, and security engineering. To enable these use cases, we’ll present a non-invasive technique to construct a detective coverage map that highlights the SOC’s strengths and weaknesses, focusing on minimizing resource requirements while still providing usable results. To accompany this, we describe a process to create a remediation plan that provides the highest return on investment by orienting on the most relevant threats and prioritizing defensive improvements based on current coverage. Throughout the talk, we will provide real examples, making it easy for those in attendance to understand and replicate at home.
Slide rule Presentation of a group System on a chip Drop (liquid) Software framework
Point (geometry) Operations research Domain name Focus (optics) Multiplication sign 1 (number) Disk read-and-write head IP address Time domain Mathematics Software Hash function Internetworking Different (Kate Ryan album) Energy level Representation (politics) Procedural programming Perimeter Vulnerability (computing)
NP-hard Implementation Group action Link (knot theory) Online help Login Product (business) Data model Malware Software Utility software Endliche Modelltheorie Perimeter Self-organization Enterprise architecture Bit Multilateration Group action Software maintenance Exploit (computer security) Product (business) Software Enumerated type Phase transition Chain Right angle
Point (geometry) Slide rule Implementation Group action Enterprise architecture Differential (mechanical device) Real number Multiplication sign Cybersex Expandierender Graph Theory Formal language Number Latent heat Matrix (mathematics) Energy level Software framework Endliche Modelltheorie Information security Perimeter Descriptive statistics Task (computing) Vulnerability (computing) Cybersex Enterprise architecture Focus (optics) Key (cryptography) Information Bit Incidence algebra Software Figurate number
Point (geometry) Operations research Slide rule Key (cryptography) Texture mapping View (database) Analytic set Mathematical analysis Measurement Data model Personal digital assistant Configuration space Information security Information security
Cybersex Slide rule Presentation of a group Context awareness Computer file Cybersex Shared memory IP address Twitter Emulator Hash function Personal digital assistant Different (Kate Ryan album) Information security Thermal conductivity
Simulation Software Personal digital assistant Decision theory Weight Cybersex Online help Information security Login Perspective (visual) Information security
Scheduling (computing) Confidence interval Weight Bit Revision control Visualization (computer graphics) Software Case modding Personal digital assistant Single-precision floating-point format Matrix (mathematics) Diagram Quicksort Metric system Traffic reporting Task (computing)
Source code Group action Service (economics) Graph (mathematics) Information Source code Analytic set Plastikkarte Diallyl disulfide Formal language Vector potential Data model Data model Graphical user interface Software Repository (publishing) Matrix (mathematics) Software testing Process (computing) Endliche Modelltheorie Bounded variation System identification Information security Information security
Scale (map) Source code Data model Visualization (computer graphics) Integrated development environment Software Normed vector space Software testing Process (computing) Information security System identification Perimeter Hydraulic jump
Point (geometry) Web page Intel Scheduling (computing) Multiplication sign Source code 1 (number) Online help Electronic mailing list Mereology Perspective (visual) Expected value Different (Kate Ryan album) Operator (mathematics) Energy level Process (computing) Vulnerability (computing) Proof theory Key (cryptography) Software developer Mathematical analysis Electronic mailing list Analytic set Planning Bit Maxima and minima Approximation Process (computing) Software Normed vector space Phase transition Right angle Procedural programming Cycle (graph theory) Figurate number Resultant
Windows Registry Confidence interval Texture mapping Blind spot (vehicle) Data analysis Analytic set Lattice (order) Mathematics Military operation Operator (mathematics) Query language Configuration space Process (computing) Endliche Modelltheorie Information security Source code Mapping Boilerplate (text) Mathematical analysis Analytic set Data analysis Bit Electronic signature Category of being Word Process (computing) System on a chip Procedural programming Information security Resultant
Performance appraisal Sample (statistics) Integrated development environment Link (knot theory) Mapping Matrix (mathematics)
Windows Registry Point (geometry) Dataflow Computer file Confidence interval Code Multiplication sign System administrator Source code Analytic set Total S.A. Internet forum Energy level Local ring Vulnerability (computing) Sampling (statistics) Shared memory Analytic set Bit Sample (statistics) Integrated development environment Repository (publishing) Function (mathematics) Right angle Remote procedure call Communications protocol Window
Mapping Confidence interval 1 (number) Analytic set Spreadsheet Process (computing) Integrated development environment Software Different (Kate Ryan album) Matrix (mathematics) Software framework Perimeter Perimeter
Point (geometry) Revision control Windows Registry Game controller Confidence interval Personal digital assistant Multiplication sign Source code Analytic set Windows Registry Perimeter Resultant
Windows Registry Slide rule Software Key (cryptography) Computer file Cellular automaton Planning Bit Directory service Endliche Modelltheorie Navigation
Focus (optics) Group action Mapping 1 (number) Planning
Point (geometry) Slide rule Context awareness 1 (number) Vector potential Pressure volume diagram Formal verification Configuration space Perimeter Purchasing Mapping Planning Analytic set Bit Basis <Mathematik> Approximation Word Emulator Personal digital assistant System on a chip Software testing Formal verification Musical ensemble Information security Fingerprint Freezing
Slide rule Focus (optics) Game controller Pressure volume diagram Analytic set Planning Price index Control flow Perspective (visual) Position operator Spectrum (functional analysis) Laser
Web page Reading (process) Slide rule Variety (linguistics) Chemical equation Web page Analytic set Online help Analytic set Flow separation Perspective (visual) Perspective (visual) Personal digital assistant Negative number Matrix (mathematics) Software testing Damping Matrix (mathematics) Position operator
Emulator Emulator Mapping 1 (number) Software testing Perspective (visual)
Emulator Group action Emulator Mapping Green's function Data structure Abstraction Focus (optics) Formal language Perspective (visual)
Multiplication sign Planning 3 (number) Gauge theory Measurement Measurement Emulator Emulator Software Telecommunication Energy level Software testing Data structure
Implementation Process (computing) Validity (statistics) Mapping View (database) Matrix (mathematics) Bit Online help Quicksort Analytic set
Enterprise architecture Computer-generated imagery Android (robot) Online help Mereology Exploit (computer security) Emulator Internet forum Enumerated type Software framework Endliche Modelltheorie Window Domain name Form (programming)
Enterprise architecture Open source Link (knot theory) Length View (database) Cybersex Video game Planning Revision control Wiki Web 2.0 Emulator Spreadsheet Internet forum Different (Kate Ryan album) Matrix (mathematics) Automation Diagram Cybersex Enterprise architecture Vulnerability (computing) Knowledge base Mapping File format Building Projective plane Gradient Mathematical analysis Analytic set Database Process (computing) Emulator Visualization (computer graphics) Repository (publishing) Right angle Quicksort Automation Navigation Vacuum
Cybersex Emulator Email Link (knot theory) Process (computing) Emulator Repository (publishing) Cybersex Repository (publishing) Analytic set Self-organization Twitter
well uh like for you to welcome Andy Applebaum he's gonna give a presentation here and instruct us on how to stop drop and assess your sock can you guys hear me okay coming out okay louder okay I'm gonna try to talk louder um cool so my name is Andy I work at mitre are you guys familiar with mitre awesome I don't have to say what mitre is cuz I always bungle then anyway I work on miters attack framework are you guys also familiar with attack that's good to hear because I have slides on attack but I don't have too many so I'm glad anyway I'm gonna be talking about kind of a methodology that you can use to try to assess your sock using attack as a scorecard and to kind of lead off I'm
gonna you know traditionally when we talk about network defense we have this tendency to treat our network as a castle and focus on the perimeter and that's not as not as true today as it was a few years ago but there's certainly a mentality that says I need to focus on my network perimeter that's all that matters if I patch and remediate all these vulnerabilities no one's ever gonna get in and the reality is that's not true adversaries will find a way they'll always find a way to get in there's always gonna be something that's going wrong and at the end of the day if you're really only looking at those walls of your network you're gonna miss a lot of stuff I think most of you
might be familiar with the pyramid of pain are you guys familiar with the pyramid of pain that sounds like a yes some some head nods this is basically things that are kind of easy for adversaries to change or this is kind of a representation of things that are hard for an adversary to change versus things that are easy for us to detect so hash values are really easy to kind of like for us to look for but they're very easy for an adversary to change IP addresses are similarly easy they just kind of you know different infrastructure domain names network and host artifacts and at the top of the pyramid our adversary behaviors or attack tactics techniques and procedures these are the hardest things for adversaries to change when they tacker networks and if we can start looking for adversaries by looking for their TTP's we're gonna do better and actually finding them and so that's where attack comes in as we're moving away from IOC s we're moving away from hash values and IP addresses and instead we're texana maizing what attackers are actually doing at the behavioral level and here's kind of what what attack is I just want to point out it's globally accessible which means it's on the Internet attack top mitre org you can go there right now you can go there at any point in time it's totally free it's available we put it out there please use it use attack attack is amazing there's
a lot of hard questions that you want to ask when you're trying to implement defenses in your network the first is how do I actually move up the pyramid of pain and implement TTP based detection you also might want to ask how effective is my defense it's one thing to just throw tools into your network but it's another to say hey here are these tools in my network and here's how effective they are at detecting things you might also want to ask what's my detection coverage against say a PT 28 or 29 since you know we've all been reading the news you know you might want to you might wonder well here's this apt they're active they're targeting people how do my defenses stack up against the things that they're doing and then as your instrumenting the sensors on your network you might want to ask hey is this data that I'm collecting all these logs I'm forwarding into Splunk are these actually helping me are these useful what can i detect by using these things then the last thing you might want to do and this is you know not not a full enumeration but you might want to say hey is this new product from this vendor that's getting all this buzz everywhere is that actually gonna benefit my network if I go implement that is that going to provide some new capability that I wouldn't otherwise have and these are all questions that we hope to address kind of using attack or there's some way to address using attack and kind of help move you from that perimeter based model and to something that's more kind of holistic where you understand more about your network so I'm gonna talk a little
bit about attack I'll try to keep it brief since it seems like most of you have heard about attack but you know traditional traditional sock defense focuses on that left of exploit killed on the left of exploit kill chain phases reconnaissance weaponization deliver and mainly exploitation attack for enterprise and that's I'm gonna say mainly attack focuses on the right of exploit kill chain phases and that's really kind of breaking down that control execute and maintain into you know high-level adversary tactical goals these are things like initial access persistence privilege escalation lateral movement exfiltration mentioned credential access right that's a fun one I like that one a lot but these are kind of breaking down those kind of high-level you like here are these kill chain phases into hey here are these adversary tactical goals and then in the attack model itself instead of just a numerating these tactical goals we also talked about the techniques that are that are the actual things that behaviors the adversary executes to achieve their tactical goals and then along with the techniques we enumerate groups you know the threat actors with links to the techniques that that we've seen them using and publicly available threat reporting literature then lastly we include a little bit of software in there as well that includes built-in utilities because a lot of adversaries love living off the land as well as custom malware all linked to the techniques that they're able to execute and so this is just a quick snapshot of
the attack the attack for enterprise matrix it's grown a lot over the years and I'm gonna have different matrices throughout this talk I think mine are a little dated I don't have initial access in there but this is I think the one that was most current as of April to the 2018 and you can see at the top level we have the tactics these are these are each column is a tactic it one of the downsides of having this model expand over time is it's harder and harder to read on a slide you have the tactics and then within each tactic you have the techniques that the adversary is that the adversary uses to achieve those tactical goals and here's an example kind of in the top you can kind of see it there you have scheduled tasks and then that kind of gets blown up in the attack framework itself we have a description we have examples we have a little bit of information about the technique in the actual model as well as specific technique implementations linking out to the threat actor groups and the software that can execute them and all this is available at attack top Mitro org I'm not gonna go in too into it but there are four key points I like to make about attack the first is that the framework is grounded in real data from cyber incidents everything is backed by either like common red team knowledge or publicly available threat reporting information that's one of the key differentiators of attack is that we're not just like enumerate these things we've read about in papers we are you know kind of kind of like theoretical attacks we're really talking about the things that adversaries do execute and that we've seen them executing attack also allows you to or one of the key things that attack does is it enables you to pivot between your red team and your blue team and I'm hopefully going to talk a little bit about that later in the talk but it basically gives you a common language that both your red team in your blue team can speak to as they're working in your network then lastly or not lastly this is number three this is my favorite attack decouples the problem of understanding what adversaries are doing from the solution you know the defensive thing that you'd want to do so we've just gone and said hey here are all these adversary behaviors here's all these things that they're doing you can go figure out what kind of thing you want to do do you want to focus on detection do you want to focus on remediation you want to focus focus on mitigation attack is agnostic to that attack just talks about and really focuses on what the adversary is doing then lastly attack helps transform your thinking by focusing on post expletive assertive behavior and this goes back to the castle model we're no longer we saying here's my walls my perimeters no vulnerabilities I'm say if this thing no the adversary is gonna do some post exploit stuff and if you start looking for these things you're gonna increase your overall security so attack is great
but how do we actually use it and one of
the things I like to say is that attack kind of sits at this intersection of four key use cases you know threat intelligence measuring the defenses detection and hunting and security engineering trying to remember what I put on this slide let me dive into it
actually detection and hunting that's really kind of talking about sock teams you know kind of your detection team really you know focusing on that detection aspect you know hunting falls in here developing analytics tooling configurations as well as kind of how you know that the analysts looking for things and and that's one of the key use cases is really kind of focusing on that detection point of view okay I'll give an example and a further slide
pentesting or maybe more accurately red teaming is another big use case and that's measuring defenses really if you're using attack your red team can go in there and help say okay you know here's what my I think my defenses are in your red team say hey here's what your defenses actually we're able to to to detect and you know one of the things one of the use cases I like to highlight here is that with attack you can have your red team actually conduct engagements engagements to emulate known adversaries and that's because of all that publicly available threat reporting data that we built off of where we say hey here's this adversary here are these things that we've seen them do one of the nice things is that you know kind of attack helps each of these different use cases inform the other use cases here measuring defenses can actually help inform your detection and hunting by telling you hey you're missing these things when you're when you're running you're running your detection and when you're doing your hunting cyber threat
intelligence is a huge use case for attack I'm actually not gonna talk about it too much in this presentation but there's a lot of cool things you can do with CTI and attack one of them you know ingesting and share be sharing behaviors for situational awareness instead of sharing like hey here's this file hash or hey here's this IP address you might want to say hey here are these behaviors we've seen associated with this threat actor maybe you do like some you know file hashes fat file hashes and other things in there as well but really is focusing on sharing those behaviors so that you have better understanding what of what adversaries are actually doing also I have a nice slide about this is identifying and mapping the changing threat landscape it's kind of like tracking how adversaries are modifying their behaviors maybe two years ago we saw an adversary use these TTP's and maybe today we're seeing them used these TTP's we can start kind of keeping tabs on what the trends are and maybe even forecast what adversaries might do in the future and CTI is really great
because it helps inform kind of both your measuring defenses because you can tell your red you can have your CTI team tell your red teamers hey here's what we think are our threat actors are doing you know the guys that we should really be cared of caring about go emulate these threat actors don't go do random things go actually focus on the adversaries that we think are going to target our networks and then they can also inform your your your detection team as well kind of from a sim perspective of hey here's the adversaries that we're worried about are we secure against them or are we not then the last one is security
engineering you know kind of informing you know one of the big use cases for attack here is in forming strategic decisions to kind of prioritize your investments and you you really want to a better way to say that might be using attack to guide how you architect your network and what sensors and what tools you deploy what logs you collect attack can help kind of net help you navigate where you should be looking and what you should be doing there so this is
probably my one of my favorite things to talk about with attack this is a notional defensive Gantt chart essentially the idea here is that we can take attack and and use it as a matrix and basically diagram which techniques we have high confidence we're gonna detect medium confidence or low confidence it's very simplistic you can obviously use like quantitative methods to to say I think I'm going to detect credential dumping as a 20 and you know schedule tasks as a 30 and then starts assigning weights and all sorts of fun you know stuff and metrics but here it's really simple where we're just using kind of a color-coded chart to say hey here's what I think my defensive coverage looks like it was great about attack as you can visualize all these TTP's all these behaviors and one like single snapshot and I'm gonna talk more about this use case in a bit but kind of
done branching a little bit another nice visualization is for threat Intel this is a chart and I don't know if you guys can read it but in pink is all the things we had really that we had an attack that had mapped back to apt 28 he only had six techniques and I think this was in 2016 or so but an older version of attack after some threat reporting I think this was about a year ago we saw 14 new techniques that we'd seen in publicly available threat reporting and this goes back to the idea of tracking adversaries and mod and watching how they might be modifying their behaviors you know this is biased by publicly available threat reporting but you can take your own networks they do kind of the same thing where you're tracking the threats seeing what they're doing seeing how they might be true
hunting is another good one this could be hunting or you know kind of purple teaming really red and blue working together attack provides a common language and here's a simple example where we have this you know the matrix for you and we're saying okay new service and from new service as a red teamer I jump to credential dumping and then after credential dumping I jump to account discovery and kind of walking through what red team's and purple team and or what the red team is doing in a way that's accessible to the blue team because they're both speaking the same language in the last security
engineering is another fun fun one this is actually visualized by a tool that we have it that's again free publicly available it's called carrot attack card mitre dat org forward slash carrot this visualizes groups as they act the group's the techniques that those groups have been seen seen to execute analytics that we have in a data and are in a repository back to the techniques that those analytics can detect data sources that those analytics need to run and then sensors that map to those things in the data model and the idea is if I can kind of expand my sensor model start thinking commercially available tools other potential source information I can start drawing a graph like this and helping prioritize what I'm doing as I'm architecting my network and choosing what things to do so we have this matrix
we have this visualization it's amazing but you know a giant question mark what
do we actually do how do we actually bring this into our environment if I'm living somewhere where all I'm doing right now is kind of looking at the perimeter I just have all these defenses and how do I get started with attack one
of the things that I that I think would be great to do is to conduct an attack assessment and I'm just gonna jump
backwards actually and the idea is you
know we can talk a lot about attack but if I can come up with this chart for my network this can help inform everything else that I'm doing but how do we actually come up with this chart how do we figure out where our gaps are where are you know where our strengths are where we kind of in the middle where do we actually kind of jump from and and
this is where you know assessments come in and the idea is yeah we can do like
red teaming and that can deck do a lot of that for you but this idea is something a little bit softer it's more of an approximation of first glance just something that's supposed to be like that to give you that starting point so you can kind of prioritize everything else you're doing and the idea of the assessment is kind of a four phase approach at the highest level is just discussion you just kind of figure out what you're looking for What's in scope what's out of scope make sure that everybody in the cycle you know from the from the CISO down everybody is kind of on the same page they understand what's going on what the what what their expectations are you know we're not we're not going in there and executing the techniques we're just trying to approximate what our coverage might be once everything is set and you have kind of a schedule and you have kind of a good rhythm then you're gonna want to actually analyze stuff I'm looking or you'd look there for mainly documentation and sensors that you've implemented things like CONOPS certain operating procedures you know do you have a document which describes hey these are the data sources we're looking for this is how I stand up a new host on the network this is these are all the tools that I'm running here all the tools that my sock operators are running you might have a giant list of all these different things that that everybody's doing getting all that documentation bringing it together and you know you can do this from two perspectives if you're in the SAC you know it's kind of easy for you to know what you're doing but if you have many different people you really need to kind of bring everything together to figure out how to do this the next is to talk to people and actually interview them it's great to get documentation and you might say oh we're running all these tools we have all these analytics for gathering all these data sources but if you go talk to people you might hear that what's actually being done in the sock is a little bit different than what people are kind of claiming they're doing it in writing here kind of things you might want to look for or known coverage you might talk to people who say hey we struggle with this or hey we're really good at this you know you know maybe some people are more familiar with attacks some are less familiar with attack maybe they can talk about that or you might just have general things like you know we kind of struggle with detecting things kind of on our internal in our internal network traffic that gonna help kind of inform how you're analyzing everything is you understand what your strengths and your weaknesses are then another good one is past performance if you have kind of successes or failures you can go off of those and see what you did there and the last thing is kind of processing everything together yeah we have all this stuff hop you know now we need to bring it all into one you know one complete picture and you don't just want to kind of get that coverage chart but you also want to develop prioritization plans that tell you what to do after you've done that there's two key points I'd like to make here the first is that this process is designed to minimize stakeholder involvement you don't want to be over burdening the sock personnel with some kind of analysis where you're spending and you're just kind of alright sit down with me for three hours a day and I'm gonna tell you what you're doing because then no one's doing their job you know the idea here is really to focus on that analysis phase and when you're interviewing people really keep that to the minimum you want an hour or two hours you know per person that that's it you really don't want to be saying all right walk me through your day-to-day because once you start doing that it's just too much involvement at the same time you want to try to maximize your use of both results and that means you do have to talk to people you can't just analyze the documentation because that only tells you so much but you know really kind of maximising how you can use those results too and the other key thing and I'm gonna talk a little bit about this later is coming up with a prioritization plan is a huge deal because it's one thing to just say hey here are your gaps and then okay what do you do like I'm missing these things what do I actually do about it where should I or should I go from here you know you don't want to just do that but you also want to come up with okay now we need a plan for making things better and that's a big part of this as well a little bit more information on
analyzing documentation you're looking here for things like tooling processes procedures methods things like that kind of the big thing is to focus on how tools are used and what analytics are running that's what you'd really want to look for and you know how tools are used that's that's hard when you're just looking at documentation but if you can get that that's great here analytics provide empirical details on what user is not cool not not detected it's easy to look at an analytic that's you know one or two you know I don't know how many lines an average analytic might be but a few lines long and say okay I think this is gonna detect this technique or this technique that can really be pretty straightforward and then specific tools those tend to use detective methods that directly map to attack and I'll provide an example with like registry based detection methods that if you say hey you know this can detect things that modify the registry okay that I've got kind of medium confidence that you can detect these things and the goal here is really to understand how the sock operates before really visits a bad word but really you know before kind of overburdening people and interviewing them if you can focus your interviews you're gonna do much better than just going in they're kind of blanket so it alright let's get to know each other and I'm gonna ask you some boilerplate questions if you do if you do the analysis beforehand then you're gonna know exactly what you should be looking for when you're interviewing you're really meeting with the security teams to understand the general readiness and I like to bucket this into three main categories the first is empirical evidence that's like hey do you know of any existing attack gaps it's kind of hard to ask and if people know of all the gaps then your jobs gonna be really easy but you know sometimes people will know one or two or maybe you talk with them and say hey what do you think of WMI they say oh you know what I think we can catch that or I you know we ran a red team exercise we weren't able to catch that the next big bucket is general evidence and that's things that are just kind of general blind spots that might be you know kind of general things that are being missed and then lastly tooling and method details how does the team operate and how do you use it configure tools a lot of tools like are they being deployed off the shelf or are you customizing them a little bit that'll change how you end up evaluating how that tool is being used when you're processing results you
tend to have kind of four big buckets of things the first is interview results let's combine jet empirical in general evidence all mapped back to attack the next is data analytics these are analytics signatures you know things like that and then you can kind of map those back to the attack model tooling and sensors it's the same thing that's just another big bucket then sock procedures that help you kind of understand how the analytics and the tools are used and that's kind of vague
so I'm gonna walk through a couple of examples this isn't a real assessment this is just kind of some some you know snapshot pictures of the matrix that can kind of provide an idea of what you'd be looking
for this first chart that this is from a there's a link down here on CrowdStrike they have something called Falcon and they have kind of a mapping of hey here's what the attack coverage is against a specific apt to the apt three evaluation in particular and this is you know if you're assessing an environment that's running this tool is it really easy because that's already available for you this provides a nice little snapshot you see we've got you know things in green those are things that are detected things in yellow you know detection was possible but it wasn't really doing it red or things that were tested those are capability gaps and then some things also weren't tested just because they were kind of out of scope and then other things weren't tested just speaking for other reasons and so this is great if this is you know available if you have if you've already done an assessment of a tool that's another common common situation that you might have is you said hey I've seen this tool before I know what it's attack coverage looks like you know I don't need to do it again and so that's a great place to start from but more often
than not you might have say okay we're running these you know endpoint and point detection things like what what's what's their coverage looked like and here's an example I mentioned about registry based defenses you might say okay here's a tool it monitors a registry it just kind of just does stuff with the registry and you're not really sure how good its coverage might be but you might have medium confidence that you know any of these things that really kind of fiddle with the registry these are things that you probably shouldn't be focusing on if you're trying to remediate gifts these are probably things that yeah it can maybe detect so I probably shouldn't worry about them so if you can look at the data sources that some of the tools are running or some of the tools are looking at then you can figure out where you should be kind of prioritizing your you're not prioritizing but really where the coverage strengths and weaknesses of that tool might live I say analytics a lot here's a sample
analytic you know this is just kind of some pseudocode you can see it's you know kind of searching for they can't see my mouse it searching for flows it's you know looking for a destination port 445 and the protocol is an SMB right protocol you know you might come into and you might look at your environment to see if you know a bunch of analytics and you get code like this you might say okay this looks like it's looking for an SMB right request you think a little bit more about it and say okay has to be write write request this can detect remote file copy pretty well you know that that'll detect it most of the time windows admin shares it's got a moderate level of comfort you know covers there it's medium and then valid accounts that's another one that okay that that'll provide some coverage of that technique and and so if you can do this for all your analytics that can help you understand what your coverages as a note I've cheated here that analytic is actually from miters cyber analytic repository we have kind of a repository of analytics it's available at Carnap might order to work so please feel free to go there and look at some analytics so if you take all the analytics that
you find in an environment you throw them all into one spreadsheet or what one matrix for you you get kind of a picture of what the overall analytic coverage is and here I've taken five different different analytics and kind of created a coverage map for those five and saying okay these are all the things that those analytics can detect more often than not when you analyze all the analytics that are running some of them or many of them might hopefully map to the attack framework but others might not do well and that's really gonna be kind of a creative process trying to figure out which of those map to the matrix and which ones don't this is a very simple example of what
you might expect when you're interviewing personnel and I've kind of cheated and done something very basic you'd probably get more interesting things when you talk to real people but in this example I'd say you know we interview them and they said we kind of have mediocre success we have kind of medium confidence that anything going across the perimeter we have decent coverage you know we have you know high confidence but maybe medium confidence and so here said okay anything that's going across the network you know anything that's really going at the perimeter or might go across the perimeter I'm just gonna say that's medium confidence of detection it's very simple and straightforward you can do more interesting things like you might talk to people and they say you know we we struggle to detect discovery and you know you just kind of highlight then say okay that's red that's something they said they struggle with but here it's a kind of you know perimeter based we're okay with it and so how do you know I've kind of
given you these data sources how do we bring it all together and here's kind of
you know we start with one of the tools we add another tool here I've taken that that Falcon I've added the registry all the things that they both can detect and they both miss and kind of bring them together so same with analytics you know you kind of can see the coverage expands as you bring each of these in and at some point you know you might have to resolve conflicts and you know essentially here I'm just building everything's nice and you know increasing all the time so if I do have coverage I just add it sometimes you might find something saying I do have coverage but then something else saying I don't have coverage in some cases you might want to prioritize not having that coverage now when you bring in the interview results here's what the end thing looks like and and one thing I highlight here is that the the coverage before this line was nope no confidence you know really really low confidence of exfiltration over command and control and that was from that tool that said if we don't have any confidence here and nothing else said they had confidence but then when I said hey you know the interview when we interviewed them they have medium confidence that they can detect all these things across the perimeter that I might say okay that's something that you know maybe we have medium confidence as opposed to no confidence that's a pretty simple example so it's not enough to just say
hey here's all your gaps you know what do you actually do how do you go about fixing your network and essentially you know my answer to that is you need a prioritization plan you need to focus on remediating specific gaps and as a simple example you know the question is if I have all these things in white all these things that have low con since which one should I actually focus on and one idea is to focus on those
that are more commonly used this is a notional chart it's very old but kind of things that are you know highlighted more those are more commonly seen this is a little notional but you know credentialed up bank file and directory discovery registry of run keys is live start folder these are all techniques that are pretty commonly seen we also just if anyone's interested in this slide we have something called the attack navigator that's free and publicly available that takes in layer files and we have a layer file that has more that has better data so talk to me afterwards and I can tell you a little bit more about that but that's one thing you can do is try to find which techniques are more commonly used obviously this is biased by whatever data is currently in the attack model
another thing is focusing on specific groups here I've taken kind of a be t28 deep Panda but you know apt 28 in blue deep panda and yellow and then techniques that both of them execute in green here I'm saying I want to focus on both of these friend actors and if I'm coming up with a prioritization plan I'm gonna focus on the techniques that both of them execute as opposed to techniques that neither of them are just one of them execute obviously you can also take another way which might say I'm gonna focus on the techniques that ap t-28 execute as opposed to the techniques that deep and execute and when you're
done hopefully you'll have a prioritized coverage map here's this again you know notional but where you highlight a technique here and there and say these are the ones that we really need to go for right away I want to focus here and you know what I these are the things then I'm gonna get the biggest bang for my buck and and that's a great place to kind of be
[Music] once the assessment is done you come up with a remediation plan I've got a lot of words here but the main thing that you want to do after that is really implementing an attack mindset you really want to move away from that perimeter you know pre exploit you know that no one's ever gonna get in moving away from that and saying okay people might get in and we really should focus on having this threat based awareness and a threat based methodology and our sock skip some of these points but some of the things you might want to do are improving coverage by acting on the coverage map it's pretty straightforward that's just coming freezing your coverage having increased awareness of your defensive gaps is really good if you have that as a day-to-day basis you kind of have more awareness of the kinds of things you should be looking for on a day to day basis that verification is important because this is you know what I've talked about it's a bit of an approximation you know going in there with a red team and verifying this is certainly helpful and so I'm going to
talk a little bit about some use cases that you can do after an assessment the first is developing analytics and I don't know if any of you were at besides because my colleagues presented some of these slides already so it might be duplicative I said the word analytics a
lot and you know when we talk about analytics they're they're great but you know there's kind of a spectrum between in analytics and indicators and I don't talk to this slide too well but analytics tend to really focus on that behavioral based perspective as opposed to that known malicious that indicators talk about they're you know there's more false positives they're broader and you tend to have kind of a lower quantity than you have for indicators we're still really useful but you know you really kind of have to target them when you're developing them and the general
recommendation for if you want to go with an attack an attack assessment to figure out what your coverage is and then start remediating gaps you should just start somewhere and here you know you know well start somewhere pick a technique and you know ideally you go from the remediation plan that you have focus on one of those techniques here's bypassing user account control and I
think this is an old slide deck and more there anyway when you're developing an analytic the first thing you should do is really read the attack page and understand the attack when you're rather when you're developing an analytic you should try to target a technique and when you do that you should really understand the attack that you're trying to target you want to look at the references for using using it thank for my adversary perspective and try to separate legitimate usage from malicious usage and that's gonna be a big one because a lot of the things that we have an attack are things that can also be done legitimately trying it out is also very important you don't just want to throw in a little what you should implement them and then refine them and say okay here's the false positives here's the false negatives trying to kind of find a nice balance between the two and then writing and iterating is also important you know maybe you write your first search and then you narrow your false positives in your iterate and
you know kind of a big hope here is that you start with your initial coverage matrix and this is great but after developing some analytics you can kind of help increase your coverage matrix instead of going in and doing another assessment you know from scratch you can take your initial assessment understand what your analytics are detecting and then update your coverage sharp based on what those analytics were actually detecting another use case is adversary
emulation you know that's kind of red
teaming ish but I I love the coverage maps in the heat maps I think they're great they're awesome places to start but they tell us somewhat incomplete picture the reality is if I say this technique is green I can't just like walk away you know clap my hands and say no one's ever gonna use this technique you really need to go in there and test hey this technique really isn't vulnerable I I can't detect this and I you know I I can't detect this you're not gonna get away with executing it and
you know attack techniques they really have many different ways of being executed and this is different for each technique you know so some of them like financial dumping there's lots of ways you can do credential dumping but reading you know the bash history that's well you're you're reading the bash history this there's different ways you can read the bash history but truly reading the bash history so coverage maps think they paint a great picture initially but they're kind of being complete and the best way to really move beyond a coverage map is to
use adversary emulation which you also might want to say is like threat based red teaming here you want to actually go and execute real techniques on your network go verify whatever coverage you have say okay I think this is green okay Red Team go emulate an adversary then execute that technique and prove to me yeah it's green you're not you're not gonna get away with executing that technique an attack is great here because again it provides this common language to not only talk with the red team but to also you know well too it provides a common language to talk with the red team but also provides that structure for how the red team should behave because we have that mapping back to the groups so using a tank for
adversary emulation there's four you know big things here first is scope attack can help you understand the scope of what the red team exercise can look like you might not just want to execute everything you might want to execute only specific things that the adversary is executing communication I've mentioned that in a few times repetition is also important if you're running a red team exercise and the red team just does whatever they want for each exercise it's gonna be hard to understand how your network has changed over time and how your red team might be changing by using attack and structured adversary emulation you can help create exercises that are more repeatable and so now you say okay here's here's what this exercise look like you know a month ago two months ago three months ago and kind of compare how you've been doing over time the last thing is measurement kind of - you can kind of see okay I caught ten techniques and missed five techniques the ten techniques I cause those were low hanging fruit those other five I really need to focus on those I'm
not gonna dive into the details too much but I will say we developed you know adversary emulation plans right now we published an emulation plan for apt three this kind of walks through how you can do kind of more how you can emulate a beat III this works at the technique level also the procedural level we have a few things in there as well there's a lot of cool stuff there and you know the big thing here is if you do want to use adversary emulation it's great to either use an existing emulation plan or come up with your own to actually tell your red teamers hey these are the things you should be doing and help provide structure to the actual exercise
once you run an exercise you'll probably get something that looks a little bit like this this is yet another view of matrix but you'll get some sort of coverage map which says we caught these things we missed these things we kind of you know maybe could have caught these things better it'll help you map back to your original coverage to figure out hey you know I'm actually gonna validate this gap or this dis strength and once
you're done once you've done that initial assessment you don't just stop there you keep doing it use a different implementation of the same attack technique and I mentioned this again someone a lot of these have many different ways that you can execute and then update your your your analytic you know if you run the technique and you catch it that's great and if you run the technique again with a different implementation and you miss it now you should update what your coverage looks like by repeating this process you can slowly improve what your coverage looks like I'm just about
closing I didn't talk about it too much but CTI is a huge thing for attack part of what we're hoping for is moving towards more or more threatened form defenses you know you takin CGI you describe things as an attack you put that out to your realistic threat model you also push it down to your intelligent you know adversary or intelligence driven adversary emulation plans to kind of help you structure those in a more kind of realistic way based on your CTI and all that feeds into an ever improving and well validated defense I've talked a
lot about attack and I keep saying attack and there's a lot more to attack than what I've been talking about I've mainly been talking about enterprise attack that talks about Windows Linux and Mac we also have mobile attack that's another framework that's again available attack top moderate org as well as pre attack which does that same kind of tactic and technique enumeration for the left of exploit behaviors this
is just a quick view you can see you know there's pre attack on the left there's many more tactics and then attack up for enterprise on the right
we have lots of resources around attack first is kind of that publicly available attack knowledge base its attack top moderate org we've also recently converted everything in attack into sixth format so now you can work with attack dynamically someone likes that because I don't know about anyone else but I actually had a web scraper that was like scraping the wiki which that wasn't fun but now it's all in stick so you can automatically write scripts and do all sorts of fun analysis do all sorts of fun cool things I mentioned the adversary emulation plans it's another thing we have we're working on some more automated adversary emulation we have a project called caldera which attempts to kind of automate the adversary emulation process you know end to end anniversary assessments that's we have a open-source version available online that's and I can talk at length about that to calderas awesome it kind of attack is kind of the thing is that it executes I want to get started I mentioned car as well the cyber analytic repository that's kind of a database we have a repository of analytics that I'll map back to attack techniques the last one is the attack navigator visualization tool you know if anyone else has tried to visualize stuff with attack it can be challenging with Excel and PowerPoint and you know matrices and other diagrams we now have a tool it's available online again it's open source that allows you to visualize all sorts of cool stuff you can do heat maps you can do kind of we need to heat maps in all sorts of different ways gradients scores hiding techniques showing different techniques emphasizing we recently added a feature where you can export the the layer that you're working on actually has an Excel spreadsheet if you do want to work it into a PowerPoint so that's that's all cool and then links in contacts there's
tons of stuff I'm not gonna go through each I'm Andy there's my email I have a Twitter account Andy plays e4 although I play the Queen's Gambit now attack lots of things at miter attack we're very active on Twitter cyber analytic repository the emulation plans caldera I didn't talk about it we have something else called cascade which kind of automates a you know threat hunting process and it builds from the cyber analytic repository that one's also open-source so please go take a look at that then a lot of stuff on CTI at the end and then the last thing
is I just want to say is mitre is awesome we're a not-for-profit organization we are hiring so people think attack is cool anyway that's it if anybody has any questions I'd be happy to take them also have stickers [Applause] [Applause]