entrepreneur and Investor with a focus on cyber security. He has Started up at gotten bought and repeated this a few times And now he is an advisor who advises people on starting up companies getting bought and repeating that he is also director at CrowdStrike and

an associate at MIT Media Lab Just checking the time to make sure that we start on time and this start talking now On the scale of cyber security, please give a warm welcome to Vincento

So hi everyone. Thanks for being here as Karen said I I've made a few changes to my career, but my background is originally technical and What I want to do today is to talk about a trend that I think we sort of take for granted

and it's to some extent obvious, but Just also underappreciated and that is Cloud scale in security specifically when I say cloud scale what I mean is the ability to process very large amounts of data as well as

spawn computing power with with He's and now that has played a role in our industry in the past decade or so But before I talk about that, I think some context is important So I joined the industry about 15 years ago and back in the days even even a place like

the Congress was a much smaller place it was some extent cozier and The community was tied need the industry was fairly niche and then something happened around 2010 People realized that there were more and more state sponsor attacks being carried out

from operation Aurora against Google to The Mandiant report apt-1 that was the first report to document how the Chinese PLA was hacking West let's call it the Western world infrastructure for IP test and

that changed a lot for for the industry there there have been two significant changes because of All of this attention. The first one is notoriety. We went from Being as I said a relatively unknown industry to

Something that everyone's talked about if you if you open any kind of newspaper There's almost always an article on cyber security boardrooms talk about cyber security and in a sense again back when I joined Cybersecurity wasn't a thing. It used to be called infosec and now very few people know what infosec even means

So notoriety is one thing but not notoriety is not the only thing that changed The other thing that changed is the amount of money deployed in the sector. So Back in 2004 depending on the estimate you you trust there the total spending for cyber security was between 3.5 to 10 billion dollars today is over 120 billion dollars. And so it kind of looks exponential

but the the spending came with a almost Like a very significant change in the type of players that are in the industry today So a lot of the traditional vendors that used to sell security software have kind of disappeared and what you have today are

Two kinds of player largely you have the big tech vendors, so you have Companies like Google, Amazon, Apple and so on and so forth that have sort of decided to take security more seriously Some of them are trying to monetize security others are trying to use it as a sort of like slogan to sell more phones

The other group of people or entities are large cloud-based security vendors And what both groups have in common is that they're they're using more and more sort of like cloud scale and cloud resources to try to tackle security problems and so

What I want to discuss today is from a Somewhat technical perspective our scale has made a significant impact In the way we approach problems, but also in the kind of people that we have in the industry today so what I'm going to do is to give you a few examples of

the change that we've gone through and One of the I think one of the important things to keep in mind is that what what scale has done at least in the past decade is It has given defense a significant edge over offense It's not necessarily here to stay, but I think it's an important trend that it's somewhat

overlooked So let me start with endpoint endpoint security so back in the 80s a few people started to toy with this idea of IDS systems and the idea behind an IDS system is is pretty straightforward You want to create a baseline benign behavior for a machine, and then if that machine starts to exhibit

Anomalous behavior you would flag that as potentially malicious This was the first paper published on on host-based IDS systems now the problem with Host-based IDS systems is that they never actually quite made it as as a commercial product

And the reason for this there were largely two reasons for this The first one is that it was really hard to interpret results, so it was really hard to figure out Hey, here's an anomaly and this is why this anomaly might actually be a security incident the second problem was it was

You had a lot of false positives and was kind of hard to establish a benign Baseline on a single machine because you had a lot of variants on our individual machines would behave So what happened is that commercially we kind of got stuck with antivirus antivirus vendors and

signatures for a very long time Now fast forward to 2013 As I mentioned APT one the APT one report came out Company and AV companies Actually admitted that they weren't that useful at detecting stuff like stacks net or flame

And so there was kind of like a new kid on the block on the block and The the buds were named for it was EDR so endpoint detection and response, but when you when you strip EDR from like the marketing fluff what EDR really is is effectively host-based intrusion detection system at scale so in other words

Scale and ability to have cloud scale has made IDS systems possible in in two ways the first one is that Because you actually now have this sort of like data lake with a number of machines you have much larger data sets to Train and test detections on what that means is is it much easier to establish the benign benign baseline?

It's much easier to create proper detections that don't detect just malware, but also Sort of like malware less attacks the other thing is that EDR vendors and also companies that have internal EDR systems have

To a large extent economy of scale and what that means is you can actually have team of analysts that can create Explanation and sort of an ontology to explain why a given a given Detection might actually represent a security incident on top of it because you have this data lake you are now able to Mine that thread data to figure out new attack patterns that you weren't aware of in the past

So this in itself is a pretty significant achievement because we finally managed to move away from Signatures to something that works much better and is able to detect a broader range of attacks, but the other thing that EDR system solved

Sort of like as a side effect is the data sharing problem So if you've been around the industry for a long time there have been Many attempts at sharing thread data across across different entities And they all kind of failed because it was really hard to establish sort of like a protocol to share this data

but implicitly what EDR has done is to force people to share and and collect threat threat intelligence data and threat and just in general data from endpoints and so Now you have the vendors being the sort of implicitly trusted third-party that can use that data to

Write detections that can be can be applied to all the systems not just an individual company or an individual machine and the result of that the implication of that is that The meme that the attacker only needs to get it right once and the defender needs to get it right all the time

It's actually not that true anymore because in the past you were in a situation where if you had An offensive infrastructure where it was servers where it was exploit chains You could more often than not reuse them over and over again Even if you had malware all you had to do was to slightly mutate

The sample and you would pass any kind of detection But today that is not true anymore in most cases if you get detected on one machine All of the sudden all your all your offensive infrastructure has to be scrapped and you need to start start from scratch This so this is the first example and I think in itself is quite significant

the second example that I want to to talk about is fuzzing and Fuzzing is interesting also for another reason which is it gives us a glimpse into what I think the future might look like So as you're probably familiar if you're if you've done any upset work in the past Fuzzing has been sort of like a staple in the upset

Arsenal for a very long time But in the past probably five years or so Fuzzing us going through some kind of Renaissance in the sense that Two things have changed two things have improved massively The first one is that we finally managed to find a better way to assess

The fitness of the fitness function that we used to guide fuzzing So a few years ago somebody called Michael Zaleski released a Fuzzer called AFL and One of the primary intuitions behind AFL was that you could actually instead of using code coverage to drive

The Fuzzer you could use pad coverage to drive the Fuzzer and that Turn fuzzing in a way more in a much more effective Instrument to find bugs, but the second intuition that I think is even more important And that changed fuzzing significantly is the fact that as far as fuzzing is concerned

Speed is more important than smarts, you know in a way And what I mean by this is that When you look at AFL AFL as an example is an extremely dumb fuzzer It does stuff like byte flipping bit flipping it has very very simple strategies for permutation

But what AFL does very well is is an extremely optimized piece of C code And it scales very well. And so you are in a situation where if you have a reasonably Good server where you can run AFL you can synthesize a very complex file formats in very few iterations

And what I find amazing is that this intuition doesn't apply just to file formats This intuition applies to much more complicated state machines So the other example that I want to talk about as far as fuzzing goes is cluster fuzz Cluster fuzz is a fuzzing harness used by the Chrome the Chrome team to find bugs in

Chrome and cluster fuzz has been around for about six years in the span of six years cluster fuzz found 16,000 bugs in Chrome alone plus another 11,000 bugs in a bunch of open source projects If you compare cluster fuzz with the second most successful fuzzer out there for JavaScript engines

You'll find that this the second the second fuzzer called JS fun fuds found about 6,000 bugs in this in the span of eight to nine years and If you look at the code the main difference between the two is not the mutation engine The mutation engine is actually pretty similar. They don't cluster fuds doesn't do anything particularly fancy

But what cluster fuds does very well is it scales Massively so cluster fuds today runs on about 25,000 cores and So with fuzzing we are now at a stage where the bug churn is so high that

Defense again as as an advantage compared to a fence because it becomes much quicker to fix bugs than it becomes to fix exploit chains Which would have been unthinkable just a few years ago The last example that I want to bring up is slightly different one So a few months ago the tag team at Google

found in the wild a server that was used there was used for a watering oil attack and It was it was thought that this server was was used against Chinese Muslim dissidents but what's interesting is that the way you would detect this kind of attack in the past was that you would have a

compromised device and You would sort of like work backwards from there you would try to figure out how the device got compromised what's interesting is that the way they found the server was effectively to mine their local copy of the Internet and so again, this is another example of

Scale that gives a significant advantage to defense versus versus offense So in all of these examples that I brought up I think when you when you look deeper into them you realize that It's not that the state of security has improved because we got we necessarily got better at security

Is that it has improved because we got better at handling large amounts of data Storing large amounts of data and spawning computing power and resources quickly when needed So so if that is true Then one one of the other thing to realize is that in many of these cases when you look back at the examples that are

brought up It actually is the case that the problem at scale Looks very different from the problem at a much smaller scale and the solution as a result is very different So I'm gonna use a silly example to try to drive the point home. Let's say that

Your job is to audit this function And so you need to find bugs in this function in in case you're not familiar with C code. The problem here is that You can you can overflow or underflow that that buffer Are your are your pleasure just by? passing a random value for pause

Now if you were to manually audit this thing or if your your job was to audit this function Well, you could use you would have a many tools you could use you could do manual code auditing You could use a symbolic execution engine. You could use a fuzzer. You could use static analysis and a

lot of the solutions that are optimal for this case end up being completely useless if now your task becomes to audit this function and This is because the state machine that this function Implements is so complex that a lot of those tools don't scale to get here

And now for a lot of the problems I've talked about We've kind of faced the same situation where the solution at scale and the problem at scale looks very different. And so One thing one realization is that engineering skills today are actually more important than security skills in many ways

So when you look when you think back at fuzzes like cluster files or AFL or Again, EDR tools what matters there is not really any kind of security expertise What matters there is the ability to design systems that scales that scale arbitrarily well

In sort of like their back end to decide to write code. There is very performant and none of this has really much to do with traditional security skills the other thing you realize is when you combine these two things is that a lot of What we consider research

Is happening in a different in a different world to to some extent So six years ago about six years ago I gave a talk at a conference called CCS and it's an academic conference. And basically what I my message there was that if Academia wanted to do research that was relevant to the industry

They had to talk to the industry more and I think we're now reached the point where This is true for industry in the sense that if we want to still produce significant research at places like CCC We are kind of in a bad spot because a lot of the innovation that is practical in the real world is happening in very large

in very large environments that few of us have access to And I'm gonna talk a bit more about this in a second but before I do there is a question that I think is important to to digress on a bit and This is the question of is

Are we change are we if we change significantly as an industry? Are we in sort of like a new age of the industry and I think that if you were to split The industry in phases we left the kind of like artisanal phase the phase where what mattered the most

Was security knowledge and we're now in a phase where we have this large-scale expert systems that require significant Significantly more engineering skills that they require security skills But they still take input from kind of like security practitioners And I think there is a question of is This it or is this the kind of like where the industry is gonna stay or is there more to come? I

Know better than to make predictions in in security because Most of the times they tend to be wrong But I want to draw a parallel and that parallels is with another industry and it's machine learning So somebody called Rich Sutton who's one of the godfathers of machine learning brought an essay called the bitter truth

and in that essay he Reflects on many decades of machine learning work and what he says in the essay is that? People tried for a very long time to embed knowledge in machine learning systems

The rationale was that if you could embed knowledge you would have a smart you could build smarter systems But it turns out that what actually worked was where things that scale arbitrarily well with more computational power and more storage capabilities and so what he realized was that what actually worked for

Machine learning was search and learning and when you look at stuff like AlphaGo Today AlphaGo works not really because it has a lot of gold knowledge it works because It has a lot of computing power it has the ability to try to train itself

Faster and faster and so there is a question of how much of this can potentially port to To security obviously security is a bit different as more adversarial in nature So it's not quite the same thing, but I think we're we are We have only scratched the surface of what can be done as far as

Reaching a newer level of automation where security knowledge will matter less and less so I want to go back to the AFL example that I brought earlier Because one way to think about AFL is to think about it as a reinforcement learning father and what I mean by this is In in this slide what AFL was capable to do was to take one single

JPEG file and in the span of about 1200 iteration that were completely random dumb mutation go to another well-formed JPEG file and when you think about it

This is an amazing achievement because there is no knowledge of the file format in AFL And so we we are in we are now more and more building systems that do not require any kind of expert knowledge As far as security is concerned the other example that I want to talk about is the cyber Grand Challenge So DARPA a few years ago started this competition called cyber Grand Challenge

And the idea behind cyber Grand Challenge was to try to answer the question of can you? Automatically do exploit generation. Can you automatically do patch generation? And obviously they did it on some ball toy environments But if you talk today to anybody who does automatic expert generation research

They'll tell you that we are probably five years away from being able to since it to automatically seen to tie synthesize non-trivial exploits Which is which is an amazing achievement because if you asked anybody five years ago Most people myself included would tell you that that time would not come anytime soon

The third example that I want to bring up is something called Amazon Macy Which is a new service service released by Amazon and what it does is basically uses machine learning to Try to automatically identify PAI information and intellectual property in the data you store with AWS and then try to give you a better

Sense of what happens to that data So in all of these cases when you think about them again It's a scenario where there is very little security expertise needed. What matters more is engineering skills so

everything I've said so far is Reasonably positive for scale is it's a positive scale is a positive sort of a case for scale But I think that there is another side of scale that it's worth touching on and I think especially to this audience is is important to think about and

the other side of scale is that Scale breeds centralization and so to the point I was making earlier about where Where is research happening? Where is real-world applicable research happening and that happens

increasingly in places like Amazon or Google or large security vendors or some intelligence agencies and so what that means is The field the barriers to entry to to the field are are significantly higher So I said earlier that I joined the joined the industry about 15 years ago back then

I was still in high school and one of the things that was cool about the industry for me was that as long as you had a reasonably decent internet connection and a laptop you could contribute to the top of the industry you could see what everyone was up to You could Do research that was relevant to what the to what the industry was working on

But today the same sort of like 15 16 year old kid in high school would have a much harder time contributing to the industry and so we are in a situation where but because scale breeds centralization we are in a situation where we will likely

Increase the barrier of entry to a point where if you want to contribute meaningfully to security You will have to go through a very standardized path where you probably do Computer science and then you go work for a big tech company And that's not necessarily a positive

So I think the same Kranzburg principle applies to scale in a sense where it has done a lot of positive things for the sector But it also comes with with some Consequences and if if there is one takeaway from from this talk that I would like I would like

you to To have is to think about how much something that it's pretty mundane that we take for granted in In our day-to-day Has changed the industry and how much that will probably contribute to the next phase of the industry not just from a technical standpoint It's not just that the solutions we use today are much different from what we used to use

But also from the kind of people that are part of the industry and the community That's all I had. Thank you for listening

Thank you very much. We have time for questions So if you have any questions for Vincenzo Please line up behind the microphones that are marked with numbers and I will give you a signal if you can ask a question We also have our wonderful signal angels that have been keeping an eye on the internet to see if there are any questions from either

Twitter Mastodon or our IRC are there any questions from the internet? We'll just have to Mike for Microphone number nine to be turned on and then we'll have a question from the internet for Vincenzo And please don't be shy line up behind the microphones to ask any questions now

It's on but actually there are no questions from the internet right now There must be people in the room that have some questions I cannot see anybody lining up I mean chance. Do you have any advice for people that want to work on cybersecurity on scale? I mean, I just had to think a lot of the

Interesting research is happening more and more like tech companies and similar And so as much as it pains me it's probably the advice is to think Either whether you can find other ways to get access to large amounts of data or and and computational power or maybe Consider us into one of those places

We now actually have questions at microphone number one, can you hear me? Yeah. Thank you for the great talk You're making a very strong case that information at scale has benefited security, but is there also statistical evidence for that?

So I think well It's it's a bit hard to answer the question because a lot of the people that would they have an incentive To answer that question are also kind of biased But I think when you look Metrics like well time in terms of how much time people spend on attackers machine That has decreased significantly. Like it's it has statistically decreased significantly

as far as The other examples I brought up like fuzzing and similar. I don't think I as far as I'm aware there hasn't been any sort of like rigorous Study Around where now we are we've reached the place where

Defense has kind of like an edge against offense, but I think if I talk to anybody who has kind of like some offensive security knowledge or They work in in offense

The overall feedback that I hear is that it's becoming much harder to Keep bug chains alive for a very long time. And this is in large part not really for for countermeasures It's in large part because bug bugs keep churning So I there is there isn't a lot of statistical evidence, but from what I can gather

It seems to be the case We have one more question from microphone number one So thank you for the interesting talk My question goes in the direction of the Centralization that you mentioned that the large like the hyperscalers are converging to be the hotspots for security research

So is there any guidance you can give for us as a community how to retain access to the field and contribute? yeah, so So I think it's an interesting situation because more and more there are open source tools that allow you to gather the data But the problem with with these get data gathering exercises is not too much how to gather the data

The problem is what to gather and how to keep it Because when you look at the cloud bill for for most for most players It's it's extraordinarily high and I don't unfortunately, I don't have an easy solution today I mean you can you can use pretty cheap cloud providers, but it's it's still

Like the the expenditure is still an order of magnitude higher than it used to be and I don't know Maybe maybe academia can step up. I'm not I'm not sure We have one quest question from the internet and you can stay at the microphone if you have another question for Vincenzo

Yes, the internet asked that you ask a lot about fuzzing at scale, but besides OSS fuss Are you aware of any other scaled large fuzzing infrastructure? That is publicly available no, but when you look I mean when you when you look for instance of the participants for

cyber grand challenge a lot of them were effectively using significant amount of CPU Power for fuzzing So so I'm not aware of any kind of like plug-and-play Fuzzing infrastructure that you can use aside from OSS fuzz But

There is a law Like as far as I'm aware everyone there that does fuzzing for for a living is now has now access to significant resources and tries to scale fuzzing infrastructure If we don't have any more questions, this is your last chance to run to a microphone or write a question on the internet

Then I think we should give a big round of applause to Vincenzo. Thank you