DDoS attack detection with open source FastNetMon Community
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
Title |
| |
Title of Series | ||
Number of Parts | 542 | |
Author | ||
License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
Identifiers | 10.5446/61589 (DOI) | |
Publisher | ||
Release Date | ||
Language |
Content Metadata
Subject Area | ||
Genre | ||
Abstract |
|
00:00
Web pageView (database)Task (computing)Game theoryComputer virusPresentation of a groupDifferent (Kate Ryan album)Data centerProduct (business)Router (computing)Multiplication signAngleConfiguration spaceCASE <Informatik>System callAverageRevision controlDistribution (mathematics)WebsiteCross-platformDenial-of-service attackLaptopExpected valueSoftware engineeringDomain nameAsynchronous Transfer ModeSoftwareProgramming languageComputer configurationCommunications protocolSoftware maintenanceLevel (video gaming)Server (computing)Integrated development environmentDescriptive statisticsTerm (mathematics)Compilation albumType theoryPoint (geometry)Endliche ModelltheorieService (economics)Software developerInformation securityCybersexArithmetic meanSystem administratorProcess (computing)MathematicsContent delivery networkLatent heatOpen sourceInternetworkingCartesian coordinate systemCharge carrierAxiom of choiceReal numberPoint cloudOnline helpGroup actionMereologyFormal grammarComputer animation
08:12
Execution unitEmailSoftwareTelecommunicationLogicMultiplicationLatent heatData centerLevel (video gaming)CASE <Informatik>Denial-of-service attackRouter (computing)PhysicalismService (economics)Communications protocolGroup actionDifferent (Kate Ryan album)Game theoryStructural loadComplete metric spaceOpen sourceChannel capacityProcess (computing)2 (number)Multiplication signNormal (geometry)Total S.A.10 (number)Computer configurationGraph (mathematics)1 (number)Right anglePerspective (visual)Interface (computing)Band matrixInformationVector potentialInternet service providerPoint (geometry)BuildingSingle-precision floating-point formatBit rateCausalityBitComputer animation
16:17
Service (economics)Latent heatSoftwareInternetworkingArithmetic meanAbstractionSystem callBlock (periodic table)Denial-of-service attackService (economics)Server (computing)Point (geometry)Single-precision floating-point formatInternet service providerCharge carrierBitDiagramTraffic reportingWebsiteComputer animation
19:31
Execution unitService (economics)Group actionIntegrated development environmentSoftwareOpen sourceComputer animation
20:15
Group actionInternet forumScripting languageProjective planeComplex (psychology)DemonCommunications protocolInformationType theoryDirect numerical simulationGroup actionDirection (geometry)Open sourceSoftwareDataflowEmailNP-hard2 (number)StapeldateiDecision theoryElectric generatorShared memoryMereologyINTEGRALQuicksortSingle-precision floating-point formatTask (computing)PlanningOcean currentOrder (biology)Router (computing)Computer animation
24:27
Program flowchart
Transcript: English(auto-generated)
00:06
Hello. Thank you for coming. I'm very happy to see lots of people here. I hope you will enjoy my presentation. So, I'm Pawel, and I will talk about DDoS detection using open source tool.
00:21
So, first of all, why am I talking here? I'm software engineer. I got formal education as software engineer, and since the beginning of my career, I started working on open source. And my first programming language was quite unusual, I would say. It was Perl.
00:41
So, not a fortunate choice for so many people, but for me it was a way into industry. So, I worked for domain name registrar, I worked for cloud compute company, I worked for internet exchange, and finally I got a job for global CDN provider, and I ended up working in cyber security.
01:00
So, what I'm doing now, I'm in charge of development of cyber security product for network security, and this product is called FastNet mode. So, I would like to start from brief description of what is FastNet mode. FastNet mode is an application. So, it's a very first thing to clarify.
01:23
It's cross platform application, and why I'm saying cross platform? I mean Linux, Macros, FreeBSD, OpenBSD. It's not yet on Windows, but it's still. And main purpose of FastNet mode is the DDoS detection for networks. From technical perspective, FastNet mode is implemented using modern C++.
01:44
Back in time, it was quite interesting story when FastNet mode was started. Very, very first version of it in 2013 was implemented in C++11. But because of compilers in some way, not so very modern distribution, we had to move back to C++98.
02:02
Since then, we still support modern versions. No reasons to maintain compatibility, it's very outdated stuff, and now it's really good for C++. It's kind of C++, you actually enjoy hacking, cruising, and changing, if you prefer to do so.
02:23
I know this feeling. When you hear about new stuff which may be relevant for you, it's the first urge to maybe, should I try it now, immediately? Because what is the point to read the documentation? What's the point to hear my presentation? If you can just install it right now? It was a very long journey, and I would like to thank our maintainers who made it possible that FastNet mode has so many distributions.
02:49
For almost every single popular distribution, used mostly for server environment and production environment, you may install FastNet on just a single command. So, you may start it right now, if you prefer.
03:01
And if, for some reason, your distribution has no latest version of FastNet mode, or you want to just install it, for some distribution it's not covered by all official packages, there is an installation tool. Let's go forward, because most of the time we talk about what our tools and our products can do.
03:25
I would like to start from an unusual angle. I would like to highlight what we can do, because it's important. Because there are so many tools for DDoS detection. There are so many angles of DDoS detection, it's the detection part, it's the mitigation part.
03:41
It can be implemented on premise and in cloud. And before we go into details what we are doing, we need to highlight what we are not doing. And if you have any issues with your website or your blog, I'm sorry, we can't help you. It's not the point of FastNet mode, it may indirectly help your carrier or service provider.
04:03
But for your case, it's better to use cloud services, because it's not that complicated to move sites around. Because normally, if I'm talking about really enormous sites, it's quite easy to move them to a content delivery network and then cover it from the desk. And if you have some issues with DDoS when you play on your Xbox or Playstation, also I'm sorry, we can't help you.
04:29
I explicitly decided to add this slide, because I have too many questions. And I think it's one of the real serious problems in modern days, because you cannot play because of DDoS.
04:40
And if you use managed service provider, it may be public cloud, it may be private cloud. And when I say managed, it means that somebody is in charge of keeping your service running. And in this case, it's very unlikely that you have access for your network, I mean, administrative level of access.
05:00
It's ability to change policy, inspire policy, change router configuration. And in this case, it's better to escalate to your service provider, like call for help. We have something, we have some problems, help us. And finally, what FastNet mode can help you? FastNet mode here is not to protect specific service, FastNet mode here to protect your network.
05:21
And when I'm saying to protect network, it has very different meaning from protecting specific entity. Main purpose of FastNet mode to keep up time of all network in general. And when I'm saying in general, it means that keep it running for 99% of customers, eyeball services behind the specific network.
05:41
And I'll explain how we can do it. What kind of attacks we can protect your network from? Again, there are so many types of attacks, there are so many opinions about classification of attack. I'm not going to go into details about what kinds of DDoS attacks. I would like to focus it from well described OSI model approach.
06:03
So what we can help you? We can help you with IPv4 and IPv6 at the same time. If you still use IPv4, please don't, please move away from it. And in terms of layers of OSI model, we can help you only with levels L3 and level 4. And if you have some specific ideas what is the option to filter out traffic using like HTTP or 2 or 3 protocol equipped by TLS,
06:30
it's better just to try to just present it like Suricata. And it's because FastNet mode is little bit autoscoped. Because main purpose of FastNet mode to detect volumetric DDoS attack. And when I say volumetric it means at least hundreds of megabits.
06:44
But mostly in general case for average size of DDoS in modern days it's around 8 gigabits. And in some cases it's exceptionally high, it's maybe hundreds of gigabits. But on average it's like just few gigabits. And that's purpose of FastNet to detect this kind of attack.
07:02
So what is the very first step when we assume that network is under DDoS? Because when I'm saying assume, can we say for sure is it DDoS? Because in so many cases how we actually can observe DDoS attack against our network. Like you check your phone, it's not working.
07:21
Like your website is not working. You check like laptop in your office and for some reason something doesn't work as expected. Or customers calling you. And first step is to confirm that actually DDoS. Because it may be not DDoS, it may be fire alarm in your data center. Why it's extremely important in the sense that it's actually DDoS?
07:41
Because it may be something different and in case of fire it's way more important and way more different kind of actions to remedy the DDoS detection. And if you know by accident that some people, your colleagues working in data center right now and it's like the same timeline. You receive call from customer like something doesn't work.
08:03
It's very unlikely that it's DDoS. It's maybe caused by misconfiguration. Because there are so many ways how we can figure downtime in our networks. And okay, we covered most of the sources which can cause network DDoS.
08:21
Network downtime is actually DDoS. And look, even this one is not DDoS. This one is, it can cause havoc. It can bring down whole cities, countries, data centers. But it's still not DDoS. And what is the first, how we can say like this one's for sure DDoS.
08:41
And graphs. The only way to be 100% sure is graphs. And by looking on this graph, if you know like okay, my network generates like 100,000 packets per second or like 100 gigabytes. And if you can see spikes by 20 gigabytes, it's very unlikely it is caused by something normal.
09:01
It's very likely it is DDoS. So it's first level of remediation. It's first level how FastNet1 can help you. FastNet1 can say for sure in this kind of dashboard that you are under DDoS. And then you can action it appropriately because you are well prepared. You know what you can do. And what we can do in this case.
09:23
FastNet1 provides lots of different dashboards. And the main benefit of those dashboards is that they're built not on physical level of network. Because when I'm saying physical level of network, I mean port counters, load for specific interface, load for specific router.
09:40
And what FastNet1 can do, it's more overview of your network from logical level. When I say logical level, it's more from networks, prefixes, specific services. And in this case, FastNet1 can provide a required amount of granularity. It's like total traffic for your network. In this case, you can see total incoming traffic.
10:00
And in case of any spikes here, you may see it almost immediately. It's one of the benefits of FastNet1. It's not historical data. It's data which actually was just received from your routers. It's almost real-time data. And so in the same case, again, from a logical perspective, instead of seeing what is the load for specific interface on my router,
10:24
you can see information about how much traffic you have for specific prefix. And you're well aware what kind of service is trying in specific prefix. And so you can understand something wrong with this specific prefix. And again, latest level of granularity. You may find island traffic for per-host because you may know that for specific prefix, you just have two services, very important services, right?
10:48
And then you can check what is the load, for example. And you can see immediately again in real-time what's wrong. If you can see spike for this specific service, okay, we found victim, sadly.
11:00
And FastNet1 graphic capabilities, they include complete support for InfluxDB, GraphIt, and plenty of Grafana dashboard. I would like to thank the community for contributing so many great dashboards. Because when we started this idea, we implemented a few of them, quite basic ones. But the community did a really great job by playing through them. And actually, most of them are way better than our official dashboards.
11:24
And what is the source of this data? Is it AI or something different? No. So we receive this information from routers or switches in your network. And from the perspective of protocols, we support almost all available protocols in the market.
11:41
And of course, one of the most popular ones is NetFlow, IPFix, SFlow. And in case of last resort, if you have no NetFlow or IPFix in your network, you can try to use PortMirror. For all cases, FastNet1 can handle a really significant amount of traffic. And there are plenty of confirmed deployments of FastNet1, exceeding at least two terabytes of capacity in total.
12:07
So after you got all the information, you may check it manually. For example, again, at this moment, this FastNet1, you will see what is your total load, what is load for specific network, what is load for specific host.
12:21
And for small networks, you may find immediately what is the victim. Because in case of a small network, you know, okay, I have 12 services, move between, and you can check one by one. Can we do it for DDoS detection? And this one is just a very not very precise map of the United Kingdom.
12:46
And you can see there are lots of interconnections. It's just the largest country of the planet. But you can see the amount of interconnections, it's incredible. Even for medium-sized internet service providers or telecom providers, they may cover at least multiple countries.
13:03
And you can see the amount, even towns, even regions, it's incredible. And if you talk about networks covering multiple European countries or multiple countries in maybe, for example, Asia, it's an incredible amount of locations, an incredible amount of entities.
13:20
You cannot check, for example, we are under DDoS, you know for sure. Let's check every single one plus million city in Europe. We cannot do it manually. It's just impossible. Every single time moving from large cities, we need to move to regions. Then we need to check household, by household, because this specific attack,
13:41
it may begin specific person playing Fortnite game in this specific building. You cannot do it manually, unfortunately. If we move a little bit to data centers, data center normally, as we can make here, it's single building, maybe huge building, but it's still, it's just one building. It's not like, it's not scattered over like continents, it's not scattered over like thousands of kilometers.
14:06
Is it easy to find out? No, unfortunately, because sadly, in data center, you may have more entities, more potential victims of DDoS than actually for large telecom networks. And what we can do, of course, as I mentioned, you can manually check every single host available in your network,
14:24
because we already got pretty great dashboard, and we have real-time data coming from your routers. What is the logic? What is the way how we can actually find that? Again, we have data about what is a bandwidth for specific network, what is a packet rate for specific network,
14:41
and we can check every single host in our network and find out. And again, in case of data center and large telecom networks, it's impossible to do it manually. That's the reason how FastNet1 can help you. FastNet1 can do it for you, and it can do it really fast. For almost all protocols supported by FastNet1, we can offer detection time in less than five seconds.
15:05
So it's not about FastNet1 can say, look, you're under DDoS, because it may be clear from graphs. At this point of time, FastNet1 can find out what is a specific service in your network, which is under attack right now. And you'll have this information in five seconds.
15:21
So why it's important, like five seconds? Why? Can we wait a little bit? Have a cup of tea or coffee and wait? Unfortunately, we cannot. That's the main problem, because back in time when I started working with DDoS attacks, it was around 2008. You can wait for around half hour when DDoS attack starts from something like 10 megabits, maybe then 15 megabits, 100 megabits.
15:46
Then we have a cup of coffee, wait a little bit, then 20, something like 50 megabits, 100 megabits. It's gone. And now what we can see, attack can escalate from like 100 megabits to tens of gigabits in like a few seconds. And human being, unfortunately, I had to admit, I cannot handle it so fast.
16:05
We need some machines, because people who actually run DDoS, they have lots of automation. And without having automation in place, we cannot defend it. So FastNet1 provides this option for you. And instead of checking every single host in your network manually, because
16:21
it's still an option, you can verify when you receive reports from FastNet1. You can check graphs, like is it DDoS, is it looking like DDoS, because FastNet1 inside, it uses very simple rules. Like if specific host in my network generates more than 5 gigabits of bandwidth, and if specific host in my network generates more than 100,000 packets per second, it's clearly DDoS.
16:46
And after detection, what we can do, and very first step, which is available for every single carrier on this planet, and fortunately, it's free. This thing called BGP black hole. BGP black hole needs a little bit more clarification, how it works.
17:04
And because of name, you may guess, if you put something into black hole, you'll never see it again. And that's the point. And how FastNet1 can help and can rely on BGP black hole to stop DDoS from network. At the beginning of my presentation, I mentioned that FastNet1 is here to protect your network, not specific servers.
17:27
And it's really important, because BGP black hole can be described in many words, because it's quite complicated abstraction. But I would call it, it's like religion sacrifice made by network engineers to keep their network running.
17:43
Why is it sacrifice? Because when you do black hole, because at this point of time, for example, for our network, we have 20,000 of hosts. Let's imagine every single host, it's residential building somewhere in Europe. And we know for sure, we are receiving DDoS right now.
18:01
And our server is degraded. Our customer is calling us. Our site doesn't work, nothing works fine. And we can find out what is the victim of this specific attack using FastNet1. And we know specific host, which is IPv4 or IPv6, which is a target of this DDoS attack. And what we need to do using BGP black hole, we need to stop all traffic from coming to this specific host.
18:27
Which means if it affects disabling and unplugging this specific customer or service from the internet. And that's how BGP black hole works. It's not about firewall, which may block attackers.
18:41
In this case, we literally, manually, voluntarily block target of our attack just to save our network. And that's only purpose of FastNet1 to stop it and do it automatically for you. And after you stop it, and you can see exactly on this diagram.
19:02
So we maintain uptime of network. And everything keeps working by sacrificing just one host on your network. And it doesn't mean that you just block it and go away. I can email to customer, look, we blocked your service, we can help you. There are so many ways how you can actually keep this host running.
19:22
But again, before you apply some actions, create plan, what we can do. Maybe you can call some specific providers to provide defense for it. Sorry. So you need to have some actions. And better to apply these kind of actions in quiet environment.
19:44
Instead of having to deal with 20,000 of calling customers every single minute, you may block specific target, you may keep uptime of your network back. And when your network is back to operation, in quiet environment, in way quieter environment,
20:01
nobody calling you, nobody calling you, decide either cup of coffee or tea, what is the option, what you can do for this specific customer. And then, how Fastnetmon can help you. And since beginning, when Fastnetmon was built, it was open source from very first version. And a lot of features, I just explained it, they weren't invented by our master plan or our roadmap.
20:23
They were part of community request, we received it at GitHub, because of look, there is an option, I have a problem, and I would like to solve it. So since beginning, Fastnetmon was community-driven project. And we have lots of community channels, how you can collaborate with us, how you can share your stories, how you can ask questions.
20:40
And please join all of them, and I will be happy to answer your questions. Thank you. Anybody has questions?
21:05
Hi, thanks a lot, quite interesting. So I just wanted to ask you if you ever felt the need to extend the way you collect data with other protocols, like, for example, any flavor of OpenConfig, specifications, or eventually BMP instead of BGP.
21:23
That's a great question. So the question was, is it possible to use protocols like OpenBMP or OpenConf to feed more information to Fastnetmon? In current generation of Fastnetmon detection tools, we mostly rely on traffic telemetry protocols, which actually carries part of network packet, it's maybe header of network packet, or it may be some meta-information about source port, source IP, destination port, destination IP.
21:47
And we don't use data about BGP directly. The only way how we can actually interact with BGP is that we have internal BGP daemon based on GoBGP, which actually injects information and announces orders to your network.
22:01
So we have no backward integration from network, so we have no way how we can learn information from your network. But because we offer different APIs, we offer different ways to automate and run callback scripts instead of just running BGP, you can run your own Python script and then you can learn information from third-party source and combine this information and make decisions using this information.
22:24
I was merely asking because, for example, with the GNMI, you can have a sort of retroaction on the network, so you can, based according to what you're receiving using IP fix, for example, you can have an action directly on routers, for example. Yes, it's one of the ways how we can actually use so-called callback scripts,
22:44
because when FastNet1 detects attack, it can run specific script. It may be batch script, Python script, Perl script. And in this script, you will have access to basic information about attack and information, what is the target, what is the host target, what is the type of attack, what is the prefix target. And then using any automation protocol, you can run actions on routers.
23:04
And because of most of the routers, there are no standard way how we can inject this kind of information for every single vendor available on market, and we decide to move these tasks to community, so to implement it on your own. And if you implement it, share it with community.
23:22
One second. Can you do BGP flow spec to, like, black hole? That's a great question. Back in time, we had BGP flow spec support based on XABGP, but it was, like, POC level, quality of implementation,
23:44
because it was just literally hard code, at least, for DNS and SSDP amplification. But it worked well. But unfortunately, because of complexity of working this API of XABGP's flow spec protocol, we decided to remove this capability. And now is the only way how we can actually inject flow spec rules,
24:02
because you can implement black hole using flow spec. You can run it using go BGP command line from callback scripts of Fastnet 1. Okay, thank you. Any more questions? No. Thank you very much. Thank you for listening.