How to protect your Kubernetes cluster using Crowdsec
This is a modal window.
Das Video konnte nicht geladen werden, da entweder ein Server- oder Netzwerkfehler auftrat oder das Format nicht unterstützt wird.
Formale Metadaten
Titel |
| |
Serientitel | ||
Anzahl der Teile | 542 | |
Autor | ||
Lizenz | CC-Namensnennung 2.0 Belgien: Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen. | |
Identifikatoren | 10.5446/61956 (DOI) | |
Herausgeber | ||
Erscheinungsjahr | ||
Sprache |
Inhaltliche Metadaten
Fachgebiet | ||
Genre | ||
Abstract |
|
FOSDEM 2023381 / 542
2
5
10
14
15
16
22
24
27
29
31
36
43
48
56
63
74
78
83
87
89
95
96
99
104
106
107
117
119
121
122
125
126
128
130
132
134
135
136
141
143
146
148
152
155
157
159
161
165
166
168
170
173
176
180
181
185
191
194
196
197
198
199
206
207
209
210
211
212
216
219
220
227
228
229
231
232
233
236
250
252
256
258
260
263
264
267
271
273
275
276
278
282
286
292
293
298
299
300
302
312
316
321
322
324
339
341
342
343
344
351
352
354
355
356
357
359
369
370
372
373
376
378
379
380
382
383
387
390
394
395
401
405
406
410
411
413
415
416
421
426
430
437
438
440
441
443
444
445
446
448
449
450
451
458
464
468
472
475
476
479
481
493
494
498
499
502
509
513
516
517
520
522
524
525
531
534
535
537
538
541
00:00
Computeranimation
00:18
CybersexPhysikalisches SystemSchnittmengeBefehl <Informatik>ComputersicherheitArithmetisches MittelComputeranimation
00:53
RechenwerkLoginComputeranimation
01:22
PlastikkarteOvalDoS-AttackePunktwolkePlastikkarteFunktion <Mathematik>CaptchaBenutzerbeteiligungWeg <Topologie>RoboterMereologieGemeinsamer SpeicherClientNetzadresseEin-Ausgabep-BlockStreaming <Kommunikationstechnik>KonditionszahlKomponente <Software>Syntaktische AnalyseKomplex <Algebra>ServerLoginFirewallEindringerkennungWeb SiteProzess <Informatik>MultiplikationComputeranimation
05:29
Gebäude <Mathematik>VerschlingungRechnernetzComputersicherheitEchtzeitsystemComputerkriminalitätCybersexNetzadresseComputeranimation
05:54
Web logp-BlockLokales MinimumMetadatenGemeinsamer SpeicherRechenschieberMereologieServerLoginMailing-ListeMultiplikationsoperatorComputeranimation
07:23
EindringerkennungMehrplatzsystemMereologieVerkehrsinformationInformationMailing-ListeSoftwareAlgorithmusp-BlockDienst <Informatik>ServerInternetworkingComputeranimation
10:29
ServerDifferenteServerNetzadresseBenutzerbeteiligungSchnittmengep-BlockComputeranimation
11:25
Physikalisches SystemFirewallKonfigurationsraumInformationComputeranimation
12:31
EntscheidungstheorieEindringerkennungp-BlockMAPMotion CapturingInformationStellenringLoginKomponente <Software>SharewareBenutzerbeteiligungServerRelativitätstheorieDämon <Informatik>Message-Passingp-BlockFirewallEntscheidungstheorieComputeranimation
15:17
Gewicht <Ausgleichsrechnung>KontrollstrukturEindringerkennungGamecontrollerKreisringComputeranimation
15:39
DefaultSharewarePasswortGamecontrollerBenutzerfreundlichkeitStellenringKartesische KoordinatenSharewareComputeranimation
16:20
EntscheidungstheorieRepository <Informatik>PlastikkarteNamensraumStellenringMultiplikationsoperatorDämon <Informatik>Installation <Informatik>Umwandlungsenthalpie
17:14
GEDCOMLoginPasswortStellenringW3C-StandardBenutzerfreundlichkeitBildgebendes Verfahren
17:43
LoginPasswortLesen <Datenverarbeitung>Inhalt <Mathematik>KontrollstrukturElektronischer FingerabdruckMathematikSharewareApp <Programm>Login
19:00
Ljapunov-ExponentInhalt <Mathematik>Puls <Technik>Prozess <Informatik>Elektronische PublikationMereologieBenutzerbeteiligungPortscannerKartesische Koordinaten
19:20
MereologieComputersicherheitW3C-StandardFlächeninhaltProzess <Informatik>VersionsverwaltungInformationE-MailInhalt <Mathematik>KraftSpieltheorieKette <Mathematik>Airy-FunktionKanalkapazitätProgrammbibliothekPartitionsfunktionKreisbogenVerzeichnisdienstPolygonzugOffene MengeLoginGamecontrollerSyntaktische AnalyseFaserbündelSensitivitätsanalyseMereologieElektronische PublikationProgramm/Quellcode
19:42
MenütechnikLokales MinimumMereologiePunktwolkePartitionsfunktionGammafunktionW3C-StandardSpieltheorieInhalt <Mathematik>E-MailServerMIDI <Musikelektronik>StellenringElektronische PublikationNabel <Mathematik>EntscheidungstheorieProgramm/Quellcode
20:00
GammafunktionMereologieSichtbarkeitsverfahrenE-MailVerzeichnisdienstFormale GrammatikEichtheorieKanalkapazitätBenutzerfreundlichkeitInhalt <Mathematik>DickeServerAusgleichsrechnungPartitionsfunktionComputervirusKanal <Bildverarbeitung>SharewareMultiplikationsoperatorStellenringPatch <Software>Plug inKartesische KoordinatenNetzadresse
20:59
E-MailDatentypInhalt <Mathematik>FehlermeldungElektronischer DatenaustauschFinite-Elemente-MethodeRandwertBaumechanikGRASS <Programm>GamecontrollerGEDCOMGamecontrollerProdukt <Mathematik>StellenringProgrammbibliothekFlussdiagramm
22:24
GamecontrollerInhalt <Mathematik>Lipschitz-StetigkeitSummierbarkeitSchnittmengeMultiplikationsoperator
23:00
MathematikStatistikGamecontrollerWeb-SeiteCybersexServerComputersicherheitGoogolComputeranimation
23:30
Flussdiagramm
Transkript: Englisch(automatisch erzeugt)
00:07
Hello, everyone. My name is Amzah, and this is my colleague, Sebastian. We are working at CrowdSec, and today we're going to introduce you to CrowdSec, explain to you how it works, and show you how you can protect your Kubernetes cluster using it.
00:24
So how CrowdSec was created? We start with the basic statement that, yeah, so cybersecurity, we start with the basic statement that cybersecurity is not a problem of means. As you can see,
00:43
those big companies have a huge amount dedicated to cybersecurity. They have security teams, but they get still hacked. And the reason we think that the reason why is because, sorry, because they try to fight alone, like using the superhero approach, fighting
01:03
alone attackers while those attackers are sometimes collaborating together to attack. So as we are a lot of users that want to run legitimate businesses, why not collaborating altogether to fight those attackers?
01:22
So how this collaborative IPS works? So basically, CrowdSec needs to ingest logs. So we have a lot of handled data sources, like flat file, basic. It can also act as a syslog server to receive syslog inputs. It can also read your Docker container output
01:42
logs on the cloud part. We have cloud trail on the stream processing. We have also Kafka, Kinesis, and so on. Then we have the IDS part. So the IDS part is a component, what we call the agent. The agent is here to pass the logs and detect bad behaviors. And this is where the community
02:07
aspect starts. We have a hub, like Docker hub, where we wrote our scenarios to detect the common attacks, and we share them with the community. And then here the other users,
02:25
they write their own parsers to handle new services, new bad behaviors, et cetera, and they share them with the community. You can also write your own scenarios to detect, like, a specific behavior in your internal infrastructure, for example, and not share it, of course. And once this detection
02:47
is done, we have the IDS part. So the IDS part, so the IDS part is what we call, it's another component of CrowdSec called the bouncers. And the bouncers are here to allow
03:01
you to do a remediation in multiple parts of your infrastructure. So, for example, the basic one is to have the firewall bouncer. So the firewall bouncer will communicate with the CrowdSec, get the decisions, and block them at the firewall level. That you want sometimes have a smarter remediation, like, for example, you run an eShop, and
03:26
we know that sometimes we have a lot of clients that are behind single IP. So what we want to do is to, when a bad behavior is detected from an IP, we want to push a CAPTCHA to the user, so we let humans still access to the website, but block the bots.
03:45
And of course, we have a lot of other bouncers on the cloud side, or also other bouncers that are written by us and by the community, of course, and they are also available in the hub. And when this remediation is done, we have the reputation part and what we
04:06
call the community block list. So we receive the signals, we send the signal to the community, to the central API, sorry, we receive all those signals, we curate them, and then we return to the community the most aggressive IPs. So you are protecting yourself and protecting
04:24
the community, and you benefit also from the community feeds. So, we already deal with a lot of common attacks, more than 50, web scan, post scan,
04:41
conditional card stuffing, but also thanks to the emphasis of CrowdSec, it allows us to detect more complex attacks, like, for example, or credit card stuffing, which is a very business-specific issue. So, for example, when you have, for credit card stuffing, it's
05:02
when an attacker buys some credit cards from a shady website and wants to test if those credit cards are valid. So here we go to an eShop and try to do small purchases. So you can detect this kind of attacks, also bot scalping, et cetera.
05:21
Basically, you can detect, if it's in the logs, as CrowdSec ingest logs, you can detect it. And thanks to that, we are building a real-time map, sorry, of cyber criminals. It's kind of like Waze, but in cyber security. So the more we are and the more efficient
05:41
is the radar. And, of course, it will allow us to kill the most important resource for the attackers, which is the IP addresses. Now I will let my colleague take the next
06:00
slide. Hello, everyone. So first thing, so about privacy, because, as Amza mentioned, you do send us data about the attacks you see. But we do collect only the strict minimum to be able to build the community block list. This means that your logs will never, ever leave your own server. Logs are processed
06:24
locally by CrowdSec, and you will only send some metadata about the behaviors that are detected by CrowdSec. So those metadata are just the IP that you blocked, the reason why, so for example, SSA is a good source, and the time at
06:41
which you block the attack. And that's it, nothing else. And also, this part is totally optional. If you do not want to send anything to us or the community, you don't have to. But if you don't send data, if you don't share with the community, in return, you will not get the community block list. And also, we keep the
07:01
data for the least amount of time possible. So basically, every IP in the community block list will be automatically deleted after one week if the IP does not perform any more attack on the community. And for the raw data we receive, after three months, it's degraded, and after nine months, it's totally deleted. Now, how do we build this
07:27
community block list? Because we receive reports from all over the community, but we cannot trust those reports because it's just an API. If someone wants to send us the information that ISO 1.1.1 performing SSL could force my server, we have no way by
07:44
basically to know if it's true or not. So, we build something called the consensus. So, we get the report from all over the community, and then each user in the community has a trust score. So, this score is really how much we trust you. It's built over time, so the
08:05
longer you are part of the community, the more your score will increase. But it will also take into account how active you are in the community. So, if you report just one IP per day or someone that is reporting 100 IP per day, we will just give a slightly better
08:22
score to the user reporting 100 IP per day. And we will also correlate your report with those community members. If you are the only one sending us the information about a particular IP, this IP will never go into the community block list because we cannot confirm that it is indeed an IP trying to attack servers on the internet. But if, if multiple
08:48
users report the same IP, then we will be more confident about the fact that this report is exact. We do also run some ONIPODs, and because those are fully controlled by us, they
09:04
are fully trusted. So, if someone tries to attack them, we know that this is a micro-report. We do also have some waitlists, because if someone sends us IP belonging, for example, to Cloudflare, obviously this is not something we want to redistribute to the
09:22
community. And again, if you do send us too many false positive reports like this, your score will be reduced, because we know that your reports are potentially false. And lastly, we do also have some more, um, advanced algorithms that, for example, we look at
09:42
the bigger picture. Uh, if you take a slash 24 network, and like 50% of this network is already, uh, in the community block list, well, if we see another IP belonging to this network, it's quite likely that this IP is also bad. Same thing, for example, if the
10:00
IP reported to us exposes, um, like, I don't know, a tenant server, uh, 20% of different service on the internet, again, it's more likely that the IP has been compromised. And so, when a report goes through everything, we will take a decision, and it will go into what we call the FHIR database, or the community block list, and it will be
10:23
pulled automatically by all, uh, the community member. Now, also, for the usefulness of the community block list, a while back, we ran a small experiment where we had two different servers on the internet, uh, on the same hosting provider. Those two servers
10:43
had the exact same set of services, exposes on, exposed on the internet, so, to adjust the web server and the SSH server. The only difference between them was, one was using the community block list, and the other one was not. On the one using the community block list, we saw a 192% less attack, uh, on the server detected by Protech. So,
11:06
because, basically, most of the IPs that are trying to scan, uh, your server exposed on the internet, are basically trying to do it for all the internet, and so this is just some background noise, you don't really care about it, you can just block it before it can, uh,
11:22
try to attack you. Another thing that we get, uh, asked quite often is, okay, so I want to replace my firewall with Protech. I want to, to replace my auditing system with Protech or whatever. Please don't do that. It's not designed, uh, to do that.
11:41
We don't compete with this kind of solution, but, on the contrary, we can help them to, to be more useful. So, for example, uh, if you have a auditing system telling you that, okay, so I saw, uh, command execution on this server, and it was a carpipe bash or something, or whatever, but you, you can configure Protech to pass this log to detect this behavior and
12:05
just to send you a notification. It does not have to be something about an IP address, it can be a local behavior as well. Or something, something with firewall, you don't have to, uh, go into, uh, the configuration at the IP, just at the IP in Protech, with one simple command,
12:23
and then you will, Protech will take care of pushing this information to your firewall, and the IP will be blocked. So, for the, uh, architecture, uh, of Protech, so as I'm, as I mentioned before, so Protech will ingest some logs, pass them, so this is the
12:43
role of what we call the agent. Uh, the agent will read your logs, pass them, and confront them against scenarios. So, scenario, just some things that describe a malicious behavior. For example, a brute force, someone trying to scan your website, and so on. When the agent
13:01
detect a malicious behavior, it will create an alert and will be sending this alert to another component of Protech, the local API. So, this means that the agent can live on one server, the local API can live on another server. The local API will take this alert and transform it into a decision. For example, by default, it's a four-hour ban on the
13:22
IP. But you can customize this behavior, and for example, you can say, so, uh, this IP is French, uh, it performs something related to, uh, an HTTP attack. So, instead of just banning it, I'm going to, uh, ask my, uh, bouncer to display a capture for this IP, and
13:41
for four hours, for example. Um, so, and when, uh, so, when the local API receive, receive a decision, it will be sending us information about the alert, and in exchange, you will receive the community blockage. Uh, the local API is also used by the bouncer, so the components that do actually apply the decision, because Protech by itself will just do the
14:06
detection part, it will not block anything. For that, you need bouncers, so we have, uh, quite a few of them. The most popular one is probably the firewall bouncer that will interact with the local firewall, uh, of the server where the bouncer is going to block the IP. Uh, but we do also support, uh, web server, for example, Nginx, and in this
14:24
case, because we are at a higher level in the network, you can display a capture to the user instead of, instead of just dropping the request. Um, so here, uh, it's, uh, the
14:43
following, uh, demo. So, very small Kubernetes cluster running locally with just two nodes, one agent panel deployed as a daemon set, uh, one local API for the agent to be able to send the alert, and the agent will be reading the logs of, uh, the ingress of the
15:03
cluster, in this case, Nginx ingress, and we will have, uh, a bouncer, the Nginx running on the ingress to, uh, block automatically, uh, the IP, if corsac wants to ban them. Okay, thank you. So, uh, as we have, we are, crazy guys, we gonna do a live
15:25
demo, and, uh, yeah, uh, let me just, I think I have some, so, as, uh, Sebastian said, we gonna, we have a small, uh, locally, of course, not online. Uh, thanks. Ah, yes, of
15:47
course, sorry. Thank you. Okay, so, uh, what we gonna do is to, um, so, sorry for that, but I will, uh, so, this terraform, uh, will create, uh, Kubernetes cluster
16:02
with two nodes, uh, with the ingress controller, Nginx, and, uh, the, uh, the, uh, uh, hello world app, like it's a demo application. Uh, nice, okay. So, here we can see that, okay, we have one, uh, ingress controller, and we have two nodes, okay. So,
16:28
what we'll do now, uh, I will not spend time to explain you, like, the, the, the crowd take values, because we released a hand chart that will allow you to install crowd take, uh, in a Kubernetes cluster. As I said, it deployed a daemon set, so, in
16:42
each node, it will deploy a crowd take agent, and one LAPI, so, local API in, uh, in a specific name space, and, of course, your decisions will be centralized, uh, across all your, uh, uh, nodes. So, here, we will install, uh, crowd take. So, basically, just
17:04
install the crowd take is in these values, uh, and this is the hand chart. Of course, I already, uh, import my repo in the name space card take and create it if it's not exist, the name space. So, here, if we are, okay, ah, yeah, as I popped a new
17:31
cluster, I have to wait for the, uh, image download, but, uh, it will come no more. Okay, what I can do, yeah, of course, uh, what I can do is to show you that I'm
17:48
able to access my hello world app. So, it's basically in hello world. So, it returns 200, and, okay, we have one port that is running, demo effect, as usual. So,
18:30
let's, you know, install, wait, why, ah, okay. So, now, uh, yeah, we're gonna fetch
18:45
this one. So, okay, I will, uh, follow my logs from the, this agent. So, logs minus F, agent, and, and, all right. So, here is my crowd set logs. So, now, what I will do
19:02
is to, uh, run Nikto, which is a, a simple, okay, a simple web scanner to attack my, uh, hello world application. And here, we can see that, I can even already, okay, here, we
19:21
can see that it automatically detects as it's, uh, automatically parsing the logs of the ingress controller. It detects some bad behaviors because I already installed, uh, like, uh, collections that are bundles of parsers and scenarios. And you can see that it detects bad user agent, some sensitive file access, part reversal, et cetera. So, if I,
19:43
uh, do something like that and get a shell on crowd set, sorry for the noise, let's local API, and if I list the decisions that were taken, we can see that we have, uh, six
20:03
other entries like this one because we detect several times my IP as I'm working locally. That's why we have those IPs. And this is the last, uh, behavior that was detected. So, now, what we will do is to install the bouncer because we are detecting, but we get still, uh, access to the, to the application. So, now, we
20:22
will patch our, uh, ingress controller, uh, to install the LUA plugin to communicate with the local API and get banned, of course. So, now, here, we have the command. So, we can go fast and have, and add time. So, I'm upgrading my Kubernetes, uh, my ingress controller, sorry. Uh, okay. Okay. So, now, if we
20:56
try to access now, uh, to, uh, the hello world application, we can see that we will
21:00
be, we will receive, uh, four or three. So, if I do this, oh, ah, ah, it was failing. Sorry, I didn't see that. Uh, yeah, it's the, let's wait for this upgrade.
21:49
Okay. Okay. Nice. So, it's popping the new, uh, ingress controller. Of course, it's, uh, local. That's why we have only one pod. I don't recommend that in production. Uh, so, it's deploying the new ingress controller, uh, and loading, of
22:10
course, the crowd sake bouncer. It's a LUA library that is talking with the, uh, local API, and on each request, uh, we have this check. Uh, and yes, of course, uh, I
22:45
think we ended. No, we have still time. So, we can take some questions, and I will follow. We've got time for, like, one question. So, if there is one fast question for one minute, feel free to raise your hand. Oh, okay. Uh, to, to hand, just to hand the demonstration, uh, it's running,
23:03
and now we have a four or three. So, that's it. And, uh, okay. So, thank you. This is the challenge with the demonstrations. Uh, if you have some
23:23
questions, don't hesitate. And, of course, we have some, uh, stickers, so don't hesitate to, to, to come to us.