Packet Hacking Village - Attacking and Defending Kubernetes
This is a modal window.
Das Video konnte nicht geladen werden, da entweder ein Server- oder Netzwerkfehler auftrat oder das Format nicht unterstützt wird.
Formale Metadaten
Titel |
| |
Serientitel | ||
Anzahl der Teile | 335 | |
Autor | ||
Lizenz | CC-Namensnennung 3.0 Unported: Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen. | |
Identifikatoren | 10.5446/48754 (DOI) | |
Herausgeber | ||
Erscheinungsjahr | ||
Sprache |
Inhaltliche Metadaten
Fachgebiet | ||
Genre | ||
Abstract |
|
DEF CON 27284 / 335
8
9
12
14
32
38
41
58
60
61
72
75
83
87
92
96
108
115
128
132
143
152
158
159
191
193
218
230
268
271
273
276
278
295
310
320
321
335
00:00
MultiplikationsoperatorZahlzeichenSpannweite <Stochastik>Computeranimation
01:03
GeradeRSA-VerschlüsselungEDV-BeratungSampler <Musikinstrument>Divergente ReiheOffene MengeMUDOISCPCMCIAKlasse <Mathematik>MehrrechnersystemSummierbarkeitLokales MinimumPhysikalische TheorieAbenteuerspielKomplexe DarstellungRechenschieberHecke-OperatorKlasse <Mathematik>FokalpunktComputersicherheitGraphBitDemoszene <Programmierung>MereologieAbenteuerspielProgramm/QuellcodeComputeranimation
02:17
Lineare OptimierungOpen SourceProgrammBasis <Mathematik>FestplatteHecke-OperatorVirtuelle MaschineProjektive EbeneRechter WinkelBeanspruchungServerDisk-ArrayZahlenbereichNeuroinformatikKanalkapazitätZweiBefehl <Informatik>NeunMultiplikationsoperatorBitrateFigurierte ZahlPerfekte GruppeComputeranimation
04:59
Zentrische StreckungDienst <Informatik>LastZurücksetzung <Transaktion>SoftwareSoftwaretestInformationsspeicherungKonfigurationsraumDatenverwaltungGleitendes MittelBitDienst <Informatik>SkalierbarkeitBeanspruchungRoutingLastteilungPhysikalische TheorieBinärdatenKonfigurationsverwaltungVersionsverwaltungRechenzentrumVirtuelle MaschineInformationsspeicherungE-MailFestplatteSystemzusammenbruchEinsLastComputeranimation
06:19
Dienst <Informatik>SiedepunktProfil <Strömung>COMSchlussregelVerband <Mathematik>Dienst <Informatik>Physikalisches SystemNetzadresseGemeinsamer SpeicherBrennen <Datenverarbeitung>MomentenproblemEinsVirtuelle MaschineMereologieBitPhysikalismusInformationsspeicherungRechenwerkSchnittmengeLastteilungProxy ServerServerLaufzeitfehlerHardwareTabelleSoftwareComputeranimation
10:01
NamensraumKernel <Informatik>Objekt <Kategorie>GruppenoperationKomponente <Software>Physikalisches SystemDeklarative ProgrammierspracheAlgorithmische ProgrammierungDatenmodellRuhmasseKnotenmengeServerSpezialrechnerMedianwertKreisringCluster ServicesRechenschieberTablet PCInterface <Schaltung>GamecontrollerSchedulingPerspektiveMultiplikationGamecontrollerExistenzaussageRechter WinkelInternetworkingProgrammschleifeBildgebendes VerfahrenAutorisierungKonfigurationsdatenbankRegelkreisLaufzeitfehlerRechenschieberSchedulingPhysikalische TheorieSoftwareBenutzeroberflächeProgrammDifferenteZusammenhängender GraphBeanspruchungVirtuelle MaschineBridge <Kommunikationstechnik>sinc-FunktionEndliche ModelltheorieEinsGrenzschichtablösungAggregatzustandDatenverwaltungBitTermInformationsüberlastungKernel <Informatik>NamensraumComputerspielPhysikalisches SystemMinkowski-MetrikDienst <Informatik>BenutzerfreundlichkeitKlasse <Mathematik>ServerMathematische LogikMereologieUmfangNichtlinearer OperatorProjektive EbeneInformationPerspektiveGruppenoperationMetrisches SystemZentrische StreckungSystemaufrufBenutzerbeteiligungGrundsätze ordnungsmäßiger DatenverarbeitungExogene VariableProgramm/QuellcodeComputeranimation
15:27
PerspektiveGruppenoperationService providerIntelDemo <Programm>VideokonferenzDemoszene <Programmierung>MultiplikationNabel <Mathematik>URLDienst <Informatik>Demoszene <Programmierung>BeanspruchungEndliche ModelltheorieMereologiePufferüberlaufOpen SourceKeller <Informatik>SoftwareschwachstelleRechter WinkelProjektive EbeneServerInformationBildgebendes VerfahrenRechenschieberCoxeter-GruppeSoftwareKlasse <Mathematik>SchlussregelWort <Informatik>AggregatzustandProgrammierumgebungGrundsätze ordnungsmäßiger DatenverarbeitungGewicht <Ausgleichsrechnung>Easter egg <Programm>Physikalisches SystemBenutzerbeteiligungInternetworkingCodeGamecontrollerRPCVektorraumVideokonferenzKonfigurationsraumGemeinsamer SpeicherPunktwolkeTwitter <Softwareplattform>Service providerMultiplikationRechenzentrumCloud ComputingWeb SiteWürfelGruppenoperationNetzadresseNamensraumPunktEinsThreadData MiningKryptologieFreewareSystemaufrufFestplatteInformationsspeicherungProgramm/QuellcodeComputeranimation
20:53
Demoszene <Programmierung>MultiplikationDemo <Programm>VideokonferenzBeamerMereologieDemoszene <Programmierung>GraphfärbungDateiverwaltungVideokonferenzYouTubeRechenschieberElektronische PublikationComputeranimation
21:51
Formation <Mathematik>VideokonferenzDemo <Programm>RechenschieberComputeranimation
22:59
DatensichtgerätLokales MinimumSichtenkonzeptDualitätstheorieHilfesystemBildschirmfensterDemoszene <Programmierung>SimulationDemo <Programm>TouchscreenWeb SiteWürfelBus <Informatik>Computeranimation
23:57
Inhalt <Mathematik>MultiplikationsoperatorBitNotebook-ComputerTypentheorieKlasse <Mathematik>InstantiierungComputeranimation
24:41
ZweiBenutzerbeteiligungServerNabel <Mathematik>Reverse EngineeringBinärcodeVideokonferenzProgramm/QuellcodeComputeranimation
25:30
BenutzerbeteiligungServerSystemprogrammBinärcodeBinärdatenTypentheorieGamecontrollerProgramm/QuellcodeComputeranimation
26:12
Diskrete-Elemente-MethodeSpielkonsoleFestplatteInstantiierungDateiverwaltungInteraktives FernsehenZahlenbereichGamecontrollerProgramm/QuellcodeComputeranimation
26:55
MAPServerBeanspruchungPhysikalisches SystemDienst <Informatik>Fahne <Mathematik>TypentheorieVerzeichnisdienstMotion CapturingAutorisierungMultiplikationsoperatorNamensraumZählenDigitales ZertifikatWürfelBitComputeranimation
28:08
BeanspruchungDeskriptive StatistikSelbst organisierendes SystemNetzadresseVariableProgrammierumgebungNamensraumDefaultServerTypentheorieGamecontrollerRechenzentrumCloud ComputingFahne <Mathematik>FehlermeldungAutorisierungProgrammAliasingMailing-ListeMultiplikationsoperatorDigitales ZertifikatNabel <Mathematik>Fortsetzung <Mathematik>Dienst <Informatik>Gebäude <Mathematik>Token-RingTouchscreenDifferenteVirtuelle MaschinePhysikalisches SystemSoftwareentwicklerWürfelArithmetisches MittelRadikal <Mathematik>Computeranimation
31:40
p-BlockMailing-ListeAbfrageNamensraumProgramm/QuellcodeComputeranimation
32:26
SchlüsselverwaltungLoginVirtuelle MaschineGroßrechnerApp <Programm>Mailing-ListeDemo <Programm>Fahne <Mathematik>Bildgebendes VerfahrenGerichteter GraphSynchronisierungFehlererkennungBeanspruchungHackerDeskriptive StatistikBenutzerbeteiligungSoftwareentwicklerMatrizenrechnungServerNabel <Mathematik>SoftwareschwachstelleNamensraumDateiverwaltungRoutingDienst <Informatik>Inhalt <Mathematik>Data MiningSkriptspracheProgrammierumgebungGamecontrollerFächer <Mathematik>AbenteuerspielComputeranimation
36:04
Public-Key-KryptosystemSchlüsselverwaltungCodierung <Programmierung>MereologieRechenschieberPunktwolkeGroßrechnerSoftwareentwicklerDemo <Programm>ProgrammierumgebungVirtuelle MaschineZweiComputeranimationProgramm/Quellcode
36:59
Verdünnung <Bildverarbeitung>Fahne <Mathematik>SynchronisierungVirtuelle MaschineKontrollstrukturProgrammGrenzschichtablösungVideokonferenzBitTouchscreenNamensraumSchlüsselverwaltungHackerFortsetzung <Mathematik>GamecontrollerSoftwareentwicklerWasserdampftafelVirtuelle MaschineGebäude <Mathematik>DiskettenlaufwerkEndliche ModelltheorieInterpretiererComputeranimation
38:48
VideokonferenzTouchscreenNeuroinformatikBildschirmfensterBaum <Mathematik>RechenschieberComputeranimation
39:38
Leistung <Physik>BildschirmfensterSichtenkonzeptZeitbereichGesetz <Physik>BildschirmmaskeSoftwareschwachstelleApp <Programm>Message-PassingComputeranimationProgramm/Quellcode
40:20
Open SourceSchlüsselverwaltungDifferenteSoftwareschwachstelleProgrammfehlerElektronisches ForumDatenbankApp <Programm>BildschirmmaskeWeb-SeiteMessage-PassingComputeranimation
41:14
TypentheorieMessage-PassingSchlüsselverwaltungBildschirmmaskeSkriptspracheRobotikMultiplikationsoperatorComputeranimation
41:56
SchlüsselverwaltungWeb-SeiteDienst <Informatik>Funktion <Mathematik>ServerBenutzerbeteiligungBinärcodeMultiplikationsoperatorProgrammierumgebungInterpretiererComputeranimation
42:58
Nabel <Mathematik>BinärcodeSchreib-Lese-KopfPunktMultiplikationsoperatorWeb-SeiteAuswahlaxiomRadikal <Mathematik>VerzeichnisdienstBitrateVideokonferenzPhysikalisches SystemComputeranimation
43:44
DateiformatBildschirmfensterInklusion <Mathematik>MereologieBitSpezifisches VolumenVerzeichnisdienstEinsOvalComputeranimation
44:31
MenütechnikBildschirmfensterGamecontrollerVerzeichnisdienstBaum <Mathematik>BitZählenVariableDebuggingDienst <Informatik>AliasingNamensraumParametersystemTypentheorieDatenverwaltungProgrammierumgebungComputeranimation
46:15
Inklusion <Mathematik>SichtenkonzeptFormale GrammatikDateiformatTouchscreenRechenschieberMultiplikationsoperatorPunktKonfiguration <Informatik>Computeranimation
46:56
MenütechnikFront-End <Software>InterpretiererBinärcodeDämon <Informatik>CASE <Informatik>Elektronische PublikationGamecontrollerSchlüsselverwaltungRadikal <Mathematik>SchnittmengeComputeranimationProgramm/Quellcode
48:22
MetadatenMehrrechnersystemBildschirmfensterRechenwerkDemoszene <Programmierung>SichtenkonzeptPunktwolkeFront-End <Software>AuswahlaxiomObjekt <Kategorie>Projektive EbeneMetadatenVirtuelle MaschineToken-RingAdditionInstallation <Informatik>MultiplikationsoperatorSystemaufrufAutorisierungProgrammierumgebungTypentheorieDienst <Informatik>DefaultMailing-ListeCloud ComputingInstantiierungMereologieDemo <Programm>ComputeranimationProgramm/Quellcode
51:14
SichtenkonzeptBildschirmfensterPay-TVHill-DifferentialgleichungMagnettrommelspeicherNormierter RaumMenütechnikOvalKonvexe HülleVerschlingungLineares zeitinvariantes SystemDienst <Informatik>ZählenVersionsverwaltungTwitter <Softwareplattform>SystemzusammenbruchProgramm/QuellcodeComputeranimation
52:13
SystemverwaltungDienst <Informatik>HilfesystemTypentheorieHecke-OperatorSpezifisches VolumenZählenMAPYouTubeSoftwaretestDisk-ArrayNetzadresseInstallation <Informatik>Kontextbezogenes SystemCloud ComputingInformationsspeicherungProzess <Informatik>Trojanisches Pferd <Informatik>MetadatenZusammenhängender GraphVarietät <Mathematik>InformationToken-RingAuswahlaxiomNabel <Mathematik>Computerunterstützte ÜbersetzungVideokonferenzPunktwolkeMailing-ListeBitrateHalbleiterspeicherFaserbündelComputeranimation
Transkript: Englisch(automatisch erzeugt)
00:00
So now, this is an interesting timing for a talk because I want to introduce an old friend of ours who is, this is the first time we're going to have a talk, but then immediately you're going to have to hop over to the next room for a workshop. So this is like the prelude to your
00:26
workshop, isn't it? Without much ado, Jay Beale, everyone. Hi, hi. Hi, hi. I'm going to try really hard to stay within range of the mic, and let's
00:43
see how I can do with that. I'm not really good at staying in one place. I kind of like to walk around. If you've seen me at many DEF CONs in the past, I like to dance. So anyway, with that said, I'm going to talk to you about attacking and defending Kubernetes. It's a topic that's become incredibly important to me and dear to me and interesting to me.
01:04
So I have a bio slide, but we all know bio slides are just big C's of text. So I have a graphical bio slide, which basically says this is stuff I've done or stuff I'm interested in. I really enjoy, this is stuff kind of going back way back to the, back in the
01:27
early, early DEF CON talks and nowadays teaching a class at Black Hat and enjoying a heck of a lot of work at Ingardians as a CTO who likes to still be a very technical CTO.
01:41
So anyway, so let's get on with what you came for. So what are we going to see today? We're going to talk about, we're going to talk about Kubernetes. I'm going to do a bunch of demos. We'll do a little bit of choose your own adventure if all my, if the tech works, the choose your own adventure part is entirely new. So the nice thing is
02:01
cutting edge fresh. The downside is, uh oh. So hopefully this is, hopefully this goes really well. So I'm going to talk about what Kubernetes is. I'm going to talk about how you, what you attack and how you attack it. And then a little bit about how you defend. We're going to do some demos. Let's just start. So the first thing I do anytime
02:21
I talk about Kubernetes is talk about what the heck this thing is for. And because, and I want to make sure I'm coming through on the mic. Am I coming through on the mic in like the back of the room? Perfect. Okay. So what the heck this thing is for? And um, and you know, people are like, I know that this is hot. I know that I should
02:40
use it. That's kind of the same way we all felt about Docker. So, you know, why, why do we use this or what's made it so big? And the, basically the, the Kubernetes came out of, in a lot of ways it came out of what Google was doing with the Borg, right? That it's an open source project that initially got released by Google and now has many,
03:00
many, many contributors. Um, but Kubernetes basically came out of this need. We know that Google, we know that Google had just a ton, has a ton of computers and a ton of workloads. To be clear, you know, it's not exactly the most transparent company all the time, but they've, they have some public statements they've made. One is back in 2014, um, they were launching 2 billion containers per week. If that were evenly
03:24
distributed, it'd be 3,300 containers per second. Um, what's a container? It's like a workload. It's a program that's going to run for a little while and do something. We don't know how long that while is. But if you're launching 3,300 a second, you can't have a human being decide which virtual machine or which host, you know, each program,
03:42
each workload is going to go on. So, um, you end up needing a tool for that. Now, what else? Um, well, they did this. They were, they were launching all of those workloads onto 2.5 million servers. At least that was their number back in 2016. And in 2016, hard drive failure rates were just right below 2%. So if every server had just one
04:04
drive, and that's a big, you know, hand waving thing, but every, if every server had one drive, that meant they had 133 drive failures a day, or one every nine minutes. So now you've got, now you've got a resiliency problem, right? Like, you wanted to guarantee, you know, you wanted to guarantee so many nines of
04:23
uptime, and yet your physical machines are going to, you know, entire physical machines are going to die on some regular basis. Like, say, RAID notwithstanding, one every nine minutes. So what features would you want if you wanted to be able to stage a ton of workloads over a ton of servers, figure out how to use the
04:44
servers at the greatest, as close to their capacity as you could, so you weren't buying lots of extra machines, and you wanted to make sure that as programs crashed, and as, and as hard drives failed, and so on, that you still had availability. So you end up with what Kubernetes is, which is
05:03
basically a software-defined data center, and we do that by container orchestrations. So what do you get? So basically, this is your feature set. This is what you end up saying. You end up saying, well, sometimes I have a lot of people checking their email on Gmail, or searching right now for the thing
05:21
that just happened in the news, and sometimes I don't have as many, so I'd like a whole lot of scaling, I'd like that horizontal scaling, I'd like to be able to say, I have four copies of that workload, no wait, I have 400, no wait, I have 50, and so on. And you get automatic bin packing, which is that, like, which workload should go on which machine, so we can keep all the machines as full as we can. And you get self-healing, like that hard drive crash thing, and you get
05:44
a whole bunch of other cool features. You get, like, automated rollouts and rollbacks, where you can basically deploy multiple versions at once, and see if one of them gets, see if one of them ends up producing more sales, or more of whatever it is you're trying to produce, or see if one's buggy, and then be able to basically route all the traffic to the one that isn't, and so
06:01
on. You get, you get the other kinds of things you would need if you're going to do microservices, so you get service discovery, and you get load balancing, and then you get orchestration of your storage, and you get secrets and configuration management. So that's kind of, that's what the purpose was. I'm gonna do a little bit more theory, so just kind of, like, what's a pod, what's
06:21
a container? So, right, we, when we're talking about Kubernetes, we're talking about containers, but we say pods a lot, and basically, pods are the smallest unit of computing Kubernetes, so all the containers in a pod are sharing an IP address. The only reason you put multiple containers in a pod is if you want them to absolutely, positively land on the same host, like
06:44
they have, they have need for very, very quick network access, or what have you. Maybe they need the same local storage. An odd thing about pods is all the containers in a pod share an IP address. Okay, so then you've got Kubernetes nodes. Simply put, a node is any system you're gonna run Kubernetes on,
07:02
and a node is any of the, any of the systems in your cluster, whether those are virtual machines or physical hardware. The nodes run a, well, first and foremost, the nodes, to be a node, you've got a, you have to have a container runtime, so you have to have something like Docker, or run C, or what have you, something that's going to basically take instructions about, take
07:22
instructions from Kubernetes about creating containers, and create them, and maybe destroy them, and so on. You also need a kubelet, and a kubelet, Kubernetes kubelet, a kubelet is just this thing that runs on every Kubernetes node, and it ties the, it ties the container runtime, or the node, into the rest of
07:43
Kubernetes. It talks to the API servers, and so on. Then you've got a kube proxy, and that thing manipulates IP tables rules to create cool networking. It gets into that a little bit more. And then services. Services are big, because in
08:03
Kubernetes, and services give me a great, give me a great occasion to kind of describe how you talk about the names of things in Kubernetes. When we're doing the attack demos, we're to be talking about this pod, and this pod, and this pod, but for the most part in Kubernetes, you don't ever think about which pod names you want, right? It's, the whole idea is that the
08:23
individual pods, right, the containers, they're very, very much supposed to be cattle. They're not supposed to have names. They're supposed to be interchangeable. I'm supposed to be able to trash them, start new ones up. I have to, I'm going for, I'm going for that. So what does that mean? I mean, if I want to talk to a pod, what do I do? You don't. I mean, you might, but rarely.
08:44
Mostly, you talk to a service, and a service is mapped to multiple pods. Think of it as an internal load balancer. Now, now, if you've got, like, why am I, why did I say that would get me into something cool? That gets me into labels, and basically, you have a, you have a load balancer. You can't say this load
09:02
balancer should be pointed at these five pod names, right? First, you're gonna have a whole lot more. They're gonna get added. Second, those pods are gonna die at any moment. Third, you're trying to go really, really fast. Remember, you know, not 3,300 containers per second being launched, right? So what do you do? You end up basically labeling everything. So you label all the pods. You give
09:22
them labels, and those labels are basically, sometimes they're by convention, sometimes they're whatever you chose, but you label the pods, and then you create a service, and you say everything that, you know, this service is gonna be a name, an IP address, a port, and everything that goes to that name and port, or IP address and port, is going to be sent to one of the
09:42
pods that have the set of labels the service specifies. That does kind of open an interesting question, like, if you are allowed to stage pods onto a cluster, could you start intercepting other people's traffic by just giving yourself the same labels that their services are looking to land track, to land things on? Okay, interesting. Last thing we talk about, last big, big, you
10:06
know, kind of simple glossary concept we need is namespaces. The namespaces are just a logical grouping for things. So we'll see those. They're often used, they're often used to separate projects or separate teams, to separate an individual, you know, maybe to separate, well, anything
10:23
that you, anything you choose. So there's some way of chopping things up, and it's important to say Kubernetes namespace is not the same as a Docker or Linux namespace, so much so that when I teach classes on this, one of the other folks from Guardians, Jared Fredes, asked if we could please call Kubernetes namespaces space names, since Docker and Linux, you know,
10:44
already have namespaces. So it was, you know, don't over, don't overload the term. So here's a quick glossary slide. I'm going to say a tiny bit more on theory, and then we'll have some fun. So, so Kubernetes is basically works in this declarative model. You don't generally say, you're going to see me
11:02
generally, like, hey, do this thing right now. But you don't generally tell Kubernetes to do a thing right now. You don't say, hey, take this, you know, take this container image, start a container on that node over there with it. Right? You tend to say, you know, it's, it's very, it's very delegating and hands off, right? So the idea is you say, Kubernetes, I, I have this program, I've
11:25
created an image, maybe I've placed that image on Docker Hub, maybe put it on some other container registry. And I'd like you to now, like, download that in, like, have some nodes, download that image. I'd like at least say five to be running it. Like, if you need more, scale up. But, you know, I want five, no
11:44
less than three, like, let's, you know, just do me a favor and get that running. And Kubernetes says, sure. And it's got a bunch of, like, control loops that basically first make that the state of the world. So you describe the state of the world you want, it, it basically spends its whole life trying to keep the state of the world where you want it, where
12:01
you've said it has to be. And then it basically loops. And it says, ah, a machine fell down. I guess I'll have to reassign those workloads. Or, ah, looks like something's not responding. I'll have to, you know, I'll have, maybe I'll kill that off and I'll start something else. So you describe the state of the world. Um, I think this is basically my last theory
12:20
and, and I hope I haven't given you too much. Um, so the kind of target components. There are a bunch of components you'll target. The big ones that you hear about are especially on the first two slides here. So you'll target the, the API server, the thing that everybody's basically doing all these, all of these, ah, HTTPS and God
12:41
forbid HTTP requests against and asking to do things. Um, and, um, the cubelet, which we said bridged the, bridged each node into the system. The container runtime, you can often think of Docker and I'll often say Docker, but there are many, there are multiple container runtimes. Um, and that container runtime's the thing that
13:01
pulls the container images from the registry, you know, like Docker Hub or what have you, and instructs the kernel to start up a container. You have, you have an etcd server that maintains all the state. And if you ever get access to that thing, ah, yeah, there's not really a lot of chopped up authorization. Ah, so if you get access to it, you kind of own all the, you own all of the, ah,
13:20
persistent, um, idea of what the system is. Um, and so that's very good for you. Ah, a Kubernetes dashboard, um, that was the thing that got, ah, as, as I recall, that was the thing that got Tesla in trouble where their dashboard was available to the internet and, ah, ah, I think was allowing anonymous access to anything. Um, but that's a web interface, as I, um, I think
13:41
it's being deprecated. Um, there's metrics components and different kinds of cluster services that'll give you good information. Those, those are your usual targets. Um, just to throw in two more, two more components that people, that it's really helpful just, you know, if you're operating Kubernetes, defending it or attacking it are useful to know. One is the controller manager. And the controller
14:02
manager, it makes it sound like it's, like the controller manager would have lots of separate controller programs. And it kind of does, but they're all just compiled into one big, you know, they're just compiled into one big program. So the controller manager has all of the controllers. The controllers are the things that basically do those loops and say, is everything the way it was supposed to be? Is everything the way it was supposed to be? Ah, something's
14:22
different. I'm supposed to have five copies of this. I've got four. I'll go talk to the next one, the scheduler, and say, excuse me, scheduler, can you find somewhere to, can you find somewhere to put this? Um, can you find a node we can put things on? We tend to target these components only through the API server. Um, so
14:40
attacks on Kubernetes, um, this is where we start to finally get to attack and defense. Attacks on Kubernetes, they start from, they start from the perspective of compromised pod. So you can attack, you can attack a Kubernetes cluster from across the network. And it's even less effective than attacking
15:01
most companies, you know, um, across the internet has been, right, lately. It's, it's, we've gotten pretty good at perimeters. Kubernetes is pretty darn good at perimeters because, for the most part, it doesn't allow any, there's nothing, there's no traffic being allowed in unless it's had, unless it's had, um, some kind of ingress defined for it. So, um, so for
15:21
the most part, you attack a Kubernetes cluster from the perspective that you are in a workload, that you're in a container and that container is compromised or otherwise under your control. So, what are the, you know, what are the, what are the, what are the ways the attacker might end up with that kind of control? One is, um, yeah, as you'll see in some, in our demos, this is the, this is the
15:41
vector I like to, I like to show. It's the, uh, the attacker found a vulnerable thing that you were hosting in Kubernetes, got remote code execution, and now they're in a container. And from that container, they may have the ability to do much more far-ranging things, because we're in an orchestration system, a very automatic one, they may be able to do far more wide-ranging
16:00
things than if they just busted that, you know, than if they just busted a, a, you know, some web server, some, or something that was in a, that was in a cloud environment or in a data, or in a, or in an on-prem data center. So, how else would the attacker have gotten in? The attacker might have just phished somebody. Brad Giesemann,
16:20
who I'll, whose name I probably just butchered and whose Twitter handle I'll have at the end of these slides, has a great slide in one of his presentations about how kubectl apply and that a URL is the new curl URL into shell, right? If you, one of the great ways to phish somebody would be if you put up a how-to article on Stack Overflow
16:42
or what have you, and the, you know, part of that article was, oh, just apply my, you know, apply my YAML from this URL. You can get anyone to apply the YAML from the URL, then you've got places for your crypto mining to go. So, and our attacker could just be some internal threat actor who's looking to escalate
17:00
their privileges. I often, you'll often hear me use the word user and attacker interchangeably on the defense side. That's not really because I think all my users are hostile. I just don't know which ones of them might be. I don't know who's being coerced. I don't know who's, you know, getting curious. I don't know who's actually got an axe to grind. And I think we all know least privilege is a good
17:21
idea at this point. So, okay, so if an attacker's gonna be in a pod, what are they gonna do with that? I'm gonna give you a few, I'm gonna give you a few things. There's even more, I'm sure. So this is kind of the general way. So they could use just the network access that they get provided by the pod to access other services. And I'll say very, very, very
17:43
often, especially in the microservice model, a Kubernetes cluster has a bunch of things that are all supposed to talk to each other. And it's actually, it can be a little difficult or at least take some work to sit down and say, okay, what should and what shouldn't? So once you're in the cluster, you have access to a whole bunch of things, many of which
18:02
might not be intended to be communicated with by anyone on the internet or anyone in the, anyone that the service that you've pwned is trying to, is trying to serve. So, what else? So the attacker could attack other containers that are in their same pod.
18:21
Multiple containers in a pod, they share, well, I told you, they share an IP address, they share a local host. So it's a cool Linux namespace thing, which we won't get into unless somebody asks me later on. What else? So the attacker could make requests to the API server or to a kubelet and ask those to maybe run
18:41
commands in another pod, maybe even interactively, to start a new pod that has privilege. And that privilege could be all kinds. It could be a privileged pod, which means it'll start on the host and have access to system devices and be able to, among other things, maybe mount the hard drive of the node. But there's all kinds of
19:00
privilege you can get out of starting new pods, and I'll show you that. They might gather secrets that Kubernetes provides to pods, because Kubernetes has a whole thing where it needs to provide configurations to pods, because you're really supposed to make those pods come up with a static image. So the idea is the pod starts with a static, totally
19:21
replaceable image and pulls whatever information it needs from elsewhere. You might connect to the Kubernetes dashboard to perform actions. Maybe it wasn't accessible from outside the cluster, gosh hope. What else? The attacker might find a way to interact with the etcd server and change the cluster state. A nice way
19:41
if there are network access control rules, if there are network policies in place that are getting in your way, if you can talk to the thing that stores them, yeah, you could potentially change them. And the other one, the one that's really fun to me for anything that's either in, you know, it's either in a cloud
20:02
provider or even in an on-prem cloud, is that the attacker might interact with the cloud service provider. And because many workloads will, one way or another, and I'll show this to you too, will have privilege on the cloud provider. So I'm going to show you a bunch of demos. And these demos are real.
20:23
They're often things that we do in classes I teach. They're also, I'm going to tell you later on, we have an open source project, and it's an intentionally vulnerable Kubernetes cluster. Call it, like, damn vulnerable Kubernetes cluster. This one's called Bust-A-Kube. We've got an Easter egg Bust-A-Move
20:40
video that you can find if you go to the bustacube.com website. So some of the things I'm showing you are basically available as scenarios that you can play in this open source free damn vulnerable Kubernetes cluster. So I've done a thing where I've made my videos part of the slide, and so now I'm going to see if that actually works.
21:03
Please work. And if it doesn't, we'll have to use YouTube one way or another, or something else. Okay. Color me very embarrassed.
21:23
Let's just see if this one's the one. You should always test your demos, even when you record them. So I tell you what, we're going to find another way to do it. And I am going to, let's see, I'm
21:42
going to unplug from the projector for a second, because that means I'm going to go and look at my own file system, and I'd like to not share it.
22:21
And here we go. Just embed the videos in the slides, they said. It'll be fun, they said. Anybody old enough to have played Warcraft, and you click on the guy, and he's like, what? And you click on
22:41
him again, and he says something else, and he says, you know, join the army, he said. See the world, he said. Okay, so here's our demo. And hopefully entering, oh, well, that's not a demo.
23:03
Is that a demo? That's not a demo either. Anybody seen my screen? There we go. Let's just hit this on mirror and
23:21
call it a day. Okay. Cool. So, that's not the right, don't mind me, I'm a, I'm clearly a noob. I may need
23:43
alcohol. Let's, so this is, this is one of the, this is one of the bust-a-coob, this is one of the bust-a- coob scenarios. So you go up and you see a WordPress site. And the WordPress site, I will tell you, is hosted in, is hosted in Kubernetes. And you fire
24:03
up maybe Durrbuster to go looking for hidden content, and Durrbuster takes a little bit of time, but we can run it, and let's go a little faster. And what we find is backdoor.php. And so when we surf to backdoor.php, this is what happens. I get this thing
24:21
where basically this WordPress instance, like so many other WordPress instances, has been pre-compromised for your convenience. So I type ID, and it says you're dub-dub-dub data, and hopefully I shut down Durrbuster because it always chews a ton of resources. Something we always say in class,
24:40
they're like, my laptop's running slow, have you turned off Durrbuster? No, it's 500 requests a second. Well, that'll, that'll do something. So I'm going to stand up a quick little, a quick little web server in Kali, because I found a, I found that reverse shell. So I'm going to stand up a little web server, and my purpose
25:01
of the web server is to be able to, you know, hand a, to be able to hand a meterpreter, or a, you know, or if I wanted a, you know, whatever, some other kind of reverse shell. But I'm going to go for a meterpreter, and so I just put in here like, hey, could you please pull this meterpreter binary I've
25:20
created from my web server, and while you're at it, make it executable, and while you're at it, run it. And I like to fast forward my videos, because who doesn't? And go look at the, you go look at the web server, and Mrs. Bin has been pulled, the meterpreter binary, but also a copy of kube control, the, the command line
25:40
utility that you, that you use to, to do damage, I mean, to administer or use a Kubernetes cluster. So I'm going to pull kube control, and the other thing I'm going to do is just take, make sure that, that it's landed here. So here's Mrs. Bin, my meterpreter binary, here's kube control, cool.
26:06
And now if, if recorded J will type a little faster, we'll run the meterpreter binary. And I'm going to go over to my metasploit console and actually be like, oh wait, I have an incoming
26:21
meterpreter that needs to connect to the console, I should probably start the console. This ordering is just great, huh? But that's often how you work. So I'll interact with that session, and I'm now in a pod. I know it doesn't look like it. It looks like an ordinary metasploit,
26:41
you know, pwned metasploit instance. So, but I see my copy of kube control, I'm happy, and when I run mount, I get a tremendous number of file systems. So I'm not looking at one hard disk or what have you. And a bunch of them put there by Docker and by Kubernetes. And so the one that's really interesting to me
27:00
is this one that says run secrets Kubernetes service account, IO service account. As soon as I see that directory mounted, I know that I'm going to have some level of access to the API server, and I'm happy. So I'm going to go, and first I'm just going to go and collect my flag. I've built this scenario as a capture the flag. So let's look at
27:21
our little flag, and it says, hey, you've found the server's backdoor. You might notice this isn't your usual Linux system. You're in a cluster, my friend. It's time to bust a kube. This container doesn't have any more flags. Can you find your way to another pod? Hint, hint. So cool thing in Kubernetes is, like I said, one of the things you can do is see if you can move to another workload. So you're
27:41
in a container. Can you move to another one? So I'm going to see what my service account can do. I'm going to go and look in that directory, in that run secrets Kubernetes IO service count directory. If recorded J will type just a little bit faster, and I look at that namespace, and it says you're in the marketing namespace.
28:01
So that directory has a token. It has the certificate authorities public cert, and it has a name, it has a description of what namespace you're in. And that namespace, like I said, is kind of a way of organizing. So we're in the marketing namespace. We'll come back to that later. And then I'm also going to go and look the environment variables that get pushed into every pod.
28:20
And my environment variables will tell me where to find the Kubernetes API server. So there's this assumption in Kubernetes that every workload will need to ask Kubernetes to do things. I don't know if that's really the greatest assumption, but I think let's just call it not the strongest of defaults. So I've got the API server's IP address and port,
28:45
and I'll clear my screen, and I'm going to set up a quick little kube control where I'm saying alias kube control the path to kube control, the token pasted in from this long path to the token, the certificate authority path, the namespace, and the
29:00
server that I just got. And so every time I type kube control, all that stuff will be passed in. So let's just kind of skip and let's see what did I do. So I said let's do a quick get pods and get pods is my my is honestly my go to just figure out if if my setup is working. And so it says oh there's two
29:20
pods. You're the WordPress one and I ran host name and it says that's the one you're in. And then the WordPress MySQL one. And that's another one. So let me see if I can move over to it because that was what the hint said. So I'll do kube control exec IT looks a whole lot like Docker exec IT huh and what program I'd like to add to that container. I'm going to add a shell.
29:40
I get this kind of error but I'm in the I'm in the container. I've just moved over and I've probably moved over to a different host. Like I'm probably somewhere else in the data center now. So there's another flag and
30:07
we'll
30:49
go
31:09
privilege wider than just what one thing had. So I did get nodes, and I found out the IP address for node. Here's a name, but this node has an IP address. This is, this one
31:21
is, this one's running in a cloud provider. And there's a cool thing. So you can talk to the kubelet, and this is, thank gosh, finally not a default, but some clusters will have it. You can talk to the kubelet on, it's got a couple ports, 10.2.50. I'm going to go and say, can you give me a list of running pods? And it gives me JSON,
31:42
because everything in Kubernetes is JSON and YAML. So I'll take that, and I'll run that again, and I apologize for this big block here. So I'll run that again, and I'll add some JQ, JSON query, really good tool to learn. And I'm going to say, give me all the pods, kind of better formatted, give me their name
32:02
and their namespace. And what I'm looking for is to see if there's a pod that's in, that's in another namespace besides the marketing one. And I quickly find one, there's a dev pod. And that dev pod, so Kubernetes loves to tell you about
32:26
anything you want, and that's really, really nice as an attacker, especially if you can script it. So I've got dev sync, I've got dev web, these are two containers that are running in the same pod. I've got the image they came from, so they're up on Docker Hub, you could pull them yourself if you wanted.
32:42
And as an attacker, that's often what you're doing, is basically saying, well, what workloads are running, and can go look in container images and see if there are any secrets or anything that would be really helpful to me. It's very, very nice of the defenders to hand you descriptions of their workloads. So I'm going to look at that, I'm going to run another command, and this is, I did running pods, I'm going to tell the
33:02
kubelet, hey, would you mind just executing a command for me? So I'm going to pass it run, and then the namespace, the pod, and the container name. And I'll do command equals id. And I get an error code that says basically no, either there's no shell in there, or I'm not allowed. I will tell you that there's no
33:21
shell in there. So, but remember, there were two containers in that pod. So I saw dev sync and dev web, and that's a really common thing you see outside of containerized environments, like you have a web server, and you have something that's, you know, something that's there to pull it content in from wherever that content's being created or stored. So I was hoping I could get to dev sync,
33:43
because it'll have that, it'll have the credentials for pulling content, it'll have an SSH key. I couldn't, so let me try dev web. So dev web, I can run commands in. I ran id, it says, yeah, that thing's even running as root, very
34:01
nice. And so let's try a different command. Let's look at the file system. Oh, look, another flag. I like flags. I love doing CTFs, and I love doing CTFs, actually, like, first doing them as an attacker, and then going and basically doing them as a defender, and saying, how many, how much of the
34:23
attack can I break with proactive stuff, with stuff that you could have thought of before you knew that you had the vulnerability? So this thing says, hey, you're not in the marketing department anymore, you're in the development department's name space, so can you go take over the development department's non-containerized machines? And they're big fans of the matrix, so it's named mainframe, named for the machine's
34:42
mainframe that may or may not have been in the second and third movie, depending on your religious beliefs. There are many people I know who acknowledge the truth, as they believe it, that there were no second and third matrix movies. Only quests for more money. So here we go. So we've got, we're trying to do get pods.
35:05
So I am, I'm in one pod, asking the kubelet on another host if it and I'm asking that pod to run kube control commands. I find out, okay, dev app isn't allowed to list pods, but it's kind of cool.
35:24
I now know that I've got a third service account. I've got, I've got the dev dev app one. So I'll try get secrets. Let's see what secrets are available. Oh, nice, there's a mainframe login and an SSH key. Well, the two pods in there, the two containers in there are going to likely have access to the same secret.
35:44
They've been placed in the same place. They don't necessarily need it in dev web, but what the hell. Let's go and ask for a copy of that secret. And I promise the next demo is hackers themed and choose your own adventure. So we're going to have some fun. So here's our SSH key.
36:04
And you're like, that doesn't look like an SSH key. And that's because it's a secret. It's, it's opaquely encoded. It's base 64 encoded. So we can decode it. And boom, there's our SSH private key. And we can do the same thing with the other
36:21
secret here. So we can find out basically what is it used for. So I'm just going to fast forward a little bit and tell you that it's the, that it's a user, it's a user and a host in AWS, but not part of the cluster. So now we could SSH into that. And we're in the developer's mainframe. And it says, hey, you made it to the machine
36:43
mainframe. Can you, for extra credit, can you get credentials to take over the whole cloud environment? And, and we can do that. But I'm going to set the rest of that aside because we do that in another demo. So, so let's see. Let me go back to the slides for a second.
37:03
And here was our, and I'm sure that I'll be, because of the vagaries of having the videos, you know, in a separate program, I'm sure I'm going to be doing a little bit of going back and forth and having the whole screen do this. So here's our attack path. We got an interpreter in the WordPress container. We moved into my SQL container with the
37:21
kube control exec. We used the my SQL container's totally unfettered network access inside the cluster to reach a kubelet that we wouldn't have been able to reach from outside the cluster on the master node. And that kubelet let us basically take a look at a, take a look at, take a look at run commands in another container, in another name space.
37:42
And from that other container in that other name space, we found, we figured out there was a, there were secrets. There was an SSH key, so we got that SSH key. We authenticated to the high value developer machine and later on we ended up getting to take control of the AWS account and we're going to skip that right now because we're going to do that
38:00
in a much more fun way. So we're going to play hacker's choose your own adventure. And I'm going to drink some water for a second because, whoa. So, I like the movie Hackers. It's a, it's a fun movie. It is an oldie but goodie. It is thoroughly realistic. We all know that whenever we're hacking
38:23
we, we see like strange translucent 3D models and, and you know, of buildings and we fly through a city and, and it's really cool. And all of that was on, was clearly done with the technology that had Apple twos and floppy disks. So, I mean, I remember 3D rendering back
38:42
then. I wasn't doing it. I was watching people do it and yeah, no, not on that. So here we go. So we're going to start here and, and again, this is where I have embedded video. So we will, unless I click on that and it works. Let's see. Oh, hey, look at that.
39:03
Now, can we full screen that? Is there a way? No. So we're going back to oldie but goodie and here we are. So first, this is going to look a little familiar. So we've got a, we've got, what's that?
39:24
Oh, well, you're on the slide and I'm on the, and I'm on the video. Okay, boom and switch, come on. I'm going to a Windows computer soon.
39:42
And that's a lot for me to say, trust me. So anyway, here we go. So we've got the, we've got the Gibson control room. We have Penn from Penn and Teller who's apparently a techno weenie, according to Ben from Short Circuit,
40:01
who somehow was playing an Indian and talking about Mr. Johnny Five. But Mr. Johnny Five here was the plague. And and we have this nice little form that's called the Gibson control room. And the Gibson control room clearly lets you like leave a message. So it's based on a, it's based on a actual on a Kubernetes documentation vulnerable app.
40:24
They didn't mean for it to be vulnerable. But it's only a little vulnerable. So I've added a vulnerability to that. So the, so what we've got is I look at the source on that page, and I find that the form is basically being provided by some JavaScript. So if I go look at that JavaScript, what happens when you submit the form
40:46
is you're referencing guestbook.php, you're sending to guestbook.php, you're saying, hey, command set key messages value this. Okay, really? You're gonna let me, you're gonna let me decide what, what key I'm gonna set?
41:01
You're not just gonna have me set whatever key, you know, was meant to be set by this form? So that was their bug. And what I've added to it is I have something that actually runs code out of a different Redis key in this, you know, back in Redis database. So we go and take the Gibson control room. And if we were to type a message, you know, it shows up. This is the intended, this is the intended purpose.
41:23
But instead, we're going to paste that same thing that happens when you submit the form, except instead of the message key, we're gonna try a different key. Or we'll try a, let's see how it works. We're gonna hit foo and we'll put in just some key, some random key.
41:41
Oh, cool. Okay, so JavaScript's probably updating that key. Let's try going to status PHP. So I'm asking you to imagine that we did a Durrbuster or looked at robots.txt. We found a status PHP. I'm not doing it because, well, I'd like to spend our time on Kubernetes. So I go to the status PHP page, it says, hey, no command to run. I'm gonna set the Redis key, the Redis command key to this, so
42:03
that you can reload the page and that will get run. So if I reload the page, I get the output of, you know, I get the output of crawling that web server. It says all services are operational here at, what was it, Ellington Mineral, or Ellington Mineral, Ellington, I think. So, well, as long as there's a command key, let's go and set it.
42:23
And I've set my command key now to environment, to run environment, like we did before to find a Kubernetes cluster, to find the Kubernetes API server, and here we go. So we know that once we're in a workload, if we reach out to 10.96.01,
42:40
we'll get to the Kubernetes API server. So I'm gonna go a little further, kind of similar to last time. I'm gonna curl in, I'm gonna tell this thing, excuse me, would you just pull in a interpreter binary I created and connect back to me? So I have it pull it in, I set that key, and now when I run status, the page hangs. I love when pages hang, when I've just given them binaries,
43:03
because it generally means the binary is running successfully. So if I go over to my terminal where I've got a Metasploit handler waiting this time, I find there's a session waiting for me. And the first thing I'm gonna do before I start interacting with a shell there, I'm gonna upload a copy of kubectl, which I've stored on this Kali system in this directory.
43:25
So I'll upload kubectl, and I'm gonna upload some YAML, which we'll get to later. So I've uploaded some YAML, which we may or may not get to later, depending on your choices, and I've asked for a shell, hit PWD. Looks like my video's cut off by one character.
43:41
I hit ID, it says you're www data user. I try SSHing, and, or I, well I don't know what I tried, because it's, because that was the end of that part. And now, we're gonna do this part so that we're a little bit gentler. I'm gonna ask you, so, do you wanna try, the YAML files that I put up were,
44:03
were ones that would mount a volume on the, that would try to mount a volume on the host. Like they'd try to mount the, the host SOM directory. The old, you know, Docker, you know, start a container mounting the host. So, do you want us to start the host mounting pod, and see if we can take over, see if we can take over a node,
44:22
or do you want us to exec into another pod? I heard a host, I heard the host mount first, so we'll try that. So, let's see. Boom, and boom, here we are.
44:43
We're in that front end, and we'll set kubectl executable, so we can run it. We'll go and set up our kubectl, we'll try to run kubectl apply, realize, wait, we've got to set all those parameters we set before. Sometimes you get lucky, and so I'm gonna take that same directory,
45:06
that same run secrets kubernetes io service account that's being mounted into the pod, I'll set a variable called dir to that, export dir to that, just to make this a little bit easier to type, a little more manageable. So, if I look at the dir, got the ca cert namespace token, just like before.
45:22
Let's go and run alias kubectl is gonna mean that, same way we did before. So, server's out on 109601, that was what we found from that environment variable. And now, let's look at our alias.
45:42
That's that token being pasted in. So, kubectl get pods, cool, we've got a front ends, we've got front end pods, we've got a redis master, we've got an Apache status, that thing we saw before, we're in the front end, so let's try launching a pod. Kubectl apply, I like it as kubectl attack, attack pod, and it says, sorry,
46:07
your service count front end isn't allowed to create resources, wah, wah. So, and this is where I get to my slide, so we'll do that real quick.
46:28
I was considering for this one, oh, damn it, let's just keep the screen mirrored. So, I was considering for this one, doing a spandex quote from the movie, and I decided that wasn't a sentiment I wanted to voice.
46:47
So, I see that I am, I see that I'm going to make sure that I'm moving on time. So, at this point, we'll go back to our original, we'll go back to the other thing we could do. So, our two options, where do we want to launch a pod, or exec in the back end? So, let's exec in the back end, and, or let's try execing into another pod.
47:05
So, we'll go, like we did the other, like we did in our other demo, try execing into a pod, keep wishing we had a better terminal, kubectl get pods, I see that there's a redis, there's a redis, that redis setup.
47:21
Remember, I was pushing redis keys. I might try to investigate the redis and see what else, now that I've got direct, I've got more direct access to it, but, because redis is very often, well, almost always unauthenticated. But, let's go into the redis pod. So, I'm rooting there, I'm redis master,
47:40
I'll grab a copy of the interpreter binary I've been using, just because that's a very convenient way of putting files into something. And, I'll back around that, and I'm going to, I'm going to switch over, now that I've got the, I've got the redis master, that pod running my interpreter. So, I'm going to switch over and catch that session.
48:03
And so, in that session, I'll go and upload the same kubectl that I did before. I'll also upload that attack pod that I just used before. And, I'll also attack, add something called attack demon set, just in case we need it.
48:22
So, now, we get another choice. So, you're in the back end. Do you want to try host mounting pods, or do you want to check out the cloud APIs? Cloud APIs, cool. Okay, so, here we are.
48:41
And, I'm going to show you this in two ways. And, can I get a rough time check? How much time will you let me be late? Oh, less than five. Okay, so, this is where I normally kick on the speed control. So, okay, so, we're in a pod, we're in that redis master pod.
49:00
And, we say, hey, I'd like to talk to the metadata API. So, every cloud provider gives you 169254, 169254. That part's common. What the calls are that you make against it, they document wonderfully. And, you read the documentation. So, this is on, this is on GCP. So, I'm going to go and say, can you tell me what the project ID is?
49:21
I need that project ID to do API calls. But, then, the more interesting one is usually where you say, would you mind, in addition to the project ID, which I'm going to set into a variable, would you mind telling me what service accounts are assigned to the machine I'm on, to the machine this pod's running on? And, there are two, but default is mapped to that.
49:42
So, I'm just going to use default because it's easier typing. And, I'm going to go and say, okay, for that service account, would you do me a favor and just give me a bearer token that I can use to act as that service account? And, it says, sure, here's your token. It expires in, like, a half an hour or an hour,
50:01
but I've got a token. So, I'll set a variable to that. And, now, what I'm going to do is say, let's go, and I've set that token to that. And, I mean, I need to refresh that token. So, most of my commands will pull that token over and over, every command. So, I'm going to use that token, type out authorization bearer token,
50:20
where the token's provided. Say, I'll take JSON, and now I'll say, hey, for this project, I'd like a list of buckets. And, it says, oh, there's only one. Okay, well, what's in that bucket? And so, I basically say, give me a list of objects. And, the way I do that is it's like slash B
50:40
for a list of buckets, then it's the slash B, that bucket, slash O for a list of objects. And then, for those objects, I can run through them. Now, on this one, I don't find anything that's all that useful. I don't find what I find in some cloud environments. I'm going to show you in another demo, somehow.
51:02
But, I do find a kubian V. So, I can pull, in a metadata instance, for this cluster that was set by the installer. And, when I do that, what I get out of it is, among other things, a couple service count tokens for the cluster. And, that ends up leading, which I won't now show you,
51:22
but that ends up leading to win. And so, just for the end of our, for the end of that, that then leads to our crash and burn. So, if Ming will let me have one more minute, I will show you a cooler version of that.
51:42
We have a tool that we're, that we keep soft releasing, but never kind of, like, putting up on Twitter. So, it's called Paratties. And, Paratties will, if I, if I actually, just,
52:01
I'm just nervous, so, it's in here, but somehow I'm, let's do this. So, Paratties is basically a remote access Trojan
52:22
for Kubernetes. You run this, and it does all kinds of things. And, I'm gonna start this from the beginning, but it is under two minutes, and then I will get the heck off the stage, and tell you that you should ask me for stickers. So, it's got a bunch of things. One of those I had said, okay, 13,
52:41
go get the IAM credentials from GCP. The same thing we just did, get the cloud service account. And, then, come on Jay, type faster, highlight faster. So, that's for that same one I just got. And now, let's take that, and let's pull, let's go looking in GCS, in the storage buckets,
53:02
for service count tokens. Oh look, this installer, this one was cops, in left, left service count tokens. Therefore, all the components, including a cluster admin, which would be perfectly useful. And so, that's the like, basically, that's probably roughly what happened to Capital One recently, where it was,
53:22
talk to the metadata API, get your cloud account, use that to go rating for information or access. So, anyway, so now once I've done that, Paratties will basically store every count it gets, and then you basically say, okay, I'm gonna switch context.
53:42
And you go and you go through and you collect all of the service count tokens, then you basically have this big old store, that's your loot, and it keeps it in memory. And you run around and basically get more access and get more access. We've even got a, this video will be up on YouTube, but we've even got something that basically takes over the nodes.
54:01
If it's allowed to mount a host path volume, it'll go and add a cron job that will send a netcat shell back to your, back to the IP address of your choice. So once we have this cluster admin, we're allowed to, we can go and do everything which included like, hey, now show me all the secrets. And all the secrets includes all of the tokens, all of the service counts for every component.
54:23
And Paratties will not just grab, will not just list these and get them, but they'll also store them and let you use them. So it's a fun little pen test tool, and it's up on GitHub, we'd love some help, we'd love some help producing it. So it's an ingredients tool, we do a ton of Kubernetes work in Guardians, and I will now get off the stage and take questions all the way out in the hallway.
54:42
So if you see me, please ask me for a sticker. Okay, thank you. Yeah, you got your bundle.