Exploring Database Containers
This is a modal window.
Das Video konnte nicht geladen werden, da entweder ein Server- oder Netzwerkfehler auftrat oder das Format nicht unterstützt wird.
Formale Metadaten
| Titel |
| |
| Serientitel | ||
| Anzahl der Teile | 542 | |
| Autor | ||
| Mitwirkende | ||
| Lizenz | CC-Namensnennung 2.0 Belgien: Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen. | |
| Identifikatoren | 10.5446/61813 (DOI) | |
| Herausgeber | ||
| Erscheinungsjahr | ||
| Sprache |
Inhaltliche Metadaten
| Fachgebiet | ||
| Genre | ||
| Abstract |
|
00:00
MultiplikationsoperatorEreignishorizontDatenbankComputeranimation
00:38
GoogolDatenbankServerVolumenArchitektur <Informatik>PunktwolkeNotebook-ComputerStandardabweichungSoftwareRechenwerkProgrammierumgebungPhysikalisches SystemLaufzeitfehlerCodeSocketDemo <Programm>SpezialrechnerSoftwareRechenwerkDämon <Informatik>DatenbankClientOpen SourceEndliche ModelltheorieSpezifisches VolumenServerLaufzeitfehlerPunktwolkeMereologieSystemplattformProgrammierungEinfache GenauigkeitKundendatenbankKartesische KoordinatenApp <Programm>AppletDifferenteWort <Informatik>MultiplikationComputerarchitekturZusammenhängender GraphTwitter <Softwareplattform>Computeranimation
04:36
Architektur <Informatik>PunktwolkeMultiplikationGebäude <Mathematik>Prozess <Informatik>Dämon <Informatik>Analytische FortsetzungCachingCloud ComputingDienst <Informatik>Kontinuierliche IntegrationBildgebendes VerfahrenKartesische KoordinatenElektronische PublikationClientGrenzschichtablösungMultiplikationsoperatorZusammenhängender GraphDatenflussCASE <Informatik>KonfigurationsdatenbankVirtuelle MaschineComputeranimation
08:31
Virtuelle MaschineComputeranimation
08:59
ServerInformationsspeicherungDokumentenserverSpezialrechnerGruppenoperationSchlüsselverwaltungDatenbankTabelleEindeutigkeitLesen <Datenverarbeitung>QuellcodeBildgebendes VerfahrenKartesische KoordinatenServerVersionsverwaltungProgrammierumgebungVirtuelle MaschineEinfache GenauigkeitAggregatzustandMereologieOpen SourceDatenbankRelationale DatenbankSpezifisches VolumenWurzel <Mathematik>Computeranimation
14:55
VolumenQuellcodeSpezialrechnerSicherungskopieSpeicherabzugBefehlsprozessorDatenbankWiederherstellung <Informatik>Migration <Informatik>InformationsspeicherungPhysikalisches SystemROM <Informatik>Inverser LimesServerVerzeichnisdienstSicherungskopieKonfigurationsraumSpeicherabzugCASE <Informatik>PhysikalismusSpezifisches VolumenStellenringProzess <Informatik>MultiplikationsoperatorOpen SourceKartesische KoordinatenMigration <Informatik>AggregatzustandKonfiguration <Informatik>DatenbankAutomatische HandlungsplanungURLInformationsspeicherungSystemprogrammPerpetuum mobileZentrische StreckungWiederherstellung <Informatik>Elektronische PublikationTypentheorieSystem FPhysikalisches SystemPersönliche IdentifikationsnummerSystemzusammenbruchComputeranimation
20:51
Mini-DiscComputervirusDatenbankKonfigurationsraumInformationsspeicherungHalbleiterspeicherSpezifisches VolumenVirtualisierungSoftwareStellenringInverser LimesVisualisierungNichtlinearer OperatorOpen SourceInstantiierungMereologieSoundverarbeitungSpieltheorieGenerizitätSprachsyntheseAbfrageEinflussgrößeVorlesung/Konferenz
27:12
Vorlesung/KonferenzFlussdiagramm
Transkript: Englisch(automatisch erzeugt)
00:05
Hi everyone. How are you? Who is hosting the weekend? Good? That's nice. I'm happy to be here. It's my first time in Europe and it's the first time that I will talk in English for the first event in person.
00:22
This is pretty nice. My name is Edet Bugja. I am a technology evangelist at Percona. This is a very basic and friendly introduction about databases and containers. About me, I am from Peru in South America. I am working six months in Percona.
00:46
It's an open source company. We create open source databases free. I am a global woman teachmaker. I was nominated as a Docker captain last year and I am a database and container enthusiast.
01:04
You can follow me on Twitter and LinkedIn. I used to post about containers, Kubernetes, open source. For the agenda today, we are going to see about containers.
01:24
We will see Docker architecture. We will see the workflow between the components of Docker. We are going to have two examples of how we run in a single Percona Server MySQL container. We are going to run multiple containers for Percona Server MySQL.
01:44
We will see the Docker volume, how this is important in this world of databases and containers. We will see backups, restores of databases and best practices. Let's start it. What's a container? How many of you know Word with Docker?
02:08
Yeah, a lot. Okay, that's nice. Docker or do you use other tools? Yeah, there are different kinds of tools for containerized application, right? But a container is like a single unit, lightweight unit of software that package everything that you need for your application.
02:30
When we run application, when we build application, we know that we need a lot of package, right? If you are running, for example, if you are building a Java application, you need libraries, dependencies, many things to run your application.
02:43
So everything has to be containerized in a single unit of software and this is going to be isolated for other things, like your infrastructure. And the good thing is that your container can run on different platforms, in your laptop, in your server, in your cloud.
03:02
With this, we end with a problem that we have when we say, hey, your program runs, yes, this works just on my computer. But no, it has to run in different platforms. We don't need to have this problem to dependencies and other kind of things when we test our application in other platforms.
03:26
There are different tools, as I say, for containerization. We have like container runtime interface, for example. We have container D and we have Docker that is the tool that we are going to focus now.
03:43
All these tools are also in the Cloud Native Computing Foundation ecosystem. If you see the landscape, you will see a lot of tools there. There is a part for containerization and there are more than three. There is a lot of tools for that.
04:03
The Docker architecture, it works like a client server model. We have the Docker daemon, which is going to process all the commands. It's going to start to listen to the client always and the client is going to send a request to the daemon through the REST app.
04:27
With this model, the Docker daemon can also manage network containers, images, and Docker volumes. If we go more in detail, we will see that we have the client, the daemon that is also called the engine of Docker,
04:44
and we have another component that could be your Docker registry, the public, which is Docker Hub, where all the images, official images, are published and also we can have our own private registry in case we don't want to share it with the public.
05:01
So in this case, this is the flow of a component. For example, if we do a pull, we are going to try to bring the image from the Docker Hub into the Docker daemon cache. If the Docker daemon doesn't find the image in cache, it's going to bring it from the Docker Hub.
05:23
But if this is in cache, it's going to take just that and start to process. The same with Docker build. Docker build is going to, when we run Docker build from the client, the Docker daemon will try to take a Docker file. A Docker file is a recipe with a lot of instructions where we put all the commands to run our application and deploy it.
05:47
So the Docker daemon is going to take the Docker file and build it, build the image, and if we want, we can also run it. We run, we will create a container. The container is our application that is already alive and is ready to make connections or petitions.
06:10
One more thing here is that we can have everything in our host or we can have clients, remote clients that could make petitions to the Docker daemon.
06:23
So container benefits. There are pros and cons, but now I'm going to focus on these benefits that the containers give us. So one of these is we can reduce costs with this because we can run several containers in a single infrastructure.
06:47
The infrastructure that we have because of the technology of containers is different than the virtualization. In virtualization, we use the hypervisor, and when you create virtual machines, it consumes more resources from your infrastructure.
07:06
But when you use containers, it's very different. You are using the technology of containers to make it possible to run a lot of containers in a single machine. So for that reason, it's possible to reduce costs. Also, the containers are very friendly with continuous integration and continuous delivery process.
07:26
If you have like a big application, a monolithic application, and you want to run containers, you want to integrate it in the DevOps process, this is going to be hard. We have to work like microservices to make each service as a container and include it in the continuous integration, continuous delivery process.
07:51
It's easy when we build our application over a container. It's easy to kill it, it's easy to create it again, it's easy to fail, and the process is fast.
08:05
Another benefit is the multicloud compatibility with the time several companies try to migrate to a hybrid cloud. They just don't want to have everything on premise. They also want to scale. They want to grow.
08:20
So for a reason, they opt for cloud, and containers fit very good in this. You can install Docker. I know you did it. You can just use your distro.
08:40
You use Debian, CentOS, everything, so you can go to the official Docker documentation and easily look all the steps. When you install this, it will install the Docker client, the Docker daemon, and other tools that you will need to use Docker in your local machine.
09:01
We already talked about containers, right? But this talk is about exploring databases on containers. We are going to talk about MySQL, which is a disk-based relational database. We know that it's a database. And to run MySQL on containers, we need to understand how volumes work.
09:23
Because the most important thing running databases on containers is the data. If we lost the data, we lost everything. For the next slides, we are going to focus on this part.
09:40
We will use the image of Percona server for MySQL. Percona server for MySQL is open source. It's like MySQL, but with more nice things. You can use it. It's open source. It's in Docker Hub. So we will use this image, and we will create a Docker container.
10:03
We will see how it works without volumes. We will see the layers in Docker, and then we will create a persistent volume, and we will see how it changes in the layers of Docker. So just here to see that if you want to have an image, it's necessary to have a Dockerfile.
10:26
Who did a Dockerfile before by yourself? Yeah, that's good. A Dockerfile is a recipe where you will put everything for your application. So you need this to create an image. Then you need the image to create your Docker container.
10:43
There are three essential steps here to remember how Docker works. Okay. We will run a single Percona server MySQL container. We will use Docker run to create the image, right?
11:06
No. We don't use Docker run to create the image. We use Docker run to create a container. So we use this to create a container, so we will do dash d to say run this container in the background.
11:25
I want to use the terminal, and I will call it Percona server one. I will pass it like the variable environment for the root. This is not a good practice here.
11:40
This is just to show how we are going to create a container, and we will use this official Percona server for MySQL. With this, I am creating a container, right? I'm creating a container with this one, okay? So if we go to Docker image ls, this is going to pull the image of Percona server, and then it will create the container.
12:05
That command is going to do two things. It's going to bring the image from the official Docker pack, and it's going to create a container. So if we see Docker container ps, our container is up.
12:22
After we have the database, we need to update. We will add databases, we will add data, we will change registers, we will have transactions, many things that we can do like a regular database.
12:42
Okay, if we run a single Percona server in MySQL container with no volume, how it works? In layers. If we see this in green, there are layers from Percona, Percona server image. This is the image that we pull it, that we can change. This is just React only.
13:02
We can change this, but on top of that, it's going to be created a layer, a new layer. This layer is React only, I can add data. This layer is the one that will contain all the things that I am doing in Docker on that image, on that container.
13:21
I added a new database, yes. I created new registries, I deleted, I added transactions, all this is going to save it here. But what happens if I don't have volume? My container is ephemeral, right? It will die, it will crash, my machine will crash, and all my data is going to be lost.
13:40
I will lose all the data. We will see how it works with multiple containers. To run multiple containers with the same image, if we see this is the same image, the same version of the image, we will just change the name of this container. Also, we can change another thing because this is a database, right?
14:02
What thing we can change it? They run in a port, right? In which port my SQL used to run? Yeah, so I need to change the port for the other container to avoid the conflict.
14:20
Okay. How it works in layers? The same. We will use the same layer, we will use the same layer for Percona, Percona server, which we can modify, but on top of that we are going to have two layers more. One of the first container that I created and the second for the other that I can add.
14:43
I can add data, I can change things, but once again, if I don't have volume, this is going to die. But this is how it works if we want to create an application when it doesn't matter if we save the state of this application.
15:02
Volumes open the door for a stateful application for running efficiently in Docker. This is important. Percease data in databases is really important for this kind of application because sometimes we think that like Kubernetes and other things was created for a stateless application,
15:21
but now we have options to use stateful applications on containers and this is one of the reasons. Create volumes. So it's pretty easy to create volume. We can create a volume just with dash b or dash dash volume and we can say it, we can create a Docker volume with Docker run and detach.
15:43
We will call it Percona server, the same process. And when we say dash b, we are saying, okay, this will be my volume in a host in my local data directory and this one is going to be inside my container.
16:02
So this is like a mirror from this image. And how it works? In layers we have the same, the layer that we can modify and on top of that we are going to create another layer, but in this case we are creating the mounted volume in bar leaf MySQL.
16:29
There are other directories that we can create the volume. I am just adding as an example this because in MySQL we have configuration files,
16:40
we have logs, we have other things, but for that we want to create these volumes for all of those things. I am just adding as an example bar leaf MySQL, which is also a directory that is very important and this local directory is the one that will be in my host, but it is not recommended because if your host crashes,
17:00
everything crashes too with your volumes. It is preferable to run it in a remote host. Okay, two backups. Who here makes backups?
17:22
Okay, I use a very easy way to make backups. I use it for logical backups, my SQL dump, just in the container, and for physical backups, we use in the company Percon extra backup, which has more features to have that physical backup.
17:43
And for restore, I will use also my SQL dump. And we don't use Percon extra backup in this case because it has a lot of pins. For backup, I will execute a backup in a container
18:01
that is already running. Percon server backup is already running. Let's see that we created, and we are executing docker exit IT to enter into the Percon, in that container, and type that command, my SQL dump, to create a backup of the database.
18:22
So the backup is going to be in that file, dump SQL, and the same process with restore. We can take that backup, and this is a different container. I'm going to restore the .SQL file in a different container. In this case, in Percon server restore, using my SQL,
18:43
just that command, my SQL. Okay, best practices or some recommendations to use a container database. One of these is that we can keep constantly monitoring
19:00
our database and the whole system because we don't know when we are going to don't have enough resources for our containers. We should be aware of that or have notifications to say, hey, you don't have enough disks, you don't have enough memory, so provision or try to scale in your resources, so we should keep monitoring. Using some tools for that, for example, is PMM.
19:23
We can use open source monitors to monitor our databases and containers. We can store this data in persistent volume outside a container, if recommended, not inside a container because it's easy to create plans for recovery. We can restore the data easily, also, and fast.
19:45
We should limit the resources of utilization of our containers. Our containers, we know that they are small, but also we should limit when they are a lot, and we should regularly have backups of the database
20:04
and store those backups in a different location. And have a plan on migration and disaster recovery is really great. In that case, having a monitoring tool helps a lot, and one more, that's all.
20:25
You can find me in LinkedIn and Twitter, everyone. Okay, we have time for questions.
20:41
If you absolutely need to leave and you can't wait until the talk is over, please do so as quietly as possible so we can understand the questions. Thanks. Hi, thank you so much for your talk.
21:01
It was really interesting. I'm wondering what kind of limitations do you see when you're speaking about having a database running in containers? There is storage limitations, CPU, or something else? Guys, can you please be a little quiet so we can understand the question?
21:20
All right, I will try without the microphone. Yeah, you. The people can hear. I was wondering, first of all, a really cool talk. Thank you so much. My question would be could you maybe talk us through some kind of limitation that you can see when you're running databases from containers? Those are some kind of limitations that you see
21:49
when you're running to the solution of running databases inside containers, right? You cannot really run very big database. You probably will have a problem with what kind of limitations you can run in containers.
22:03
So, the question is about what limitations you can run into with database containers. Yeah, I don't want to say this, but it depends really on the business.
22:20
You want to invest a lot of money in infrastructure because at the end, the volume that you have is not going to be part of your container. It's going to be outside, and it depends on you. You want to invest a lot of money to save that data. It's good. You want to replicate it? Please try and be quiet while we are asking questions.
22:43
Are there any more questions?
23:10
There is one more question from the back, so please be quiet. Thank you. Hello. I wanted to ask, did you notice any kind of performance issues?
23:23
Did you benchmark things? Did you identify some kind of overhead going on when you containerize a database like MySQL or other kind of databases really? Sorry, I didn't get your question.
24:00
All right, I'm just going to ask again.
24:02
When you containerize a database, be it MySQL or Postgres or any kind of open source database that you may have tested on this kind of setup, did you notice any kind of overhead compute memory or disk essentially, where you can see that the database performance
24:21
or operation is significantly affected by the fact of being containerized? I'm not sure about that, but if you use open source tools to monitor your containers on databases, you can have a visualization of these things. If you don't have enough resources,
24:42
it can show you alerts or things like that where you can figure out where exactly is your limitation. Okay, so for example, did you run benchmark? Could you help me?
25:04
Okay, could you help me to answer? Okay, my friend is going to help me to answer this. All right, thank you. Yeah, thank you to you. Hey, so usually the performance degradation is around two, three, four percent. The issue is more about how you configure the database,
25:23
kind of storage, if it's local or network storage. But the virtualization is minimal. It's like running on an EC2 instance. Okay, so there is an impact, measurable, at least you say around four or five percent,
25:41
but you say that's not going to be the, that there are configurations we can do to try to avoid that. Do you have any kind of paper or any kind of resources that we might use to avoid those kind of bottlenecks? If I got correctly, not much. The measure that we do in databases is measuring TPS.
26:05
So you will notice if you're running benchmarks, with sysbench, for example, three percent, like if you are running 1,000 queries per second, you will get 980, 990 queries per second when containerized.
26:25
Okay, and do you have any kind of recommendations, kind of generic recommendations you can do so that when you run a database in a container, here is what you can do to try and negate some of the performance bottleneck that you guys have noticed. To be honest, on real-day activities,
26:45
I would say 99 percent of the performance will come from how you configure MySQL. The containerization is like just a small piece of the game. You can make more effects by modifying the database configuration.
27:05
Alright, thank you very much. Thanks to you.