Monitoring and managing Containers using Open Source tools
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Title of Series | ||
| Number of Parts | 45 | |
| Author | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/54640 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
openSUSE Conference 202024 / 45
5
9
14
16
19
26
28
34
35
40
00:00
Information securityProduct (business)ArchitectureMetric systemData managementTotal S.A.Computer-generated imageryRead-only memoryComputer networkoutputBefehlsprozessorFunction (mathematics)SpiralShared memoryContent (media)Metropolitan area networkFocus (optics)HoaxCone penetration testObservational studyCuboidHypermediaPoint (geometry)Condition numberCartesian coordinate systemWorkstation <Musikinstrument>Task (computing)Information securityData managementSemiconductor memoryMereologyComputer chessStatisticsLine (geometry)Level (video gaming)Metric systemLocal ringRule of inferenceMachine visionStreaming mediaAreaMatrix (mathematics)Presentation of a groupText editorTwitterSystem administratorBefehlsprozessorLatent heatSoftwareCore dumpProcess (computing)Utility softwareFunctional (mathematics)Medical imagingView (database)Integrated development environmentMeasurementMathematicsTrailProduct (business)Different (Kate Ryan album)Computer animation
05:01
StatisticsComputer configurationGoogolVolumeSummierbarkeitStatisticsSemiconductor memorySingle-precision floating-point formatGastropod shellInformationMetric systemRhombusBefehlsprozessorVirtual machineHookingCartesian coordinate systemRevision controlSoftwareWeb 2.0Function (mathematics)Client (computing)File formatTotal S.A.Process (computing)Remote procedure call2 (number)MultiplicationSpacetimeUtility softwareFront and back endsInformation retrievalMathematical analysisUniform resource locatorDefault (computer science)User interfaceRepresentational state transferWeb serviceGraph (mathematics)System callArithmetic meanPersonal digital assistantSoftware testing40 (number)Observational studyAreaShared memoryOpen setInterface (computing)Water vaporCone penetration testContent (media)Point (geometry)Term (mathematics)Task (computing)State of matterComputer animation
10:02
IRIS-TPhysical systemInterface (computing)Inheritance (object-oriented programming)Real-time operating systemShape (magazine)ParadoxRadical (chemistry)User interfaceMetric systemMultiplicationSource codeXMLProgram flowchart
10:38
Computer-generated imageryData bufferKernel (computing)Maxima and minimaRootTorusBefehlsprozessorPrincipal ideal domainSynchronizationIcosahedronWindowProxy serverAnnulus (mathematics)Just-in-Time-CompilerDependent and independent variablesFibonacci numberPersonal identification numberIntegrated development environmentVenn diagramExecution unitPhysical systemBit rateSet (mathematics)Point (geometry)Data storage deviceMatching (graph theory)ChainDecimalCartesian coordinate systemVirtual machineMathematicsBus (computing)Client (computing)InformationMedical imagingContent (media)SpacetimeMassStability theoryMetropolitan area networkLatent heatSoftware testingInformation securityComputer fileSystem administratorCombinational logicChannel capacityElectronic mailing listComa BerenicesDuality (mathematics)Sign (mathematics)Uniqueness quantificationTask (computing)OscillationSequelTouchscreenGame theoryGastropod shellSocial classAlgebraic closureDistortion (mathematics)Workstation <Musikinstrument>Stack (abstract data type)MetreOpen sourceLevel (video gaming)Shared memoryOnline helpSquare numberMaizeInterface (computing)Type theoryBasis <Mathematik>Forcing (mathematics)FreewareSystem callCommunications protocolVideo gameScaling (geometry)Suite (music)File systemDomain nameCategory of beingINTEGRALWhiteboardForm (programming)Error messagePressureAxiom of choiceSoftware bugDatabaseIdentifiabilityProcess (computing)Thread (computing)Data managementEstimatorSoftwareMotion captureSheaf (mathematics)BefehlsprozessorIntegrated development environmentLoginComputer configuration2 (number)Data structureGroup actionRule of inferenceElectronic signatureView (database)MetadataFilter <Stochastik>Expected valueRun time (program lifecycle phase)Kernel (computing)User interfaceSoftware developerSelf-organizationFlow separationDifferent (Kate Ryan album)Mathematical analysisNamespaceSource codeJSONXML
19:38
Local ringRing (mathematics)Total S.A.VolumeExecution unitArchitectureMetric systemData managementBack-face cullingOrder (biology)Matching (graph theory)Metropolitan area networkObservational studySelf-organizationAreaComputer filePressureShared memoryLocal ringNeuroinformatikImaginary numberConnected spaceInternet service providerData storage deviceType theorySoftware testingCore dumpCombinational logicGoodness of fitInterpreter (computing)Internet forumInformationMoment (mathematics)MassWorkstation <Musikinstrument>FreezingSocial classSystem callError messageSemiconductor memoryCartesian coordinate systemView (database)Coefficient of determinationWhiteboardLine (geometry)Uniformer RaumMaizeLink (knot theory)Physical systemPresentation of a groupComputer configurationIntegrated development environmentInterface (computing)Multiplication signVolume (thermodynamics)Radical (chemistry)Remote procedure callDifferent (Kate Ryan album)Medical imagingProjective planeService (economics)Instance (computer science)Object (grammar)SoftwareWordUser interfaceElectronic mailing listCloud computingField (computer science)Windows RegistryStatisticsOpen sourceSet (mathematics)Sheaf (mathematics)Domain nameGroup actionComputer animation
Transcript: English(auto-generated)
00:04
Hi, my name is Jose Melo Ortega and this presentation is about monitoring and managing containers using open source tools. My name is Jose Ortega, I'm from Spain and in the last year I'm interested in DevOps
00:22
with focus on security, you can follow me on Twitter and LinkedIn where I usually share content related with security, hacking, ethical hacking, Python and containers. Also I am technical writer and this year I have published a book related with DevOps and
00:42
container security with focus in Docker and Kubernetes tools ecosystem. This is the book I have published with VPD editor and in this book basically you can learn about Docker as a platform for the deployment of containers and Docker image
01:01
taking into account the security of applications. Also you can learn some specific tools about monitoring and administration in Docker with tools such as cadvisor, 6day, Portainer and Rancher. This could be the main points to discuss in this talk, basically we are going to talk
01:26
about challenging containers and architectures distributed from the point of view of monitoring and administration, most important metrics that we can use to measure container performance. Later we will review the main tools for monitoring and management of containers such
01:47
as cadvisor, 6day and Portainer and we will make a little introduction to Rancher as a platform for administration for Kubernetes. Well, the question at this point is why do we need to monitor our Docker containers?
02:07
Well, it is obvious that monitoring is an essential part of Docker based environments. Among the main reasons we need to monitor our containers, we can highlight detecting issues and troubles at an early stage to avoid risks at a production level,
02:27
changes and upgrades can be done in a security way as the whole Embedded Minisys track. And another reason that we can find is to refine or refactor our applications for
02:46
better performance and robustness in general when we are deploying our application in different environments. Well, monitoring containers are one of the most challenging activities related
03:05
to container-based applications. Monitoring application performance ensures that both the container reset applications and infrastructure arrive at optimum levels. For example, monitoring historical data and CPU usage are useful to recognize strengths that lead to recurring
03:26
issues or bottlenecks. The DIs analyze some tools for Docker monitoring that provide a dashboard for visualizing data per container like CPU, memory and network.
03:42
The sense of monitoring philosophy is to ensure that a running process is performing to the expected level. Among the main metrics, we can highlight the use of CPU, memory, input-output,
04:03
network requests and in general the network traffic that we can monitor with specific tools like TCP-DAM or Wireshark, we can use these metrics for measuring
04:31
the performance of our application. The Docker engine itself provides access to most of the core
04:41
metrics we are looking to collect as part of its native monitoring functions. For example, running the simple command docker-stats will give you access to CPU, memory, network and this utilization for all the containers running on your Docker host.
05:03
In this way, docker-stats achieves this by picking up metrics from the docker-diamond, that is the main process that Docker runs for creating the different containers from an image.
05:24
Docker-stats basically shows the percentage of CPU precision for each container, the memory use and total memory available to the container, so that we can see total data sense and receive
05:42
over the network by the container. These statistics can be used to see the behavior containers during execution and information can be useful to verify we need some restrictions on the resources that we will apply to the containers. Docker-diamond also provides
06:03
remote REST API. This API is used by the client to communicate with the Docker engine and can also be invoked by other tools such as CURE or Postman REST client. Basically, the API provides a stats endpoint and we can use it with the container-ed or the name of the container
06:30
to invoke this endpoint. For example, we could use the stats endpoint using the container-ed
06:40
id in the URL and in this way we are obtaining the container statistics in JSON format by default and in the returning information you can get detailed information for memory usage as well as information about the CPU. If we continue with the review of the
07:06
main tools that we have in the Docker ecosystem, one of the most known is a shell visor that is a container monitoring tool from Google with native support for Docker containers.
07:21
Basically, it's a diamond that collects, aggregates, processes and sports information about running containers. Shell visor is easy to get up and running as it can be deployed in a single Docker container. All you have to do is run this Docker run command to execute
07:43
a shell visor container and expose the web interface on port 8080. Once launched, shell visor will hook itself into the Docker diamond running on your host machine and start gathering metrics for all of your running containers including the shell visor
08:05
container itself. The best way to think of a shell visor is that basically it's a graphical version of what you get when you execute the Docker stats you have seen before. Basically,
08:22
shell visor is the graphical version of Docker stats. Among other things, shell visor allows you to monitor resource isolation parameters, historical resource usage and network statistics.
08:40
Shell visor is a very useful tool that can be set up in a easy way to discover the resources consumed by the containers' applications. For example, we can display the graphs related to the CPU, memory usage, network input, output and this space utilization.
09:04
Shell visor dashboard shows data for the last 60 seconds only. However, multiple backends such as Prometheus and InfluxDB are supported that allows long-term storage,
09:25
retrieval and analysis. Shell visor provides both a web UI and REST API to monitor Docker containers as well as integrate metrics to an external application via web service endpoints.
09:44
In the Docker container section, you can see the URLs of the containers that are running on the Docker host and if you click on any of them, you will see the resource usage information for the corresponding container.
10:05
In the Docker ecosystem, we can find other tools. For example, CTOP is a tool developed in Golang that provides an overview of real-time metrics for multiple containers in a graphical way. Other interesting tools like for Docker
10:26
monitoring is Lexi Docker, that is a terminal user interface for both Docker and Docker Compose built in Golang. Another interesting tool is not related directly with monitoring, but
10:48
for example, if you need to explore a Docker image layer by layer, you can use this tool. It's called Dive. This tool allows us to explore Docker image, the content of each layer
11:03
that make the layer image as well as the size and persistence of image efficiency. For example, you could show chunks in the file system for each layer and estimate image efficiency for tasks related with analyzing the size of image. When selecting a specific layer
11:28
inside an image, we can see the layer details and the folder structures of this layer. Also, we can see information related with the command that is generating that layer,
11:40
image size, potential wasted space, and image efficiency score. We continue with other interesting tools. In this case, we are going to review what provides the 6D. 6D is a startup that has many tools related with monitoring, analyzing the
12:09
security of containers. Basically, 6D contains the most in-depth integration within the Docker ecosystem and tracks data directly from container metadata to enable security
12:23
and Docker monitoring. From the point of view of monitoring, Docker containers with this tool, we can control the behavior of containers. 6D works by capturing system calls that are
12:41
made within the Linux kernel. This provides 6D with visibility into how applications work with containers. 6D, this is the architecture, can operate both with real-time data, previously captured data, and install a signage on each host which takes advantage of system
13:05
calls and kernel filters and rules in a signature database to identify suspicious activity or anomalies inside the containers. From the monitoring point of view, 6D provides
13:20
some specific tools for helping the stacks. One of the open source tools we can find is called 3-6-D. These tools allow you to analyze and capture the system calls of various types of containers such as Docker, Linux containers, and the underlying processes and threads along
13:46
with the use of resources at the CPU, memory, and network levels. This tool shows all containers running on the host and you can drill down into a container to see what resources
14:03
individual processes are consuming in a specific container. Another interesting tool that provides 6D is called Falco. This solution is oriented to container security monitoring. Falco is the
14:29
foul, for example, is the foul Kubernetes thread detection engine and has the capacity for detecting an expected application behavior analysis on threads at runtime. Falco allows you
14:45
to monitor a container application host and network activity just as we would do with a combination of Unix tools such as Snort, TCPdum, and Strace. Basically, Falco combines all these
15:02
tools in a specific command. As we can see in these screenshots, it provides a rich set of the foul events, search calls, and filters and lets you customize the rules for them.
15:21
With the following command, for example, we are obtaining all the containers in execution and internally what these tools is identify the system calls, execute in different user namespaces, and reports the name of the container for each namespace.
15:45
The interesting thing about Sysdig is the ability to filter a facet on information from a Docker container. At this point, it provides filters available to inform about system status, application protocols such as HTTP or MenkaJ, CPU usage, and network activity.
16:08
When trying to expect, for example, when trying to expect network traffic in a similar way, we can use tools like TCPdum. We could use Falco with the HTTP log command for
16:21
monitoring HTTP requests from containers. We could also filter all the system calls executed by Apache and MySQL processes, indicating with the proc.name property the names of the processes that we want to filter. Well, to take the full advantage of container
16:49
benefits, organizations need the appropriate management and administration tools that allow them to manage the full lifecycle of the container and ensure the availability for both
17:02
development and production environments. Within the Docker ecosystem, we can find some interesting tools for developers for managing and managing containers. Among the main tools for container administration, we can highlight Rancher and Portainer. These tools are similar.
17:26
Well, Portainer is focused on managing the lifecycle of your image and containers with Docker, and Rancher is more focused in administration, not only the containers, also your cluster,
17:47
and so on. Among the main advantages that Rancher offers, we can highlight, it allows you to create as many environments as you need and manage uses and roles in different
18:02
environments. Also, it allows you to select the container orchestrator from several such as Mesos, Kubernetes, or DockerSwan. It provides more flexibility to administrate your cluster. Rancher basically provides a web interface to control containers,
18:27
and the main dashboard shows a starting, a stop, and running containers. The best
18:41
of Rancher is that it provides a user interface with a lot of many options for administrate your containers and creating an environment where your containers will be secured with high availability and with high performance.
19:06
The containers, this is another screen capture where we can see the container sections that list all your running containers. For example, you can open a shell into a container
19:21
and do other actions like stop, restart, and delete the container, and other options like view logs and clone the containers. It provides many options to manage the lifecycle of your containers. The second tool I'm going to comment on is Portainer. This tool basically
19:48
allows you to manage your containers, image, volumes, networks with a user interface that allows you to manage the different Docker environments in the same way that we secure a
20:02
Docker pool or Docker run from the terminal. With this tool, we can do it from a graphical interface, as we will see now. Portainer can be executed as a lightweight Docker container on top of a Docker host or a Docker cluster. We can actually run Portainer as a container,
20:26
which gives us the added benefit of being able to experiment without worrying about installation and dependencies. And starting Portainer as a container using Dockerclay is as simple as executing Docker run or using Docker Compose configuring or Docker Compose YAML.
20:51
In this file, we can highlight some options like creating a volume named Portainer data for persistent storage. And also in the Docker Compose, we can see that we are forwarding TCP port
21:08
900 from the host to TCP port 900 in our container for accessing the using interface. The very first time we are configuring Portainer lets us choose the type of connection we can use
21:29
for handling the container. We can choose between different installation types like local or remote Docker environment. Also, we can use a Portainer agent or connecting with
21:44
other cloud providers like Azure. This is the main dashboard where we can see containers, images, and services that are running in the Docker environment we have selected
22:01
and have been created in the Docker instance. On the containers section, we can see each of our running containers along with some basic information about each one. In this section, we are able to execute several instructions that we usually execute through the command line
22:26
such as starting, stopping, or deleting the container. Clicking on a container name will drill down into the container details section and give you a lot of information about it
22:41
while also allowing you to create a new image based on your container or change other settings. At the top, we have different actions like start, stop, kill, restart, pause, resume, and remove. At the bottom, we have links for view, other information for logs, inspect,
23:06
and start execute, or other commands that are available in the Docker domain. Here we can see, for example, details of a container like port configuration,
23:25
environment variables, and labels. In the volume section, we can see a list of our volumes where we are able to remove them or add new ones and for each volume, we can see the name,
23:43
the star, the driver, and so on. In the word settings, we can see a list of all the Docker networks that have been created with details about each. In the image section, we can see a list of images on your Docker instance and so we can create and manage our own image. For
24:06
example, pulling an image from the Docker Hub registry and clicking on an image will take you to the image details which will show you a lot of the details about each image.
24:22
In container statistics section, also we can see the use of CPU, memory, and network. Portainer is compatible with Docker engine and Docker shard. If you want to
24:43
to test Portainer, Kata Koda provides a scenario for deploying Portainer to Docker shard cluster. Basically, you need fields to deploy a Docker shard cluster and
25:02
over this cluster, you can execute Portainer for testing for deploying a container with high ability over this cluster and testing this,
25:21
monitoring this container with the Portainer user interface. Finally, for conclusion, we can see that all of the tools we have covered, we will be able
25:41
to get you started with container visibility and monitoring and the object is to keep an open eye on what is going inside these containers in order to optimize our applications and systems. Choosing the best monitoring solutions depends on the combination of
26:06
tools that we have, our challenge, and our needs. It's up to you to decide what combination of tools is needed to collect information that will help your monitor and
26:22
optimize your application and systems. The tools that have come into this presentation are not maybe coming out the best for your needs, but I have tried to review
26:42
the best open source tools that you can use in your projects. For conclusion, if you want to know more, there are some books that are related with this topic,
27:04
but for example, monitoring Docker, this book will show you how monitoring Docker containers with some of the tools we have reviewed in this presentation. And that's all. Thank you to the audience and the organization. If you have any questions,
27:27
you can do it through the organization and I will try to answer the question.
27:42
Thank you very much and bye.