Present and future of Ceph integration with OpenStack and k8s
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 542 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/61645 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
00:00
DisintegrationState of matterDemo (music)HorizonComputing platformPoint cloudSoftwareComponent-based software engineeringMultiplicationComputer hardwareData storage deviceComputer networkStatement (computer science)Open sourceCloud computingSource codeHypercubeOperator (mathematics)Scale (map)Service (economics)Operations researchOnline helpEuclidean vectorGroup actionDemonVolumenvisualisierungVertex (graph theory)Data managementOnline service providerMachine visionComputer configurationCommon Language InfrastructureInterface (computing)Module (mathematics)Declarative programmingConfiguration spaceDefault (computer science)FingerprintProjective planeDemosceneSoftwareDifferent (Kate Ryan album)Operator (mathematics)Data storage deviceMereologyINTEGRALDemo (music)Service (economics)BitMultiplication signInternet service providerCloud computingInterface (computing)Connectivity (graph theory)Open setProcess (computing)ScalabilityDescriptive statisticsScaling (geometry)Medical imagingMiniDiscVolume (thermodynamics)Front and back endsMaxima and minimaFlow separationGroup actionPoint (geometry)Interface (computing)Point cloudStrategy gameVariable (mathematics)XMLComputer animation
08:48
Common Language InfrastructureOperator (mathematics)Computer configurationInterface (computing)Module (mathematics)Component-based software engineeringVertex (graph theory)Declarative programmingConfiguration spaceFingerprintDefault (computer science)Distribution (mathematics)Service (economics)Formal grammarElectronic mailing listUniform resource locatorSummierbarkeitControl flowComputer wormDemonDevice driverMessage passingNetwork socketSynchronizationProcess (computing)Client (computing)LastteilungServer (computing)Proxy serverAsynchronous Transfer ModeData recoveryScalabilityCommunications protocolWorkloadPerspective (visual)Computer networkDemo (music)MIDIEmailEndliche ModelltheorieLimit (category theory)Point (geometry)Maxima and minimaStability theoryVirtual machineAddress spaceTunisLevel (video gaming)WorkloadCASE <Informatik>CuboidBootstrap aggregatingConstraint (mathematics)DemonClient (computing)Service (economics)Computer fileSlide ruleAdditionDescriptive statisticsInternet service providerSoftwareConnected spaceCombinational logicStructural loadTerm (mathematics)Game controllerDevice driverPlanningInteractive televisionOperator (mathematics)View (database)MathematicsRevision controlAreaShared memoryGateway (telecommunications)IP addressVolume (thermodynamics)DemosceneRun time (program lifecycle phase)Projective planeVariable (mathematics)Interface (computing)Computer architectureLastteilungProxy serverComputing platformCommunications protocolComputer animation
17:30
Computer animationProgram flowchart
Transcript: English(auto-generated)
00:18
Thank you, let's welcome Francesco on the present and future of Ceph.
00:26
Okay, thanks everyone. This session is about OpenStack and Ceph integration. With our quick glance to the Kubernetes world, here's a quick agenda. I'm going to talk about
00:41
what have been the integration with Ceph in the OpenStack system, in the OpenStack community in general, what's the status of the heart, what has been changed with the Ceph ADM in the bare metal world and what means having Kubernetes in this picture. There is also a demo which is, I'm not
01:00
going to show the demo, but it's linked in the session so you can feel free to take it later. So, for those not familiar with the OpenStack project, it's just pretty old at this point, it's the infrastructure as a service. As you can see on the right side, there are a lot of projects being part of the OpenStack ecosystem. Each of them provide interfaces to each other and
01:25
they cooperate together to provide processing, storage, network resources. You basically can build your cloud infrastructure using all these projects together. It's open source, it's now 13 years old, so there are a lot of projects that are stable now. Ceph is part of this picture
01:49
in the sense that you probably don't see this picture very well, but in both computes, in the storage part, we have Ceph that basically can be used as a backend for Nova,
02:04
which is the compute processing component, so we can provide ephemeral disks using Ceph as a backend. We have Manila providing object, there is a good providing file, Swift providing object,
02:21
there is a long story with the integration with Rados gateway, glance for images. So, basically all these components you see there in the storage area, they can use Ceph as a backend and this is a justification to have this integration with these two technologies.
02:42
So, why the integration and why it's relevant? There is the HCI, either converting infrastructures, operators can save other resources, co-locating the compute part of OpenStack, the infrastructure, and OSD nodes. This means that you can save hardware,
03:06
you can have both technologies together serving as an infrastructure, as a full operational infrastructure. And a couple things, this is funny, I just asked Chad GPT why Ceph
03:23
is relevant and why the integration of these two projects is interesting and I just put there, you cannot see probably this part on the right, but it's basically my marketing sentence on why this thing makes sense. And there is scalability, resiliency, it's scalability and all these
03:47
kind of keywords that we like a lot. But this session is about orchestrating services, deploying OpenStack and Ceph together. There have been a lot of deployment tools and strategies
04:01
over the past and I want just to focus on Ceph Antiball and Ceph ADM because Ceph Antiball has been the official tool, the most useful tool in the past and now with Ceph ADM things have changed a bit, especially in the OpenStack ecosystem. So previously the approach was,
04:22
I need my infrastructure as a service, I need to deploy OpenStack and Ceph, I want to describe my entire cluster with a lot of tons of variables and I push that magic button that makes it so. So Ceph Antiball was there in this picture during the deploying of OpenStack,
04:44
there was this particular part where Ceph Antiball was triggered to deploy Ceph behind the scene. So the drawback of this solution is that if you need to change anything in your Ceph cluster, you have to run again the playbook, the Ceph Antiball playbook, because there is this
05:03
Antiball layer that manages everything for you and it needs to stay consistent with the status of the cluster. So basically the human operator is supposed to provide variables and maintain those variables, which is a bit different. Also it affects scale down, scale up operation
05:26
and day two operation, especially day two operation. I want to change something in my Ceph cluster, I need to run the deployment again because I can rely on the Antiball hidden potency basically. But with Ceph ADM things are a bit different, the status of the cluster is
05:45
not made by tons of Antiball variables, there is Ceph ADM which is able to keep the status of the cluster, watch the cluster and take an action. I want to deploy a new SD whenever I
06:01
attach a new disk, I can do that and I don't have to run deployment again or doing any fancy day two operation with my tool that is supposed to manage my infrastructure in general, which is made by both OpenStack and Ceph. And this changed a bit because we had the Cephantiball
06:25
world where we describe all our cloud infrastructure, we pushed that magic button and everything was deployed and sometimes broken. But now operators are more aware about the steps,
06:42
so things have changed because you have to provide networks and networks means that you want to manage your hardware, you want to build your networks, you want to use this specifically for the Trivolo project where we integrated Cephantiball in the past and now we're moving
07:03
to Ceph ADM. And now people should provide networks, should provide metal, the description of the nodes and they are just separated steps. The storage Ceph is deployed first, so you can start deploying your OpenStack infrastructure with a minimal Ceph cluster already
07:22
deployed and when you deploy OpenStack you can say I have Cinder, I need volumes, I need the volumes pool and I can finalize my cluster creating and configuring the Ceph cluster accordingly. I have Manila, I need CephFS, I can deploy MDS doing other stuff behind the scene,
07:43
but you're basically aware that according to the service you're deploying in your OpenStack infrastructure you can enable pieces in your Ceph cluster and you can just tune it accordingly. At that point we still have the Ansible layer managing OpenStack and all the
08:01
infrastructure in general, but at that point the Ceph cluster is seen as a separated entity, so it's like having an external Ceph cluster even though you have the same tool deploying both technologies together. And Ceph ADM and the manager, the orchestrator, is that piece in Ceph that is supposed to provide the interfaces where the operators
08:26
can interact with and it's basically the slide. We still have Ansible doing everything on top, but the orchestrator interface is what you can rely on to make changes in your cluster
08:43
without having to redeploy everything again around the Ansible that can take a lot of time if your infrastructure is large. And we don't have any, the operator is not supposed to keep and maintain any variables in the Ansible system, you can just interact with the Ceph ADM CLI, create a
09:05
spec which is a little definition of the Ceph service that will be translated by the orchestrator and it will be deployed as a new daemon in the Ceph cluster. So this is about why it's so easy,
09:21
because you have Ansible, at some point you can just bootstrap a minimal Ceph cluster with this common bootstrap providing a monitor IP address because networks are already there at that stage and you can create a spec file that is basically the description of the nodes that should be enrolled in Ceph and you can just apply them. It's even easier rather than
09:45
running an Ansible with a lot of roles, execution time which is long enough, and data operations are complicated. They are complicated because, not just because of this slide and this is the interaction with the Ceph ADM CLI, you can query the cluster and see the
10:05
status, you can see where daemons are placed, you can list the hosts and manage their labels and assign roles to these hosts. You can do a lot of things, but the real point here is that
10:21
with Ceph ADM we don't have the need to run again all the deployment of the OpenStack infrastructure. An example of this of how projects are impacted by this change is Manila. It's not just because we need a new version of LibreBD, we need to be compatible and we
10:43
are changing from Ceph Ansible to Ceph ADM, but just because we are doing architectural changes to the use cases provided by OpenStack. Manila is that service that curves out CephFS volumes and provides them to the virtual machine with the antennas, which means that you have a dedicated
11:03
network that you can use to mount your shares and behind the scene we have CephFS or NFS with Ganesha. In the past Manila was supposed to use D-Bus to interact with NFS Ganesha and it was complicated because we had to run privileged containers, we had to use this
11:26
interface to update and manage shares using Ganesha as a gateway. From an architectural point of view, we had an active passive model made by Peacemaker and systemd. You basically had
11:44
Peacemaker honing the virtual IP as an entry point and then one active Ganesha, even though you have more than one instance and with some constraints with systemd. Now with Ceph ADM there is an interface with the NFS cluster and Manila can use a new
12:06
driver to interact with this component, so we don't have to do D-Bus anymore, we don't have to do D-Bus to the Ganesha container anymore and that's the new model where we rely on the ingress daemon provided by Ceph ADM and this ingress daemon is made by HAProxy and
12:26
Keepalifed. Keepalifed honing the VIP as an entry point HAProxy for load balancing across the and distributing the load across the NFS Ganesha server. It's a better approach, we still
12:40
have some limitations on this area because considering that Manila is an infrastructure service for Obestock but providing shares within the tenant virtual machines with a dedicated network, we need client restrictions to avoid other tenants mounting the same share
13:01
and there is an effort doing the proxy protocol in Ganesha to make sure that we can use client restriction with this new model which is the current limitation basically or at least there is some effort to provide floating stable EP addresses to the ingress
13:22
daemon and skip the HAProxy layer which is always an additional hope and in terms of this can be something that has an impact of course. Last thing, at this point we had Ceph ADM what Kubernetes means in this world. We have
13:46
look as a way to deploy Ceph within Kubernetes regardless of how Obestock is deployed we have combination of these two things together. You can have convergent infrastructure where
14:02
Obestock control plane is virtualized or it can be containerized so basically using the same model, the same approach to deploy both it can be useful because it's a unified approach to manage your infrastructure. It's easy, deployable and reproducible because Kubernetes poses a
14:27
standard approach to deploy things so we don't have any more that flexibility that today is trivial but it's easier from that point of view and the Ceph cluster can be shared
14:40
between Obestock and Kubernetes. We have different workloads. Kubernetes is PVC interfaces provided by Rook. Obestock is mostly RBD and your workload runs on virtual machines and it's usually outside so you have to reach from the compute node the Rook cluster,
15:01
the Ceph cluster deployed by Rook within Kubernetes which poses some networking challenges that they can be managed using host networking through so you are using Kubernetes as a platform to deploy your Ceph cluster but you are still relying on the host networking to reach it
15:25
and provide RBD to the outside workload and that's the point of this slide. There are some things that are not mentioned here like some tuning in Rook that is supposed to be done to make sure that there is Kubernetes in the middle so it's not natively the native Ceph
15:47
cluster we usually have so at this point the thing is that we should do some tuning especially in the HCI world because Azure Convergent is still one of the most popular
16:02
use cases and HCI is provided out of the box by Kubernetes. You can tag your infra nodes, you can deploy Rook, you can assign those nodes for OSDs, that's it. But at that point you have to make sure that both your cluster and the connection is available for the outside
16:23
workload. So this can be done, it's done by this demo, I'm not going to show this but it's all there, it's all described, it's all I was describing in this slide so you can have your Obestack infrastructure deployed with DevStack or Triple O and it's still bare metal
16:47
and it can consume a Ceph cluster deployed by Rook using RBD. You can still use the CSI actually to provide PVC interface, it's not something that it's mutually exclusive but
17:03
it's just a good exercise to see how these two technologies can work together in the future probably. And yeah, just some additional resources for who is interested in looking at the slides offline and some contacts for people in the Obestack world if you want to
17:25
dig more into these experiments. That's it, thank you very much.