The Container Storage Interface, Explained

Video in TIB AV-Portal: The Container Storage Interface, Explained

Formal Metadata

The Container Storage Interface, Explained
Title of Series
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Release Date

Content Metadata

Subject Area
Over the next year, all Kubernetes persistent storage will be moving to CSI plugins. But what is CSI? What's in the specification? How does it work in practice? The rapidly evolving Container Storage Interface specification has finally slowed down, reaching its first stable release v1.0. With it, all vendors have rushed to update their drivers or write new ones. But what does this mean to us? How does CSI benefit the ecosystem? What are the final features? In this talk we’ll cover this and more, going over the objectives and feature of the CSI spec, as well as some of the Kubernetes flows. By the end of the talk, you’ll be up to date with the current CSI spec, be able to deploy a Kubernetes cluster with a CSI plugin, and know the elements in Kubernetes’ CSI, their relationship, and the part that each one plays.
Standard deviation Scheduling (computing) Presentation of a group Group action Mereology Data management Different (Kate Ryan album) Computer configuration Information Fiber (mathematics) Physical system Software developer Electronic mailing list Data storage device Bit Connected space Type theory Data management Process (computing) Interface (computing) Volume Resultant Spacetime Topology Statistics Game controller Implementation Service (economics) Channel capacity Device driver Data storage device Mass Architecture Workload Operator (mathematics) Service-oriented architecture MiniDisc Plug-in (computing) Computing platform Computer architecture Plug-in (computing) Game controller Standard deviation Multiplication Dependent and independent variables Information Server (computing) Interface (computing) Volume (thermodynamics) Line (geometry) Device driver Software Personal digital assistant Network topology Communications protocol
Gateway (telecommunications) Code Multiplication sign Source code Parameter (computer programming) Mereology IP address Front and back ends Timestamp Mechanism design Different (Kate Ryan album) Single-precision floating-point format Network socket File system Active contour model Information Process (computing) Data compression Social class Physical system Source code Mapping Block (periodic table) Open source Data storage device Range (statistics) Parameter (computer programming) Menu (computing) Bit Connected space Type theory Data management Process (computing) Repository (publishing) Volume Metre Dataflow Topology Functional (mathematics) Game controller Freeware Service (economics) Real number Channel capacity Device driver Data storage device Graph coloring Number Revision control Latent heat Operator (mathematics) Touch typing Plug-in (computing) Computing platform Computer architecture Context awareness Multiplication Dependent and independent variables Information Demo (music) Idempotent Interface (computing) Plastikkarte Coma Berenices Volume (thermodynamics) System call Equivalence relation Film editing Personal digital assistant Mixed reality Point cloud Social class Enterprise resource planning
Scripting language Game controller Touchscreen Information Electronic mailing list Device driver Volume (thermodynamics) Directory service Mereology Mathematics Different (Kate Ryan album) Plug-in (computing) Asynchronous Transfer Mode
Decision tree learning Game controller Mobile app Information State of matter Idempotent Device driver Volume (thermodynamics) Bit Parameter (computer programming) Cartesian coordinate system Login System call Metadata 2 (number) Causality Different (Kate Ryan album) Representation (politics) Website Right angle Object (grammar) Logic gate Plug-in (computing)
Web page Point (geometry) Implementation Email Mapping Interface (computing) Multiplication sign Software developer Patch (Unix) Electronic mailing list Data storage device Data storage device Lattice (order) Device driver Number Goodness of fit Mathematics Different (Kate Ryan album) Repository (publishing) Energy level Devolution (biology) Plug-in (computing) Computing platform
good morning and thank you all for coming on a Sunday morning my name is
got a lair and I would like to start with a quick raise of hands from those of you who have used persistent storage in your container workloads it should be on yeah thanks okay so seen that many racing hats probably I'm not the only one that thinks that it's a little bit of mass the we have way too many interfaces to connect our storage to our container workloads and sure some of the interfaces are nicer than others but it seems like the the interfaces well not all that well thought off from the start at the beginning we thought that we didn't need persistent storage in our containers and later on we started adding them adding them as we need it and we ended up like this which and storage vendors that have to have a lot of different drivers so it's anytime we simplify this and this is where the CSI or container storage interface comes in it aims to be the sole interface for a storage in any container platform that's its same so but having we heard this before like a standard it suddenly comes a new standard that tries to be d1 and fails to do so and a year after you find out that it's just another one that comes into a list so the question is should we care about CSI is it actually going to make a difference and is it here to stay and that's what I'm going to try to March for this presentation over the CSI spec the first release that came out in remember and we'll see how its it works and what are it chances to make it one of the things that CSI is actually trying really hard to do is to be a storage agnostic storage vendors their deployment options their features and in this effort it supports many different infrastructure options we'll go over the two most common one the first one is an architecture where all the storage well the storage can be accessed from all your notes so every node can access to create delete and also connect the volumes in this deployment we will have CSI plug-in service running constantly on your nodes instead of having common line calls that are called from the orchestrator The Container Orchestrator or Co you have a service and using Google's RPC the orchestrator can make the request this way this way we decouple the implementation and also makes it possible to for the plug-in to be more efficient it is the responsibility of the CSI plug-in to support the management of the resources as well at making them accessible to the nodes this is a simple architecture but in many cases our deployments are a little bit more complex so we have our storage that have has different networks for the management and done from the data path so we will also have in our Orchestrator deployment different types of nodes we will have infrastructure nodes that can actually access the management to create and delete resources from the storage and we will have container workload notes which only need to know about the storage transport protocol to be able to connect to the 2-day data paths for example if we have a s kasi they only need to have the ice cuss initiated they don't really need to know what back end they are using just that it says kasi this is your connection information go connect these are the two most common deployment architectures in CSI and now we're going to have a look at the features the features were the developed or chosen between the orchestrator developers the vendor the syringe vendor developers and third parties that were also interested in the CSI spec so it is a joint effort between all the parts it is not only the orchestrator that decided a we're going to be doing this so everybody's interests are represented in here sorry in here these are the features in we can think of them as two different groups one group is the features that are meant to assist the orchestrator in doing its job when interacting with CSI plug-ins and the other group would be the features that are meant to manage the actual resources that we want in the assistant or in the helping features we have an info feature that helps identify the different plugins so that the orchestrator can know which controller goes with which node plugin because you can have multiple CSI plugins running and it needs to match them so that the requests go to the right place then we have a capabilities feature because CSI actually well most of the most if not all the features in CSI are optional that way it can adapt to all the different storage solutions so we need a way to check the the plugins and see what features are actually available in a specific plugin that it's deployed then we have a probe system that allows us to check the health of each of the different plugins they are a way to find out how much available storage we have in a topology feature because we need to know which nodes can access what which storage you wouldn't want to for example try to create a container in in a node and connect it to a fiber channel volume when the node doesn't have an HPA and it cannot connect it doesn't make sense so we have a topology feature that allows the orchestrator to do smart scheduling of the containers according to their volumes it's going to be using finally for the results type of operations we have two different type of resources and we can create delete and list volumes and a snapshot and also we can get the stats on a specific volume to see how much available space is there and also attach and detach the volumes these are basically all the operations that we have in the currencies I expect but the CSS spec is alive we already
this is version one but we already have new features in there for example we can already increase the size of a volume which is not here but it will be released on the next version we don't have time to go over all of them so I'm going to focus on three that I think you'll stay pretty well the different mechanism of CSI the first one it's the volume creation which like all storage related CSI functionality it must be idempotent this means that you can receive multiple tanks the same request from the orchestrator and the CSI plugin must make sure that it only creates one volume in this case and it always returned the same volume so the orchestrator will pass seven different arguments to the to the CSI plugin the first one it's called name and you can think of it as a request ID because it will be unique for each request and if you receive multiple times the same call you gonna have the same name this is what it's used by the by the plugins to actually attain idempotency thank you meters which is a mapping that allows the CSI plugins and the storage vendors to expose their advanced features this is how they support beam storage agnostic while at the same time allowed the storage to show their fancy features for example if you have a storage that supports compression in the CSI plugin says in the documentation hey you can use a key called compression pass it through and your volumes will be compressed then the CSI can expose that and it will come as a parameter this parameter are opaque to the orchestrator so the orchestrator doesn't care just take them and give it gives them to the CSI plugin so they will be different from CSI plug-in to CSI plugin and you have to go to the documentation to check what you have available then we have the source argument which allows it to define what type what type of what kind of volume we want we can create three different types of volumes empty volumes glom volumes which are volumes created from another volume and bonds created from a snapshot then we have capabilities which is the way of how the orchestrator tells the the CSI plugin how these volumes are going to be used are they going to be mounted used as a block device are they going to be mounted as a file system what kind of file system do we want to be there is it going to be used by a single reader single writer real writer multiple reader single writer you name it so the orchestrator must ask the controller what it wants and if the controller is able to do it it will return the ID of the volume to get the actual size and also accessibility to make it to be to tell where this volume is actually accessible from this is a simple flow it's a synchronous call you call it creates and when it's created you get a return value for a little more complex flow we have the attach the touch operation which if you read the the CSI spec you will find out that they don't talk about attaching and detaching they talk about publishing and and publishing volumes and that's how they refer to it first we will see a flow in the first architecture architecture we saw where every single node can access the the back end and so the container we will the orchestrator we will call the CSI plugin first to check the capabilities and see okay you are using architecture one every single node can access the backend so immediately goes to the node where it's going to create a container and it tells please publish this specific volume and make it available on this target and then that's it it is exposed so it is the node published obligation to make it accessible on a specific target but what about architecture to where we had displayed the functionality between the manager and the nodes now we have additional function first the control capabilities and you find out that you firstnet need to call the controller so you get a little more info because the controller doesn't know where it's going to be publishing this volume so you first querying the note what you're going to do place the con in the container get information and with the idea of the note you then call the controller and you tell it hey please publish this volume to this specific node for example if you were create a publishing nice Casa volume here is where you would be in exporting and mapping the the volume to the specific node with the initiator name and the IP address and from the controller then we call the node publish and make the final connection but the node publish if you have a multi reader multi writer can be called multiple times on the same node so if you publish first you publish for one container you get one call then you call create a second container accessing the same volume you will get a second published call so this is something that the expect is very clear you may receive multiple calls for node publish on the same node for different volumes so to assist the CSI plugin what you have is an an extra an optional call that can be made if the CSI plugin asked for it if it's called not a stage and it will go between the controller publish and they and the node publish calls right in the middle it will always be called on the node where we are going to be run in the container and this allows drivers for example that use NFS to make the actual mount of the volume on the staiin face because they know that this is going to be called one and only once per no damper volume and they can make it they can mount it in a staging path that is passed by the orchestrator and then when they receive a call to the node publish as many calls that they want all they have to do is be mount they stay in path and they are done so they don't have to to keep any kind of tracking how many containers are using this volume on the node they just know first I will get a note a call to know the stage i mounted there then B mount and the unpublished is the exact opposite you get a node unpublished they just need to amount the beam mounts and then when they get a call to the node at any stage it is when they actually amount the external storage this is a little bit
more complex if you have all the early in architecture to plus notice staging it is more complex and finally we have I have chosen the snapshot creation because in for most drivers for most plugins did will be this will be a simple synchronous call the orchestrator calls and say hey create a snapshot the driver cuts there's nap shot it is saved and it returns it is a single skull and the snapshot is ready to use but the CSI spec introduces one additional feature which is post card processing this means that your snapshot can be cut it is ready you return to the color as a synchronous call and tell it okay I have made a cut but the snapshot is not ready I have to do additional work so it returns in ready to use a false value so the the orchestrator knows that it this is going to be in a synchronous scope and it is going to be running in the background in the plugin and it can take hours to complete let's say if you are updating a your is an upshot to the cloud for example I suppose that processing and it is the orchestrator responsibility to pull and check is it already ready to use can I use it can it use it and since we like all the resource operation this is idempotent we have a very simple interface because you will be receiving the same parameters and you can easily check with the name if the operation has actually completed this concludes the overview of the CSI spec that I had in mind and now you like to show you a little bit of the specifics in in the orchestrator platforms right now the CSI is supported in docker kubernetes cloud foundry and messes and I'm going to give a brief overview of how kubernetes implements ESI kubernetes has decided to implement it as a mix of sidecar containers many cycler containers in in a cubelet the the kubernetes agent so it has code in a cubelet and it has side cards that you have to include in your pod so in for the architecture number 2 where we have the controller and the node we would have a part in the nodes running the your CSI plugin code as a service then you will have the no driver racer which is in charge of registering the CSI plugging into cubelet so it knows that it's actually running on that node in a likeness frog that can hoop the in to govern a dispro system to report the health of the whole pod the lightness probe is basically a gateway that receives HTTP requests and makes calls to be a G RPC to the probe feature that we described earlier so HTTP requests check the GRP see the status of the plugin and we turn it via HTTP on the controller or infrastructure notes we will have our controller side of the CSI plugin optionally also the lightness probe and then we start with the external provisioner which is in charge of watching your persistent volume claims and triggered of workflow of creating of the or deleting them if it is creating it will check the persistent volume claim or PVC and check the storage class join the information from the two and pass it along to the CSI plugin they if you remember we had the parameters in the in the create volume call that expose the extra features of the of the vendors they come in the story they are specified in the storage class so you can specify your compression equals true in the storage class and external provisioner will pass it to the controller CSI plugin and then we'll create a PV will bound it and it will do the whole operation to attach we have the external attacher C sidecar that monitors volume attachment and calls the controller publish call so this is only necessary if you have architecture number two where you have published on the controller side and on the node side and this is the first part of the attachment the second part is done by cubelet which thanks to a no driver racer knows how to what is the socket to talk to the node CSI plug in and makes the G RPC calls from that node to complete the attachment call in publish or staging calls finally for the snapshot we have the external a snap shutter which monitors the volume is not is not shot and the volume is not short class basically does the equivalent of the external provisioner it joins all information parameters from the from the snapshot class a the request from the volume snapshot and makes the call to the controller CSI plugin right now we have this is version one now we have the resize so we have yet another sidebar which is the the I think it's resizer external resizer and now you'd like to do a quick demo of ember CSI which is CSI plug-in that implements supports like 80 different backends it's it supports CSI 2.3 and once here on the same container and the example it's one one of the that are included in the repository it it launches kubernetes 113 with ember CSI 1.0 and using an LVN backing I used LVM because I don't want to favor any storage vendor and it deploys one infrastructure node and to work low notes it's recording because I didn't want to risk it and hopefully just a second because it is a physical quality and know why it's terrible all right let's see if it's uh this is not going yep this is not going well anybody can see anything nah all right awesome I cannot make it bigger let's see if I could get it's just a second
mm no same thing here right just a
second and this is slightly better or if
it's the same sorry about that basically I'm going to describe it seeing as nobody can see it clowns the repository changes that the directory into the examples one in runs a little script that uses vagrant Libert KVM and unstable to deployed to make the deployment I have sped up the the ansible part because it takes like 20 minutes so now we have completed the deployment we SSH into the master node and now we're going to check that the pots are actually running we have the controller in two node pots on the controller we have five different containers I'm not running the lightness Pro container so it's provisioner attacher is nap shutter the actual CSI plugin and instead of the lightness and run in CSC which is a command-line tool that allows you to make requests to your CSI plugins directly you don't need to go through kubernetes this is useful for debugging like for example you can run our list volumes without actually going through kubernetes on the nodes we are running the driver racer as I mentioned the Ember CSI container running on node mode and again the CSI container the CSC container the tool now we have the driver racer register with kubernetes use with cubelet using right now it's a CR D which is CSI node info dot C side of the story dot okay okay at eight s dot IO and here we can see that it has register the two notes that we have and that the controller is not registered there now we are going to split the screen to see
on the lower half the logs from the controller and we can see that we have received multiple probes we receive multiple probes because each there is a snap shutter and they attach here each one probe the the plugin to check that it's working so they are coming from different side carts and now we are going to create a PVC yeah see that the new cause all right we get a request to get capabilities because to confirm that the controller can actually create the volumes we get a little more capabilities plugging info and finally we receive the RPC create volume request with that is the name the what we call the request ID the name argument which was the request ID which we use for idempotency and then we create a volume and we return there that ID so we have now PVC that it's bound and now we're going this is a the gate volumes is an internal representation for ember CSI to to store the metadata of the volumes in kubernetes now we create an app that it's going to be using that if you see we receive the controller publish call and we return the value we will return that that the volume has been exported and mapped by the by the CSI plugin so it can be used for from that specific node then on the upper half we the login of the note site that where the control in the container is going to be started and we see that the driver is using a stake in face so we get a call or - not a staged volume and it returns now it is request the request come get the state's volume then a published volume and the volume at this stage is already published in the target path so kubernetes can already use it we check the status of the port and we see that the application hasn't is not ready in we wait a little bit and the application is running we can check the volume attachment then that is the object that they're at attacher sidecar is monitoring and we see that it it is there and this concludes the present day
the demo last second let me take it out
from there
so to conclude at the beginning we were asking should we care about CSI is it going to make it is it going to make a difference if you have a look and see the support that it's getting from the orchestrator and the storage vendors the number of Orchestrator platforms that already support it and the fact that all the new features are coming through CSI and the orchestrator are not bothering to add these features to their other interfaces only CSI is getting these features it is clear that they are betting on this and we already have support from the orchestrator we have a good number of plugins so in my opinion it is going to make it this is what it's going to look like storage in the future for containers and if you want to check it out easily you can go to member CSI and try it thanks and if there are any questions I think we have like two minutes for four minutes okay okay the question was how can we track for example as a CSI plugin implementer determine the the devolution of the CSI spec they the development is being done in a github repository they have a mailing list and they have now I don't I think they have increased the gardens to by quickly or monthly meetings everything is open to everybody and you can check the status of the pull request how they are advancing and check also the the meeting agenda to see if they are discussing a topic you can attend the meetings so usually the best way to if you are not as much interested in contributing to a CSI but implemented it the best way is to just once a week to go check they did hub repository see the new patches that art went in and say ok these are the changes now we have a way to resize and we can resize offline on online let's see what my backing can do and just implement it does that answer the question ok anybody else I don't think they publish it sorry the question was whether they publish a road map or not and I don't think they they do I haven't found it I know they when they are in during the meetings at some point they decide okay we won this feature this is a blocker they decide the priorities but it is not published as such you can probably you can check the the the levels on the PRS and see if they what little they have been assigned as a blocker as a nice-to-have or us this is going on the next release you don't have a page or something with this disorient you have to query the PRS yep I'm out of time thank you very much [Applause]


  356 ms - page object


AV-Portal 3.21.3 (19e43a18c8aa08bcbdf3e35b975c18acb737c630)