oVirt and OpenStack Storage (present and future)

Video thumbnail (Frame 0) Video thumbnail (Frame 1047) Video thumbnail (Frame 2118) Video thumbnail (Frame 2868) Video thumbnail (Frame 3998) Video thumbnail (Frame 5876) Video thumbnail (Frame 7564) Video thumbnail (Frame 8760) Video thumbnail (Frame 9874) Video thumbnail (Frame 10599) Video thumbnail (Frame 11426) Video thumbnail (Frame 13194) Video thumbnail (Frame 13925) Video thumbnail (Frame 17003) Video thumbnail (Frame 17848) Video thumbnail (Frame 18637) Video thumbnail (Frame 19896) Video thumbnail (Frame 21969) Video thumbnail (Frame 23164) Video thumbnail (Frame 24588) Video thumbnail (Frame 26096) Video thumbnail (Frame 26775) Video thumbnail (Frame 28319) Video thumbnail (Frame 31039) Video thumbnail (Frame 32156) Video thumbnail (Frame 33181) Video thumbnail (Frame 34434) Video thumbnail (Frame 35761) Video thumbnail (Frame 36518) Video thumbnail (Frame 37315) Video thumbnail (Frame 40746) Video thumbnail (Frame 42131) Video thumbnail (Frame 42952) Video thumbnail (Frame 44610) Video thumbnail (Frame 46349) Video thumbnail (Frame 49890) Video thumbnail (Frame 51304)
Video in TIB AV-Portal: oVirt and OpenStack Storage (present and future)

Formal Metadata

oVirt and OpenStack Storage (present and future)
Title of Series
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Release Date

Content Metadata

Subject Area
This session will cover the current status of integration between oVirt and the OpenStack image repository (Glance), analyzing the motivations, the low level implementation (including Keystone authentication), and ideas for the future. This presentation will include also an ample part dedicated to the future work and ideas to introduce the integration with Cinder (the OpenStack volume manager). * Introduction to oVirt Storage Architecture * Glance Integration Motivations * Glance Integration Deep Dive * Ideas and Future Work on Glance Integration * Future Integration with Cinder * Roadmap
Authentication Presentation of a group Musical ensemble Computer animation Data storage device Lebesgue integration
Computer animation Decision theory Data storage device Lebesgue integration Price index Limit (category theory)
Data management Computer animation Software Open source Order (biology) Virtualization Diagram Cartesian coordinate system
User interface Area Presentation of a group Service (economics) System administrator Multiplication sign Projective plane Virtual machine Data storage device Interactive television Cartesian coordinate system Particle system Workload Arithmetic mean Computer animation Software Different (Kate Ryan album) Computer network Bus (computing)
Service (economics) Computer-generated imagery Information Computer file File format View (database) Decimal Virtual machine Representational state transfer Metadata Computer animation Bit rate Order (biology) File system Right angle Lebesgue integration Form (programming)
Category of being Addition Type theory Computer animation Information Lebesgue integration Computer architecture
Computer animation Computer-generated imagery Personal digital assistant Single-precision floating-point format Right angle Template (C++)
Computer animation Computer-generated imagery Data storage device
Arithmetic mean Service (economics) Computer animation Computer-generated imagery Software Internet service provider Network topology Prisoner's dilemma Phase transition Lebesgue integration Right angle Web browser
Point (geometry) Authentication Service (economics) Computer animation Internet service provider Right angle Descriptive statistics Connected space
Domain name Service (economics) Computer-generated imagery Weight Multiplication sign Keyboard shortcut Moment (mathematics) Data storage device Electronic mailing list Planning Demoscene Latent heat Arithmetic mean Computer animation Personal digital assistant Different (Kate Ryan album) Operator (mathematics) Telecommunication Internet service provider Touch typing
Domain name Computer animation Computer-generated imagery Personal digital assistant Structural load Planning Representation (politics) Volume (thermodynamics) Heat transfer Demoscene Metadata
Service (economics) Computer animation Information Computer-generated imagery Internetworking View (database) Transport Layer Security Moment (mathematics) MiniDisc Representation (politics) Mereology
Point (geometry) Domain name Process (computing) Computer animation Information Operator (mathematics) Interface (computing) Data storage device MiniDisc Metadata Template (C++)
Computer animation Computer-generated imagery Data storage device Streaming media Metadata Demoscene Gradient descent
Email Service (economics) Computer animation Computer-generated imagery Personal digital assistant Right angle Heat transfer
Volume Arithmetic mean Mechanism design Server (computing) Computer animation Computer-generated imagery Computer file Block (periodic table) Personal digital assistant Volume (thermodynamics) System call Gradient descent
Multiplication Computer-generated imagery File format Moment (mathematics) Expert system Mereology Rule of inference Template (C++) Volume Sparse matrix Arithmetic mean Computer animation Operator (mathematics) Chain MiniDisc Data compression
Web 2.0 Word Sparse matrix Arithmetic mean Computer animation Computer-generated imagery Lecture/Conference Data compression Template (C++)
Revision control Volume Multiplication Computer animation Computer-generated imagery Mapping Order (biology) Gene cluster Volume (thermodynamics) System call
Authentication Area Service (economics) Mapping Computer-generated imagery Information Block (periodic table) Patch (Unix) Moment (mathematics) Set (mathematics) Price index Directory service Volume Computer animation Personal digital assistant Kerberos <Kryptologie> Buffer solution Videoconferencing Lebesgue integration Mathematical optimization Spacetime
Service (economics) Computer animation Computer-generated imagery Object (grammar) Associative property Identity management
Area Service (economics) Web portal Computer animation Personal digital assistant Data storage device Price index
Group action Backup Service (economics) Device driver Mereology Volume Latent heat Strategy game Lecture/Conference Computer hardware Lebesgue integration Computer-generated imagery Block (periodic table) Weight Projective plane Shared memory Data storage device Volume (thermodynamics) Instance (computer science) Demoscene Film editing Computer animation Personal digital assistant Video game Right angle Game theory Cycle (graph theory) Sinc function
Area Computer animation Operator (mathematics) Data storage device Planning Lebesgue integration Connected space
Distribution (mathematics) Computer animation Link (knot theory) Computer-generated imagery Network topology Arithmetic progression
Computer animation Friction Lecture/Conference Multiplication sign Network topology Lebesgue integration Quicksort Frame problem Template (C++)
Dataflow Complex (psychology) Implementation Computer-generated imagery Direction (geometry) Data storage device Volume (thermodynamics) Line (geometry) Human migration Computer animation Operator (mathematics) Lebesgue integration Right angle
Point (geometry) Dataflow Scheduling (computing) Multiplication sign Planning Formal language Revision control Word Digital photography Computer animation Lecture/Conference Right angle Lebesgue integration Whiteboard Local ring Form (programming) Stability theory
Web page Computer animation Counting Planning Code refactoring Lebesgue integration Price index Rule of inference Plug-in (computing)
Computer animation
hi my name is that economic and I'm working for editing the albeit
storage of group and I am here to present you the current integration between all the and OpenStack storage so presentation is must about
our integration between over an OpenStack but I will give you on short introduction about older than all the factors to get started on what the other in then I'll go over the present currently integration so what what we have today we
deal with and Lance integration always
in the story decide of importing exporting unless you induce a current limitation and then I'll go over the
what we would have in the future but with the glands future integration of our new teams that we can add and improved their a keystone out indication from and how he is also involved with storage is somehow all used and similar integration in the coming future so just start what is
already this is the definition that you can find around pretty much everywhere already is the virtualization management application that manages harbor no storage and
network resources that are in your that center in order to deploy ripple
machines but but it's open source really under the appearance of the Apache License and this is
a quite old diagram that you can find but nothing like we have some new things going on but that this is
pretty much the idea that we have and 2 different web interfaces of the administration particle and the user portal to interact with it and API than i j bus application that is the engine that controls the mean the that the centers and then on each node of the cluster yeah the descended takes care of controlling those no and I make them I will available and usable from the engine to manage basically everything from storage is the network at anything that the ripple machine would need to actually run I what these
OpenStack allergies cloud computing project that provides an an infrastructure as a service and also controls to large pools of compute storage and networking resources Apache license and I go over and by an Fundation that was established in September 2012 of there might be some overlap in the 2 projects on their tailored for different workloads soul today makes sense to make then leaving together in the same are infrastructure so that is why we need to integrate them are I want to go much further into the differences but of because the time is really limited so this is the idea so the present then you need some interaction between the 2 and them come as well as a sea OpenStack can be can have 3 different areas of projects in computer networking and storage we will focus on storage obviously the in the scope of this presentation and more specifically will look at the
lens and how that committees of the In the integration readable right so what these glance glance provides services for discovering reducing and retrieving virtual machine images that are either as a RESTful API for squaring of again induced me to be that and actually downloading that image of they can be these images can be thought of as a then place if you're familiar with that concept in the order of can also be used as back-up and has different Beckett's eventually ranging from the simple file system to so weak supports several the formats and containers formats and this form of his rope people and ISO and container formats as bear and craft you're probably familiar with these if you already use also of view and it supports the major metadata and
basically these some of these are some additional information that you can have on the images that are soaring glance on you can pretty much go over them yourself in the and decimation chew of them because they're kind of important in the integration but 1 is the size of this size has to be intended as the real size of the image of the file that is transferred uploaded or downloaded from that and the status and that's a special status this you with that and it is when you have the metadata they these rating glance but the image is not there yet because these are important for the integration so the we also have an
some an additional properties
that now more
standardizing wishing to about the and some addition information but the
is as the architecture of the hypervisor type and some other information on we decided not to use these common properties to store all the specific information about because we wanted to actually have a full integration we of OpenStack so that that information wouldn't have been co consumable by and by OpenStack so we are not storing any additional properties in in that's for now
and so the interesting use cases for over 2 that can be so
that we wanted to integrate we are importing and exporting single images or templates from and to LANs right for the interoperability with OpenStack and I We can be used for or are they out the images
of the ends and of and basically also use it as i all of the main to
talk of a over overtime something specific that is called the eyes of the main there and their used to store eyes images for the for the and the fraud against but we can try to think ahead to leverage plants also for that 1 and then the
final goal that is freedom reaches is to import and export of a full the ends and and 10 plates supporting completely and Phoenician multiple these and snapshots and so for the
initial phase we just focus on the 1st and find here which is importing exporting single images and the basically the debt that
integration has the of I
implemented using something that came along in
all its 3 3 and they're called
external providers so all in all there to tree we introduced the concept of external providers that are on while services that we are able to understand and integrates with right and and basically both open-set image glance and network as and also formant has been like they been integrated as an External providers in all the right so we have a new tab in the well this is the meaning you I and this is where you can add a new works some providers and this is this is how it looks like when you have glance and integration here so they have the dilemmas external provide when you define
it you have several feels that you have to feeling unwell name description you out of the
service and the authentication right you can also test the the connection at that point and this is pretty this pretty
much get you started it and as soon as you uh that these external provider and plants external provider you will also find and you
are storage domain in the storage tab and that is exactly the same as the mean that you just added and they're somewhat special but they basically are unattached to a storage pool up until now we always had domains that were and what touch to our certain that the center of the but in this case the they're not attached to any specific that the center and you can use them in fact 2 are important export images between and different that the centers what we for some operation that we do have from the all their perspective and we needed to interface with time OpenStack services but OpenStack provides at the moment I only by some bindings for that and and gene in over it is written in Java so we also started contributing to OpenStack domesticate as a mean of communication between the engine and the in the OpenStack services but the we tried to sterilize it and add up so the subcosts for things that we needed and so we can diving in 1 of the operation that we are actually I where intercity and aligned to be in the integral from basically and the the glass image discovery right so we want to lease the images that are provided by the blast of the uh service and basically when you select the plans of stars domain in the UI you will be prompt a weight and the you will have a list of images that are available or that you can access and that lets the so what happens behind the scene is a is engine
that the is the 10 the goes keystone requested and efficient gets token and then goes glance be blessed provider External provided that you defined earlier to get the image least there some kind of caching involved here and then eats the displayed in the U I and as a showed you before in so once you listed these images that are available in mind a glass service you
probably want to import them so what you do you can select 1 of the image and you can basically take the new button that is important and what happens is that the and you will be prompted we done and I Delaware you can select what that the center of 2 years and in which domain you won the image to be imported to and what quarter about 2 years for that image and what happens behind the scenes is
engine again going to he started to work with the token but it goes and fries and discover the image metadata to glance so that it can create on the internal representation of these images inside the engine and after we have better or a representation of yet of the images that the engines we go and issue the comments to be sent to create a new volume so that's the container where we will store the image and then we will ensure the load image I that we'll starts being that the transfer from plans to so we some will go the glass and download the image data into the volume that we just read so there is a special case of our where
basically yeah what blends the provide it was the the the the the major ritual size and so as a said before basically what blast the view is the size of the image but the thing is that you were not able to discover in advance large how big is the is is the image once the attach it to that the and and so on how large is the least that'd be the end see island these is needed for on the internet that a representation that we keep off the beast inside the engine so we also need this information and at the moment what we do is we go and downloaded the initial part of the image and we discovered that people had there and we peak for the virtual size of the disk said that's my provide some additional information about the middle size in the future but these basically works also for regular http so this is a more compatible just we a also other HTTP service that they're serving an image and we will see later on that we can support also that how these
in part to the image will look seen in and in all that I've
basically I can be found in their disks but tab and basically the metadata easement stained most of the and then some other uh information our forwards a with falls like for example the interface that is the to yield and based at that point you can create of Yemen or create a template in 3 4 we automated the process so it can be imported right away as a template and and so so at this point we can also look at the other and at the other end of the operation that is out
symmetrical to the user is exporting the SCS to blends and it's pretty much the opposite so now you go to the disk that any selected and 1 of the disks and you can export them out to a glance storage domain are you will be prompted you had prompted again with the dialog where you can select the glass the main i and that's
behind the scene will pretty much do the same engine go to key stock and well it will cheer the image as I mentioned before basically we prepare all the metadata about the image in glance but the real upload is done by the descent I thought the engine sense and upload the image in our quest to be descended then streams the data from the volume of that we have in our shared storage into the yeah
and investors these will look like as a regular image and some of them into the enticement pain like the need for example on and this could be used III is based on DVDs
and support for sharing images of others has been added specifically for that but it can be leveraged for other and metals of transfer of images and 1 so we have 2 new or API eyes that we use Our in and use their ability imagine upload image and we can issue different methods for the transfer and 1 of these these http that is the 1 that is used for the uh for glance right and we can also specify an additional headers but to and
so well as I said in this way we give support both for blondes ahead and also for all the other it's the service and there's another special case you know that we have to cover that
is when we download an image that is Q call on a block the main and so
this is team provision means that we create volumes there are a lot of lot the
means we use of the and we create
volumes that are I'm not entirely larger as the varietal size of the of the image we just created them with chunks that then our extended so as we as the and start using it and if fact to implement basically came provision on block a block devices and so what happens is that the 1 that is created is just probably 1 gigabyte of in size and what happens is that the descent has to go to the HTTP server for blends and download and peak for the In this case the actual size of the file that would be transferred and then it will extend the volume to the correct size before downloading In those in all these mechanisms we
have few limitations and
we are at the moment is limited to and not being able to expert images with multiple volumes and this means that we cannot export these that have snapshot or that are babies team provision on a template and this is because basically we want to export 1 1 image that is full contain that while we are using external snapshot 4 volumes right and and so it means that we if we want to explore the use of these images we would have to collapse them but I have 1 slide about that later on there is no alive VM disk as part of the the VM should be down and but uploading templates is always possible and there's no sparseness support from find the means for rule format and the images are always preallocated means that you're not punching holes in 0 since years of zeros at the moment so we download them from obviously compression with the what would help in this case in and we cannot resume our partial download of the and so we are moving out to the future and if I these are they ideas for the future improvements of the lands have we can add a few simple operations like deleting and images of a chain or land squash that is what I was speaking about so support these that have a
snapshot and upload the image
compression to save boundaries if and also supports sparseness probably and and well integrated upload and download via a web the meanings portal so you can upload the image you sort download images from your desktop car and they will be streamed and into the volume in and as I mentioned earlier restored IC images glance and use them that you lean and albeit as either the means basically an important phobia and some templates so and just few words about these questions a squashing and compression of
the of the multiple volumes images that we have been
over so as we see here we have an image that has 2 snapshot and that basically the base volume can be either or Q as a matter but snapshots 1 is much too far shoot you call volumes and In order to rule basically is um squashed the emergency just the latest version we have to compute their uh map of the clusters that are gathered around these 3 volumes and there will be a the support for the zinc in
thinking the image but it will give us a map of blocks and then ask other around in the in this volumes and then we will read them sequentially and after buffer rising then we can compress them and string them so this is just 1 of the video on how to deal with the problem without using some additional space where we have basically store the the image of just for the optimal of other improvements in the area of he's done our well all 3 for week to this out larger factoring off indication and directory services out integration gives you the I
do entity of the user and the directory service provides up on sets of information about the user the user of being the the decoupled so now you can use them of mixing and matching basically you can use the Kerberos authentication with the other the 3 service if you want to or any other combination of and keystone be keystone indication as is cast as 1 of the user beckons but at the moment we have patches of upstream and if anyone wants to contribute to the over data them and finish them is welcome to this so but this is the current status and I will probably have it in the future from 1 of the interesting use cases for this integration is that we can consolidate get indication keystone
for was that the centers albeit an open now we can also associate keystone users and tenants to all their 2 entities and will
basically be of specific per permissions and roles to objects that we have in all the earth like 1 of the and saw images for these and reuse the same keystone identity is when we're interfacing with these uh with the services with the OpenStack services on soul this
is probably how it will look like
I'm out indigenous this to keystone and you will be granted the BSS the access to the and web portal in this case for this picture but to the rest EPI or anything and then supposedly later on you can go to the services you're using those out indication credentials and the token probably and other services that in the storage area that we want to integrate sweetie is
the US and seeing the service that provides basically an
infrastructure for managing volumes and since policies for some big because previously it was part of the of
the compute node of no no since for has been in separating 2 our specific a project itself that is sitting there and then the volumes are persistent 3 right block storage devices that I most commonly are the through ice cuts for now we have some other that our slightly different for example Glaeser that them are attached as is shared and as a share of our mounted action is shared of volumes so and and in they have a life cycle that is independent from the VM instances so you can create the ends that are using these these volumes and destroy them and create other new instances that are using it but there the the coupled by and supports snapshots and the snapshot can be used for new volumes and back are generally archived in things we and the the use cases seen over to all we we can think of being part of importing exporting images from scene there but probably the most interesting 1 is sharing seen volumes for interval before the interoperability with all OpenStack and in that way we we will leverage the singer volume drivers that they have that for the support of storage appliances for IBM that up hardware than you C and D in these weight gain also the storage of floating capabilities the coming week the scene the volumes and we can also think to unify snapshot and backup strategies in your that the center also the for both older OpenStack world of 1 part of the integration would be to of a game going to the stuff for that indication and do some
simple operation like listing creating and deleting Wellington in seeing the and the later on uh when
we actually want to use them and we will come after we created over here was identified abundance that that that we want to use with the with some over to the we will issue will either set of the connection or created the and we'll and bad some way off yeah connecting to their areas while Toda storage that is provided by seeing the so this is pretty much what we're planning as future integration with OpenStack what I want to mention that we also have a toll tomorrow about networking integration between albeit an OpenStack to networking so that we
complete the picture on and I have useful links for you mailing list that if you're interested
and this progress self having glance over that our as our cuisine tree
for images that you could start downloading yourself or
your albeit a sense from a so
basically we will have images of the most important distribution there and you will be able to from from your installation to
download these templates are using glance any questions the some sort
wait for the mike so high how you think the integration with snow would take time we are planning it for a so I'd be all of ideal all that for release seems to be the so that's a pretty much the time frame the albeit for trees and that from friction and Center for example cults and up by you trying to going to support such things the we have not specific details it is out of how much that work so basically how much time would
have I not much and and so basically the
DVD probably integration reader of seeing there is that we have many flows of and complex and operation that we do on storage that there are specific to all their like for example on like a snapshot for alive storage migration that these are in the current implementation of our retailer to how holiday me at all how old the images are stored in the storage and how they look like in all your right and all this flow as my they don't map it all to in the so an easy integration now it's in there would be to leverage what we have raised today there are direct lines so we're ready support direct land the on and we have some operations there that we support but that's the easiest the integration so you basically can use in the volume says
that it plans for that reasons for all the
other complex flows it will require time on differently thank you at that board books 0 the release form of
0 is related to rather than at time or those 2 schedule to match so already and the log of the public about the who about what you also support my in word freedom father actually so as you see in your 7 holes rust 7 or some sort of as a hostess plan for or 5 for engine itself to run probably with the more of the interesting point probably using it as a host the the but what can we expect that this that and the right of way like it is the way you do a lot of back porting language to criminal stuff like that that you had some features from the photo to those release the backported to the actual or package of over the the I think we usually it's it's not an issue so much for orbital or have which is the directed version of it because we usually rebase each minor version so we don't have to do a lot of back porch for features was of the back Warsaw forced balization and we're going on at them in upstream lowered as well so just that board freed up free but free and I stabilization this for the current over the so a lot of pop back what about move upstream and downstream so so so all of the world so what would you that's too for sale at transcriptional should the the I joined a bit late but the time is the the the thing about local integration and all those things but um I've got a lot of customers from which all still road please don't
beat me up but that would still running the yellow pages of rules in this would show all plain Unix user back and
without any L of integration we got did some people oppose its hackles on that to extend the the probabilities of grouping public that is there any integration plan for just simple a Unix user by counts which are embedded extended uh regarding public egorical this and that you can use our as local mentioned there a lot of than refactoring of indication that can you can write a plugin for anything this so the the