AV-Portal 3.23.3 (4dfb8a34932102951b25870966c61d06d6b97156)

Exporting Ceph Object Storage data to the outside world

Video in TIB AV-Portal: Exporting Ceph Object Storage data to the outside world

Formal Metadata

Exporting Ceph Object Storage data to the outside world
RGW Sync Modules for fun & profit!
Title of Series
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Release Date

Content Metadata

Subject Area
RGW (Rados Gateway) is the HTTP REST frontend to Ceph, exposing a S3 and Swift API. From the Kraken release of Ceph, RGW introduced the concept of sync modules which allows for forwarding data and metadata to an external tier. This allows for interesting analysis of metadata or archival/backup solutions without the need to support these in Ceph itself. We'll take a brief look into the design of sync modules, peer into the crystal ball for what the future holds and also cover the currently available ElasticSearch and Cloud Sync modules, which allows data to be exported to external clouds supporting a S3 like API including Amazon S3. Ceph is a distributed storage platform that is a contender to become the future of software defined storage, providing unified access to block, object and file interfaces. Rados Gateway (abbreviated RGW from here on) is the Object storage component of Ceph, exposing an all familiar S3 and Swift APIs for object storage. Since the Kraken release, RGW introduced the concept of sync modules which allows for forwarding data and metadata to an external tier. This allows for interesting analysis of metadata or archival/backup solutions without the need to support these in Ceph itself. We'll look into the basic design of sync modules, pointers to look for should you ever need to write one, cover the existing ElasticSearch & Cloud sync modules, and see what the future holds for RGW Sync modules.
Synchronization Object-oriented programming Website Right angle Modul <Datentyp> Data storage device Endliche Modelltheorie Object-oriented programming
Open source Object-oriented programming Surface Price index
Point (geometry) Implementation Group action Beat (acoustics) Multiplication sign System administrator 1 (number) Insertion loss Mereology Login Goodness of fit Inclusion map Mathematics Object-oriented programming Endliche Modelltheorie Website Monster group Multiplication Mathematical optimization Tunis Time zone Electronic mailing list Sound effect Basis <Mathematik> Bit Lattice (order) Database normalization Website Text editor Directed graph
Point (geometry) Implementation Module (mathematics) Computer file State of matter System administrator Insertion loss Event horizon Revision control Frequency Object-oriented programming Computer configuration Different (Kate Ryan album) Queue (abstract data type) Object-oriented programming Diagram Endliche Modelltheorie Identity management Time zone Mapping Structural load Electronic mailing list Extreme programming System call Uniform boundedness principle Arithmetic mean Process (computing) Personal digital assistant Order (biology) Synchronization File archiver Website Video game Modul <Datentyp> Quicksort Cycle (graph theory) Directed graph
Canonical ensemble
so i'm going to talk about the exporting of objects initiated the outside world so of its avoid this think models which we introduce and that his country.
i'm so this is a brief talk it's only for fifteen minutes it's just the going to be talking with a g.w. and then have briefly about mighty site and then what think models are and in the recycling parties are the implemented right now.
what i see me really he already knows what surface because this is easily save woman it's pretty much the end of the play would resume people have attended enough talks would save and. i guess everyone knows what has the ability is so it doesn't object so expensive cluster and and that this was a less well so if the industry be a it recently translates your he should be her scores in too late for liberals cause.
and india you have the experience of p.p.a. being exposed and then of objects are not exactly when one is one of those objects from you would have for other over the normally the subject of implementation of uses apples and buckets and stuff like this and then there's already have the ecosystem of stuff. in its decline doing that can be leveraged already so you don't need to develop but the e.p.a. and the easy it is a good enough so these are the place that we're exposing. and yeah that a lot of for of history like features supported so you have a mighty part of goods here object were shunning gardens and you can lifecycle objects you have support for inclusion competition started with sites from luminous of which were made ready to search for the last six search. so from jewell release on woods we supported the concept of my the site which is this a geographic dependency costs if gustav objects to each and that is the basis for all of the sink models in excluding be does the outside world.
so on. basically my days a year lows for you know a geographical redundancy by a lake with his think beatification. so what are so it's not exactly on its income people excited he doesn't always have its unconsciously think i come to that i'm so you have the concept of for something called tunes his own bit of fun but items of the one would be any of you at a group of schools usually is a selfless the. it would be a zone and then you are resurfacing beat up within his own group and which is a collection of the ones and there's always a zone it should be elected as a master's in which would be this was a fruit for me to teeter and made our data that result we synchronous to the mosque the so it always goes through must also that is the. point which i mention we knew when i said i think of the matre d. does always synchronous of the monster but it's always a single honestly transmitted to the second we are the views and beat as always think honestly transmitted via the edge of abuse. so the basic optimize rebuild this on top of the fact that you didn't actually does it always very frequent in a cluster of it has made a deep the changes are not so much frequent and to me that you may believe that he just being to have a list of eight effect like when you create a you set out a container something like this it's usually something that would. have to be laid across to every of the selfless to where they're so that's the the mice with his objecting just as something that you would have no people as if this that and you know i think honestly transmitted i'm so we have made a c.b.e. in a local plus the of a peanut and with mr if you talk of what kept consistencies. so under the hood what was basically implemented as you have to mean the kinds of logs recently so you have loss would be ready to meet editor and admin logs and then every time this immediately the change this a notification same to them would soon which then goosen gets.
the object so you think one of those increases the majority to log in to. so you we basically have an ability to know before you've didn't want to win when there is already to change and this is the concept that rebuilt the think many years and drop off i'm so. i think autism says there will be on top of the mighty say a thing work on since we have the ability to actually you know synchronizing as england's leno deflated there are many maybe the team just and beating just for the cluster. we can leverage that and then a month they can actually not instead of writing to those afflicted itself just the latest the take a loss to something like a classic searcher of you in the late because to a different global though you make a notification to something like that. so the before i give you might be said itself was befriended a sink model which is to be forcing model in which it on when you actually create a i g. w. and. currently of all you think what is identity plus dozen requests in the woods. on and usually almost always a create a thing party was on on its own zone. some which was he was a plate of you as if plus the four just processing anybody goes something like this and you know design europe occasions like that. and in a few want to get started writing a single you should give us a thing for billion people to log think many of which that's not being other than just log the options and that is a great starting point if you wanted i started eating when you don't. i'm and that is the basic idea behind this includes and all we just going to cost to i mean the releasing point is we have been isolated and so any question so far. and apparently not so i was thinking that his injury is a classic search which recently since a mediator object of objects to plastic search so the reality of a good notification an object is created so this object made a twelve percent. the classic search. we are six was so and use a b.p.a. to the hygiene abuse of it being used to itself can make a last six or charities and that is actually forward it to a succession you get to this once back which is again for would have to do so on and that's a plus the administrator you can actually have moved interesting verizon because it's like finding out you know how many have roots happen if we have or. i like you know what sort of me to mean uses those what sort of files and stuff like this which is not usually possible if you want to just use that money via. yet it's a basic a diagram of our which looks like you have a second be as the reviews when we should be the essex its own and this is a standard diagram for the new think what you would always have a second reason that would be the was one for the think we're doing it was just what would it do what i was in the latest in this case it's a classic said.
i would think what you'll have it basically los you for the mighty kloden see have thought so you can actually back up. your street i mean your own eyes the beauty totally different plaudits up with industry that b.p.a. they come as on its feet says of any of the load that connects with a if the e.p.a. and it so technically you can even its was i mean the sender be taken as one of the gw without having to write it up to a mighty site of a. interview. and you have a configurable mapping how you map a user and a bucket in this cluster to the most is to your the local of stuff. and this has been used he says when you have a complaint you see this as something together you want to have the to blog i mean back to two different way that was something like this. this is a new feature a novelist which is an archive think model so i figured he supports is a feature called or object wishing which is in his sleep. the which recently allows for every object to have different versions so every that i immediately different the object that any would you pick but i do you plead with not actually be needy object but the different versions so you can do it back and journal be between different missions so the archives think model actually allows for example if you haven't. the good for different sites in every state has its own concept of a different version of jack's you could just do away with wishing indeed local slated to open just have one site which is the stark raving everything so there is no delete in the archive site itself just our capability which of your good that comes to it. so you can actually avoid watching at the local clustered in just two which me into the more clijsters who that's one of the use pieces without cave and then you would with have use cases where you want ok we'll have that sort. and in the this is also under the new money let's interviews and not the lists. so you have the steps that some pubs a model where you can basically subscribe to notifications. efficient events on a bucket so right now the basic notification reason we support the ico object create and you know just delete and of course if you're dealing with version objectives that believe market. i'm. and you can actually forwarded to a different teacher deeply and the queue. can be so will this actually improve list of the others and to visit should be making it then time for at least a point when to leave in order so that is about to sink model and for more details there was a talk in q one in seattle and that should actually what is a thing for you. and that's mostly my talk on. and the settling for the extremes of community and then christians. i. you know. i hear so. i want to live. but years old and out of pocket. i call on when it needed on the phone but it brought back to the working life. so the question was so without the more you use up with something that getting so that you can actually quite back separate the of and you know i was a pretty good at something better. initial implementation. so getting support business coming up a notice. the ending plus i gave it could be possible of human in that state the wii but it still has to be just about so. the revealing what looks itself is that you actually have would have i mean life cycling object to a different view and in what you would have to do is they want to get out caving in a way that this particular pool which is actually getting period as the one that is getting archived and back where you can sort of to achieve this. in its was implementation but you might still have to figure out that they might be tougher just go around. i'm here the said interest. any other questions. i like thanks to. i.