0 views

Citation of segment
Embed Code
Purchasing a DVD Cite video
Series
Annotations
Transcript
lots of stuff that you can plan is going to be talking Marconi OpenStack hearing notification service and you know there's a software engineer the clouds the thinking so thank you I'm so I
mean I I'm both nervous and excited to be here on so I'm working for that are mostly around virtualization I have been contributing to the over it's mostly around the storage
area and to OpenStack to them work on
a project so today I want to talk to you about colony which is occurring in notification service for open and so what we're going to
talk about today is why do we need a messaging service for OpenStack what is my point I want to give you a high-level overview of what to on actually looks like and it will also give you some might use cases about how you can actually diploma Corning and use it in your own cloud environment so let's
talk a bit about are the project's history so the president doesn't really have a big history it's pretty I think it was I started on January 2015 Albayrak space and Red Hat but it was incubated into OpenStack come during the price so sad that cycle of is currently production and we really hope it will get into the next generation to open standards and so
1st of all before we start talking MIMO corny I want I want to talk a bit about OpenStack sold
here you can see a picture of the OpenStack services so of OpenStack is basically of a set of services that are running distributed machines throughout your infrastructure of around and they're pretty much independent to 1 another but they also need to talk to 1 another so currently very using centralized and message broker in order to do that and what we want to do with work on is introduced to you and alternative to this are message broker in cases where the message broker is just not good enough for not secure enough for you use case I'm from so
as I said we have a lot of independent services in OpenStack and there are also a lot of our messaging technologies which means different languages and we want to have 1 unified way for the services to talk to 1 another so
basically we have a missing piece in OpenStack of what we want to have a couple of things so 1st of all we want to have a queuing service for OpenStack we also want to be able to have and notification as a service this means that we want to service to be able to publish messages to other services in OpenStack also we want to have a very lightweight messaging API out which means it will allow us to integrate our services you using 1 unified API that will be a really simple to use and it will cause no extra cost of but you're infrastructure while the blind so how are we going to do that some of you are probably
wondering on from talking about just yet another messaging broker because we have so many why why would you need just another 1 and there is also a really nice joke about it are you probably know and so on and so I want to
reassure you that you don't have to worry about American we are not aiming to replace any existing messaging technologies what we are aiming to use our existing messaging technologies and to sit sit on top of them and use them also Marconi is not any means to be a task manager we have salary which is distributed dust manager and Marconi aims to talk to salaried is able to talk to 2 sorry and we want you to see but to sit on top of Marconi and use it also my point is not a cue provisioning services it will not allow you to install any of your underlying technologies used still have to install everything yourself and deploy everything himself he was still use the messaging technologies you're using today but you will use it broke on on top of these technologies so now that
we know what we're colonies not I let's talk about what we're he's so my money is a really simple and lightweight RESTful . API and what it means to do is to unify your existing our messaging technologies what do I mean by unified it means not just to sit on top of them but also that you could use different messaging technologies in your infrastructure in it and in your
deployment at the same time the so what we're going to do
is on how to create an open source alternatives to SQS and as soon as I do you know these services by Amazon OK so I am basically as few as an SNS are both as services by Amazon but as the as is a simple during service and as soon as a simple notification surveys all in the first one provides you to use up and the 2nd 1 provides notifications they're both separate services and what we aim to do with Marconi is to supply these 2 services as in 1 single service which is the recording and we end up point to be used as the application running on opens the clock so now I want
to talk to you about a few really exciting use cases we have former colonies so the first one is actually to deploy using them according service so you will just have your own action service and you will be able to sell cues to your users I also like the rest of them and use cases are a means to be used by our OpenStack services and so on the 1st 1 is the rise notifications so when you start instance and whether it fails or succeeds our eyes and our get semantification so on you are able to do it today but I'll walk like I horizon is just bowling but we want to be able to get notifications so it will be able to do that are using Marconi also we have thus customers service so that it generates events and ask statistics based on notification he gets from the other OpenStack services I am currently does that using this centralized message broker of OpenStack on sorry so it is using the same GP message broker and it uses a really low level API and what we aim to do is to give it a more high-level API so it sits on top of that messaging technology of and we want to add to our against agents are intercommunications which means you have a lot of gaze agent running inside of an instance and if you have a fairly you will want to be able to communicate to the other guest agents or to services in OpenStack are currently you are able to do that using them message messaging broker but in this case it's just not secure enough so we want to introduce an alternative all so now
I want this to move to a high-level overview over whether corny actually looks like solve mankind's architecture is pretty simple on it's composed of 3 layers are the transformed the API and the story and and a transferred is the actual protocol other declined stocks to and the API introduces you to them of colony of research says and the actions it can perform on those through the protocol and the storage are the actual messaging technologies that recording talks to the underlying messaging technologies we're using so a really important thing that I want to mention about American architecture is that it's composable so you can outplay with it much like as if it were labeled breaks it is obviously lagging based in order to allow you to do this on an age value has to conform to a well-defined API so you can just use their transport and the storage you want to use and just are used a suitable so on top of the
transport layer we have that the authentication middleware is actually provided by a 3rd party currently we support to authentication methods which are at keystone and basic HTTP so we keystone you basically have all of them will determine the features they'd already allows so you will be able to use are multiple talents in multiple project under the same Oconee deployment at the same time on obviously you can write your if if you want to you can write your own authentication that can just write a flagging from according to use it the on the transport layer we currently support and that our production protocol which is http this is what we use how we plan to support we targeted TCP for generally on on on the API layer we expose appointees research says so the messages are the main and most important resource in the colony that is actually what Marconi is all about it's about delivering messages so these messages can be red posted and claims from acute which is a logical entity on and you can claim the message from the queue this means that all when a worker claims a message from a Q the other workers can process those messages at that time now also you can configure all of your messages accusing claims in terms of teacher so the stars there are at RT actual messaging but acknowledges you using which you will deploy by yourself in your infrastructure so currently we support to messaging that technology the first one is that 1 with the being and the 2nd 1 is still carrying out we have both of these 2 items already and excuse me the so as you alchemy is not really recommended for production environment but it's not really good for a she stands out because of its performance so what we use a mortar recommend for our production is the moment it be planning MIT currently used in that are Rackspace Cloud services and and we also had targeted have already support for Don OpenStack generally so Monge DB will allow you to have fully durable accusing really persistent cues and on the other hand you will have radius which will allow you in memory support so if your application needs a really high throughput the to allow you to use so let's move on to
how you can use and deployment colony in your own infrastructure so we have 2 ways to do that the 1st 1 is using a single storage faster so you will have multiple recording nodes running in parallel EU regional structures on top of a single storage clusters and the storage can be whatever you want whether it's moment already used or whatever our future application needs you just have to write the like informal money and he just he chooses according to replication needs another way to deployment corny is using storage pools and this means I just think before you will have multiple announced are there still running in parallel but instead of sitting on
top of only 1 storage class today will have a student job of multiple independent clusters so you
can choose either classes to be whatever you want and you can configure mercati to talk to you to these classes on the cue based this means that if you want to you to be fully durable you will choose it should be on among the DB Search cluster and if you if you want to be in memory and to have a high throughput you will use them Our ready storage cluster the so
let's move on to the really great things about my so and the 1st thing is that obviously it's open source so we all hear a lot of open source but it allows us to have a really simple a unified API which means by you will be able to use multiple messaging technologies but it is also providing us with the 4 guaranteed free accused of which is something that rescuers does not provide so obviously this feature on depends on your underlying storage and currently all of the storage is support supporting the after the before granted and sensor Marconi is configured on a queue basis you can I make sure that you you will be free from but we also provide you with storage pools are which allows you to use different messaging technologies at the same time and our control that triplet of your applications also a really important thing Obama is that it's really easy to scale just as long as you have enough nodes in your infrastructure you will be able to have as many more pronounced as you want and as we talked earlier about the use cases of of target terminal money to be used by open such services and that sense it's really lightweight and easy to install and displays nicely with everything else you have be in your infrastructure so you just have 2 your and it is so before
we close I want us to talk a bit about our plans for the future roadmap for Data General release so something really excited about having a and flavors which are just like in the novel compute flavors that allow you to configure instance so the q flavors will allow you to configure your so if you will just be able to create a Q and choose a flavor whether you want to choose to be a fully durable are to be in memory you will be just able I had to choose a flavor and then when will choose the right storage classes for him also we aim to have like migration of cues so In case you haven't really have you read or have the right to you and it's basically just killing storage pool you will want to be able to migrate to another storage pool without any downtime to your application and I don't think we need to do that we already discussed is having already that will allow us in memory cues and allow for the future we aim to have aimed keep support we have a lot of discussion about this topic upstream I was also planned for about generalities but we had some problems so if you have some knowledge around the idea of messaging again and a unifying technologies then we'd really like to hear from you and to for you to join in the discussion upstream and
so after this style OK really hope you got a preview of what American is and what they actually aims to do and if you're already using reclining which really allowed to hear feedback and if you have some new and interesting I use cases after the stop and I would really love to hear feedback to our can conduct so that the the do you have any questions without need questioned this think that it thank you very much we can to get them he is also the size of the so kindly
recordings used in production in their Rackspace Cloud services we're not really familiar with any other a production environments but it is production you are just welcome to install it and try and right so while I understand that economies of them so well and of some of my committee supposed to be a like core building block of the stocks and to make some before requirements uh what else makes it and they can kind different than older being solutions consistent with people questioning it's not uh competitive let's say and secure Kafka or all the messaging acknowledges so call it's not yet another committee cannot express service in competition with an acute and cover that so if the user during integration and things but it is not another message broke down of what I
mean by that so it is basically just giving you another level of isolation and all of your messaging program so I obligation be able to just use a really simple and high level API and not the only that underlying messaging technologies API so if you're currently using the name the worker and decide it's not good enough for your application needs and you want to change it then you will just be able to deploy another message knowledge like let's say for instance just installment of being you want really high kind I have fully durable cues you you will just deployed in technology and you won't have to change any of the of conditions will win in the Cuban can open political economist Robert computer whatever other service to whatever it wants us storage room with what of the reliability of the different parts of the final product is the most popular 1 but not the only 1 that is that you will be able to use the and also I know it's that's the 1st thing by so when you say that my colleague could be used to let different of the sex called to him that needs to 11 new driving also messaging actually I I don't really have assurances from that so I really like like 1st of all they are thinking and any more questions OK thank you very much I think
Benachrichtigungsdienst
Web service
Software engineering
Computer animation
Point cloud
Open set
Area
Benachrichtigungsdienst
Web service
Red Hat
Computer animation
Data storage device
Projective plane
Virtualization
Open set
Point (geometry)
Standard deviation
Enterprise architecture
Product (category theory)
Spacetime
Electric generator
Red Hat
Cycle (graph theory)
Projective plane
Point cloud
Bit
Web service
Maize
Computer animation
Integrated development environment
Personal digital assistant
Cycle (graph theory)
Message passing
Web service
Exterior algebra
Computer animation
Personal digital assistant
Order (biology)
Virtual machine
Bit
Set (mathematics)
Service-oriented architecture
Benachrichtigungsdienst
Web service
Message passing
Computer animation
Telecommunication
Web service
Independence (probability theory)
Message passing
Formal language
Independence (probability theory)
Point (geometry)
Web service
Computer animation
Web service
Queue (abstract data type)
Service-oriented architecture
Data management
Task (computing)
Task (computing)
Computer animation
Multiplication sign
Point (geometry)
Statistics
Group action
Open source
Archaeological field survey
Horizon
Point cloud
Horizon
Cartesian coordinate system
Event horizon
Web service
Message passing
Centralizer and normalizer
Arithmetic mean
Exterior algebra
Event horizon
Personal digital assistant
Personal digital assistant
Energy level
Statistics
Service-oriented architecture
Row (database)
Read-only memory
Group action
Enterprise architecture
Multiplication sign
Authentication
Control flow
Image resolution
Web service
Term (mathematics)
Queue (abstract data type)
Data storage device
Computer architecture
Authentication
Multiplication
Product (category theory)
Moment (mathematics)
Projective plane
Image warping
Cartesian coordinate system
Message passing
Radius
Computer animation
Integrated development environment
Data storage device
Order (biology)
Communications protocol
Middleware
Cluster sampling
Multiplication
Dialect
Moment (mathematics)
Gene cluster
Parallel port
Independence (probability theory)
Student's t-test
Cartesian coordinate system
Independence (probability theory)
Single-precision floating-point format
Process (computing)
Computer animation
Data storage device
Single-precision floating-point format
Vertex (graph theory)
Data storage device
Data structure
Row (database)
Social class
Cluster sampling
Scale (map)
Read-only memory
Game controller
Open source
Characteristic polynomial
Multiplication sign
Open source
Basis (linear algebra)
Cartesian coordinate system
Open set
Independence (probability theory)
Radical (chemistry)
Web service
Computer animation
Data storage device
Personal digital assistant
Web service
Vertex (graph theory)
Electronic visual display
Data storage device
Social class
Read-only memory
Feedback
Planning
Bit
Instance (computer science)
Cartesian coordinate system
Human migration
Single-precision floating-point format
Summation
Human migration
Computer animation
Data storage device
Personal digital assistant
Personal digital assistant
Right angle
Queue (abstract data type)
Social class
Installation art
Computer programming
Building
Product (category theory)
Block (periodic table)
INTEGRAL
Consistency
Decision theory
Expression
Instance (computer science)
Mereology
Cartesian coordinate system
Computer
Open set
Web service
Message passing
Computer animation
Integrated development environment
Data storage device
Core dump
Energy level
Row (database)
Condition number
Loading...

Metadata

Formal Metadata

Title Marconi
Subtitle Queuing and Notification Service for OpenStack
Alternative Title Marconi - OpenStack Queuing and Notification Service
Title of Series EuroPython 2014
Part Number 108
Number of Parts 120
Author Kaplan, Yeela
License CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
DOI 10.5446/20049
Publisher EuroPython
Release Date 2014
Language English
Production Place Berlin

Content Metadata

Subject Area Information technology
Abstract ykaplan - Marconi - OpenStack Queuing and Notification Service Marconi is a multi-tenant cloud queuing system written in Python as part of the OpenStack project. Marconi aims to ease the design of distributed systems and allow for asynchronous work distribution without creating yet another message broker. This talk aims to give the audience a broad look at Marconi’s design and technologies used. ----- Similar to other message bus frameworks, Marconi's main goals are: performance, availability, durability, fault-tolerance and scalability. Besides providing support for queuing and notification services through OpenStack, Marconi aims to ease the design of distributed systems and allow for asynchronous work distribution without creating yet another message broker. This talk aims to give the audience a broad look at Marconi’s architecture, design, technologies used, development process, and discuss the issues it adresses.
Keywords EuroPython Conference
EP 2014
EuroPython 2014
Loading...
Feedback

Timings

  349 ms - page object

Version

AV-Portal 3.7.0 (943df4b4639bec127ddc6b93adb0c7d8d995f77c)