Merken

Distributed locks with Python and Redis

Zitierlink des Filmsegments
Embed Code

Automatisierte Medienanalyse

Beta
Erkannte Entitäten
Sprachtranskript
and so well thank you were coming from
all everyone did you notice that there are 3 jobs at the same time with water distributed to name but a robot involves that
have some sense of humor for sure well 1st of all like to introduce myself as a it was already did so on the software developer and the maintenance guy sort of focus token Poland my main always argument twisted and salary uh what they're doing at the companies to providing good business to business the telecommunications solutions right we hosted conferences so we so that basis and a lot of our main product is a contact center so it nowadays you have multiple ways to communicate with each other in their social media and the SMS text messages well there are e-mail and our phone calls so a contexts and there is a way to unify all these 2 the gain that much of the back your customer there's so from this I will
show you the moon principles of implementation in distributed logs in inredis using Python and this show the group usage in area life applications so everyone knows or the nest anybody doesn't know what to read this is OK there are some hand so I will weekly referred to that the this isn't always dual database it's basically a key-value storage but the key can be on the strength of the value the the more complicated data structure for the sake of the stock most important our string unless they're pretty much like biking counterparts and also reduces a simple support for a message passing and so what 1st of all I will introduce some case studies and I will show you how to implement it correctly
distributed binary semaphore so-called new text and later on some and finally I will give you some tips and there will be some time for questions so uh 1st of all this is so this is the definition
that I came up with a few days ago no biggie door on these he stock yesterday has David right example of locking keeping safe the reference counters your case would be that we have a shot database connections between friends and and we would not like to other friends sending some personal comments while in the other trade was and try to carry out the transaction so this is basically what looking is easy to protect the shared resources no so In the case study will concern a context center
hope nobody expected that so in the context of the of the war the people of course and to reduce refer to them as agents there's so an agent can the response or on originator phone calls and e-mail messages text messages or carry charity tap conversation and this kind of stuff well and and mine advantage of contacts and that it automates all this kind of work so I need to keeps the agents easy so they are not looking at the floor or a sailing OK and but do these discounted more family are and have replaced agent with over John W. that stands for a worker and 80 our task so you could say that this looks like the best QE who doesn't know need best QE solutions strikes Our q Germans are the only ones doesn't hardly any of them no there is good so basically another concept and this is the problem we are trying to solve the net influences
similarity you know the stuff you can apply a working solution that it will be great right run a dark couple things that you have to worry about 1st
of all work there is an optimal structure and it can choose whether to water or not if he can simply log out of the system and you will not be available anymore and this is a normal situation this is not the failure of system or some kind you can for somebody to stay in and war that this the people and well the tasks must must be prioritized and this is a heavily desired
feature from our clients so yeah I don't know how to use any other news distributed that's curious like so there was supportive of further only salary you can partially achieved this by looking to different cues and that's all so it is also not sufficient last mostly mind-blowing is
that 1 worker can handle multiple tasks simultaneously and from I'd I don't mean concurrently but simultaneously so in parallel at this from point of view of dust monitor so this will be viewed that way of course a human cannot carry
out to phone conversation at the same time but why not right and e-mail and speak to someone that's fine uh so our our solution and depicted as a whole
and centralized Task Manager and it's right and you lose using twisted and the radius will serve us looks to the server and will hold them uh so the 1st thing I like to say is about to binary semaphore is only 2 states can be rock locked or unlocked to get you to lock it I will refer to this as acquiring and to looking just releases the same as that standard but the library who has background with multiprocessing golden different library yeah this is quite a lot and it was then Monday talking this
yeah there's quite a few things uh so when you and it will be stopped other registration and and all calls acquiring or release will be non-blocking from by 2 so the 1st use
case is about his keeping state of a worker uh whether he's busy or idle so we know meaning that lock is released it's the same as idle and and so on and the lock will be acquired to when it does Canada will sign a task and it was release of so it is over in and the 2nd case is about preventing race conditions and they wanna mentioned that the worker would like to look out sometimes and I don't know go for a break or something so that he can you do it using this mechanism because it can acquire a lot before Task Manager do an audit will simply do not to release it after finishing it's the previous task and OK so as soon as this is stored in this threads the string we will use to reduce comments that a set to restore along so the stream and then get to retrieve it what we need just optimistic every single column on the right is is an atomic plant so no 1 else can disrupt your data uh that is so the 1st draft I think we have a crown this was something like that so we getting that log checking if we can lock it actually and then do it but there is a problem because noise between get and set there is some instruction running and entered this as a whole is not atomic so when someone else might has changed this in the mean time and this is a problem and it is not acceptable them so we introduce some kind of red is transactions that this is not the real transactions in a sense of SQL databases but it allows you to execute a bunch of comments in an atomic weight and we issued the command watch and giving them the name so this will be carefully watched during the transaction uh and finally you further we cannot do do anything then we on which is sold this connection will be free of side effects later In the next step would be to introduce the transaction notions multi and these actually will denote transaction people the things that should be done conditionally only if I watched he has not been changed so I guess that's quite simple staff OK then important part that's just sometimes would like to know whether the state change where when we look at the work he was find it eventually but uh when we do not uh when we are extremely it it must know when the worker street so again actually perform some action like for example lucrative for the next task for it and to do receive unifications we issue subscribe content but to be more precise on the bike inside so this is the best manager another state that is written in twisted there is some kind of magic like uh look like they're ready subscriber will use handle their stops subscribe and the actions for them for you know and this is the event-driven using their very ugly metal name must receive which is certain common and and by overloading method you actually defined In the case of for and so on the 2nd part will be
about distributed semaphores so in the
summer for is different from working that way that it maintains an internal counter uh so it means that you can be released or acquired too many times for it we define a semaphore with counter to we can acquire it twice and also really sick twice much the original counter cannot be exceeded by far the calls and this will be stored as friends list and the calls to it will be blocking and so the use cases that you can handle and multiple task at the same time we have a stated before their use well what is not sufficient and the use of a bunch of close binary and some office would not be sufficient to so this is similar to string operations but now we are using our approach which stands for the right so it will up and an element to and you know better pop which stands for blocking right spot so it will pop an element from the list and are pushed to serve as the as a release operations and the looking the right of of the acquiring operations so when there's nothing there and means that semaphore has been acquired from maximum number of times then their common will block until there's something there some in the but part would not change anything because it to react only on defense eventually you can implement your checking whether the Center for a free or not and there are cases when you might need to state of semaphore after changing it
it's you could use previously showed will the exact blocks but that's not always the case and alternative is to write a simple last week you see that it's very much like right and in this case except this lost local so it means that reallocating the local variables so it will be a garbage collected after this blog and this script only releases our long and to return the land that we have left I will say about there's warning need the exact blocks and the lowest rates of all our atomic indirect the problem is that blocking innovations in a atomic weight have not nonsense and this time so it would mean that the red this instance has to be blocked for ever to serve this 1 so making verbal beside mother exact will return new so it's not hanging by the client or and the developing such a lost it will registry responded with an narrow uh or alternatively you can use and arable so it's non-blocking feature it will return new ideas new if there's is looking at the wire or value if it has a some
final remarks and before you brush and implement the solutions in your software uh this care about starting conditions and make sure that there blocks are properly instantiated before using them and study carefully control flow in your application to not introduce too many lots or to less and they also please work up a restoring stick procedure when something goes wrong there might be cases when the state the art in red is is not the final
ecosystem so you might want to fix that somewhere for example we've birds this state in an SQL database some kind of history of activity of a given water so we can restore state properly after some kind of failure so well that's all for me and now it's time for some questions you have some coffee thank you
you the don't have you consider using sets and X for the lock the set annex commands the means of an right is
used for all of yeah I be we have read about this and this is described also on the register conditions other no about well this is an expiring comment as far as I know yeah you can use it as a non inspiring commanding going into sediments by yeah but
uh mean it
somehow we didn't have the governing I can't remember now why I'm about knowledge was sufficient for this kind of walking yeah but it's an alternative for sure short OK have you will try to look at other
implementations besides the do the not the implementation and reduce client you mean besides the
river right yeah this is still the summer this class as well as the ideal lead would be to write your own looks server and vital part of the case is that some called which we have and the most is reading each the soul because to be something outside the hi how
well when you're talking about distributed semaphores and lots are we talking about of so our 2 locks in 1 location on 1 red instance and the workers can be distributed or model locks some force he had this is year that's a good question
and I hope that someone asked their uh well and basically we use a single red instances for serving our lots of due to different limitations that we do not need more resources but distribution and you can achieve is by idea charging credits instances and this is done automatically by using for this cluster starting from registry point all uh or replication which is also in understand but when I prepared assignment distribution by the workers of distributed and in this are different kind of resources that might achieve a lot so acquire it does this process a question OK
any more questions you have you tried it I'll have you thought about using ZooKeeper and I'm not entirely familiar with it but my colleagues at work use it and it's supposed to that which is you know synchronizing the few distributed of those instances of anything to with only you know but get long resources but uh well we didn't
and that I have read about there's something the legs of keeper just yesterday on all really stand but I will assure you consider that self-interest suggestion but no I don't have any experience with the 2 but it seems like a good idea yeah if it would be simpler or better in some cases why not replace yeah because it's used supposedly to you know that you select the name node in the Hadoop cluster for example so this is also a of this is readers to the system and only you know you have to you think about it and that synchronize it so that supposedly for that yeah I will allow investigators for sure and if I came up with something interesting issue I would give a talk at the next occasion of the this year I think OK I think he there any more
questions that aware that you have 1 minute and and yet she ever seen anyone talking on the phone and right in the medicine time yeah myself deaminase to make
this is you know there are some studies that show that human that carrying out use the simultaneous tasks these the complete to reduced by its brain capabilities so we did it was there was a comparison in the book I have right that to he behaves as if you had a brain of 60 6 years old trial sold you know I really don't find personally that multiple task at the same time our means super queuing feature model that's what the business wants yeah yeah because they respond with some of the life of there is maybe the any more questions I guess weak at 10 and I think you're going to present
Roboter
Metropolitan area network
Prozess <Informatik>
Wasserdampftafel
Red Hat
Vorlesung/Konferenz
Baum <Mathematik>
Computeranimation
Roboter
Telekommunikation
Gruppenkeim
Implementierung
Kartesische Koordinaten
Login
Computeranimation
Softwareentwickler
Speicher <Informatik>
Datenstruktur
Beobachtungsstudie
Parametersystem
Videospiel
Datenhaltung
Token-Ring
Biprodukt
Kontextbezogenes System
Fokalpunkt
Quick-Sort
Softwarewartung
Flächeninhalt
Basisvektor
Hypermedia
Dualitätstheorie
Schlüsselverwaltung
Baum <Mathematik>
Message-Passing
Beobachtungsstudie
Einfach zusammenhängender Raum
Semaphor
Datenhaltung
Speicher <Informatik>
Kontextbezogenes System
Binärcode
Menge
Computeranimation
Metropolitan area network
Transaktionsverwaltung
Message-Passing
Mailing-Liste
Rechter Winkel
Baum <Mathematik>
Informationssystem
Managementinformationssystem
Umsetzung <Informatik>
Schaltwerk
Familie <Mathematik>
Systemaufruf
Ähnlichkeitsgeometrie
Kontextbezogenes System
Computeranimation
Data Mining
Eins
Task
Task
Endogene Variable
Übertrag
E-Mail
Baum <Mathematik>
Message-Passing
Portscanner
Task
Client
Task
Wasserdampftafel
Vorlesung/Konferenz
Physikalisches System
Datenstruktur
Computeranimation
Kreisbogen
Portscanner
Task
Umsetzung <Informatik>
Task
Punkt
Sichtenkonzept
Gruppe <Mathematik>
Vorlesung/Konferenz
Parallele Schnittstelle
Baum <Mathematik>
Computeranimation
Kreisbogen
Binärcode
Radius
Subtraktion
Server
Semaphor
Systemaufruf
Systemaufruf
Computeranimation
Zeichenkette
Task
Datenmanagement
Task
Programmbibliothek
Server
Vorlesung/Konferenz
Semaphor
Ereignishorizont
Baum <Mathematik>
Aggregatzustand
Mereologie
Multiplikation
Gewicht <Mathematik>
Stab
Gruppenoperation
Geräusch
Aggregatzustand
Computeranimation
Task
Streaming <Kommunikationstechnik>
Multiplikation
Datenmanagement
Task
Kontrollstruktur
Thread
Phasenumwandlung
Implementierung
Software Development Kit
Soundverarbeitung
Einfach zusammenhängender Raum
Managementinformationssystem
Kraftfahrzeugmechatroniker
Datumsgrenze
Logarithmus
Datenhaltung
Semaphor
Ausgleichsrechnung
Menge
Zeichenkette
Transaktionsverwaltung
Eigenwert
Rechter Winkel
Konditionszahl
Mereologie
Baum <Mathematik>
Zeichenkette
Aggregatzustand
Multiplikation
Mereologie
Gewicht <Mathematik>
Web log
Extrempunkt
Zahlenbereich
Oval
Element <Mathematik>
Computeranimation
Task
Variable
Trigonometrische Funktion
Mailing-Liste
Task
Äußere Algebra eines Moduls
Skript <Programm>
Skript <Programm>
Semaphor
Implementierung
Nichtlinearer Operator
Semaphor
Systembereichsnetz
Mathematisierung
Systemaufruf
Mailing-Liste
p-Block
Bitrate
Systemaufruf
Office-Paket
Sinusfunktion
Portscanner
Rechter Winkel
Mereologie
Bildschirmsymbol
Aggregatzustand
Instantiierung
Euler-Winkel
Datenhaltung
Wasserdampftafel
Kartesische Koordinaten
p-Block
Aggregatzustand
Algorithmische Programmiersprache
Computeranimation
Software
Kontrollstruktur
Vorlesung/Konferenz
Bildschirmsymbol
Baum <Mathematik>
Aggregatzustand
Arithmetisches Mittel
Menge
Rechter Winkel
Konditionszahl
Baum <Mathematik>
Computeranimation
Vorlesung/Konferenz
Baum <Mathematik>
Computeranimation
Client
Rechter Winkel
Klasse <Mathematik>
Mereologie
Server
Implementierung
Vorlesung/Konferenz
Baum <Mathematik>
Ordnungsreduktion
Computeranimation
Distributionstheorie
Subtraktion
Punkt
Prozess <Physik>
Semaphor
Computeranimation
Informationsmodellierung
Forcing
Datenreplikation
Inverser Limes
Vorlesung/Konferenz
URL
Baum <Mathematik>
Instantiierung
Konfigurationsdatenbank
Knotenmenge
Güte der Anpassung
Physikalisches System
Baum <Mathematik>
Computeranimation
Instantiierung
Arithmetisches Mittel
Task
Beobachtungsstudie
Videospiel
Informationsmodellierung
Rechter Winkel
Vorlesung/Konferenz
Paarvergleich
Baum <Mathematik>
Computeranimation
Roboter
Metropolitan area network
Red Hat
Computeranimation

Metadaten

Formale Metadaten

Titel Distributed locks with Python and Redis
Serientitel EuroPython 2015
Teil 103
Anzahl der Teile 173
Autor Buczyński, Sebastian
Lizenz CC-Namensnennung - keine kommerzielle Nutzung - Weitergabe unter gleichen Bedingungen 3.0 Unported:
Sie dürfen das Werk bzw. den Inhalt zu jedem legalen und nicht-kommerziellen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen und das Werk bzw. diesen Inhalt auch in veränderter Form nur unter den Bedingungen dieser Lizenz weitergeben
DOI 10.5446/20200
Herausgeber EuroPython
Erscheinungsjahr 2015
Sprache Englisch
Produktionsort Bilbao, Euskadi, Spain

Technische Metadaten

Dauer 22:29

Inhaltliche Metadaten

Fachgebiet Informatik
Abstract Sebastian Buczyński - Distributed locks with Python and Redis Traditional methods of coping with concurrent programming problems are well-known and described in literature. Many programming languages, including Python, contain in their standard libraries tools and primitives such as semaphores and can spawn threads or subprocesses. However, in the face of increasing interest in service oriented architecture and building distributed systems, that span across many independent server nodes, emerges a need to adapt traditional solutions, so they can be applied in the new environment. In this talk I will share my experiences gathered during building a modern contact center - highly concurrent system, which requires certain resources to be accessed exclusively by several self-contained components.
Schlagwörter EuroPython Conference
EP 2015
EuroPython 2015

Ähnliche Filme

Loading...