Merken

Bringing PostgreSQL towards zero downtime migration

Zitierlink des Filmsegments
Embed Code

Automatisierte Medienanalyse

Beta
Erkannte Entitäten
Sprachtranskript
but thank you very much for those things alone so yeah and then at the Reebok was this migration may or may not be a problem for you that's my % you hear something which we believe is so quite new quite another thing in what way a week and handed down to and that's
sold the ratio the 2 more consensus tree is not Lorentz invariant and so we're doing so that unites the
restaurants and bars let's things right this statue and actually I'm also allege 1 of the commenting and procedures and use of mono engine so strange diagrams much money would present something of prosperous that's this was usable both and there may we would see that's actually using polymer batteries that energy can help you also to the new concepts to maybe to start some um who here is using prospect is making migration the other 1 you're lying and um pool of already had problems with migration from the dominance of events the major events of the here is that 2 main cases with migration so we can have something quite simple you may extend new table new column batteries and generate the kind of stuff was just quickly but it's going out quite fast and 1 of the problem very small founder of marriage himself that is some possibilities also to combine basically 2 versions of gold and combine actually make migration serialized that when we burn something in the next 1 we bring back which made etc. which is something quite different because that's the some gave us some of which you guys who raised and lastly know about so we have a very different business logic humorous make a strategic change in the company which requires the use migration which unfortunately needs a lot of also data handling and these several the entries of the dozens of minutes of downtime kind of stuff nobody really likes to do so that even more if you have some number take oriented companies have for this we're going to use prosperous 9 for and strictly 9 4 9 5 some of the things that you will know see a lot of about logical decoding OK yeah it's unfortunately
not so no 1 that is actually really great from basically logic decoding is the level of replication the 1 last person who is using representation in press the 1 from last not from mixture right it's about audience so basically when using replication there's some write-ahead logs which is a system of course was to make replication work and you have different levels of applications some hot standby this kind of things that you already about this logical decoding is basically next level was introduced and for which was basically to describe what is happening and at this point and reputation in a logical way to save meaning precisely His name is annotate greater than having some blogs and so this those square of that goes with it and you can extend it with all of the applicants and um what is also very nice and what this is replication that's the other than that it's just to replication and dressed in other than which 100 replications maybe with some parameters and site that's on both Chris you can have slots which means the difference so there is a soul can read from different slots which you can stop at different times which can have different levels and even more so than lobster all you all the trust but so slowly introduced starts to perform and to make some sense I hope we have basically this is what is
possible our toolbox we're going to use what adjacent which is a great tool which was not written by certified the 4 of which we're using because there is some hope for the compilation something just proposed is going to be tied across 9 4 alternating made it's actually really great 1 of the the writer had lost so this internal of the possible expectation of language and have them you understand the movement of the BG migrated to look sorry module with the jury vitamin cycle to this is where here and which analysts to do some sensor stuff that 2 main modules basically which 1 is the switching from a traditional physical replicating said to logical replicating 2nd must this basically when you have to do is notice that this vertical replication you are not anymore you can choose not to be any more in a master-slave and have to master and here we come to treat that we say that the replicating to Mars and Jason receiver which basically was about some of the and of lesson transformation so what we're going to to have a migration which of whatever size we had the then just because the very few seconds of downtime is the following we start with classical replication switch logic verification we also then on new switched secondary former slaves which is non master repeatedly logically we apply the migration looking at then we replace the migration we posted replay what we understand the time transforming data on the slide then we have a variety of applications by some applications that can't size and we write directly to the numerous this is the concept I hope you follow this becomes graphs and then like so this is starting point idea is this this the 2nd part of the state of which change of its way basically both communicate 2nd method also delivers the old state becomes and you must migration then we enabled basically water adjacent in size is actually a C writing directly communicating directly with the possible would replace that we go through what about this is that major migration which states what needs to be have transformed the removed and everything is possible and right what we want to be written in the new areas that have been used to and basically when you are the time you have your data which is transferred to the old like and you'll notice is transferred them but we still right there make a small downtime switched have room redeployment even more that's easy cases make which have you could make the choice directly to the you must although you can do whatever you want with it actually fuel production you have to have the right to have 3 or 4 you can say that identification and prosperous very easily so that you must that you have because security matters as we yeah I guess that continued directly with the light they move said some very urgent questions next we other especially longer life question during a special OK and I think I would like to that would that's who knows me motivate this uh I tried to make it somehow reasoning to with the Koran we are on the 1st on the left because you grow you use when I'm talking to master visited him and vibrant and I have to adopt containers for master and slave establishment of a state in which in is the green 1 is this the usual and the 2 last economy for some yeah the texts are right so that here we're going run faster it starts with some classical called I could also have a list of but you don't have the tools the database is quite high something you reconstruct that and understanding of the and and and a I and on a here we so in which 1 given that of I was write where and the interesting thing but there is 1 of the 1st things that just try this way when you think about how to was my mistake had needed in its present form but for the most simple and fast is uh right here we continue so we can see that these are the kind of the wall right that love which is the kind of thing that OK this is written because of the has and here
we go on I just got to clean that have so so we can use and interesting but what do we have to have the knowledge 1 alright that here we can basically my name of you like 4 think our policy that it's not previously is we need some random because you never know what they actually and what we can do the same thing here the excuse me I to the structural change you also that you believe and cheating and have the same thing in the back alright so as you see that have wonderful replication replication works
and you may know what it's just too make it even more obvious we can do something like this the it's going that's what this but we and now
we can do something else where greatly we're gonna in said something to the um still accused of moved here
between the whatever I'm going to go through these windows parents just prior to this so I think to explain with it but what it's doing is basically creating their own data and inserting some value of those mostly so I think that's the density of check again we can
also use I think that think the should be this alright and also here I think right so again this is a quite trivial but not so this is a the case where it's that is basically your customers which continuous steps you want in various ways and to do you the product of such migration so you're gonna use the 2 interacting with little in the numerator and use this as components of yeah right unintelligible to switch of of and the I see some of the application of this means that the right is that so we have we can define themselves the various sources of energy so special etc. and uh we can define some structure for the testing thing the I think to think that there is something but we created this led the which that we use the trigger 5 so this is the classical way in the past but when you have application using device to make state people must of have this is also in case you want to have some form of the given that the so now we have must and we're going to change this would OK so have no reason to I'm going to go back to this is the stages state of the of use before so this is the master of the master students must master status this thing that this led this is a list of things that have be a master so we don't have exactly the same in the kitchen it's not exactly what we want to do is to do some fancy since things which basically all migration so for example we got out of the and had a new kind of and eventually we can also learn so this this is the language and we can some data we rely is that you without the old info and thank you just particularly 224 roles that this sounds where and now we can go to the next step we're gonna implementation because it's gonna be too long of this what we can continue of parents this so and this dataset and all the rest of the time who is jailed receiving basically no worse in the states and not the case in which people because that's of the data migration that the new model is doing nothing more than the other 2 so which to basically with what we want which was basically I was quite fast for why because basically it went back all which were missing it replaced with and now it's going to slowly because like on the fly as you can see adding info and for the values that were based on the action this 1 this 1 here uh so we will not have my which because much from basically no life there to must and the 2nd 1 is working you could already right some of the data if you really wanted to have 2 systems right and things or synchronizing you can try going to talk about the basically what you would do at that time the moment where you switch you are you making it more small content to be sure that there are no parameters right the nature of and you switch your system in which the goal behind so there is no way we can actually that the sort of we can make sure that he 623 and on the old master also 623 and so on on the slaves we can also some of you Madam Justice green 1 we can also check whether it's someone which has no where near it was not and there 0 I see what's good news just when you go to a right so that used by the system must it was and know but this time of nature some words about the future for sure would like to develop an open source of it's the baggage the last thing I think it's time to develop and so we implemented so that really on these versions style some components having a column for the state security we need a lot more details for a more complex stuff we need some serious testing subset the communication and things on but later in the week with integrated with by the Fremont and eventually so if there is interest that the community willing to recognize that it's it's possible to have a general package which which is basically check the migration 5 and ultimately trade to transfer of objects who so those who have that for energy that of that was in fact I would like to thank offense metaphor is the college market word of on but that's not like
thank for your I think adult and able to show the code that does the transformation from he also kind of blogs screen 1 more certainty so this is
basically the structure see that this is is about this uh
and so this is basically the 2 main
components and the make major and I implemented we haven't asked that transform advanced which we just so against 1 or the other operations and this which facts basically how you do the tragic right and so this is like the base that and you will for example uh here the you can see well but this is basically the way you would have to add new data which of you in this way the the average of the and and there is a very simple man is a status to the other side was the right uh but basically you have plans for you reduce of whatever you want and the receiver its connects directly to build a lot of writing and right so you Jason which is better than another is OK all William but and of course there are some operations that we have thanks for the talk
that did just that some the solution for more advanced so structures like age store or a full text search FIL and full-text search I don't think that things can go wrong but store must be more complicated legacy from basically to right the the transforming images this is a quite flexible structure I guess everything is possible but I did this so I can this the Jason it's still Our analysis and I just such of a possible set up for a hot standby of of of groups could be implemented using similarly so hot standby it's level of replication and the logic reputation is basically the higher so logic replication coding to that would also put the presentation on their on also right inside and with Richard and we have also some links to the arguments but basically this is just the levels of annotation and when you do with the logic of repetition you have included the full of hot standby it's just something on top of use this you get this 1 step and you can think of thanks for talk and I'm not quite understood it do you know when do you know that you can speak to your application 1 denote the replays done this is basically something which formalize not automated I don't know if it's completely all made of everything you could listen is written on them a and check the moment where nothing was written so it depends which kind of fade out you have uh you could also use some kind of changing mechanics for the new 1 if you have a group green uh deployment system you could even do really hard so that led to the main problem there is actually a sequence that it's actually except sequences there's nothing which prevents you really completely of making a complete hearts which we sequences you would have to attributed this by restricting them on state to a lower space which is high enough for what to expect to coming from the master it's true that surrounds where chemistry um as basically you would that down you the data inputs and to have a system that you money to well enough that you know well enough you know then when is the right point to start right you must the only problem could be the some some in between sequences and this is why you should make sure that you this needs to be really tough walk on the old master that's basically do you want to have it for moderate cost I haven't is this possible to have the interplay between 2 different database systems and in peace synchronized was was was was that it is possible to think about synchronizing constants something else like animals my Creole each my extra dimensions so that we can model for example if it's a chapter Jason that the what um I think it's quite obvious that you can do it so it's basically be out of school for that nothing to do with the matter the migration it itself that uh since you can have this flexibility with the replication starts and iterative replication and even multiple possibilities you can definitely used something slightly similar to that do indeed right some multiple to some like for example with the adjacent to so unless their mom wouldn't have raised against been tried that's the interesting and by graduates possible even more I remember in mind we would have documents which are more complex and contains data from several tables you can have some that's intelligent this is also the table right now I see prevent also even transform on the right what was the now question to looking to see if can so I don't know about that so that I may have to admit that there are a lot of other persons rubrication master master schemes and prosperous and so we wanted to basically to repeat afraid all these tools which all came to be the best also on on how to sponsor was supported by prosperous so we wanted to to work actually was created internal itself was just sitting also be next becoming developments and yet is system of that there might be for for the purpose of all the different solutions at the end of questions OK so
Migration <Informatik>
Orientierung <Mathematik>
Subtraktion
Server
Invarianz
Datenanalyse
Mathematisierung
Versionsverwaltung
Zahlenbereich
Maßerweiterung
Kombinatorische Gruppentheorie
Mathematische Logik
Computeranimation
Netzwerktopologie
Migration <Informatik>
Front-End <Software>
Mobiles Internet
Punkt
Maßerweiterung
Tabelle <Informatik>
Singularität <Mathematik>
Übergang
Ereignishorizont
Algorithmische Programmiersprache
Keller <Informatik>
Sinusfunktion
Energiedichte
Diagramm
Funktion <Mathematik>
Räumliche Anordnung
Tabelle <Informatik>
Punkt
Web log
Compiler
Selbstrepräsentation
Formale Sprache
Parser
Hausdorff-Raum
Kartesische Koordinaten
Ungerichteter Graph
Oval
Login
Computeranimation
Übergang
Metropolitan area network
Verweildauer
Datenreplikation
Auswahlaxiom
Parametersystem
Computersicherheit
Datenhaltung
Klassische Physik
Übergang
Biprodukt
Arithmetisches Mittel
Zusammengesetzte Verteilung
Rechenschieber
Funktion <Mathematik>
Rechter Winkel
Decodierung
Aggregatzustand
Varietät <Mathematik>
Server
Web Site
Subtraktion
Gerichteter Graph
Mathematische Logik
Wasserdampftafel
Mathematisierung
Transformation <Mathematik>
Maßerweiterung
Mathematische Logik
Erwartungswert
Migration <Informatik>
Front-End <Software>
Gammafunktion
Demo <Programm>
Videospiel
Zwei
Programmverifikation
Physikalisches System
Modul
Quadratzahl
Flächeninhalt
Mereologie
Dreiecksfreier Graph
Streaming <Kommunikationstechnik>
Portscanner
Inklusion <Mathematik>
Metropolitan area network
Dualitätstheorie
Mathematisierung
Datenreplikation
Gammafunktion
Metropolitan area network
Bildschirmfenster
Vererbungshierarchie
Dichte <Physik>
Offene Menge
Webforum
Momentenproblem
Web log
Natürliche Zahl
Formale Sprache
t-Test
Versionsverwaltung
Kartesische Koordinaten
Kardinalzahl
Extrempunkt
Computeranimation
Metropolitan area network
Große Vereinheitlichung
Softwaretest
Parametersystem
Softwareentwickler
Computersicherheit
Güte der Anpassung
Quellcode
Biprodukt
Speicherbereichsnetzwerk
Teilmenge
Diskrete-Elemente-Methode
Rechter Winkel
Bildschirmsymbol
Aggregatzustand
Telekommunikation
Gruppenoperation
Implementierung
Transformation <Mathematik>
Code
Informationsmodellierung
Bildschirmmaske
Migration <Informatik>
Bus <Informatik>
Vererbungshierarchie
Zusammenhängender Graph
Inhalt <Mathematik>
Datenstruktur
Touchscreen
Videospiel
Open Source
Mailing-Liste
Winkel
Physikalisches System
Quick-Sort
Maskierung <Informatik>
Objekt <Kategorie>
Energiedichte
Wort <Informatik>
Tabusuche
Nichtlinearer Operator
Dualitätstheorie
Automatische Handlungsplanung
Computeranimation
Sinusfunktion
Metropolitan area network
Mittelwert
Rechter Winkel
Verschlingung
Zusammenhängender Graph
Datenstruktur
Gravitationsgesetz
Große Vereinheitlichung
Metropolitan area network
Gammafunktion
Subtraktion
Folge <Mathematik>
Punkt
Momentenproblem
Gruppenkeim
Kartesische Koordinaten
Kombinatorische Gruppentheorie
Mathematische Logik
Raum-Zeit
Computeranimation
Übergang
Migration <Informatik>
Datenreplikation
Datenstruktur
Softwareentwickler
Speicher <Informatik>
Bildgebendes Verfahren
Analysis
Kraftfahrzeugmechatroniker
Parametersystem
Datenhaltung
Nummerung
Physikalisches System
Störungstheorie
Ein-Ausgabe
Binder <Informatik>
Konstante
Ablöseblase
Aggregatzustand
Tabelle <Informatik>

Metadaten

Formale Metadaten

Titel Bringing PostgreSQL towards zero downtime migration
Alternativer Titel Bringing PostgreSQL towards zero downtime migration with Python
Serientitel EuroPython 2015
Teil 142
Anzahl der Teile 173
Autor Rigal, Matthieu
Lizenz CC-Namensnennung - keine kommerzielle Nutzung - Weitergabe unter gleichen Bedingungen 3.0 Unported:
Sie dürfen das Werk bzw. den Inhalt zu jedem legalen und nicht-kommerziellen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen und das Werk bzw. diesen Inhalt auch in veränderter Form nur unter den Bedingungen dieser Lizenz weitergeben
DOI 10.5446/20160
Herausgeber EuroPython
Erscheinungsjahr 2015
Sprache Englisch
Produktionsort Bilbao, Euskadi, Spain

Technische Metadaten

Dauer 27:44

Inhaltliche Metadaten

Fachgebiet Informatik
Abstract Matthieu Rigal - Bringing PostgreSQL towards zero downtime migration with Python Using an SQL database offers a bunch of advantages; first of all its maturity and that it is understood by almost every software developer. But it has at least one main disadvantage. As the data is structured, if you want to modify the structure, for example on a long-running project, you need a migration and therefore almost for sure, a downtime. When you have to make a migration, to modify the structure of data for a small amount of records, it is so fast that it never gets problematic. But if you think to modify the structure of tables containing millions or billions of records, the time required to simply apply the structural change is problematic. Here are some changes we are working on at orderbird to go towards zero downtime migrations using some of the latest improvements of PostgreSQL 9.4, mainly logical replication and mixing in a little magic of some python scripting with psycopg.
Schlagwörter EuroPython Conference
EP 2015
EuroPython 2015

Ähnliche Filme

Loading...