Merken

From Legacy to Admin

Zitierlink des Filmsegments
Embed Code

Automatisierte Medienanalyse

Beta
Erkannte Entitäten
Sprachtranskript
in a in a year and a half and I move around a lot is lazy and that you guys are coming up of know you add many other options so that the end of this talk and I hope to be informative and about inspect each of which is a general comment that helps you of reverse engineer model from an existing database on
within Inspector Chris for all but I go by monodomain Chris of when I was practicing this I ran along some Fig kind of fast to trying as much information as they can on this is me there's a
professional Christian and the regular chris so that you can talk to me afterward so you know your
talk to graduate not professional cross but right now
it's a professional cross on so something to the talk that I wanted to give our don't right so don't write code unless you have 2 and my name is chris not data come up and if you do something to do something hard which I know that you probably doing it wrong and this and is something I realize I started using Django initially on when I say so when I created a custom command to populate initial data and I realize that there were fixtures of so this happens a lot when you like when you start to work on something you don't read all the documentation or you see some cool part of it you wanna do not realize that there's other things of so my advice is you know to spend as you come the general con and go of Gaia Iousy of gold forums and there's a lot of rich information there's Fletcher such gender and that's and there's a lot of like custom
apps uh templates TL have like their suggested ways of doing things so inspected B is 1 way to save a lot of time and if you work
with legacy systems so instead the reads tables and columns not rows in a database and produces a model stop I've standard out of have to manually write this code and you could you could well you know create a class that inherits from all the model and give an explicit TV DVD it like mean and and you could go through every table in your database but this just jump start the process for you it's not perfect so we'll go over why what what we got do afterwards but
it works pretty well and when you need this I and 2 scenarios have come up new this idea that I have legacy legacy system listed in this Annex 1 admin interface to it of so foreign keys are grade of the primary key constraints a great result mean always and strengthen my sequel comes with but in reality you have like an additional of validation layer to let the system that you want keep it clean state you can you reverse-engineer model at a couple validations and you'll have this slicker admin interface of the general comes with for free on the other use case that I've come up
with is a pretty management commands us so when you create management commands for going through much of rows of a database and making sure that 1 field is less than another field or you won't find all of rows that have you know a certain value the or em comes in handy for 2 for doing these things and writing custom sequel is sometimes hard of particularly if like you know it's not not designed to be a programming language designed to specific things in sequel that's another reason why this is come handy in the past so
credit created a fictitious scenario of y of why I'm doing this demo and I have a legacy system with a legacy databases and I can't touch the code for whatever reason I and I need to be able to manipulate data but still keeping a clean state and so I can have the source and there's a
lot of reasons why you India was at the source you have you have a source of to third-party product it's written rails or of you know the source principle it's impossible understand also at the end of this demo I
want I want to be able to have an admin interface that's there was little work as possible and you will have a basic right operations and 1 be a little leave the existing database intact more or less and we'll talk about how we break that when we have to break that so many ways to ways to get around it but a more or less what that leads the system it's interacting with this code to to be the same after
this also the tree from the docks
insect avait and database and whatever's pointed so the 1st thing you'll notice here is that of whatever your settings of name is pointed to death with can introspect of and it's also in this state the statement doesn't seem like it says a lot but it actually says a lot the script once metadata asynchrony model for each table on so something that you don't think about is that you often have more tables than models because of many to many relationships and inspected the inspection don't don't understand the concept of and also the most important part of this is this
feature is meant to be a shortcut not is different among generations so I am I created a
fictitious model yeah so it's pretty
basic there's a user table I made a lot of like egregious errors don't blame me I designed this to be terrible on purpose so that we could understand what would happen if you designed the is not yet a work with is the terrible manure expecting them they're not to be perfect so every user table it has 1 primary key it's our incremented it's not null and has username and then of course it has a point X that's work of we have a company table has 1 of the primary key is Otto incremented if that in a minute self-referential so a company can have a parent company and so it's interesting corner case and then this goes on before a user can belong to many companies and many come and a company can have many users and their related by this user has company of many many table so this relationship in in uterine Django you have a user model that has a many-to-many relationship or a company or vise versa but there's really no way of of work of gender inspection to know this so we're missing at what this action produces a 2nd and another thing that's terrible but this referring to the Django at least that is that has 2 primary key so has composite primary keys which is gonna be a problem for us some of the y the of Agenda basically since the you have every single model this in the of 1 ID field that has a primary key and this is doesn't fit that pattern and then profile budgetary thought community worse has a triple a composite key and there's a role here so role is meant to be a foreign key to this table role but this is of our car and the name years so as to match the role in here there's like completely worst interests and you can imagine there can be so many problems with this obser anomalies deletion anomalies like you name it so that's but smashing model and then just for fun if we have time at the end and go over this table because I created like the worst of field names possible just to see what they would produce and so the source of inspect the
be all and this is the source of new it's good size I guess so that there's 2 primary lives the 1st loop other users connection that infection table names also connection introspection is a of connection specific introspection function to get the table names so depending on what database you're using it'll get it'll user differ command to get back at list of table names but then once it has that was the table names of its agnostic this code of the rest of goes agnostic to the type of database you have so I looked at the source code for other the different introspection definitions so my sequel oracle and posters because all of them and you look at the documentation it says my superimpose grows over pretty much was before spices and and oracle has the same function introspection functions methods defined to introspect database so I don't get a chance to test oracle but I don't see the reason why when work and I was looking online it seems like a lot people driving to work so maybe I can look up that later in the talk about that but so once you get a table name and at that 1st primary loop the secondary part of the loop of the inside that loop I guess relationships gets indexes and then it has a secondary loop to loop through all the other columns and at the end of this and ardently do the job of the screenshot but the end of this it generates a collocate yields a the output of to the use an output of the column name to standard out so that already used so this was a little quiz I made
a these are the intersection Pacific commands that my seat so that gender uses that to get table names so my he goes to show tables are select tables for Oracle and something knows about these these each of these commands is not guaranteed to come in any particular order social tables just gives you a list of the tables of it doesn't give you the table sorted by you know when they were created and use the table sorted by of the name and all these are the same so we go through that loop it's basically going get up and there's just no way to guarantee what order it but so this is
what we talked about before the we use the connection introspection about to get the table names relation of new indexes and the columns but then we have agnostic at the next layer to generate the models once we have all information from our a connection and this is the many-to-many fell of
like I said this should map with a many to many on the the use of the company field depending but this maps 3 individual models so let's take a look at
what would happen if we do this so Python managed to apply it to the show so I created this in my sequel so we'll take a look at us so these are the 6 tables we started with and we haven't think the database so we don't have any of the gender-specific of tables so the question is do we wanna singer database or do database introspection 1st the database introspection looks to the database and create a model from the DB creates ah since a database of to the correct point of that were out right now or whatever we have the and and uh right now so if we run our sink database 1st order create tables that of gender needs to be run at to log in and you can't older users but then is we run uninspected days after that those tables will also be in our database and they will be introspected as well and that's not really what we want because those are those models already defined somewhere else around them inspect B 1st the so like I said goes directly standard out of which is not very useful but you can take a look at it of things so tells you to do things which is nice so I had to be be giving you talk industry this you rearrange the models you make sure that and everyone has a primary key and if you want you can remove managed falls on the doesn't really say with that's gonna do besides its can allow that gender to have a lifecycle and allowed to rename the models but you're not allowed to read DB tables which is like the Met option to explicitly state the table name is to override the convention of and then there's this line which you know I'm sure won't be important all you have to insert the output of gender and sequel custom app name your whatever so let's go ahead and put this to so I created map for this you have and let's assume we
produce this is same thing but worse in a file now of and now we can run our safety the the so the 1st from the run into is of related name collision so let's add a related name to 1 of the the namespace collisions so so it really so looking at a model of user has company user user and by the way like my sequel and generated these field and so I'm sort of the terrible have 2 ways the change them but has to foreign keys to user has company and the problem with that that is that the name space for complex so that the accesses to get the results that that will be of will be the same without a generates so battered women into 1 of them
the the announcements the about to again think a database now but so its use and related name told us to do that so was easy enough here's me now to understand me some right so sensor we have that AV-ASR database and now we can run a server
so it work now and it's running so that we have and Florian so I added something
daughter register all the models and so we can look at it right away with look at our company and create a new company so this is weird there's a company ID and I normally don't have enter my on primary keys so let's look at
what went wrong and this is a common thing of the integer field has a primary key but it's not a field and I know that it's not a field because I
created a data model and the company has autoincrement but I can
somehow edit the order implemented value which doesn't really make sense so to fix this
week kitten and our field the
back right so that disappears now the auto increment in the background and security company my schedule not just because that annoys me
I'm sorry for my use of members the now it's user the the
also going on so usernames likely Chris
and my password to the passwords talker the OK so those with the 2 easy tables to map they had 1 primary key there are incremented inmates messed up on identifying it's an auto increment field but that was easy enough to fix and even if we didn't we could assign
a value there of who the next
thing is gonna come up is these composite primary keys so we don't have a primary key listed here so there's multiple ways to approach this
of so related name we talked about
that only have to add my primary keys no also generates from primary key a
composite primary keys of so this always affects this you can drop the primary keys and keep the foreign key constraints and you can add a new field called ID and Janos so like model assumes that there is an ID field anyways and you can make that NOT NULL primary KEY in autoincrement so you're adding this extra fields to each table that has a composite key you just reducing the composite key to this extra field so I
did not memorize how to do that so from the
TV show the so i for the 2 tables that composite keys imatinib call my ID into primary key auto increment and before I did that I dropped the primary keys there existed which were those composite primary keys and then turned to quickly and for the other table for this user's company which the same thing and there some my sequel of book that I have to get around proper foreign keys so I can mean enablers afterwards but for the purpose stomach it doesn't need to be OK so I have a company I have a user and I want put that user in that company without many-to-many relationships so that that come back and cruel role and I know I want to role to the a worker bee the the now inside of my profile from unaccredited profile and given 90 and the work p Prentice misspelled B so we want this role of to be consistent with the role stable even others not foreign key relationship and it's not allow us to do that right now because there's no validation on the role field so that's not going to write a check and see what we can do about that
but
so in our profiles on added validation to ensure that happens and I'm a borrow most of this from here to that of its
foundations are the the import rules so
when import validation Aaron importer roles and if the text for the profile role does not match something that we have in our value list for the names and then we will raise validation romantic so we have 1 profile at that state and then another 1 that's supposedly in a good state or B so worker bee at 1st I misspelling so worker bee is now being validated and it says roles not found so we can but if we change it back to work at B which is in our world so the basically do validation
if something's messed up you can still you know in Python run some check a OK
and answer crater a management command that
kind of demonstrate what you could do but you know this metric man in specific to running on a website so it is wanted to find all the places where you're interact with the database right a code on in Python in in final places where that role relationship is messed up you can do that without text
do not purchase then this whole process the so adamant opposing has a handle it tries to find that apples with the notes doing it look similar profiles and for every profile make sure the role is in the the name
the so if on 1 bad apple in the primary key was 1 the OK so these are the 2 pass I talked
about us so the 1 that talked about you can remove the primary key add a new field named ID and make that the primary key and pre auto-increment field of and the other part
that I didn't specify what was that you can she created you and and change the table name to look that you an increase primary key in that you put it then it messes up your updates inserts also a platform for they took and as other was used to there's a general at that does composite keys I haven't tried out of from that look at that moment the so the runs that he
began what's gonna happen but it's going to generate models for all the stuff that we've already generated source override all the code with it so there's only meant to be a 1 time thing
so if you inspected the epicenter the Inspectorate model for built genomes
right so now the question and this is a trick
question so if anyone wants to break that by liminal will happen if give this code to another developer there were no don't think enough of a so so we have managed at the false
right now the so the angle will create those tables because is unmatched so if we gave this us advances it depends if they have a database that doesn't have those tables are then it's not that create them for them and they're gonna kind of think like what's going on here of and if they have a database that has those tables but then things will work more or less of so that's for that's equal custom command you can if you need to do this database to somebody else you want to create those of files or he gives if you wanna take out the manage forestry the manageable falls apart I
wish I had before it keep talking about the future of but this meant option manageable false so if you wanna take out that manage false than the default values manage equals so if you gave this to somebody else and you took all those out and you menacing TB for that next person would create these tables of and I want to do 1 last being at the head tap actually
and just for fun because this table I created and as we know every table gets mapped model and every column gets mapped to field in Python but what if the fields of would conflict or what they would create things the nonsensical in Django of for example the field name pass which of our car passes a keyword in Python so what's going to happen underscore underscore is going to conflict with the following our foreign key relationships so let's just take a look at what it does just
for fun justifying the field get added with these underscore feel gets added and then since both of these yet it's worth get removed from the next field even though it has 2 underscores and then the 1 underscore field is removed and then they just keep adding these numbers to it so that none of the namespaces Complex I then dear the number 12 is my sequel call name is also invalid in a affair in Python because it's just a number so number underscore 12 it at
but I think I'm at time you sure so
don't write code unless you have too many MIS Chris another Tecumseh High Country friendly guy and you do something which I know of that hoard probably doing it wrong so any academic and my back
Informationsmodellierung
Reverse Engineering
Imaginäre Zahl
Computeranimation
Information
Computeranimation
Webforum
Geschlecht <Mathematik>
Rechter Winkel
Code
Mereologie
Schreiben <Datenverarbeitung>
Information
Code
Computeranimation
Tabelle <Informatik>
App <Programm>
Turm <Mathematik>
Prozess <Physik>
Datenhaltung
Klasse <Mathematik>
Partielle Differentiation
Physikalisches System
Code
Computeranimation
Datenhaltung
Informationsmodellierung
Datensatz
Standardabweichung
Code
Modelltheorie
Informationsmodellierung
Standardabweichung
Tabelle <Informatik>
Resultante
Nebenbedingung
Schnittstelle
Datenmanagement
Fortsetzung <Mathematik>
Extrempunkt
Ähnlichkeitsgeometrie
Computeranimation
Gradient
Datenhaltung
Informationsmodellierung
Datensatz
Datenmanagement
Webforum
Skript <Programm>
Schnittstelle
Programmiersprache
Umwandlungsenthalpie
Turm <Mathematik>
Datenhaltung
Validität
Physikalisches System
Datenfeld
Schlüsselverwaltung
Aggregatzustand
Physikalisches System
Demo <Programm>
Turm <Mathematik>
Code
Datenhaltung
Befehl <Informatik>
Physikalisches System
Quellcode
Code
Computeranimation
Datenhaltung
Nichtlinearer Operator
Demo <Programm>
Turm <Mathematik>
Datenhaltung
Quellcode
Physikalisches System
Extrempunkt
Biprodukt
Code
Computeranimation
Datenhaltung
Open Source
Webforum
Rechter Winkel
Biprodukt
Schnittstelle
Tabelle <Informatik>
Befehl <Informatik>
Turm <Mathematik>
Datenhaltung
Dreizehn
Computeranimation
Datenhaltung
Netzwerktopologie
Metadaten
Informationsmodellierung
Funktion <Mathematik>
Schnelltaste
Menge
Standardabweichung
Mereologie
Skript <Programm>
Skript <Programm>
Modelltheorie
Modul
Informationsmodellierung
Aggregatzustand
Tabelle <Informatik>
Punkt
Relationentheorie
Gruppenoperation
Selbstbezüglichkeit
Computeranimation
Homepage
Datenhaltung
Informationsmodellierung
Softwaretest
Standardabweichung
Mustersprache
Rechenschieber
Vererbungshierarchie
Skript <Programm>
Modul
Tabelle <Informatik>
Schnelltaste
Turm <Mathematik>
Dreizehn
Profil <Aerodynamik>
Ranking
Generator <Informatik>
Datenfeld
Schnelltaste
Funktion <Mathematik>
Geschlecht <Mathematik>
Modelltheorie
Schlüsselverwaltung
Fitnessfunktion
Fehlermeldung
Tabelle <Informatik>
Informationsmodellierung
Fortsetzung <Mathematik>
Extrempunkt
Steuerwerk
Code
Computeranimation
Homepage
Loop
Softwaretest
Prozess <Informatik>
Datentyp
Rechenschieber
Zehn
Funktion <Mathematik>
Softwaretest
Einfach zusammenhängender Raum
Umwandlungsenthalpie
Lineares Funktional
Automatische Indexierung
Turm <Mathematik>
Datenhaltung
Güte der Anpassung
Mailing-Liste
Quellcode
Automatische Indexierung
Mereologie
Binäre Relation
Orakel <Informatik>
Tabelle <Informatik>
Tabelle <Informatik>
Turm <Mathematik>
Cursor
Indexberechnung
Mailing-Liste
Computeranimation
Datenhaltung
Loop
Geschlecht <Mathematik>
Trennschärfe <Statistik>
Code
Ordnung <Mathematik>
Informationsmodellierung
Tabelle <Informatik>
Einfach zusammenhängender Raum
Turm <Mathematik>
Machsches Prinzip
Relativitätstheorie
Indexberechnung
Computeranimation
Datenhaltung
Mapping <Computergraphik>
Informationsmodellierung
Datenfeld
Automatische Indexierung
Code
Information
Innerer Punkt
Normalvektor
Tabelle <Informatik>
Informationsmodellierung
Resultante
Punkt
Stoß
Mathematisierung
Fortsetzung <Mathematik>
Komplex <Algebra>
Gerichteter Graph
Computeranimation
Datenhaltung
Informationsmodellierung
Gerade
Meta-Tag
Funktion <Mathematik>
App <Programm>
Namensraum
Datenhaltung
Relativitätstheorie
Elektronische Publikation
Knotenmenge
Quick-Sort
Konfiguration <Informatik>
Mapping <Computergraphik>
Datenfeld
Funktion <Mathematik>
Geschlecht <Mathematik>
Modelltheorie
Ordnung <Mathematik>
Schlüsselverwaltung
Self-consistent Field
Informationsmodellierung
Lesen <Datenverarbeitung>
Tabelle <Informatik>
Turm <Mathematik>
Rechter Winkel
Datenhaltung
Server
Computeranimation
Datenfeld
Euler-Winkel
Ganze Zahl
Modelltheorie
Schlüsselverwaltung
Computeranimation
Informationsmodellierung
Meta-Tag
Turm <Mathematik>
Computersicherheit
Datenmodell
Computeranimation
Homepage
Benutzerprofil
Scheduling
Datenfeld
Softwaretest
Rechenschieber
Ordnung <Mathematik>
Modelltheorie
Meta-Tag
Magnettrommelspeicher
ATM
Modelltheorie
Computeranimation
Meta-Tag
Euler-Winkel
Zeiger <Informatik>
Systemaufruf
Computeranimation
Datenfeld
Betafunktion
Passwort
Passwort
Modelltheorie
Schlüsselverwaltung
Zehn
Tabelle <Informatik>
Meta-Tag
Informationsmodellierung
Modallogik
Turm <Mathematik>
Schlüsselverwaltung
Knotenmenge
Bildschirmfenster
Computeranimation
Homepage
Softwaretest
Standardabweichung
Kommandosprache
Rechenschieber
Hill-Differentialgleichung
Passwort
Modelltheorie
Schlüsselverwaltung
Objektrelationale Abbildung
Meta-Tag
Informationsmodellierung
Nebenbedingung
Informationsmodellierung
Datenfeld
Green-Funktion
Modelltheorie
Knotenmenge
Schlüsselverwaltung
Computeranimation
Informationsmodellierung
Meta-Tag
Tabelle <Informatik>
Tabelle <Informatik>
Tropfen
Turm <Mathematik>
Validität
Profil <Aerodynamik>
Fortsetzung <Mathematik>
Computeranimation
Benutzerprofil
Profil <Aerodynamik>
Datenfeld
Rechter Winkel
Schlüsselverwaltung
Tabelle <Informatik>
Tabelle <Informatik>
Tropfen
Ausnahmebehandlung
Fächer <Mathematik>
Validität
Elektronischer Datenaustausch
Datenmanagement
Profil <Aerodynamik>
Schlussregel
Dialekt
Extrempunkt
Steuerwerk
Computeranimation
Datenhaltung
Schlussregel
Benutzerprofil
Einheit <Mathematik>
Speicherabzug
Hill-Differentialgleichung
Modelltheorie
Bitrate
Meta-Tag
Informationsmodellierung
Objekt <Kategorie>
Turm <Mathematik>
Ausnahmebehandlung
Güte der Anpassung
Validität
Profil <Aerodynamik>
Datenmanagement
Mailing-Liste
Bildschirmfenster
Computeranimation
Schlussregel
PROM
Benutzerprofil
Speicherabzug
Modelltheorie
Gammafunktion
Informationsmodellierung
Aggregatzustand
Meta-Tag
Objekt <Kategorie>
Web Site
Turm <Mathematik>
Existenzaussage
Ausnahmebehandlung
Datenhaltung
Interaktives Fernsehen
Code
Computeranimation
Schlussregel
Benutzerprofil
Hardware-in-the-loop
Bewegungsunschärfe
Datenmanagement
Modelltheorie
Informationsmodellierung
Meta-Tag
Objekt <Kategorie>
Prozess <Physik>
Datenmanagement
Profil <Aerodynamik>
Bildschirmfenster
Schlüsselverwaltung
Message-Passing
Computeranimation
Schlussregel
Einfügungsdämpfung
Turm <Mathematik>
Datenfeld
Momentenproblem
Mereologie
Eindeutigkeit
Sichtenkonzept
Systemplattform
Schlüsselverwaltung
Computeranimation
Wendepunkt
Tabelle <Informatik>
Server
Informationsmodellierung
Lemma <Logik>
Quellcode
Modelltheorie
Code
Computeranimation
Datenhaltung
Meta-Tag
Funktion <Mathematik>
Softwareentwickler
Turm <Mathematik>
Differenzkern
Code
Winkel
Datenhaltung
Dynamisches RAM
Elektronische Publikation
Softwareentwickler
Code
Computeranimation
Tabelle <Informatik>
Informationsmodellierung
Turm <Mathematik>
Datenfeld
Default
Message-Passing
Computeranimation
Tabelle <Informatik>
Schreib-Lese-Kopf
Softwareentwickler
Namensraum
Datenfeld
Code
Systemaufruf
Zahlenbereich
Fortsetzung <Mathematik>
Computeranimation
Turm <Mathematik>
Code
Vorlesung/Konferenz
Code
Computeranimation

Metadaten

Formale Metadaten

Titel From Legacy to Admin
Serientitel DjangoCon US 2014
Teil 26
Anzahl der Teile 44
Autor Cabral, Chris
Mitwirkende Confreaks, LLC
Lizenz CC-Namensnennung - Weitergabe unter gleichen Bedingungen 4.0 International:
Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen und das Werk bzw. diesen Inhalt auch in veränderter Form nur unter den Bedingungen dieser Lizenz weitergeben.
DOI 10.5446/32830
Herausgeber DjangoCon US
Erscheinungsjahr 2014
Sprache Englisch

Inhaltliche Metadaten

Fachgebiet Informatik
Abstract For those who have not had the pleasure of seeing django's inspectdb command in action, I will create a demonstration of it's power. Django's inspectdb command can reverse engineer a set of models from a postgres or mysql database. I will demonstrate how to take a legacy database and create a quick and dirty admin tool along with a simple rest interface.

Ähnliche Filme

Loading...