From Legacy to Admin

Video thumbnail (Frame 0) Video thumbnail (Frame 1004) Video thumbnail (Frame 1517) Video thumbnail (Frame 2859) Video thumbnail (Frame 3880) Video thumbnail (Frame 4718) Video thumbnail (Frame 5523) Video thumbnail (Frame 6018) Video thumbnail (Frame 6984) Video thumbnail (Frame 7963) Video thumbnail (Frame 11396) Video thumbnail (Frame 13969) Video thumbnail (Frame 14844) Video thumbnail (Frame 15687) Video thumbnail (Frame 18895) Video thumbnail (Frame 20144) Video thumbnail (Frame 20934) Video thumbnail (Frame 21559) Video thumbnail (Frame 22040) Video thumbnail (Frame 22754) Video thumbnail (Frame 23370) Video thumbnail (Frame 23893) Video thumbnail (Frame 25197) Video thumbnail (Frame 27694) Video thumbnail (Frame 28185) Video thumbnail (Frame 30011) Video thumbnail (Frame 30825) Video thumbnail (Frame 31338) Video thumbnail (Frame 32294) Video thumbnail (Frame 32797) Video thumbnail (Frame 34175) Video thumbnail (Frame 34812) Video thumbnail (Frame 35625) Video thumbnail (Frame 36484) Video thumbnail (Frame 37552)
Video in TIB AV-Portal: From Legacy to Admin

Formal Metadata

Title
From Legacy to Admin
Title of Series
Part Number
26
Number of Parts
44
Author
Contributors
License
CC Attribution - ShareAlike 4.0 International:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license.
Identifiers
Publisher
Release Date
2014
Language
English

Content Metadata

Subject Area
Abstract
For those who have not had the pleasure of seeing django's inspectdb command in action, I will create a demonstration of it's power. Django's inspectdb command can reverse engineer a set of models from a postgres or mysql database. I will demonstrate how to take a legacy database and create a quick and dirty admin tool along with a simple rest interface.
Computer animation Model theory Reverse engineering
Information
Computer animation Information Internet forum Code Gender Right angle Mereology Writing
Mobile app Standard deviation Process (computing) Code Multiplication sign Model theory Database Table (information) Row (database) Physical system Social class
Programming language Key (cryptography) Validity (statistics) Sequel Relational database State of matter Interface (computing) Gradient Model theory Database Field (computer science) Latent heat Computer animation Personal digital assistant Customer relationship management Resultant Physical system Row (database)
Computer animation Demo (music) Code Source code Database Physical system
Computer animation Demo (music) Code Operator (mathematics) Interface (computing) Source code Right angle Database Product (business) Physical system
Scripting language Computer animation State of matter Network topology Model theory Statement (computer science) Set (mathematics) Database Endliche Modelltheorie Table (information) Metadata
Point (geometry) Group action Electric generator Key (cryptography) Inheritance (object-oriented programming) Gender Multiplication sign Keyboard shortcut Model theory Fitness function Field (computer science) Database normalization Computer animation Personal digital assistant Profil (magazine) Self-reference Pattern language Table (information) Error message
Functional (mathematics) Sequel Code Source code Electronic mailing list Database Representational state transfer Function (mathematics) Mereology Connected space Type theory Subject indexing Goodness of fit Latent heat Loop (music) Process (computing) Computer animation Software testing Table (information) Oracle
Loop (music) Computer animation Gender Order (biology) Electronic mailing list Selectivity (electronic) Table (information)
Subject indexing Computer animation Information Mapping Relational database Endliche Modelltheorie Table (information) Field (computer science) Connected space
Point (geometry) Complex (psychology) Mobile app Sequel Computer file Function (mathematics) Field (computer science) Mathematics Computer configuration Endliche Modelltheorie Mapping Key (cryptography) Namespace Relational database Gender Model theory Database Line (geometry) Computer animation Order (biology) Collision Quicksort Table (information) Reading (process) Resultant Directed graph
Server (computing) Computer animation Database Right angle
Computer animation Key (cryptography) Integer Field (computer science)
Data model Scheduling (computing) Computer animation Order (biology) Information security Field (computer science)
Computer animation
Computer animation Key (cryptography) Password Table (information) Field (computer science)
Computer animation Key (cryptography)
Key (cryptography) Relational database Model theory Table (information) Field (computer science)
Computer animation Key (cryptography) Sequel Validity (statistics) Profil (magazine) Right angle Table (information) Field (computer science)
Computer animation Validity (statistics) Profil (magazine) Rule of inference
Goodness of fit Computer animation Validity (statistics) Profil (magazine) State of matter Electronic mailing list
Existential quantification Computer animation Code Customer relationship management Interactive television Website Database
Message passing Process (computing) Computer animation Key (cryptography) Profil (magazine)
Computer animation Key (cryptography) Moment (mathematics) Insertion loss Mereology Table (information) Field (computer science) Computing platform
Computer animation Code Multiplication sign Source code Model theory Function (mathematics) Endliche Modelltheorie
Computer animation Computer file Angle Code Software developer Equaliser (mathematics) Database Table (information)
Default (computer science) Message passing Computer animation Model theory Disk read-and-write head Table (information) Field (computer science)
Computer animation Sequel Namespace Multiplication sign System call Field (computer science) Number
Computer animation Lecture/Conference Code
in a in a year and a half and I move around a lot is lazy and that you guys are coming up of know you add many other options so that the end of this talk and I hope to be informative and about inspect each of which is a general comment that helps you of reverse engineer model from an existing database on
within Inspector Chris for all but I go by monodomain Chris of when I was practicing this I ran along some Fig kind of fast to trying as much information as they can on this is me there's a
professional Christian and the regular chris so that you can talk to me afterward so you know your
talk to graduate not professional cross but right now
it's a professional cross on so something to the talk that I wanted to give our don't right so don't write code unless you have 2 and my name is chris not data come up and if you do something to do something hard which I know that you probably doing it wrong and this and is something I realize I started using Django initially on when I say so when I created a custom command to populate initial data and I realize that there were fixtures of so this happens a lot when you like when you start to work on something you don't read all the documentation or you see some cool part of it you wanna do not realize that there's other things of so my advice is you know to spend as you come the general con and go of Gaia Iousy of gold forums and there's a lot of rich information there's Fletcher such gender and that's and there's a lot of like custom
apps uh templates TL have like their suggested ways of doing things so inspected B is 1 way to save a lot of time and if you work
with legacy systems so instead the reads tables and columns not rows in a database and produces a model stop I've standard out of have to manually write this code and you could you could well you know create a class that inherits from all the model and give an explicit TV DVD it like mean and and you could go through every table in your database but this just jump start the process for you it's not perfect so we'll go over why what what we got do afterwards but
it works pretty well and when you need this I and 2 scenarios have come up new this idea that I have legacy legacy system listed in this Annex 1 admin interface to it of so foreign keys are grade of the primary key constraints a great result mean always and strengthen my sequel comes with but in reality you have like an additional of validation layer to let the system that you want keep it clean state you can you reverse-engineer model at a couple validations and you'll have this slicker admin interface of the general comes with for free on the other use case that I've come up
with is a pretty management commands us so when you create management commands for going through much of rows of a database and making sure that 1 field is less than another field or you won't find all of rows that have you know a certain value the or em comes in handy for 2 for doing these things and writing custom sequel is sometimes hard of particularly if like you know it's not not designed to be a programming language designed to specific things in sequel that's another reason why this is come handy in the past so
credit created a fictitious scenario of y of why I'm doing this demo and I have a legacy system with a legacy databases and I can't touch the code for whatever reason I and I need to be able to manipulate data but still keeping a clean state and so I can have the source and there's a
lot of reasons why you India was at the source you have you have a source of to third-party product it's written rails or of you know the source principle it's impossible understand also at the end of this demo I
want I want to be able to have an admin interface that's there was little work as possible and you will have a basic right operations and 1 be a little leave the existing database intact more or less and we'll talk about how we break that when we have to break that so many ways to ways to get around it but a more or less what that leads the system it's interacting with this code to to be the same after
this also the tree from the docks
insect avait and database and whatever's pointed so the 1st thing you'll notice here is that of whatever your settings of name is pointed to death with can introspect of and it's also in this state the statement doesn't seem like it says a lot but it actually says a lot the script once metadata asynchrony model for each table on so something that you don't think about is that you often have more tables than models because of many to many relationships and inspected the inspection don't don't understand the concept of and also the most important part of this is this
feature is meant to be a shortcut not is different among generations so I am I created a
fictitious model yeah so it's pretty
basic there's a user table I made a lot of like egregious errors don't blame me I designed this to be terrible on purpose so that we could understand what would happen if you designed the is not yet a work with is the terrible manure expecting them they're not to be perfect so every user table it has 1 primary key it's our incremented it's not null and has username and then of course it has a point X that's work of we have a company table has 1 of the primary key is Otto incremented if that in a minute self-referential so a company can have a parent company and so it's interesting corner case and then this goes on before a user can belong to many companies and many come and a company can have many users and their related by this user has company of many many table so this relationship in in uterine Django you have a user model that has a many-to-many relationship or a company or vise versa but there's really no way of of work of gender inspection to know this so we're missing at what this action produces a 2nd and another thing that's terrible but this referring to the Django at least that is that has 2 primary key so has composite primary keys which is gonna be a problem for us some of the y the of Agenda basically since the you have every single model this in the of 1 ID field that has a primary key and this is doesn't fit that pattern and then profile budgetary thought community worse has a triple a composite key and there's a role here so role is meant to be a foreign key to this table role but this is of our car and the name years so as to match the role in here there's like completely worst interests and you can imagine there can be so many problems with this obser anomalies deletion anomalies like you name it so that's but smashing model and then just for fun if we have time at the end and go over this table because I created like the worst of field names possible just to see what they would produce and so the source of inspect the
be all and this is the source of new it's good size I guess so that there's 2 primary lives the 1st loop other users connection that infection table names also connection introspection is a of connection specific introspection function to get the table names so depending on what database you're using it'll get it'll user differ command to get back at list of table names but then once it has that was the table names of its agnostic this code of the rest of goes agnostic to the type of database you have so I looked at the source code for other the different introspection definitions so my sequel oracle and posters because all of them and you look at the documentation it says my superimpose grows over pretty much was before spices and and oracle has the same function introspection functions methods defined to introspect database so I don't get a chance to test oracle but I don't see the reason why when work and I was looking online it seems like a lot people driving to work so maybe I can look up that later in the talk about that but so once you get a table name and at that 1st primary loop the secondary part of the loop of the inside that loop I guess relationships gets indexes and then it has a secondary loop to loop through all the other columns and at the end of this and ardently do the job of the screenshot but the end of this it generates a collocate yields a the output of to the use an output of the column name to standard out so that already used so this was a little quiz I made
a these are the intersection Pacific commands that my seat so that gender uses that to get table names so my he goes to show tables are select tables for Oracle and something knows about these these each of these commands is not guaranteed to come in any particular order social tables just gives you a list of the tables of it doesn't give you the table sorted by you know when they were created and use the table sorted by of the name and all these are the same so we go through that loop it's basically going get up and there's just no way to guarantee what order it but so this is
what we talked about before the we use the connection introspection about to get the table names relation of new indexes and the columns but then we have agnostic at the next layer to generate the models once we have all information from our a connection and this is the many-to-many fell of
like I said this should map with a many to many on the the use of the company field depending but this maps 3 individual models so let's take a look at
what would happen if we do this so Python managed to apply it to the show so I created this in my sequel so we'll take a look at us so these are the 6 tables we started with and we haven't think the database so we don't have any of the gender-specific of tables so the question is do we wanna singer database or do database introspection 1st the database introspection looks to the database and create a model from the DB creates ah since a database of to the correct point of that were out right now or whatever we have the and and uh right now so if we run our sink database 1st order create tables that of gender needs to be run at to log in and you can't older users but then is we run uninspected days after that those tables will also be in our database and they will be introspected as well and that's not really what we want because those are those models already defined somewhere else around them inspect B 1st the so like I said goes directly standard out of which is not very useful but you can take a look at it of things so tells you to do things which is nice so I had to be be giving you talk industry this you rearrange the models you make sure that and everyone has a primary key and if you want you can remove managed falls on the doesn't really say with that's gonna do besides its can allow that gender to have a lifecycle and allowed to rename the models but you're not allowed to read DB tables which is like the Met option to explicitly state the table name is to override the convention of and then there's this line which you know I'm sure won't be important all you have to insert the output of gender and sequel custom app name your whatever so let's go ahead and put this to so I created map for this you have and let's assume we
produce this is same thing but worse in a file now of and now we can run our safety the the so the 1st from the run into is of related name collision so let's add a related name to 1 of the the namespace collisions so so it really so looking at a model of user has company user user and by the way like my sequel and generated these field and so I'm sort of the terrible have 2 ways the change them but has to foreign keys to user has company and the problem with that that is that the name space for complex so that the accesses to get the results that that will be of will be the same without a generates so battered women into 1 of them
the the announcements the about to again think a database now but so its use and related name told us to do that so was easy enough here's me now to understand me some right so sensor we have that AV-ASR database and now we can run a server
so it work now and it's running so that we have and Florian so I added something
daughter register all the models and so we can look at it right away with look at our company and create a new company so this is weird there's a company ID and I normally don't have enter my on primary keys so let's look at
what went wrong and this is a common thing of the integer field has a primary key but it's not a field and I know that it's not a field because I
created a data model and the company has autoincrement but I can
somehow edit the order implemented value which doesn't really make sense so to fix this
week kitten and our field the
back right so that disappears now the auto increment in the background and security company my schedule not just because that annoys me
I'm sorry for my use of members the now it's user the the
also going on so usernames likely Chris
and my password to the passwords talker the OK so those with the 2 easy tables to map they had 1 primary key there are incremented inmates messed up on identifying it's an auto increment field but that was easy enough to fix and even if we didn't we could assign
a value there of who the next
thing is gonna come up is these composite primary keys so we don't have a primary key listed here so there's multiple ways to approach this
of so related name we talked about
that only have to add my primary keys no also generates from primary key a
composite primary keys of so this always affects this you can drop the primary keys and keep the foreign key constraints and you can add a new field called ID and Janos so like model assumes that there is an ID field anyways and you can make that NOT NULL primary KEY in autoincrement so you're adding this extra fields to each table that has a composite key you just reducing the composite key to this extra field so I
did not memorize how to do that so from the
TV show the so i for the 2 tables that composite keys imatinib call my ID into primary key auto increment and before I did that I dropped the primary keys there existed which were those composite primary keys and then turned to quickly and for the other table for this user's company which the same thing and there some my sequel of book that I have to get around proper foreign keys so I can mean enablers afterwards but for the purpose stomach it doesn't need to be OK so I have a company I have a user and I want put that user in that company without many-to-many relationships so that that come back and cruel role and I know I want to role to the a worker bee the the now inside of my profile from unaccredited profile and given 90 and the work p Prentice misspelled B so we want this role of to be consistent with the role stable even others not foreign key relationship and it's not allow us to do that right now because there's no validation on the role field so that's not going to write a check and see what we can do about that
but
so in our profiles on added validation to ensure that happens and I'm a borrow most of this from here to that of its
foundations are the the import rules so
when import validation Aaron importer roles and if the text for the profile role does not match something that we have in our value list for the names and then we will raise validation romantic so we have 1 profile at that state and then another 1 that's supposedly in a good state or B so worker bee at 1st I misspelling so worker bee is now being validated and it says roles not found so we can but if we change it back to work at B which is in our world so the basically do validation
if something's messed up you can still you know in Python run some check a OK
and answer crater a management command that
kind of demonstrate what you could do but you know this metric man in specific to running on a website so it is wanted to find all the places where you're interact with the database right a code on in Python in in final places where that role relationship is messed up you can do that without text
do not purchase then this whole process the so adamant opposing has a handle it tries to find that apples with the notes doing it look similar profiles and for every profile make sure the role is in the the name
the so if on 1 bad apple in the primary key was 1 the OK so these are the 2 pass I talked
about us so the 1 that talked about you can remove the primary key add a new field named ID and make that the primary key and pre auto-increment field of and the other part
that I didn't specify what was that you can she created you and and change the table name to look that you an increase primary key in that you put it then it messes up your updates inserts also a platform for they took and as other was used to there's a general at that does composite keys I haven't tried out of from that look at that moment the so the runs that he
began what's gonna happen but it's going to generate models for all the stuff that we've already generated source override all the code with it so there's only meant to be a 1 time thing
so if you inspected the epicenter the Inspectorate model for built genomes
right so now the question and this is a trick
question so if anyone wants to break that by liminal will happen if give this code to another developer there were no don't think enough of a so so we have managed at the false
right now the so the angle will create those tables because is unmatched so if we gave this us advances it depends if they have a database that doesn't have those tables are then it's not that create them for them and they're gonna kind of think like what's going on here of and if they have a database that has those tables but then things will work more or less of so that's for that's equal custom command you can if you need to do this database to somebody else you want to create those of files or he gives if you wanna take out the manage forestry the manageable falls apart I
wish I had before it keep talking about the future of but this meant option manageable false so if you wanna take out that manage false than the default values manage equals so if you gave this to somebody else and you took all those out and you menacing TB for that next person would create these tables of and I want to do 1 last being at the head tap actually
and just for fun because this table I created and as we know every table gets mapped model and every column gets mapped to field in Python but what if the fields of would conflict or what they would create things the nonsensical in Django of for example the field name pass which of our car passes a keyword in Python so what's going to happen underscore underscore is going to conflict with the following our foreign key relationships so let's just take a look at what it does just
for fun justifying the field get added with these underscore feel gets added and then since both of these yet it's worth get removed from the next field even though it has 2 underscores and then the 1 underscore field is removed and then they just keep adding these numbers to it so that none of the namespaces Complex I then dear the number 12 is my sequel call name is also invalid in a affair in Python because it's just a number so number underscore 12 it at
but I think I'm at time you sure so
don't write code unless you have too many MIS Chris another Tecumseh High Country friendly guy and you do something which I know of that hoard probably doing it wrong so any academic and my back
Feedback