Building a REST API with Django & Django REST Framework

Video in TIB AV-Portal: Building a REST API with Django & Django REST Framework

Formal Metadata

Building a REST API with Django & Django REST Framework
Title of Series
Part Number
Number of Parts
CC Attribution - ShareAlike 4.0 International:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license.
Release Date

Content Metadata

Subject Area
REST APIs are capable of providing valuable services within and beyond an organization. Django and the Django REST Framework enabled my team to quickly deliverable a highly functional REST API that was customized to our unique needs. This discussion will cover how easy Django makes it to build such an application and how to overcome potential pitfalls.
Computer animation
Presentation of a group Group action Building Serial port State of matter View (database) System administrator Direction (geometry) Multiplication sign Range (statistics) Source code Combinational logic 1 (number) Set (mathematics) Mereology IP address Formal language Web 2.0 Mathematics Computer configuration Single-precision floating-point format Software framework Endliche Modelltheorie Stability theory Theory of relativity Mapping File format Electronic mailing list Bit Representational state transfer Lattice (order) Type theory Data management Auditory masking Volumenvisualisierung output Right angle Row (database) Point (geometry) Rule of inference Number Element (mathematics) Latent heat Term (mathematics) Integer Address space Task (computing) Form (programming) Authentication Default (computer science) Addition Dependent and independent variables Standard deviation Matching (graph theory) Validity (statistics) Demo (music) Cellular automaton Interface (computing) Projective plane Database Directory service Cartesian coordinate system System call Bootstrap aggregating Computer animation Software Integrated development environment Logic Personal digital assistant Iteration Object (grammar) Table (information)
Group action System administrator Source code Range (statistics) Execution unit Disk read-and-write head Wiki Medical imaging Different (Kate Ryan album) Single-precision floating-point format Error message Exception handling Physical system Arm Mapping Software developer Electronic mailing list Staff (military) Database transaction Bit Unit testing Demoscene Arithmetic mean Data management Process (computing) Figurate number Point (geometry) Slide rule Sequel Computer file Number Product (business) Codierung <Programmierung> Router (computing) Metropolitan area network Address space Form (programming) Information Key (cryptography) Interface (computing) Line (geometry) Cartesian coordinate system Limit (category theory) System call Uniform resource locator Personal digital assistant Iteration Table (information) Building Code Multiplication sign View (database) Direction (geometry) Combinational logic 1 (number) Set (mathematics) Coma Berenices Mereology IP address Permutation Subset Web 2.0 Mathematics Software framework Endliche Modelltheorie Scripting language Email Electric generator File format Closed set Data storage device Representational state transfer Connected space Type theory Auditory masking output Website Right angle Row (database) Server (computing) Service (economics) Link (knot theory) Image resolution Repetition Web browser Login Field (computer science) Revision control Integer Software testing Proxy server Multiplication Dependent and independent variables Validity (statistics) Forcing (mathematics) Database Computer animation Logic Moment <Mathematik> Object (grammar) Routing
Computer animation
and I'm and
I'm thank you and
they differ from the presentation of the disastrous enemy out here in southern lets us since January this year that's really well 1st started using the Python and Django along so what I'll talk today about is just some of the pitfalls I encountered in building the project and how we overcame those and hopefully that will help some of you out so when I joined tasks the project I had was to build a REST API they're looking to automate follow rule
editions cells working for network team as part of a larger of the but they were looking to automate this because it was kind of a pain point in terms of a time that they were spending on the requirements for build application of support direct API calls and provide a self-service portal through just internal and they work their preference for Python and that was because some of these which is that they were getting ahead Python preloaded on those and they just wanted to go ahead and start building expertise in that language so they already had the input for this application find we had a customer source destination IP address and destination port and then we had an optional business justification where user can say this is 1 requesting this had to select a starting point comedian and they had the Python preference but they pretty much gave free rein they said look at what would be the best way to build this REST API and will go forward in that direction as I sort looking Django because it was something I had heard of something I was interested in learning I just hadn't had really the chance to do it before then it was Python-based and then also the built-in admin interface it looked pretty and it looked really nice and will like you can get with very little work and then into its research uh somewhat across the generous framework which offer the browsable API which also look very pretty will I could give you a lot for very little work but I did have someone recommend looking at flask and to me it just seemed really complex getting into it if I looked at it today but it probably be easier for me but at that time I don't know a lot about like that so I started doing some prototyping the Django generous framework get something of learning of fairly quickly as able the demo that our management team and then our internal customers and was well received so we went forward with that and so in in learning how to use Django generous framework I had a go through all the tutorials and things I think several the presenters of covered a lot of this material I go in doubt but you have your models of this is weighted defined database tables you have serializes for application we use J sounds are data format and for your views you're taking in a request and you're doing something and then you're sending back a response you your view sets which can be collections of related views the give you your Uriel's to the map addresses to places and their hours and then settings of course we can configure the application therefore the data formats you have different renderers available so 1 reason is the Jason renderer uh there is x Mallarme there some other ones out there to in of course a browsable API renderer that's a big part of our project right so for our application we've gone through 2 iterations at this point of both of those of all gone into the production environment and the most recent 1 when on last week and we've that we have some pitfalls along the way and worked our way around it and as Horgan to now so 1 thing to consider when you're building a REST API is free of use to you want to access those views you can certainly use the user in groups stable as a way of meeting membership just of the application and use sales of foreign applications state from an application standpoint of Django does allow to set whether or not you want to automatically add a new user and if you choose to do so you can even customize how wet user object gets created as in this case someone tries to get your application the travelogue and it changes as a I don't know who you are if you get the set automatically atom would do that and build it the way you want build that user object we he told it to and there's default settings for the of for us we don't want that if someone came their application and we don't know who they were well we just did want them to use application and so we turn that off of another way we were control access to our views were to just check the user data from the request so the request comes in and I think it's a request I user you get an object for whoever's accessing application or that you and so the and even if someone's not logged and you have a view that doesn't require a long and you can get an anonymous user objects through that and you can then build in logic and your view to check who is this user and what do I want them to be able to do for us where we and our 2nd iteration we added a authentication so we're actually creating Active Directory to say OK this is the user this axis and I application do they belong to a status specific Active Directory group if they do OK you can use application and if they don't they will return no appropriate response could instead so for disabling the view that something can also do a 1 1 thing is if you're writing your own custom views as a maybe you've written a get you another post you so that may take care of itself if you don't already have a certain view out there for us for use in view sets out so that we just have all the rest API actions already available that meets our internal company REST API standards and it also gives us about bit intends to give us some drawing room like right now we don't support delete method of but we haven't in there because eventually we make so if someone calls delete on any of our views were just going to return a 403 Forbidden into say you can do but depending on your application and you could certainly return a different type of the right so as we got to iteration to our requirements change and course this impacted our underlying logic for our application instead of accepting a single request on a change we were now taking a list of requests and then instead of taking just a single API before address for source and destination we now had to take that also aptly ranges and also subnet mask and in addition to that instead of just 1 value we may have a list of values of 0 and then similarly for destination port we could take a single-port or a range of ports or a list of any combination of those types of input and then business justification our 1 of our internal customers came to us and they said hey we really think that this needs to be required now we we don't want you process anything unless we have that data someone's entered something and so we had a bill that and then so in in iteration 1 we build Chart are on the web form to so to service us er self-service portal and just used the bootstrap JS for that but in iteration 2 we had 1 of the company's standard IT groups provide our friend Ian in this this group was called out to change so i t change had it's on you own user appropriately named IT change so if if our view was called we can get that users say hey this is allowed to change calling us and if they call us will we have to keep a record of every for all change that we actually make we need a talent that to a user uh so we can say hey this is i to change the called us and not to change has to provide us with the user value so that we know who actually submitted the request and now if someone directly called very API for just 1 already have that user years so we don't have to check for that value but for our IT change we say hey are you had to change did you give us a user value and so we keep going if not we return on I think is therefore 100 which is a bad bad request that data at so validators no were taking in all kinds of input now in iteration 1 that we're using basic things so we reviews in existing validator for that before addressed the syntax validation I think we just did a simple integer validation for the port numbers of we got iteration to that in quite work for us so we had to build on some regular expressions so that we can check to see the data that we're receiving that match up you know was this a range of IP addresses that this IP address come with a subnet mask a what about the port numbers was just a single port number was that of 1 through 3 something like that is very small but 1 3 thousand new as we had right on validators for that and 1 of the things we also were able to do was to just taking group validators together so for your models you can declare an actual validator for things and I think it just points to 1 early cells my understanding when I was going through the
development of as so I pointed to a single validator there was a group and within that single validator I call other relatives and for those of us if if any of those 4 is a validation error because if you have a problem on iterates validation here but you can you can of put in logic maybe you have 5 validators part of this group validated and as long as the data comes in and passes 1 of those validators is OK so you just handle any exceptions you come across a anything but if if you deem invalid you turn the validation error and you can pass in data for that so that you're telling your user hey this is why I was invalid and then they can go in and just whatever it was they were given to the application as so they can have a successful request next attempt X so for the data going around in the model of 1 thing that was covered that fall for us was in iteration 1 we're we're just 1 with single IP addresses we knew at some point we may bring ranges or subnet mask but we can pull out a 4 fold into that we just OK were were handling an IP address will deal with that we get a we we should put more thought into it planned a little bit better as so what we ended up doing an iteration to was going a bit of an umbrella model and this may not be a best practice but we found it useful within our group now purposes and this is just a big model for our API entry point which says this is all the type of data all the types of data that we may need now or eventually and we put some that in there never something we needed eventually we can make it an optional field at least for now but uh we we never see this model to the database we just simply use it for when we're accepting inputs and then we have subset models that have actual data were saved to the database for record purposes another thing to keep in mind is would you need to tell the user after you've completed an action so they called your API down somewhat behind the scenes what you need to get them back from this so you're record-keeping is a big deal we need be of map whose submit these changes why were they done as so we assess associated an ID with that so our request comes in and we do all our logic behind the scenes then we write all records to the database we associate that group of actions with a single ID we give that to the user so the user has a question later on they can come back and they can reference that ID with us if they just come back and say they we have a problem of you know your resource gave us of a 503 Service unavailable we we can't do a lot with that arm but all I guess that wouldn't apply there but if that they put in a request and accepted this request in but I can't reach of the destination IP from that source what's going on what the gives a request number we can go look it up and see what was actually done on the back-end for that and see if we have any logs associated with that was we capture something we we we had that point of reference another thinking man's air data so for the user we want to gracefully handle in that may occur and we just want to give them the right now we just given 503 paid services unavailable you should try again later or you can contact the supporting on the other hand of for our support staff we really need a lot more data than that the user doesn't really care if it's not working then that's all they need to know they can let us know they been work but we need to go back and figure out what went wrong so we tried to do 2 things at 1 of those is we trust do we capture all the information that's available at that time but that into an e-mail and send an e-mail to the support staff so that we can hopefully have a almost real-time on a lot of that year another thing which I do is to log their information that same information into a database and so then we have that record and there we can go back and look it up now have both of those methods fail we have something catastrophic going on but at least we try at so in going from iteration 1 iteration 2 we had to start accepting multiple inputs types of as I mentioned o long ago head so what we're aiming to do is we could have done an entirely different API call to accept these different types of formats but as part of our iteration 1 deliverable we already had internal teams that were looking at REP learning REP are starting to write automation that we utilize are API so we want to be is minimally impact impactful to them as we could be but so 1 1 big change was having that business justification and worse before it was optional now was required but that that was hopefully a minor hit to but we we are basically redesigned our existing API call to support this these multiple inputs and 0 we get that by taking the requester came into the view and building logic he entered examined do kind of pre check of the input we received so 1 the 1st things we do is say for the request came is the list if it's not a list we added to a list if it is less for OK and then we take for the uh the IP addresses the ports we say or these lists and if they they are list OK they're not a list we at those but those into a list and so then all of our logic this going on beyond that point can treat all all the input as if it is a list because we guaranteed idiots images made writing that the logic further down a lot easier another thing that we didn't have an iteration 1 that would have been great was have a version with our API because then iteration to we could have just had our customers calls in gives a different version number and we could have broken out logic to handle that input different so we get version our API and an iteration to get iteration to when I was versioning it I came across 2 schools of thought on how to actually version 1 of those being to put a version number in the Accept header and another to be was open and into the URL the problem we found with if we went forward with the URL is if we have some address slashed B 1 and then we moved to aversion to what we want to maintain backwards compatibility we now have those 2 addresses to deal with so we gotta keep the 1 working and then we gotta keep maybe the to work so we really didn't want to go that route but we went with just http acceptor uh coming and if a user doesn't give that to us assume Haiti were going to use a version 1 . 0 and as we go forward in a permit from here we can then branch out the logic in our code of based on a version of we actually received so for the safe process of for a model you can actually extend that uh in iteration 1 I'll talk talk this up to just learning I put a lot of logic into the model save method so it was doing a lot of things they're checking the data before actually see the data to the database so we took in extract all that out iteration to on logic is done in in in our view following a separate a logic far the but you can override say method if you need to but now as we were doing things in iteration 2 we had to save a lot more to the database of we were breaking things out into of permutations of combinations of all these IP addresses so we since we get now see these lists of input so as we started to say these entries to the database we immediately solve performance impacts is just taking a lot of time to save all that data and we're trying to throw large quantities of data and it for the input to the do cum internal will test so what we found was that would use ball create to have all these objects databases for 1 transaction the problem we had with that was that we have a lot of foreign keys so these or other objects that really needed to go into a main object are mean request object and so what we ended up having to do with the combination for any type of object we were saving were we're just going to have maybe a handful of those and they were foreign keys we just individually save those map models and then for our bigger request permutations since we have a multitude of those but by that time we already have a foreign objects uh 4 key objects because we say those we've got either foreign TID back or we just have object itself and we can
insert into and we basically build a big list of all these other objects we need to save for the foreign keys and there's were building those objects and once we have that big list we can call book create on that and we immediately solve the performance issues go away from that that we may have learned some things here general conic go back and look at me we can optimize it even more but that for us was an immediate the savior a documentation considerations of course the built-in browsable API was very powerful for doing that are internal customers have found it very valuable it allows them to play with our API we can give them a sandbox system and they can just have that as they've really like that but you can also use it to see this on different data formats so they can see the Jason response that we're going to give them and making go back and right automation of very they can handle that and process looked at swagger which is very similar to browse tilapia it's much more beautiful in my opinion but when I looked at this it was probably a mortar April time and there have been some browser incompatibility issues I think those have been resolved of but I was still wanting to for some actions just listed every REST API action and I wanted to go in customize and say well OK you can call delete on R API will really don't want you to right now so I don't want that show up in that in that when you and I couldn't find an easy way to remove that so something I'll go back to look at it may be a lot easier to do that by now another thing we do is we have our own internal team wiki which is accessible from any our company but we our document our API there were putting out uh all the different use cases so if you give us a successful request this is the type of data you can expect that if he goes a bad request here's what you'll get back if you have an error then this is what we will give you back and a way all teams are building automation based on our EPR have that available for the as so some other lessons learned along the way that is we did have some legacy databases I think another torque is already covered some of that as a were running must equal or not for a database and a we found that we could run the inspect DB command and it would actually take all the pre-existing database tables and generate model code for that of the problem we found there is we're looking to not only uphold as soon from models but also so that we can manage these legacy databases from admin interface and we needed right to it as part of our request process so if we were to save 1 of these models using the automatically generated code the expect d b had made of the auto increment field an integer field and the Django model code which meant that Django did not automatically give us an idea back after we say that models a so the quick fix we found for that was to change it from an integer field into an oral field and then we could say that moral we get that idea back and we can continue our processing using the and that may be improved in 1 . 7 I'm not sure we were using 1 . 6 5 as so another thing once again where user must equal store but we started seeing issues were set my sequel server is gone away server was there we can access the there's no problem with that of that we access the database this just seen that happen in we we can find any reason behind so we had a connection open too long a just seemed random to us in the information I could find line the appropriate way to handle this was to just close the database connection before you tried to uh having a database transaction I I don't like that response on the hopeful that maybe that will change eventually but those where was at that time and we put that in there so we had a database uh maybe a a close connection was a command our we put that in there and then we do a model that say the issue went away but then we ran our unit testing and that broke because it just in like that you're trying to close the connection to the test data and so we ended up going our settings spot and we said uh setting in their unit testing true or false as so when we run our code and we we go based on that and where you got sarcos logical actually say if you were not unit testing close that connection and then a model that's it never unit testing it just ignores that we just say the models and that this work force of for another thing we're doing is using the coverage model and that way we can run our unit test and we can actually have some nice H 2 male created where we can look at the code coverage we go see what lines of code we haven't actually tested and we can go back and try to get those and we've successfully maintain 80 % code coverage for all of our production iterations so far and will will try to keep that higher but at least 80 % of men as some other things where we had this society changes providing a web form so they're using JavaScript to build their form and it could not work with our code of until we added certain HTTP responses is really easy to do just in your view code you can add logic to add on data to the headers at the headers so we did that no problem but uh we encountered yet another issue so we had our application is available as running on its own servers running Apache but we have to support 2 different methods of accessing are application just based on our own internal limitations as so at the changes to call are API through just the server address and then any other user our company has to call it through basically a proxy and when you're trying to run both of those together on the same system as you start have some problems and so for us we tried configuring our settings falls we tried to have multiple settings files they could that they basically inherited from a master settings files and then based on the server running on based on whether action change called us or just some other user called us we would then select the appropriate settings falls set those well what I found with running Apache and uh this is something I'll deal with why go back is once the 1st call was made those settings were set so someone else comes in the settings didn't change the and so on are gonna go back and look at that and so we ended up having to a manipulator saying falls a little bit more so that a single settings for all worked regardless of whether not to change called us or just some generic user but uh another thing we get there was the browsable API as great as it is a it was automatically generating some of the links that we had and it was generating them for our actual direct server address and we didn't want that for general users they were the ones they were going to actually use a browsable API and I to change wouldn't so we wanted the general users to have lead that went to the actual correct address so use the API as we had to go in and customize the template weird customize the router for that actually generated links but the good thing there was it was all customizable those just learning how to do it that the 4 script name there that was our we basically had our internal site access . com slash are applications so you had to set our applications for script name to get that to shorten the we don't it's so enclosing at counting encode rigorous framework to be very easy entry point for myself hours if when I was at the time so I still am in some way case but is very easy for us to get something up and running quickly I we found me very customizable as odd usage needs advanced at it for me my management team was more than willing to give me the time I needed to explore and figure out how to get things to work as so that that's definitely if you have a time put into it you can that you can customize it you just have to know when that it's get e-mail to a if you want to follow with me I've got the slides posted they've been on on the the whole time on any questions here
some of the uh a