Optimizing your webapp by using django-debug-toolbar, select related(), and prefetch related()

Video in TIB AV-Portal: Optimizing your webapp by using django-debug-toolbar, select related(), and prefetch related()

Formal Metadata

Optimizing your webapp by using django-debug-toolbar, select related(), and prefetch related()
Title of Series
Part Number
Number of Parts
CC Attribution - ShareAlike 4.0 International:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license.
Release Date

Content Metadata

Subject Area
This talk explains how to perform SQL query analysis and how to rewrite your views to reduce the number of queries Django uses in evaluating your model objects and their attributes. Special emphasis will be given to the powerful methods "select related" and "prefetch related." I will highlight the problem with a naive use of the ORM, how to target code for optimization, and the beneficial result.
Momentum Computer animation Gender Software developer Resultant Library (computing)
Personal digital assistant Gender Set (mathematics) Online help Collision Code
Theory of relativity Sequel Wrapper (data mining) Multiplication sign Gender Projective plane Set (mathematics) Expected value Arithmetic mean Data management Cross-correlation Computer animation Object (grammar) Musical ensemble Endliche Modelltheorie
Group action Computer animation
Addition Subject indexing Implementation Computer animation Structural load Configuration space Parameter (computer programming) Information security Abstraction
Filter <Stochastik> Point (geometry) Functional (mathematics) Server (computing) Sequel View (database) Multiplication sign 1 (number) Set (mathematics) Similarity (geometry) Fault-tolerant system Product (business) Template (C++) Term (mathematics) Operator (mathematics) Energy level Endliche Modelltheorie Condition number Theory of relativity Inheritance (object-oriented programming) Boilerplate (text) Keyboard shortcut Moment (mathematics) Model theory Electronic mailing list Database Bit Category of being Subject indexing Proof theory Data management Computer animation Symmetry (physics) Personal digital assistant Query language Blog Pattern language Object (grammar) Quicksort Window
Computer animation Blog View (database) Bit
Functional (mathematics) Information Sequel View (database) Interface (computing) Letterpress printing Power (physics) Process (computing) Computer animation Term (mathematics) Query language Electronic visual display Right angle
Web page Computer animation Information Blog Projective plane Electronic mailing list Cuboid Energy level
Injektivität Web page Addition Dependent and independent variables Focus (optics) Overhead (computing) Electric generator Sequel Information Euler angles Multiplication sign Electronic mailing list Set (mathematics) Event horizon Machine vision Process (computing) Computer animation Software Query language Right angle Object (grammar)
Web page Computer animation Sequel Visualization (computer graphics) Information Query language Multiplication sign Expression Electronic mailing list Social class
Group action Theory of relativity Information Key (cryptography) Multiplication sign Set (mathematics) 3 (number) Line (geometry) Code Loop (music) Computer animation Bit rate Software Personal digital assistant Query language Term (mathematics) Semiconductor memory Statement (computer science) Selectivity (electronic) Object (grammar) Endliche Modelltheorie Mathematical optimization Resultant
Numbering scheme Programmschleife Computer animation Information Block (periodic table) Query language Reduction of order Likelihood function Field (computer science) Mathematical optimization
Group action Existential quantification Identifiability Computer animation Query language Personal digital assistant Interpreter (computing) Bit Pattern language
Electric generator Theory of relativity Computer animation Symmetry (physics) Personal digital assistant Query language Pattern language
Revision control Default (computer science) Addition Theory of relativity Computer animation Personal digital assistant Operator (mathematics) Object (grammar) Endliche Modelltheorie Cartesian coordinate system Resultant Mathematical optimization
Computer animation Information Personal digital assistant Set (mathematics) Online help Student's t-test Line (geometry) Parameter (computer programming) Code Stability theory
Injektivität Point (geometry) Theory of relativity Computer animation Query language Bridging (networking) View (database) Cycle (graph theory) Template (C++) Reverse engineering
and that the so just introduction as uh as really done of wind that momentum is Chris Adams and might which would be about is that in the 63rd of interest and they are not because sentences also a developer Python developer gender of when general contributor who works at a library comments on a situation that's not me and and not the last and the general result in the nineties and professional wrestler who's here for
disambiguation sake various that's neither of us just in case you're wondering and yet can actually having a very common name in seeing the collisions that can possibly happen and so we have some
agenda of gender so is great we'll of using gender on assuming and ends at gender so that the channel is really a set of tools and and they those whose work together in any interesting ways in helpful ways but at the same time that we're using when using gender will running gender code is of is that is a set of tools that work a certain
way so to integrate and it but tools can be used in good bad ways thank and this time in 1 sense is the obvious meaning but it's actually 2 things so that it might not be so obvious to you when you're when it when using them especially for the 1st time and so the general is
1 of the tools in but that gender provides less of a object relational model and easy wrapper over a sequel queries refused as usual is a much more tedious to be writing and so 1 thing to remember is that whenever you're using a tool especially mutual is we should manage own expectations for for tools so that the approach a neutral with the broad set expectations as to what they think the tool will do for them and how can we this may have no correlation with the project actually has implemented so this is just kind of a long way of saying that tools can be misused especially in Europe so you know as a music as it
would be if they did
unicorns don't exist how to allow doctors a not really perfect but they never can be perfect in this imperfect world and so we have to really understand them and not deceive ourselves especially when encountering mutual so that all the that is to the 1 panacea that's going to really get everything can understand me it and me it's really going beyond the understood by this tool that I'm using the action that's not true use URIs once we understand the tools to use it correctly so
just them alone in its other and among other things you say about is an abstraction layer notice is sent to a it is a truism instruction where um as just another user of really because they take us away from the messy details I'm also risky for the same reason they take it away from messy details so k and so we don't do many things that we use that to do before because we have the 1 that we might forget or not understand the Owens doing things that we expected to do so we yourself a
ground when you know and you have a you have you have an API itself the objective implementation you have an that Python you may be the EU listeners in some some way how did you additional security arguments work you have massive of layers when using all an let's see signal itself and you have your yours your specific dates back and which might have some quirks um might you manage indexes in a particular way might of might do things socially differently loadings from the widow's also can actually expecting things in it and that the president of the configuration so you have a here the so
1 thing that I like that GenGO is the query set up the the the the problem face that query sets implicitly give us and enter explicitly years and we can use we can use a query it we can define Greece sets we can do many things with them time we want certain things to happen and when we when we add define sets um and so said have 2 features that's worth keeping in mind when you're thinking about them was that query sets all Lucy the and those queries grace at our annual so what this is a review for people might notice a ready answer will wear these terms mean use that doesn't evaluate into it needs to so that many people you know don't get it you know do we really don't realize this and when you do this it's a great aha moment because them using what's going on behind that bind the hood we define a query set know you know my that at all we do is you know defining and instruction which is deferred but it doesn't executed until needs to a certain cases when it does and execute and in many cases where doesn't excuse the question of movement which means that if you change them together you say you know I increase that and then I say that the was returned from the operation is a brand new query set however that new proof that um it as much as possible inherits all the features and the and the properties that you want have your own okra sets you can change filters together return query set and you apply conditional have another filter that only happens in a case that conditional things and a time decrease returning and you're going to 1 I'm so you not to make that but the the the the references the increase said you can so you can store so 1 of the more you know were which I want to notify the symmetry of the function and so could create always new and so you only need the the the good examples here you know he he into operations is uh returns a query set and does not actually data so but keep in mind that you that and what's not happening in same levels what is happening this however there are some situations where query sets are evaluated and there were fewer than their an agenda that condition is just a simple 3 and you know where the other ones also but who sets that when you make a list and then when you can use an index arrange operator and and when you iterate over them so but the initial bonded to turn back here is that you can define a query set and the early review function it's only at the 1st look for example in in the in the in the case here the cruises growth and is actually evaluated the sequel sent to the database server so and so they did this given my mind doing that with with the query sets for their so I want to highlight some some of easily that so some patterns which are problematic and and often as you need to know that the pattern is problematic because they're really doing things that look very straightforward I Writing like maybe you function I am going to scrape I mean you know you year and motivation but it looks like a musical sets the way I'm expected to but actually you know what I understand what's going on but and certain certain things that but it's easy to miss so let me link appointees out here and at the so we went with a simple act which is sort of blog at kind of boilerplate and if I take away like very similar actresses models is just the relations between the models and some or 1 of the features of Django model objects is that I some when I when I act well lit up the Pollock understood the examples we can see so so we have shown that we can use the blog post and comment on a post and has submitted which is the point here is that the user model post which has and a funky the back to the blogs so you posting a blog and then you know of of writers million people like that post as I'm and comments on the post which are done by a a a particular user and I'm and you know that point back to the post that their the comment on producing model for straightforward meant so here's a view which I am is that which can be used to do just generate a list of all of the blogs that are on the side maybe this is management view or something like that which really simple I use the template that windows that that view looks really simple and we have a review of a reference to the bloody to approach for the black in there and you submitted by the person is emitted from very simple if the seems like just just how online like like really easy thing to write the push production get the same working but they don't have the like this
OK as would be expected for it so on and I just wanna talk about the detail view for a little bit of the detailed
view what is the what we see in particular blog and I yield better that the blog will flow from the if a blog on that doesn't this idea and I and I request the posts that metaphor that log and and then I went to the AI went that of the blood detail of you know that this is a more complicated is a
lot different features that may be may use 1 or the and the see who writes the blog they wanna see the comments that on the out and so this is a bit more complex that
so they generate something like this OK you know I'm just using simple and that some generator so you will come and see people like that OK so this looks a little more complex in terms of how how this is generated
so now has about which were left and thinking of you have done my job right it I've got something out of I I've I've completed and concluded the work why it works right but I would really be today as I said uh the general is is is a tool and the was function in a certain way I'm 1 of the main interfaces that and then we have to add to is through the simple quick to vesicle query language and that generous generating the generating sequel so how and find out what's goal being generated to to to to find a particular view I'm anti-piracy can measure and you never know the problems that you know and how how do you know this this information lacking and that we can fill out this and that we think it's doing this magical I know the Northumberland's doing like you know i its history the United not also it's doing my make really there right now it's it's not the disaster of our 1st assumptions we can make so you really and journalist goes not just for cannabis for most tools of and most technology and you need to measure and if you need to get an actual and print out a display of and transparency into how the tools function and what it's doing and as precise as possible the movement get here the more the more power you have is a developer and in I think about you and what we see here so we get this information it we can be naive with our tools have that's information where 1
really cool project is a gendered about to work but if you're not using it I had that Wikinews it work and have some way of getting information and that's equivalent to to the the kind of things that this provides a box and so on that level of a house that had been a good documentation and you can do that it's it's relatively straightforward the so that's user to take a look at our the at pages
sufficiently good about the blog list page
so I vision that list and notice that when the could of the gendered about to advise gives us a panel which is a it is the injects additional JavaScript and into the page and also the data about the execution the page and it is the statue we can click on the information that you know and that the time that it took to generate base you know and that details about the settings of HEP headers the request object OK this you're that I can't think for a long time of them the focus for this for this happen sequel and notice that there's 52 is being generated and then executed to generate this attitude chatter generators page which on so the rocket is the list right so we we did the thing maybe we use that when it looked really straightforward we use that and and what it what happened here is that we actually we deny that we generally quite creates and so on and in this event was because those add add add network on round-trip that has to happen that this arrest you send the query and that the the the and also the uh which is a response and all of the data so as the process each in visual queries to the lot of extra overhead for having all these all these things happening um and I are so if we
could if you could go the sequel but uh it the
page actually actually have is gendered eventual goal is to actually gives us a list of the cruiser that happen with you think about it here where the visualization gives us a time when a couple of more things just for the sake of displaying but use different check it out and so you know that now expression is great we have we have the queries here I so do you to being generated and actually you can click on a class and you can get the information that the
plus 2 that the left side of the query and also the actual line of code with the query in those but is is is generated that actually this is um not query set that's generating at that that actually happened was particularly in it was particular case is reflect statements our ability to through the following key blood that's a bitter so gentle decrease it's a lazy and also the model objects a lazy he find the relations are I'm gonna be accessed on can be loaded until we need to be or if they need to be that's what saves in terms of not and optimization maybe you never reference the submitter object there and so you need to have information that that about the user of a of a model object so that a blonde that's the use of the reference that's generating the these many many many individual queries in the more to have more courage has this is actually a very bad situation is in the initial rate is much easier this was selected it comes in
select nobody um but works by creating a single joint include refusal related object in the individual select statement for the the the the for the model itself so so that the user is going to joined with the uh with the the the bad it's in this case so this is a term that the black III and more than a million times uh has as many as users as the are and if it was is that well it's action much more efficiently you Apache you do in 1 shot and so there's only 1 network about what that that information so actually I'm much better much fewer queries and so it goes through another missing data was query and um however to include much larger result set that was our from memory relationships like a many to many feel as opposed to foreign key there may be reversed spiky when we're we're going in the opposite side of the relation so it will handle those cases handle the in terms of in one-to-one feels that's it's it's it's a helpful and we going over them in a loop
and so here we we use likelihood that we tend to want to select the the submitted and on the other plants and you could have commas and little by little field future which we will join together and get all the all the information and then so I'm so now we don't end those 2
qualities and that are and those in the drawing on on the on the great so I we just completely on a reduced number of queries and even if we have much much more users and my foot 4 blocks this isn't the increase in we just need to query so this is really really good optimization 1 and a great tool special using loops and the so the
black interpretation as a little bit more difficult
so now this was done with 44 queries and that if we take a look at the
queries in this case it's it's allow less straightforward but you just have a list and and usually do this after the top of my collection go down and CC releasing to being involved in I just try to be intelligent unidentified patterns patterns but something's happening over and over again but maybe group things happening over and over again in this case I'm just gonna around the review if you
here the patterns that you define if you take a look at these things are that this there is a similar PBXs on the comments a case in judgment comments there being a generative who submitted their common and there's
also a query on the writers of symmetry which is you remember is a many to many relations this people like the posts so we have a comment that's and post like a so the
actually is an optimization its use for this case and instead of using an inner which I'm gonna do is gonna professionals are those relations of many to many relations and it's all the benefit do the application we expect of our model is gonna join the pipeline and that's actually efficient having good data for you so there's reason was selected doesn't do this by default professional it is gonna prefetch because many many relations to that it could just the result of the massive you know the you know result set so it's much better use of this way the human professional is from any of the relation has many many past and then and many related members this is applicable to a ship as join a pipeline and prosody of this allows it prefetch many to many many to 1 objects in addition to the fucking 1 that is in Switzerland it's likely that but you use helps to help you manage them and also submit surgeon operation Gertjan I don't know what version agenda that is and that the support that I would check that but it you you can use it for those kinds of things so we put there and it
is the eye and you can use double underscore syntax in this case this is the most efficient but and our set of arguments that but there is uh that in you know at the IEEE that'll help us in optical example of other maybe more than I had ever looked I I didn't spend them a center for an optimal optimal but these these are this simple opposition this this 1 line of code is gonna is gonna save us and so the the student prefetch tune the comment stable and that this matter and it's also going to have that the comments information which can be helpful for us as an implicit do that and if you select the Likas and our online and were done uh well you will in
some ways now I'm sure you can get this uh even smaller but did you know what I'm just just it amazing that work they did that you can do if you some yeah prefectures and that certain things before others so you know you it's it's just really helpful
patented to think about and to bring into your your own tool toolkit as you as you're working to to make you could make your views better and so in some the climate it's likely that a professional added and some us and these best practices and to avoid extra queries in use and templates and that the cycle of war or man-to-man one-to-one relations and prefectural Oedipal many to many or many-to-one relations including the reverse pointy relations of yeah so I guess was that so thank thanks to this bridge if I hope this will run allow you to start using those of these tools more effectively than I have I couldn't get out so that do not so please please click and any of the thing you very much few if any point in the