Files in Django

Video in TIB AV-Portal: Files in Django

Formal Metadata

Files in Django
Title of Series
Part Number
Number of Parts
CC Attribution - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license.
Release Date

Content Metadata

Subject Area
One of the most confusing parts of Django for newcomers (and some old hands alike!) is the handling of files. Among the 10+ settings, static vs user uploaded distinction, and plethora of deployment options it’s no wonder that many people end up cargo-culting their production settings. The API overview Short introduction Go over the difference between static & media files Run through the File abstraction and the various settings Django Storage API, collectstatic etc Production & Development configuration Whitenoise/dj-static/Nginx for static files Cloud storage providers for media & static files (S3 etc, mention some popular libraries such as django-storages) CDNs Implement a storage engine together & the future Implementation - practicing what we just learned to solidify understanding Closing remarks and mention possible future Django developments.
Computer animation Commitment scheme Computer file Theory
Context awareness Scripting language Multiplication sign View (database) Sheaf (mathematics) Analogy Set (mathematics) Mereology Computer programming Attractor Software maintenance Data model Fluid statics Medical imaging Type theory Different (Kate Ryan album) Computer configuration File system Software framework Data storage device Endliche Modelltheorie Information security Modulo (jargon) View (database) Wrapper (data mining) Software developer Computer file Maxima and minima Mereology Statistics Process (computing) Computer configuration Fluid statics Data storage device Interface (computing) output Summierbarkeit Cycle (graph theory) Quicksort Point (geometry) Server (computing) Game controller Inheritance (object-oriented programming) Link (knot theory) Software developer Computer-generated imagery Data storage device Directory service Product (business) Cross-site scripting Root Internetworking Software testing Metropolitan area network Consistency Cellular automaton Interface (computing) Magnetic-core memory Debugger Projective plane Java applet Content (media) Mathematical analysis Denial-of-service attack Directory service Cartesian coordinate system System call Similarity (geometry) Computer animation Network topology Routing Local ring Library (computing)
Email Group action Context awareness Magnetic-core memory Software bug Medical imaging Computer configuration File system Data storage device Process (computing) Data compression Email Arm Wrapper (data mining) Software developer Structural load Computer file Computer configuration Fluid statics Data storage device Website Quicksort Task (computing) Reverse engineering Data compression Server (computing) Functional (mathematics) Game controller Wrapper (data mining) Proxy server Patch (Unix) Real number Computer-generated imagery Field (computer science) Template (C++) Twitter Product (business) Goodness of fit Reverse engineering Proxy server Default (computer science) Server (computing) Uniqueness quantification Magnetic-core memory Field (computer science) Cartesian coordinate system Datei-Server Human migration Doubling the cube Natural language
NP-hard Email Context awareness Installation art State of matter Multiplication sign Modal logic Source code Data dictionary Perspective (visual) Food energy Database normalization Fluid statics Medical imaging Square number Data storage device Data compression Information security Library (computing) Point cloud Email Structural load Computer file Googol Process (computing) Fluid statics Computer configuration Weißes Rauschen Website Quicksort Cycle (graph theory) Middleware Spacetime Content delivery network Point (geometry) Server (computing) Flash memory Data storage device Similarity (geometry) Diallyl disulfide Content (media) Number Time domain Traffic reporting Content delivery network Mathematical optimization Computing platform Domain name Context awareness Gender Content (media) System call Uniform resource locator Computer animation Personal digital assistant Multiset
Demon Execution unit Inheritance (object-oriented programming) Fourier series Demo (music) Revision control Peer-to-peer Writing Computer animation Data storage device Revision control File system Data storage device Installable File System
Medical imaging Process (computing) Demo (music) Spherical cap Data storage device Standard Model Term (mathematics) Hash function Source code Subgroup Front and back ends
Well-formed formula Magnetic-core memory Hash function Civil engineering
Content delivery network Game controller Server (computing) Link (knot theory) Interior (topology) Data storage device Set (mathematics) Insertion loss Parameter (computer programming) Client (computing) Traverse (surveying) Formal language Twitter Medical imaging Lecture/Conference Different (Kate Ryan album) Information security Error message Domain name Default (computer science) Email Arm Namespace Block (periodic table) Interface (computing) Structural load Computer file Content (media) Shared memory Directory service System call Flow separation Cache (computing) Arithmetic mean Computer animation Personal digital assistant Data storage device Logic Internet service provider Point cloud Website Pattern language Quicksort Object (grammar) Reading (process) Library (computing)
saying that some of them and so on and
so forth on also great and
high under Shariah for files and Django and she just needed to account for all of you guys but I left my phone theory be easily blocked it because I didn't give my phone but it will be quite low which was a person commitment and this is thousand and 0 and so just quickly
about me and the talk I've been using general for a little while 1 3 getting in the custom models also the call as was the native migrations and and just keeps getting better so I mean changing of storages for a last couple years enabling talk about it and I mentioned in a sorted by this talk is not about that but I have a lot to say to interested and so sample proceed in 3 parts that's an overview the File API and rummaging of static media storage that sort of thing some options slash recommendations for the deployment of production and in some caveats endings we wear and I will writing sword back and it will actually demo surge back and and it idea fast which is quite setting and I just networking so here and so it's got as a static India of sorry upper I'm I'm getting the stock and and somebody attractors goes I had a serious problem of knowing more understanding were grokking the differences in static media files is all the settings they're both files in static route and static URL media with media meteorol wired to and enjoying was 1 of the 1st piece of programming I really really going to for 5 years ago and so the basic differences in a sequential forward static files are files the everything else apart application you know not everything unfortunately Python if only but it CSS JavaScript image files and media files are files that are uploaded by the user they come from ittunga undress content static files your your verifying you know there it's the defined ACL ecstatic time the files profile pictures or what have you I so I think part of the reason that it was a little confusing for me and I think for the people cause I have the red agency and i've are more a lot of G developers have noticed that tree consistent confusion is they're both use the same Storage API to store them in essentially of fruit for local firm and development and oftentimes for production and but there is this very important section has important security that concerns anytime a don't interest input is that and so that static files so that if I was really means that can from static files Act which ships is that it's it's enabled by the following our project I'm Cervantes settings the settings are they all culminate in how ColLex static works and so essentially we're working with is you have your files in your static but it under static in in your local development all year dependencies applications and debug tool bar and and whatever other admin third-party applications have at present there including the end you need to include those or else you'll never links breakage and nothing more and so and so it all comes in like static and essentially the static file storage executes all viewfinders looking Troyer directories in placing a metastatic route that euros as the endpoint that hits the cell they hate us static file and if I know that he was the talk about a point that I yesterday for those they continuously was in the way that cycles served in development automatically is to run server which is also a static files magic man and it's is very insecure it's never use it for production and and it's it's nice because it just works automatically so that's the files
media files New France as a whole lot more settings on and this is largely because you're dealing with things are coming in from the Internet and there's things you 1 a control things there are important to think about you know denial of service attacks that sort of thing she is so like saying media root and you're always confusing for me 1st the analysis crude and set your own in so much as side is work like places on the static files at the end and media root is where your media files and up but these are different because you really don't want someone to upload a static a media file that has a pack of your static file that they get search all the users may be the sum upwards japery Jess and suddenly their jobs because executing in your context if there was a fallback so there it's enforcement framework these have to be different has they have to be different you or else I for whatever reason it's not actually clear to me this is something I add to it to every project but it's taken the documentation which is of course fantastic the best definition person himself or project really and and it ad serving of media files to your using the same same tool same tooling same view as static files that to development and you can see attracted settings that debug because I never use run server especially never use Ross server for serving your this media files 1 introduction so the 3rd major piece of this that unifies them is the Storage API and it's a pretty straightforward affairs has really changed that much since 1 . 0 modulo you know that times awareness and a couple of things for max later names and such and the cost of the file system storage where you know when you're defining your media roots your static routes you see all of a sudden you upload a file an offer there be a real you probably forgot to get ignore and then he went back and it to get ignored because you really don't want upload your urine test a year using for development which I've done before and so there's a brochure for interface and not all of it is required I the and yet so honored demonstrate a bit or or show that a novel and and data is which is probably the most popular a package that he is influence interface varies Baggins basically is a wrapper around the party libraries that and this interface of so generally speaking
files in court I was acting out under such where place this side you go 1st last and but I think that it's kind of unhcr unlikely it's not super common to work with the file directly on but what it actually shipping you is a of Django core files file which is just a very the wrapper that those things I can't enchanting so usually this is the sort of thing that gets uploaded and you're working with you working with an image killer in any field that file your action working with the file file file and true stories my template and so on and so 1 to mention that you can specify which storage use is a default file servers people file storage is what controls where things the sword that is upward images and and file fields by default but there is 1 caveat all nodes or 1 annoying bug which is that I find that you don't want use the same file storage in development of production and if you specify separate storages maybe need different ACLs really different headers or whatever it is and those it's realizes migrations what a locally and that's using know you offer reaction is an open issue to turn call our storage until callable so you can you could pass your callback function here and which would which would solve that issue the people of in many many issues on on storages about this and OK so that was the API over real and now I would like to talk a static file serving I so we deflation use run server driven or why it's a warning in the docks a lot and so so what what what has to be done what's next and and of course we want all the nice things that they borrow from websites the refasten perform and things like caching headers things like and hashed patch files so there's the unique things like allergies of compression and modification which plugs into collect static that actually like usually something the compressor whatever three-halves R on trends coded to see assassin minify and so 1 option which I think is totally validates users proxy whatever using for your reverse proxy anyway she and says reverse proxy considerable to slow loris so users proxy genetics and 10 was for you and there's
there's a lot of goods about a lot of things to say about this the main reason 1 important reason you what you wanna do this I'm seduced load on on on Django Python is slow you know as can send files so writer Django request that could be better served you know answering uh doing or enquiries in looking in the API is in all sorts of context switching back and it is highly optimized for this and kinds In user accesses ceteris proxy I deployed a lot applications several you it's popular and it's not not really think that you have access to have also done a lot of work with AWS and I I done Apache and 4 and it's it's use engine next for this arm it's is quite fiddly and sort of best practices change tarts get right and easier to get wrong which a doubling of the same thing it's not going to compromise USA probably yeah by it'll get slower and you won't really realize it is you think I I can figure well I'm done I so there are other options
another possibility and is white noise which is a fantastic package In burst the seen from my perspective 2 or 3 years ago before that what use DJ static and the sort of thing people probably use runs are at some point I'm sure but I have a white noise is is it is you was there was about but possibilities here but I think sometimes shows dangerous and white noise just works it's very easy to install a ships at all the best practices including cheese is probably hash files caching headers Python a brollies is a new compression under them from google it's space on like a dictionary of common thanks for the number of being happening coding for Jesus and and there's actually a open to get of Virginia to somehow integrate white noise in to a court which would be really great because right now it's it's the party and when is pure Python and just works squares and so 1 of the drawbacks of this of course is you you do indicating Python DAT do in Buenos it's ships its gender middleware that sees the you know the is source of static static URL and rich but it is the file and so a common thing to do is if your site is a small no traffic in this is fine it really it's not important I think there's a lot of future optimization of that OMB and we know when no sensory headers so but it's quite as behind a CDN content distribution network so aliases frontiers Akamai's a lot and basically servers everywhere the closer to users their request of a request come in for the file if they don't have it that rather from the origin server in this case from j be away noise and they'll catch their actor at the edge so it's very fast very nice especially if you know your servers on Virginia and you have users in Hong Kong I and 1 thing that's common which I should do not recommend always serving cycles anniversary of it's really common people said set you this use 3 storage from from doing sort is an asset that differ location locations and and this is what they do it's very brittle and so you have to clear Amazon API the somewhere to so you can actually you know go ahead and deploy it doesn't you need to keep some manifest us it'll build up with the same files every time was making a lot of exists API calls and the so I actually honestly think that if you're using a CDN with white noise at 0 little this work better
that being said for media files you should definitely not use your own domain and there is a big security rests that people can craft all sorts of fun malicious content that yeah has the a politician jaded but is J. bag it's it's some flashes who knows there's a million of the attacks I will put out a nice paper saying you know this is why we host content from a different domain and you'll see a lot of states to this you know that you have for example a common you have from a static devil that come but not static . 2 subdomains still can execute the same security on the same context were really report y and here's the same-origin policy I so this is the sort of thing where I think origins or similar S 3 is perfect and you still use a CDN because I had noticed that I think esterase fine further I've had whatever sites that users are heavy e-commerce and when you load a full-page as 3 is really not meant to be a content serving platform it's like in a a fantastic objects or before serving content exist I don't have the deny the throttling there's a lot of bottlenecks hard and so then I put a C D N a from I S 3 bucket and adjust it works really nicely but it also had a really really really fun but with necessary and course headers in image tags and loadings energy jobs strips so you can answer that sometimes I had to called my friend to water from
1st cool so now assets that's the fund from Fourier from so I like I said I'd be changing the storage is about you back against Asia SST-PT whatever I so I thought let's run a sword back and that was way too ambitious does I just bang must 2 hours and I barely done but I'm so I give as the interplanetary file system I love the name the when the name alone mean I do this but I you know it's a distributed peer-to-peer controversial version you'll storage but every buzzword imaginable and if you saw the file coined initial quite offerings these guys the so I just want to show what very quickly and I'll take questions and what it looks like so let's the
so here I'm running and I this demon and I quickly through together a
demo which has the very standard the Book of the yeah yeah a which has very standard model I think that every everyone should have a small least may be hidden and non-admins accessible and the and I wrote up and please and I use this anymore I can I I wrote I roll it must be 1 of the most insecure source back ever and I'd give us some process based and so I did a source contemplatively by hash and so I ever I redo the docks and quickly put together ikea fast-paced storage backend I so this is what this looks like I would like to go just to cap parties
and I have a couple of images and I don't eat that's in know term with cap because our gap that self-adjust party and that and you save then indeed you can see
and this is now saved in my
IP fast back and so I give us running locally so I took a krypton blockchain please do not answer the questions that we could have outside the core over whatever doing later in if the so who questions
to the that you mentioned earlier putting content delivery network between US 3 objects that people your serving you elaborate on that more share so I'm the general principle which I'll however quickly would be that you then your your links on you'd use you'd settings where your links or you're on site URL rather is is that in the case of the conference like know CloudFront died that such whatever on and solve your links that researcher the user would be those would be links linking to cover so when you actually when the user clicks reviews of downloads you know was an image tire or have you it would go to call from cloud from which had its local cache it doesn't exist it would go to your server server which of the trend that setting also denies caching headers saying hey conference to the past this you know cash for 30 years and and then at at every time that that a request came back to its to CloudFront would already be there so wouldn't even have to build your server so thanks I'm for separating uh media content user uploaded media from the actual static false does it also include uh like PDF files or or blocks the user up with the the most saved on your own domain yeah object I mean any any user Content I would say is do I mean PDF files and it's really hard to validate a PDF files of the evaluated read you could use live magic to read the header just does it ends in PDF does mean doesn't mean anything basically and the others is binary I it's so great talking with different from for media of loads how you stock at different users from uploading overriding other users files yeah I so did it depends I at there is a for example the 1 of the arm API methods and that interface for storages is get available so in that your sword backing could check does this name exists maybe 1 uses feel to upload their own file and in that case I'd say you probably need some domain logic in front of it because in that case the business logic it gets complicated there's a there's a fight the setting to do this uh recent like the bottle back and with just this is changing animal names I for a lot of files for loss of it's it's really nice for me to enable the like you go down directories so I use the upload to language can take a call back said say OK put it under this name space which I know I know what the user control I say this and user as ID in which case it's almost like they had their own separate file thing and there is some trickery which is why it's so is the use of a library where you can imagine someone can upload a file it serves a dot dot dot dot and then suddenly they're doing a directory traversal it is dangerous I so that's why it's sort of any user stuff uh sorry security related stuff is usually not the best interior on I think there's too much like nobody should write it in the community but I think it's it really needs a lot of thought put in today do you have any recommended patterns and best practices around that the world files that people will yet so I I just as looking at this for for a couple clients and so for the cloud stuff it normal and to make you know your your pocket for an answer is case is private by default the seal and then you have to have unsigned euros but but anyone with the SI neural which you can include a timeout can get it from past that you really end up having to do something where you put it because at that without that you know that something out that Amazon is meeting with you and I am sure it exists rather than as just the error the provider mostly from past value and of having to just put your own view of foreign which you know it gets much slower because then use do laughter self-pity fetch the file and then you need to survey I was it's coming directly from Amazon In juggling to give the link to the user and and you can you can go to that directory until custom of your have the the the hatched Signed uh your parameters it doesn't matter but there is I mean there is and I have my users of the lakes expire after an hour I've already had model people say broken well I can make it never expire but then the link itself it's never stop writing more depend on your tradeoffs they want to look at yeah yeah questions yeah I think Josh I think there will you might
check that and