Add to Watchlist

Full Stack Python


Citation of segment
Embed Code
Purchasing a DVD Cite video

Formal Metadata

Title Full Stack Python
Title of Series EuroPython 2014
Part Number 82
Number of Parts 120
Author Makai, Matt
License CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
DOI 10.5446/20003
Publisher EuroPython
Release Date 2014
Language English
Production Place Berlin

Content Metadata

Subject Area Computer Science
Abstract Matt - Full Stack Python There has been a lot of noise about being a "full stack developer" recently. What does the full web stack look like for Python and how do you go about learning each piece? This talk will guide you up the layers from the server that handles the web request through the JavaScript that executes on a user's browser. ----- This talk distills information from the open source guide [Full Stack Python] I wrote into a 30 minute talk on web stack layers. An approximate timeline for this talk would be: * 5 min: intro story * 5 min: what the web developers need to know about virtual servers, web servers, and WSGI servers * 5 min: what do web frameworks provide? * 5 min: what are the most important parts of your web application to analyze and monitor? * 5 min: static files and execution on the user's browser * 5 min: concluding story and resources to learn more This is a high level overview intended for developers who are new to Python web development and need to understand what the web stack layers are and how they fit together.
Keywords EuroPython Conference
EP 2014
EuroPython 2014

Related Material

OK thanks everyone has this talk is going to go about 25 minutes so if you have questions afterwards all just the up here we can talk afterward so I don't think we have too much time afterwards for questions so this talk is based on I would say that I have been created for the past year and a half
but just a show of hands how many people red full-stack python . com before 0 wow OK only a only few hands of but at this rate and so hopefully introduces resource which spent a significant amount of time on and explains all the layers of the pipe stack deploying Python web applications so oftentimes the reason why I created this website is because there were so many times I would talk to new developers even if they have worked in a different language Ruby Java and they were wondering how do I create a Python web site and web application and then the deploy put it out on the internet so they were learning the pipeline syntax they were starting to understand again need to pick up a framework if I'm not going to write a framework myself I wanna create a web application of pick something like Django off last a work with Python learn the syntax and then iterate on on my concept and and and so many people have your ideas for what they want to create for the web applications and they create that the initial concept and this now I want to
put this in the hands of the people but how do I go about doing this and you start explaining things like well in several ways the server and then you need to have in a virtual private server and working with them and you're drawing a diagram and all of a sudden it ends up looking something like this which to someone who is new to web development can be very intimidating and so on this diagram here if it's
difficult to see can actually just go ahead and download that it's on the
front page of force that I found right now and there's a PDF of it so if you want men and and click on the links to different sections certainly go ahead and do that will take you go back to the this
page for a moment here it's very intimidating for someone who is new cost coming from a different language moved over the Python several years ago from the job the world and there all these new terms that even know I was an experienced programmer I was still questioning what what to these things mean and so when I'm working on new developers and explaining to them these concepts from essays OK this is sort of the end the let's wait the slate clean for now and let's start with what you've been working with what have you been working on so if we take away all that stuff for
the moment and will explain it piece by piece how you deploy a Python web application you start with this whiskey the framework must just call web frameworks and this is what it looks like so if we take a look from from your perspective for looking at this the clients from the left side may have a mobile applications accessing the application programming interfaces CPIs other applications are doing machine-to-machine communication with J. Sonner XML and then you have the traditional Web applications people accessing them not accessing web applications on the desktop or the browser or just through the mobile Web I've labeled the middle box internet it's a little bit deceptive here because while certainly a lot of the communication is going on on the internet this may also represent things your hosting yourself so if you're hosting 1 of your own services it's also technically on the internet on a different server you're communicating to it through the internet it's all lumped together and that center column and then the 1 on the right is your web application deployment what components you need in order to get your site life and running and understand how people are using it so let's start with the web framework that the thing that most
people start with is Django because it's just recommended agendas that great documentation they're walking through the initial holes application you can build a blog with there's also been some popularity with with flask now and so these the probably the 2 big frameworks as for what people are recommending to new developers in Python there certainly other Greek frameworks out there and some people start with them bottle Purim all those but but for right now these the ones it seems like it comes up over and over again region or some classes you can move to something else later these 2 2 frameworks represent opposite ends of the spectrum on 1 side that gender which says batteries included everything you wanna do when you're creating a web application you should be able to do with the framework the core framework itself on the other hand the other end of the spectrum is blast which says let's have a but have a small and a lot of extensibility make it really easy to add extensions to the framework itself you see this in in for 1 example user authentication not every web application needs user authentication so fracture last does not have an official user authentication model built into core there are many extensions that will help you you that last login but that's not built into the core itself a separate piece that you would install as a separate dependency words Django has an official weighted you use your user authentication so these represent these spots and the spectrum of the great part is we don't really have to to choose which framework want to run with which web server were answered with the server and the reason for that is because of which the the way that this was outlined whiskey stands for web server gateway interface and it is of that
standard type his proposal standard which talk about a 2nd the way that we would look at this from our
deployment diagram is the ways the server is what is going to be executing our Python code whether that is the code that we've written for the framework for other dependencies that we have again
this is a perhaps standard 3 3 3 and then 3 3 3 3 the 2nd 1 was added for for Python 3 support but mostly just deals with some unit code but it basically says the 3 3 3 if you're doing that right you're still using Python to your good you find the 2nd 1 was just added because there's some changes with unit code in Python 3 you need to adhere to an order for you to be a adhering to the whiskey tax standards here's how
this works how was the
works and it is actually a very simple but I recommend if you're doing Python web development to read these standards are actually very very readable especially this 1 in particular and what it's saying in 3 3 3 and then the 2nd 1 is you have a web server were also called the gateway sometimes also called with the container and it is invoking a cold object on the ways the application also sometimes called the framework there those those terms used interchangeably within those pets standards and the reason for this is because before we had the standards before we had was the we did not have a uniform way to execute Python code on the server and so you could not get 1 Python container to work with another framework interchangeably now we can because of this Standard and this is an interface standard which the servers and the framework you may have heard some of these of web server was the servers rather green unicorn is 1 of them my whiskey if you're
working with Apache and new whiskey micro was to you with the someone needs to clarify exactly how you pronounce this 1 but this is 1 that 1 of the original ones that was created for executing pi now
we need a place to deploy our application we understand what was the container is if executes code we understand we're creating a web application a Python web application in ways the framework and we need a place to put it and that's for servers and operating system common so this this
large rectangle we have here on the right side is our server were you may have a platform as a service we can use those somewhat interchangeably there's a few different variations of this 1 is is the
bare metal server if we take this laptop in the front row here and we look it up to the internet and we have a static IP addresses doesn't change over time we point oracle Domain Name System to that that laptop we can run our web application of that's all we need that's often considered bare-metal server or just a bunch of servers that we have in a closet somewhere there's also virtualized servers which means we have a an actual hardware server and the more virtualizing sort of many operating systems on top of that logic that larger server this infrastructure
as a service that Amazon Web Services Rackspace Asher IBM Blue next and Platform as a service broker python anywhere the purposes of this talk we're just going to be talking about just think about the other variable servers in a closet somewhere where virtualized server that you've rented from know the dissolution 1 of those services were going to be running Linux on top of it some sort of variation of Linux I personally use Ubuntu all the time synthesis
Great Red Hat's there's all sorts of different distributions it matters what matters is does it work for you and but ultimately we're going to be using the next 4 deployment so that's another box that we will look at as being running inside of our server now the
1 thing about this diagram here is that there are all these clients and the not yet talking to a web application they need to talk to a web application through a web server and the Web
to web servers that are out there are Apache into next certainly there are others but these are the top 2 ones right now out on the internet that we are most likely going to use Python 1 and we reconfigure these so that they would
pass through certain requests to do with the server
let's say for example we write a blog application and we decide that our domain name whatever example . com forward slash blog is going to be our our Python application we would configure the web server so that it passes all forward slash log request on was onto a which was the server that will execute the code along with our blogs and then will pass back results if this is just a web pages spring it's just going to pass that HTML from the whiskey server to the web server and then that would be passed back to applying for just generating J. Sun for example if we just have a machine readable version of our blog will pass factors the JASigning from the ways the server to the web server back to the calling invoking client now
the thing here is what about our code right so we need to get
our code on that server somehow and that's resource control comes in so view source control when we were just developing application
checking files and but we also use it as a part of our deployment process so we can get the latest version of our code for specific version specific tag our approach on 2 are production attached servers a couple of them that are mostly use now we
begin in material is just should distributed version control systems and that's the ones that most Python developers and using today but they're not being forced to use something that is you know from 20 years ago or or something like that
OK the way that this looks is we may host or source control
ourselves when they have a game server set up by different clustering full our source code down onto a server from that source control history when they also use something like the harbor BitBucket that is just going to host it and will pull it down and from those those servers instead of posting ourselves and maybe we pay them money so that they can hosted as opposed us having to rent rent another server or build another server ourselves when we
pull down the code we have application dependencies which was stored in a file was the defect of our requirements that txt file which would look something like this
using Django 1 . 5 . 4 reasons will help a library reasoning requests and straight all these different libraries we had specific version numbers that were using now we need to get these dependencies on for servers well there's 2 concepts here 1 is a virtual and and virtual and
useful way of creating an isolated pipelined execution environment so is basically the way I like to think
about it it's just if you have a system copy of Python installed that's not going to be clean it's your only version of Python virtual and gives you unlimited number of copies of your Python interpreter with their own set of dependencies so a foreign consulting environment we've got 3 clients 1 of them is using legacy Django 1 . 2 another was using 1 of the war and other ones agreeing to 1 . 7 we have 3 virtual and 1 for each of those clients and they have all the stuff dependencies not only Django but whatever version numbers of other libraries and they're using in each corresponding virtual and now to get files into the virtual and we could technically poster server with all the distributable as for those those versions of the most likely we're going use paideia it's not to be confused with type I type is a different interpreter which is a just-in-time interpreted this is high PI which is a centralized Python hosting repository which is that I find origin here's how this looks like
also added just a little like sticking out when I add something new this diagram because the diagram does tend to get busy over time I just added a little sticky note to show what we're changing every time they show this diagram as were building up so in this case were going down the source control from the source code from source control were putting it in our virtual and or putting it on the file system it's being executed in the virtual environment and were pulling down dependencies into the virtual and from I usually through something or something like that were easy install we're just pulling those versions those version dependencies down through from PI PI and putting among ourselves so now we've got this whole execution environment and the web servers handling traffic Physical view the big missing component here and
that is the data so we need a database something or other so those 2 forms usually that were storing data in and most the time
restoring just traditional data in a relational database so that's the 1 on top and then there's also all sorts of musical databases our data stores from that sometimes you going to call them single data sources because they're so ubiquitous now relational
databases most likely were going to be using my sequel or post-processed I tend to use PostgreSQL as the
time seems like what a lot of the Python developers are using now it is certainly my simple is about option as well and you kind of have to give the flip back-and-forth particularly for pipeline consulting environment but if I start a new project and it's my choice attendees stress now from sequel perspective most people are thinking like OK document or induced on my godmother DB where I don't have the same constraints that would have been a relational database but this also includes things like red and right is a key values key-value pair store that is just in memory and that's often used for just if you have a long-running database transaction is 1 story aggregated results to just put in a key value pair store like credits and that'll just pull it out very quickly for memory because relational databases are going to to actually retrieve those that the data that you want now with something like what would you be there's been a lot of talk over the past few years about what no sequel data source can do for you the way that I like to think about it is germ changing the where your constraints are so instead of having a schema in a relational database which is enforcing foreign keys and you have to have you just a structured to database if you database migrations was people say I would do all that stuff on user document or entities like-minded EB or react and I'm just going to understand use that and that will free me of those constraints the problem is is that that doesn't actually remove the constraints altogether it removes the constraints up to the application layer so when you change your schema sure you're not running migration but you have to account for that code and so the complexity shifts from being at the database level at the application level such as something to be aware of it doesn't mean it's not about option for your use case which is think through that fairly was a couple technologies and their browser based some unfortunate for lack of time we will sort of displays over these JavaScript and CSS you're building any sort of
web applications this is what is going to be rendered help help your web application be rendered in the browser CS cascading style sheets from just a layout perspective and and JavaScript for dynamic interaction and also things like background acid or angular for this is more dynamic 1 page applications were seen to be a little bit more popular in certain
industries now if you're just reading API you really don't need to worry about that just sending machine readable chase sonah externality animal to not even be worrying about those front-end technologies there's 1 other thing about this it's a content delivery network so and once set up you could have a web
server serving up these files but in another set up you could have a content delivery network like Amazon CloudFront or Akamai to serve up the static files for you that takes a load off of your service so that's 1 option as you trying to scale up you can say I want my engine extra Apache Server serving up JavaScript and CSS and that 2 content delivery network 1 more thing to
complete this task used and this is often the forgotten its like of something that happens like after that really got my if you request response cycle is taking way too long on it's hanging on this API call you're probably if you're doing something it takes a long
time then you probably should be doing that outside of the HTTP request response cycle so use something like Manfred salaries tends to be the most common task you the plotted
taskmaster was bit lighter weight in or Q which is originally intended just Paredes OK so
what that does that completes the right hand side of this diagram the task queue allows you to execute Python code outside of anyone who requests sending initiative you request for you responding to the HDP with to be responsible so if you have a long-running deities maybe you have to calculate all sorts of stuff across millions of records that are on that in a regularly scheduled task job you're not
going to run that when someone says give me the results of this query just gonna run it's a every 15 minutes for every 30 minutes and you just can't do it on an ad-hoc basis so rather than during a request response OK there's a few other pieces that we talk about here on so we've got our application up and running like this is a pretty standard set up for most of the Python web applications now 1 thing to note is that this is very extracted from a server perspective most of the time you're going have database on a separate server then web server you may be scaling out may have 5 web servers with the with the server's running on top of them and they're all talking to the database so you could take a look at this right so right hand side and there's multiple ways to scale that were not a concern ourselves that at this point but there's a lot of free information on how you scale how these architectures just think of this as these are all the pieces that you need and eventually you can break them apart so the scale that so we've got people coming to our web application and now I wanna know like what are they doing right so web analytics is this is where web analytics comes in and says that all through encompassing philosophical question of is there anybody out there and they give us 3 graphs and charts and
like numbers and stuff and we can see like only obstacle people are actually coming to my web application and using this is amazing so we can see like what are the traffic trends and if we if we put out a new version of people using it more or less how power interacting with us so the way that this person is that we would have web analytics usually holds posted on the on another server so like Google Analytics a third-party third-party hosting and they would take data from the browser and they would just aggregate all those metrics create all the research for you there's also something like makespan Ortiz metrics and also serve and you can also do this now we do analytics as well where you can send data from the servers as well put in the traditional sense of you basically just slap a little javascript align on your but HTML page that's rendered and then that is going to sound through JavaScript from the client-side analytics and data to Google Analytics services so that basically the set of the user OK so now cool we
got like where there using this but now to a problem that there isn't and what we do know so that's for logging and monitoring come and the suffix overlap to some extent the way that I like to think about
it but the I don't know if there is a hard definition for this the logging piece of it is generally by events so I want to lodge every time I have my 100th visitor or 1 log when I get a certain error or I 1 allotment database query takes longer than a 2nd this specific events that were looking out for words on the monitoring side it's basically just monitoring possibly in the background so that later on it there's a problem and you wanna see what's happening you can just look at all the data that possibly collected over time by Nietzsche that's running on your server so the way that the system with and really host monitoring and
mining solutions yourself there's also a 3rd party ones like New relic incoporate from honoring standpoint and informal lines simpler there's loudly and long and many other services like that and so you can post them yourself or you can use a third-party service and this is essentially the set up that you use your server is going to be sending data and all also to the browser tends to send data will often have a little agent that runs in the browser is sending data to a monitoring service just passively in the background the some other topics here
that are important as well we unfortunately we really have time that died in them but just in
general you be thinking about what is the deployment lifecycle look like if you have a pull this of up virtually cut off the the bottom with all other topics but I'll just tell you real quick what those are just thinking about the point in life cycle we check code and to be picked up by continuous integration server to be taxed tested and there should be some automated deployment process so we all have to manually can do the deployment and going along with that is configuration management tools made public shaft salts that interval with that is going to allow us to do is set up the server the server or servers in a certain speed and automatically you revert to a certain state or the ball varsity through scripts rather than just manually right and tuning things like changing patches and by S is eating into the server other things caching is allowed as a way for improving the performance of your web application arm and then there's just other topics that are out there that are just of certain things like well what are the resources do I need in order to learn about certain frameworks besides despite the gender now if you take a look at this pdf on the web site each 1 of these is actually a link so you can go into each topic and see what the best resources and plaintext explanation for what each topic is to going back to the beginning when I have been working with all these junior developers who were like I'm just getting started on with Python and building my application of this great idea and I want put in a lot of times they're looking for this is like a plain language explanation of what these topics are and I would
encourage people to when you're writing open-source software and you're working on these things Alzheimer's beginners are looking for just a plain language explanation of what something is analogies that allow them to better understand it
so often and I get e-mails about this so often in the Python but also in other
ecosystems the programming
world we try make things sound really complicated when facts if we just use in plain language it will help people much better tube understand those topics so what and trying to do with that I found that come from I've had about 70 sixers thousand people visit so far this year has been also to thank you guys in the audience have have read that and I encourage the rest of you just if you're wondering about these topics and linked to all sorts of other topics on there above the back on the website and also on this talk is well my name is not and developer than petroleum here's my contact information I've tweeted out this presentation right before
it came up your freedom from Indian however Twitter and here's my the rest my contact information thank you very much much much don't take a couple questions
but but time for us and so on and questions now the questions that I think very much in the search
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation


  445 ms - page object


AV-Portal 3.8.0 (dec2fe8b0ce2e718d55d6f23ab68f0b2424a1f3f)