Add to Watchlist

How we switched our 800+ projects from Apache to uWSGI

10 views

Citation of segment
Embed Code
Purchasing a DVD Cite video

Automated Media Analysis

Beta
Recognized Entities
Speech transcript
and about from his matches couldn't tell us about using you is which I've recently had very good experiences from FIL thank you for coming to this stock I know it's hard to get up this early you know this be applied is still the midnight everybody so yeah I also had problems today OK uh 1st the question who is already using OK it's nice I know that prevent here prevent yeah this is the lead developer of here is you can we give him some philosophy so the talk is called how we switched over 800 + projects from attention to you is the so who we I work for the company called I love it
said web agency rational emergency but we do websites mainly we
are part of island group which consists of 3 companies I love targeting icon it I love this pronounced just like I was not a love because some people ask me about it I don't know why just so we is situated in Russia in Moscow we have more than 120 people in and we made more than 800 projects during the last years I personally work as a candidate for the hell of company and this is 1 of the largest Russian weapons OK my name is much and as I noticed that I work even Russia Moscow and also the author of by amendment
package which is currently the most popular feature complete bycatch to excessive data from randomized from in and architect probably you see my life in talks Monday which is the successor of gender would you define a package of which provides for that matter table petition for general be owning and SQL from users and when we just started we used on the beach for developing our projects with giants on the front end
and pension we've remote beach on the back and which was the best available with hostage duplications at that time it was optimal but then after several years we realize that there are a lot of problems and downsides repeats itself so we started to search for another language and just by the just by and for several reasons which are beyond the scope of this talk of and because we already had already working in Japanese and Apache environments so we decided to choose between what Python which was not completely dead and the band onto the time and what we see which was also there but it's corpse was to work and you know it was more feature-rich and had monitoring patients so we chosen what we and societies here about the applications we mourn with you we used to use what we see in the because it allows allow us to derive applications in separate processes and was also the augmented weight from what we see the author himself but actually everything works but we had several problems with that action with OK mean problem was that had projects written different versions of PhD in Python and the only way that we found we
you know we had to have 5 Apache instance this 1 for each version listening and different words which was really a nightmare support another problem was we
use the automatic called loading during the development of reading and 1 so we we use general parameters for a project by so we want you general building maps and we wanted to develop we need more research because we wanted to be sure that everything will work as expected that we deployed production services and there are 2 modes in the Malt with basically 1 call and that it won't and as far as I know 1 needs to restart the whole attaches to the goal changes and there is there's a demon mode which we were using and you have to use you touch common to the With the entry point to rule the source but also there was the site from what view author which was hidden nodes you mean the docks and it was the 1 100 was something quite soft custom-made Python script which you wrote and we had to include description of projects and we will the animated so the sum of the problem you know but still it's not to 2 thousand 14 year so there must be a more appropriate way to do that and actually to the moment on an active development all the more reason he started again there are several new versions already and
started from . 1 there is there is a general obligation body which has a 1 will be the common to the general which allows Python you know manage violence and what was the real intent is common but this only for general and it's actually the same old 100 wise what was aligned script which is not just became the part of official package of what we see so nothing new OK another problem was then during development we create a lot of different branches and the
in want them to be available for example will the like branched off project . com and we also wanted each branch Girondist separate human process because if 1 branch has some problems and 10 of these problems can crash human process the other developers should continue to work say the solution we came up with to have a he took which generates separate Apache config file for each new branch and then restarts Apache is a branch gets deleted them the config file is is that to and we have to look at that again no OK if we have 20 branches we have 20 almost absolute
similar config files yet except for the different view process name can do other things In this is only 1 project so what if we decided to change something in the config file of project we have to read all these 20 config files know this really painful time perhaps there is another week in hole there is another way but we we spent almost a month whom could just find it so maybe somebody knows the stone and we're interested of
yet another problem more problems so when we add a new concept file we have to use you ought because eventually can plot configuration files dynamically but there are actually 2 problems with that 1st but there is still a little time when this web server don't respond to requests this time can be from milliseconds to seconds to based on various factors and 2nd if there is an error in your configuration file the whole so they will crash yes there is a concept comment which can
check the semantics of the file but this is only the simplex so if it's not perceived experiments of logical level then the whole graphical observable branch to decide 10 yeah was clear sign small problems you
know may be subject to but still the first one it better configuration files modeling and a lot of people when me including who thinks that you know if you look at the better integration files you will need some time to understand what's going on in there especially for the yes on complex once so that that hard to consisting properly OK so the have yet you have to be really Apache experts to configure it in a way that it could compete with other web services in areas like in of memory management utilization so not every system knows how to do that the yet old you know even my grandma was used in building yes is 2 . 2 version was released in the 2004 yes there is a key basis to develop this branch by actually the most box and 2 . 4 version was released in 2 thousand 12 and yes it's fixed a lot of problems and limitations for example no memory management became much better the final edit and duty to declare config file variables so for object files beginning between 0 to stand yeah what we use seems to be better at the time when we was looking for a solution but it's actually developed
together so this is not really a problem and all these problems so made us to start looking for solutions and completely quickly found the solution it's called here is the soul of the review it's a morning project coming from not mistaken it started
somewhere in between the 2 thousand 902 stuff 2010 each has a
really fast development cycle and new features are constantly added it supports a lot of languages
including Python and survive pipeline Jonathan's forthcoming coming support speech you look through roughly aligned go has support for be a danger in 1 from this field that applications and what sums to and that works you know all indexes we use these these these and other causes like Mac OS hilarious and so on and Janis supports you with the protocol directly and reason the best performing political community but it also has a total features will discuss some of the most interesting later in in just a brief introduction how we can install it's really easy so you just go download the latest version of the they that's the
majority of you these features are available as points which is also really cool and if you want to have the maximum amount of flexibility and use only the minimal amount of resources to just create a model beauty which is the recommended approach and I'll show how to do it from the source because I respect each depositors may not always have the latest version or may not be available ones so we just don't log these here is to think of if you want to change its publication and you know you can answer the to you the following the objectives the you can't
search quarantining file in then you create impetus course now we have to do the work of were already
consists of several packages that are most likely to be named by everyone but if your thing that you're not everyone but you can also customize what exactly the chordal consists of and the compiler and then we needed to build blood to blood full but 2 . 7 3 . 3 times and let's do 2 points will be and already each this that but anyway we just completed installed process it's actually it's very simple now let's see how we can solve the problems that we had with Apache and more recently with the help of you 1st yeah multi-version 1 this is really cool this was the biggest problem for us all and
is just works and you really you
can borrow as many points as you need and it just works terms it yeah remember that 100 while slides Python script which was used to model is key to my changes and
RealTravel doing process you is he has just the no simple option you just to specify how many seconds here we want to be you know this there is a need to scan for your call changes and true everything automatically there is also a natural option including so normal custom weight homes greets know which you have to search and including project finally of we also had a problem we with used
with the devil process you can think of the dynamic and we need to separate process for did right so we had to generate up to the single config files and UBC has this thing called temporal which was the solution to this problem of basically so a special instance of with you that you want to especially those assembles forms stop rewarding inferences known as vessels on demand by default the emperor conspecific uterus for supporting configuration files but it is accessible using imperial wanted to that means that you can store configuration was addressed you know Mondrian publishing 0 and you and so on there are many options why but see example for the deuteron much of what we started is the
emperor instance-and and passed from 1 of yeah the falling due to pattern now if we had a new file to this dude during the emperor will automatically support and new vessels and if we modify the file the emperor we hassles and if we the file the vessel will be killed in all this will happen absolutely automatically there is no need to use some you know Stockholm and so on and now what if we have an error analysis integration files a chilly nothing bad will happen and to the other vessels the emperor will just say that these vessels gross you know and once spawning new instance of it so you then paradise himself then obviously altercation vessels on died with him of so how will this actually help us with this problem with dynamic with the the process
problem yet but should we also create the same insane amount of configuration files project 1 for each branch yes and no will use complete config files and creates a link this is an example of a strictly convex file 1st we define our variable called project you and said you the path to our
project then we find that will be used in Python 2 . 7 bits of what in our other Python project related stuff which should be nothing new to anyone knows how Python apps work the interesting thing here is the present that magic variable which is the good thing could without extension but we don't know about you but for me this index is much much cleaner you know compared to batch configuration files can also is a bonus my brain doesn't hurt after I work with this configuration files so so that means that from now on instead of generating separate config file for each branch of and we
just create to the project's complete file and and yes they will be to this will give us 23 means for 20 branches still but the mean Apaches there if we need to change something which ended in 1 place in the complete config file instead of changing every you know config file generated for Apache yeah and and also because the values in the human motor which is monitoring our directories for all the files when using link is created a new vessel is called automatically and if we do it this thing then the vessel is skewed also automatically no
need for any restart Commons and so on we also started from home 1 . 9 . 1 you cannot tell the emperor to form that asshole only after the 1st request has been made combined with the Ireland and I liable options so this allows us to have a really truly on-demand applications which is also called OK was briefly talk about some of the year is this
interesting features so we can implement artist dealing with
here is values in the woodlots said and and
promotes the value and I'll options so it's a combination of features you know the idea is that when side use this cycle is having in your vessels just can't handle it they can ask emperor to enter brutal want and give them some help right so they could win the battle you know interval the request to and after the load is normal again there look all x and everything is fine again but as if 1 . 3 there is an alarm
subsystem it allows the developer or sysadmin to announce some special conditions the of the various channels for example you may want to get modified Java administering terrible or you can walk files for example so that there are a lot of also options to configure 1 subsistence system there is also a nice features for well in Python modules you know that having multiple versions of python package
model is very common in and the 1 we use manipulation by the best for using Richland's but for example you used use us another option is called Elliott insisted let's say you have imports of foreign bar modules in a lot of places in our court and we want to make some modifications to word keep our regional from environment using that for whatever reason we can create experimental 4 instrumental 1 will use and make changes in them and alias them in when you know you will making import food and water instead of input in the original files experimental for an experimental 1 models will be inverted there are a lot of cool features a lot we need a lot
of time to you know talk about all of them but briefly there's also crontab
yeah there is balanced clustering subsystem of system which allows you to automatically do indeed some heavy costs through separate threats it also has a lot of different blog means for different tasks and it integrates with almost all that well known web services like non-Japanese that should you life http the MoMu to and so on and so so on it has
also generated integration and 1 in which shows the status of here is you know allows restart all query sketch and there is more functionality in development also it has reached configurations system it's
supports configuration files in any XML yellow adjacent and so you can choose what you like what is that if for bring it has more than 30 magic variables for all sort of things In all environmental variables placeholders and and so on you can even do simple more often in place holders you can reach hundreds of other files from your config files so you can write 4 cycles and the statements and all the files you can't agree on variables in your Python scripts and use them in the and much more to the goal of OK finally which was used to generate power of you know there is no easy
way unfortunately there is no such magic tools that will translate all your Apache config files to you is the ones and how we do that with our you know what 800 projects well that we write
our projects into several groups that have equal or almost equal so that's a config files then you wrote the script that generates similar and looks for you is if we group the 2 Gus approximately 2 days and the to switch fall projects to you with the unknown that service then we started to run our function that can see Vol them pass then whenever there was a problem that we just looked at and you know we this way which Interpublic fast for you with the and we will when we were sure that everything works on their service the made this feature here is gonna production service so that's probably the that but again there in by hand and fortunately conclusion that in the conclusion that want to say that that she is actually not that it's a very good and stable reps so which is
used by a lot of people and if it's suitable and you need a suitable for life situations and you can happily use it if it works for you do not blindly trust people that see stocks go try this or that use of the collapse of the and just to know your brain and think do you really have any problems with your batch and if no then we've happily with it and don't listen to anyone In this talk I explained when we switch so we can have any problems with the pitch performance but we had problems with these features so we could do everything we wanted with Apache but wasn't happy with how we could do this In what I'm trying to say is that you should choose an absolute by features and not by benchmark the role of here is viewed has a much better memory management then pressure to you probably something in performance and also it's for pointless to benchmark of because usually the all perform approximately the same if they configured properly and 99 cases this is an application that is written incorrectly and has performance problems and not web server you know which just serves in addition to the world so the if B if I feel
that thank you max so we got time for a couple of questions them glutamic but come here to so I want have to the last and you know was yeah but so the question
is if it's maybe was better to compare with the real mode you know we're not used read the whole Texas thing here the yes yes of course in front of so we we had dynamics in front of and the patch with more with you on the back and in our have again Janet on the front end and was he on the back and speaking about the comparison if it's a fair comparison I think yes it is because more recently is just you know thing to serve Python applications but he just won't work without a page itself so of course and then well I don't think the comparison space so yes you can you can use the which is proxy yes you can but if you know the problem was that it's not very comfortable to work with a pension you have to search for some work-arounds always to make everything work we didn't like what about the the problem 1 of the things that you want this is 1 of the giants and part of what we call 1 on the other hand 1 this yeah but do drink on each it can't surf you know any other type of obligations Python yeah so it's so maybe you know you can compare that with you with because it's you know they therefore web service basis of this support of of language but acknowledges and so yeah this is where this comparison the question of course track thank you thank you letters you
know
Goodness of fit
Computer animation
Lecture/Conference
Web-Designer
Projective plane
Matching (graph theory)
Maxima and minima
Berlin (carriage)
Rational number
Projective plane
Coma Berenices
Mereology
Local Group
Computer icon
Maxima and minima
Local Group
Web 2.0
Pointer (computer programming)
Computer animation
Authorization
Website
Musical ensemble
Group action
Gender
Multiplication sign
Magneto-optical drive
Projective plane
Debugger
Coma Berenices
Stack (abstract data type)
Complete metric space
Cartesian coordinate system
Weight
Table (information)
Formal language
Maxima and minima
Revision control
Derivation (linguistics)
Arithmetic mean
Video game
Integrated development environment
Authorization
Subtraction
Demon
Point (geometry)
Building
Service (economics)
Line (geometry)
Ring (mathematics)
View (database)
Source code
Parameter (computer programming)
Revision control
Inclusion map
Flow separation
Mathematics
Authorization
Scripting language
Descriptive statistics
Product (category theory)
Mapping
Scripting language
Software developer
Moment (mathematics)
Projective plane
Code
Instance (computer science)
Word
Summation
Revision control
Vertex (graph theory)
Website
Reading (process)
Asynchronous Transfer Mode
Demon
Process (computing)
Computer file
Scripting language
Line (geometry)
Ring (mathematics)
Software developer
Magneto-optical drive
Projective plane
Code
Branch (computer science)
Coma Berenices
Branch (computer science)
Bit rate
Coma Berenices
Mereology
Electronic meeting system
Configuration space
Scripting language
Aerodynamics
Process (computing)
Subtraction
Separation axiom
Server (computing)
Demon
Process (computing)
Computer file
Divisor
View (database)
Multiplication sign
Magneto-optical drive
Projective plane
Branch (computer science)
Similarity (geometry)
Coma Berenices
Bit rate
2 (number)
Plot (narrative)
Web 2.0
Lecture/Conference
Configuration space
Dependent and independent variables
Process (computing)
Subtraction
Error message
Building
Computer file
Structural load
INTEGRAL
Scientific modelling
Multiplication sign
Branch (computer science)
Total S.A.
Semantics (computer science)
Declarative programming
Revision control
Sign (mathematics)
Web service
Simplex algorithm
Electronic meeting system
Utility software
Energy level
Cuboid
Configuration space
Physical system
Area
NP-hard
Proper map
Basis (linear algebra)
Expert system
Memory management
Limit (category theory)
Variable (mathematics)
Fault-tolerant system
Error message
Logic
Crash (computing)
Configuration space
Object (grammar)
Pointer (computer programming)
NP-hard
Magneto-optical drive
Projective plane
Configuration space
Cycle (graph theory)
Software developer
Java applet
Survival analysis
Cartesian coordinate system
Formal language
Field (computer science)
Formal language
Revision control
Subject indexing
Summation
Causality
Speech synthesis
Cycle (graph theory)
Communications protocol
Multiplication
Erlang distribution
Point (geometry)
Maxima and minima
Installation art
Momentum
Computer file
Scientific modelling
Source code
1 (number)
Core dump
Binary file
Revision control
Maxima and minima
Computer animation
Mathematics
Object (grammar)
Units of measurement
Local ring
Point (geometry)
Plug-in (computing)
Process (computing)
Building
Multiplication sign
Revision control
Core dump
Login
Online help
Local ring
Multiplication
Compiler
Point (geometry)
Slide rule
Process (computing)
Ring (mathematics)
Scientific modelling
Projective plane
Code
Login
Weight
System call
2 (number)
Mathematics
Term (mathematics)
Revision control
Normal (geometry)
Scripting language
Multiplication
Mobile app
Default (computer science)
Asynchronous Transfer Mode
Process (computing)
Computer file
Assembly language
INTEGRAL
Mathematical analysis
Dynamical system
Directory service
Instance (computer science)
Inference
Event horizon
Single-precision floating-point format
Configuration space
Pattern language
Aerodynamics
Error message
Form (programming)
Batch processing
Computer file
Computer file
Projective plane
Branch (computer science)
Bit
Complete metric space
Template (C++)
Subject indexing
Blog
Linker (computing)
Touch typing
Network socket
Software repository
Configuration space
Extension (kinesiology)
Local ring
Separation axiom
Computer file
Projective plane
Branch (computer science)
Branch (computer science)
Directory service
Complete metric space
Cartesian coordinate system
Template (C++)
Arithmetic mean
Lecture/Conference
Linker (computing)
Configuration space
Local ring
Asynchronous Transfer Mode
Computer animation
Structural load
Combinational logic
Branch (computer science)
Cycle (graph theory)
Local ring
Template (C++)
Multiplication
Computer file
Java applet
Logarithm
Software developer
System administrator
Scientific modelling
Aliasing
Magneto-optical drive
Water vapor
Letterpress printing
Exponential function
Revision control
Word
Mathematics
Computer animation
Integrated development environment
Module (mathematics)
output
Configuration space
Modul <Datentyp>
Condition number
Physical system
Web service
Video game
Computer animation
Multiplication sign
Blog
Subtraction
Separation axiom
Physical system
Task (computing)
Computer file
INTEGRAL
Software developer
Disintegration
Variable (mathematics)
Functional (mathematics)
Power (physics)
Integrated development environment
Lecture/Conference
Query language
Statement (computer science)
Configuration space
Free variables and bound variables
Knowledge-based configuration
Scripting language
Cycle (graph theory)
Quicksort
Product (category theory)
Service (economics)
Scripting language
Projective plane
1 (number)
Repetition
Functional (mathematics)
Local Group
Flow separation
Wave
Writing
Configuration space
Scripting language
Mobile app
Email
Addition
Server (computing)
Slide rule
Server (computing)
Multiplication sign
Memory management
Coma Berenices
Weight
Cartesian coordinate system
Benchmark
Maxima and minima
Web 2.0
Video game
Benchmark
Personal digital assistant
Process (computing)
Right angle
Pressure
World Wide Web Consortium
Web page
Email
Trail
Pairwise comparison
Server (computing)
Spacetime
Slide rule
Patch (Unix)
Debugger
Basis (linear algebra)
Dynamical system
Coma Berenices
Weight
Mereology
Cartesian coordinate system
Formal language
Maxima and minima
Web service
Lecture/Conference
Data type
Asynchronous Transfer Mode

Metadata

Formal Metadata

Title How we switched our 800+ projects from Apache to uWSGI
Title of Series EuroPython 2014
Part Number 16
Number of Parts 120
Author Tepkeev, Max
License CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
DOI 10.5446/20006
Publisher EuroPython
Release Date 2014
Language English
Production Place Berlin

Content Metadata

Subject Area Information technology
Abstract Max Tepkeev - How we switched our 800+ projects from Apache to uWSGI During the last 7 years the company I am working for developed more than 800 projects in PHP and Python. All this time we were using Apache+nginx for hosting this projects. In this talk I will explain why we decided to switch all our projects from Apache+nginx to uWSGI+nginx and how we did that. ----- The talk will start from describing the setup we had for the last 7 years, i.e. Apache with mod wsgi for Python projects and mod php4/5 for PHP projects + nginx. I will explain why we used this setup for so long time, what problems we faced with this setup and what solutions we tried to solve them before switching to uWSGI. Then I will tell about uWSGI, what it is, how it works and what features it has. I will show the comparison of configuration files, how simple it is to configure uWSGI compared to Apache. Lastly I will explain how we managed to switch all our 800+ projects developed over the years in 2 different languages with 2 major versions changed (PHP4/5 and Python2/3), how this switch simplified our development and administration of this projects, the improvements we got in memory management and other areas.
Keywords EuroPython Conference
EP 2014
EuroPython 2014

Recommendations

Loading...
Feedback
AV-Portal 3.5.0 (cb7a58240982536f976b3fae0db2d7d34ae7e46b)

Timings

  631 ms - page object