How To Build a Python Microservice Without Losing a Job
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Title of Series | ||
| Number of Parts | 118 | |
| Author | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/44769 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
| |
| Keywords |
00:00
GoogolPoint cloudBuildingProcess (computing)RootMathematical optimizationReading (process)Extension (kinesiology)Stack (abstract data type)Group actionOrder (biology)ResultantDifferent (Kate Ryan album)Service (economics)Software testingSoftware developerCodePhysical systemMedical imagingBoss CorporationPoint (geometry)Video gameRight angleJava appletSelectivity (electronic)Mathematical optimizationLevel (video gaming)Session Initiation ProtocolDampingComputer architectureBitMetropolitan area networkQuicksortWeb pageTraffic reportingGodBuffer overflowFormal languagePattern languagePrototypeMultiplication signExpert systemProjective planePoint cloudAxiom of choiceCASE <Informatik>Natural numberProcess (computing)GoogolStack (abstract data type)Thermodynamischer Prozess2 (number)Web 2.0Human migrationDebuggerStructural loadComputer clusterComputer animation
05:47
Software frameworkProduct (business)Component-based software engineeringMUDOperator (mathematics)Software frameworkDebuggerComputer configurationDefault (computer science)Point (geometry)Formal languageResultantConnectivity (graph theory)Boss CorporationLevel (video gaming)System administratorProduct (business)StatisticsPattern languageScaling (geometry)Fitness functionService (economics)Stack (abstract data type)SynchronizationFlow separationStrategy gameJava appletVideoconferencingQuicksortWeb 2.0YouTubeSoftwareLoginBenchmarkCASE <Informatik>AbstractionInheritance (object-oriented programming)BuildingLimit (category theory)Interface (computing)Line (geometry)Stability theoryThermodynamischer ProzessProcess (computing)Right angleChainProjective planeVideo game consoleSoftware developerRevision controlAxiom of choiceComputer architectureTowerSlide ruleoutputArithmetic meanElectric generatorNormal (geometry)Computing platformCuboidMultiplication signGoodness of fitHacker (term)Software testingWritingWebsiteDirected graphWindowComputer animationLecture/Conference
14:55
Server (computing)Function (mathematics)Information and communications technologyLogicPolygon meshHill differential equationProcess (computing)Information and communications technologyService (economics)Standard deviationConnectivity (graph theory)Pattern languagePhysical systemDirect numerical simulationTask (computing)Polygon meshLoginView (database)Centralizer and normalizerMessage passingWeb 2.0Communications protocolContext awarenessError messageCartesian coordinate systemTelecommunicationOperator (mathematics)Electronic mailing listQuicksortCoefficient of determinationDatabaseComputer architectureGame controllerCASE <Informatik>Reverse engineeringUltraviolet photoelectron spectroscopyTrailSimilarity (geometry)Multiplication signRoutingDifferent (Kate Ryan album)BitSoftwareWordInheritance (object-oriented programming)Dot productWebsiteMobile appLevel (video gaming)Direction (geometry)Element (mathematics)Server (computing)Uniform resource locatorMetadataÜbertragungsfunktionComputer fileFault-tolerant systemHard disk driveSoftware frameworkThermodynamischer ProzessChainFormal languageProduct (business)MereologyProjective planeFunctional (mathematics)Independence (probability theory)Dependent and independent variablesVirtual machineLogicJava appletINTEGRALOpen setVideo gameRootXMLUML
23:55
Multiplication signComputer animation
24:23
LoginSynchronizationService (economics)Boundary value problemClient (computing)Projective planeCodeCASE <Informatik>Control flowPropagation of uncertaintySoftware developerSoftware testingLoop (music)Communications protocolPoint (geometry)Physical systemRight angleFehlererkennungMultiplication signLambda calculusComputer configurationFerry CorstenSoftware frameworkPresentation of a groupError messageLibrary (computing)Connectivity (graph theory)DatabaseLevel (video gaming)Functional (mathematics)Pattern languageLogicSubsetProduct (business)INTEGRALCache (computing)2 (number)BlogMereologyUnit testingInstance (computer science)ChainCartesian coordinate systemEmailSound effectAsynchronous communicationWeb 2.0TwitterServer (computing)Lecture/Conference
Transcript: English(auto-generated)
00:06
Thank you, dear friends. It's so nice to see you. Let's get right to it Did you notice that executing the same actions in different order can often lead to different results? Even if we take very very correct actions that
00:22
Normally get us to beer or some other wonderful results and do that in the wrong order We can get into a disaster How often have we heard oh shit? I should have done that first the famous thing and it especially applies in the software development because as we all know premature Optimization and what is premature? Where is this point? It is pretty hard to predict sometimes
00:44
So what we're going to do today is we're going to talk about some not so obvious aspects of building the distributed system of micro services in Python and as well as in other languages without failing much because I know that's like Yes, if if you are full-time employed getting fired is pretty hard
01:04
If you are freelancers gets easier Fortunately we have now this whole micro service. Hype makes things much easier You can fail in so many ways so getting fired is really getting there, so I will be also Sarcastic sometimes, so if you recognize yourself, it's not you
01:20
I'm not I don't mean anyone I mean myself in the first place because I went through this whole thing and let's start simple We start with a stage zero before even the whole micro service thing starts Imagine that the big boss is calling you. Hey, we have to talk you think am I already fired or what's happening?
01:41
No, you're not. He says it's time I Was thinking I was reading report. It's time that we go micro services At this point you might think hey, but sir we just hairdresser saloon We accept bookings only on the phone anyway like what are we going to split into micro services
02:02
Don't do that Don't do that. That's humiliating the boss. He perhaps read so many Gartner reports about success stories He was thinking how he will brag in the bar with his boss friends like hey, Jimmy Microservices now What do you have and you offer him some Django thing no? Just do what he wants because otherwise if you don't then you fail on the stage zero
02:25
You will not even get that job as the freelancer is particularly relevant And now you think I'm talking a bit extreme maybe but I have a real-life case I worked for one big and famous company, which will I will obviously not name they hired me to fix their architecture and
02:41
When I presented my first draft, they said alright, but where are micro services? So What's micro services is that the requirement they say of course I mean looking to the future now Should supposed to be an expert so good well should understand that Microservices are not so much a tech solution as an HR solution when you have
03:02
Thousands of people and they have to cooperate on the same project efficiently That's a very good way to go or if the load is such a difficult nature But you guys you have like one request per second big time They were disappointed we thought we were hiring an expert and you're offering us a monolith lost the job
03:24
So let's just agree micro services take it as granted. It's hype. They will come don't resist it accept it and Start forming your happy and motivated team that is just eager to jump right into that right into the migration How would you normally do such a big new beginning you would perhaps read a book and there are many good books
03:44
You have a good selection also of the cloud container patterns available now published everywhere even for free, but the book I mean The book is boring who reads the book. It's 719 for God's sake we have Google Google is a new book. Google is all we need Why would you read 500 pages in advance? Your team will be so upset. So if the team wants to just
04:04
prototype, you know Freedom of choice and the freedom is a basic human, right? They will be upset if you don't let them prototyping you might lose your job So don't do that. Let them prototype because Google driven development is actually not that bad It's not as bad as stack overflow driven development. Who knows the difference?
04:25
Difference is that stack overflow is when things are already on fire. So if it's Google they are sort of planning it in advance Right now, let's go a bit more serious. Now. Let's go deeper down to the stack stack You have to design it from scratch and here to be correct
04:41
I have to say stacks because we have a new fancy polyglot infrastructure We have many languages and it's not just about the languages Because every language has own stack every language has own tools own build process. And now we have many of this wonderful You know when you push Python code you perhaps do pilings you perhaps run tests you package it somehow
05:06
Flake it whatever build a docker image. That's one way of doing it now Imagine the next thing you have is Java you'll get to know Maven the beauty of Maven or ant You will pack things differently some jars or wires or whatever they're using and now
05:26
Imagine we have front end as well in the stack. You have JavaScript Wow, you would have to minimize it add shims can get a nice stuff Babel other things a wonderful life and then if it's really really front and then the web back
05:40
Google grant. Oh, that is so much fun. I have a perfect illustration for this I took from the JS Congress in Munich It says there are milk products that last longer than some JavaScript frameworks that totally reflects the situation of the front end these days So yeah, your stack will look perhaps like that and just share it with the ops ops will be so happy to see it
06:01
They either jump out of the window or throw you out of the window. So get prepared get your team with you So that it won't be you But of course, of course You can always say it's better than a monolith an old ugly monolith this nasty Jenga tower standing out They are ready to fail We're going to fix this with microservices and we've been way more elegant
06:24
There is a surprise waiting for you because if you if you migrate from here to micro services 90% of cases you will get many little Jenga towers instead of one big All new is each own stack on with each own box and with own hacks to keep it somehow stable and alive, you know
06:43
now imagine that There are just we have micro teams right imagine. There are just one or two people working on each Jenga tower It's a good thing cannot get fired because they depend on you heavily right now, so They will probably not just not fire you though the smart boss would probably vaccinate you against everything in this world because their business depends on every every little piece of this and
07:05
You're basically irreplaceable. I Love it as a result what I call it is an epic zoo of technology Imagine the front end is and react statistics is an R and then you have some admin site PHP legacy you have Accounting made in Java you have to maintain that whole thing somehow and this components
07:24
They they bark some components meow and another components roar and yep. Sorry I forgot the fish fish is important, but it just writes logs so can keep silent now Imagine how can you possibly manage this whole thing? How can you keep this component sane and well for keeping it saying we're going to the next chapter
07:44
Even deeper even more concrete We are going to repeat some wisdom of bigger companies that did it in the past already Imagine the new team member comes and says I Don't care. It's all in Python. I want my new fancy microservice in JavaScript what you have to do
08:05
Beware to offend him or her Should be polite and you just use the wisdom of Netflix Netflix had this issue They had the whole thing done in Java it worked perfectly until they hit certain architecture limits So they had to split into micro services which were Java micro services
08:21
And then they grew grew grew and eventually they had to deal with other languages as well So they developed a very very smart strategy that works for them at the beginning. They said of course at Netflix You're very welcome to pick any language you like you you have a total freedom in doing that But just just in case just in case that language you'll pick will be Java
08:42
You'll get the failover you'll get the discovery you'll get logs. You'll get monitoring all for free automatically, but if it's not Java well You're very welcome to implement it in your language of choice and at this point most of the developers said Maybe Java is not that bad option What I wanted to say is that at the first stage
09:04
Alright, let's say you have to split your monolith into many components But that doesn't mean you need to have that zoo right away You can still stick with one or maybe two platforms that will be well tested that will have a well-defined process on the ops side of building things on testing things and on deploying things so
09:22
It costs a lot especially on the in the beginning to integrate every new process In your architecture, so if you can avoid it just avoid it at the beginning and you can pick just one reliable well-known language such as Java and work with that Kidding kidding no worse, so let's say we take Python
09:42
We need still to decide on the exact tools that we are using in Python to write microservices and here I'm just going to quickly jump over the most popular framework and choices that we have Let's start with Django. That's not a joke if you're damn good in Django. You can write the microservice in Django I know people who did that
10:01
It feels to me like going grocery shopping on a truck Nonetheless Nonetheless yes, you can do that still because Django is still like this very very flexible plug-in architecture Which you can customize or you can throw use the things out, and you have wonderful pieces like an admin panel there
10:21
Automatically, but the problem is that if you take Django by default it has so many components, so if you think of that Actually each component could be a separate microservice So it doesn't really make much sense to use Django as it's shipped to you. It only makes sense if you cut Things you don't need out Then it's fine and speed wise I mean we don't have to discuss Django
10:42
We all know its advantages and disadvantages, but it is still doable So here many would suggest flask flask Everyone loves flask flask is so minimalistic It's fits so well the whole idea the whole concept of microservices because it does just one little thing and it does it right I? Like flask flask is it's okay, it's not the most fast web framework, but it's a comfy one what I like in flask is
11:07
asynchronous networking and This is particularly useful for microservices because when you have a chain of requests from one microservice to another and they are blocking and One thing fails and the whole chain is blocked and does not serve more
11:22
Requests because potentially I know it gets out of the workers in the pool. That's bad. That's not cool So I strongly prefer and suggest you to use the synchronous tool so that you are not blocked If one of the components is blocked And you don't have a limited amount of slots of the parallel requests that you can process at one point of time So there is a solution there is quart
11:41
Quart is the replica flask is just a synchronous Copy of flask let's say I didn't check it out in production. I just know it exists I played with it. It has a very same API as flask does I think they are Developing it further now But it's a good option nonetheless as I said I didn't check it in production flask
12:04
I did so if you were a flask person, and I know flask is a Much better framework because it has a cooler logo, and it's minimalistic It's like yeah, it's 500 a minimal thing does all you need looks fancy for those who do not need to compensate You know with a big scale
12:22
Totally love flask now. I'm just going to quickly talk about other options there. There is hug who knows hug Hug is great hug is based on Falcon Falcon They say by Fred by the benchmark that Falcon is the fastest web framework in Python by amounts of requests per second
12:42
It can handle Falcon wins. It's synchronous, but it's very fast and Then hug is a sort of abstraction over that simplifies building API. It's like Django rest for Falcon makes developing restful endpoints very easy Generates you the common line interface right away because remember your microservices not necessarily have to communicate over HTTP
13:06
It could be that it's very useful if you have something like Amazon CLI Available right away you create a normal restful endpoints you have a console interface Automatically, that's pretty handy. I used hug in production does a great job, then there is tornado
13:22
tornado is 15 year old now as far as I remember it's super stable. It's Synchronous, it's well tested well documented. It's like more More the older version of async IO. Let's say That supports Python 2 as well which for some cases because usually when you come to the project that
13:42
Wants an architecture upgrade the chances are that they have a lot of components written in Python 2 so with tornado you can have fancy synchronous communication right away since to that six they Sort of assimilate the yield from or async await thing was just normal Python generators, so you can use yield
14:01
You can use yield With Python 2 and it just works very similar to the way a sync IO does I'm not putting a sink I own slides because that is so obvious like that's the default thing you'd go with a HTTP and a sync IO That's now the default way to go now the project grows and
14:21
Thanks to Python of course at some stage We hit the point as Netflix did where we cannot resist getting new languages and new stacks anymore So what can we do then? then We should use the container patterns, and there are books published their articles published and also many many videos on YouTube about that
14:41
I'll describe three most popular patterns that are used and that I checked personally Let's start with a site car site car is sort of Extra container that you put next to your main container that would enhance it To be more precise here is how it looks like imagine that you have a main container Which is a web server and your site car container is a log saving
15:06
Application why would you need to separate it you you might ask you can have your main Application and something like a sentry agent that collects your logs or kibana or whatever the the trick is that Every web server even super old one and super legacy one can write logs to the hard drive and
15:25
Not every old framework has sentry integration, so we go a bit another way, and we say let's just assume Every app can write logs to the hard drive And we take another container that would pick logs from the hard drive and send it to whatever cloud aggregation system
15:41
this way we separate totally the responsibilities of these two we say that the web server should just do the web related job and save logs as it can and it's a job of the Sidecar container to pick these locks ups process do the whole routing find the right cluster to submit it
16:01
This sort of work. This is the first pattern that you can use and the second one is ambassador It's a very similar concept sort of a big important fella That you have to speak to instead of speaking to some even bigger entity This is especially useful when you are dealing with some remote resources like a database or a redis cluster
16:20
Ambassador will hide it for you Ambassador resides on your local host it runs next to your main container And you only communicate with the ambassador it simplifies a lot because in your super huge application You perhaps have a super huge redis cluster. That's Every component that speaks to has to maintain you either need the IP or DNS
16:43
And if it's failing then you need a new route So this is all job of the ambassador and the master is always on local hosts the same port So if anything fails, it's a job of an ambassador to get the new route Find maybe do some graceful degradation in this case or cashing things as well can be done on the ambassador level
17:03
So this is also a pretty common pattern very useful when you're writing big big Distributed systems it is particularly useful because as I already mentioned when the new language comes in It does not affect the main container You can have a main container in Java and you can have your ambassador in any other language
17:22
They communicate of a standard protocol standard ports. They're totally running independently now finally the adapter the adapter looks like like ambassador, but the reverse ambassador if the Ambassador pattern represents an application your main application with a simplified view of the outside world outside cluster
17:41
Then the adapter does the reverse it's Simplifies the view for the outside world of your application to give you an example imagine the monitoring case like perhaps we want to have a health check on our main application and different frameworks different applications serve from different protocols
18:02
They have different ways to check if the app is alive or it's down Some would provide you a URL like slash health or whatever some would write in a file in Unix That would say file is there then I'm up and some would just not do anything You would have to do PS outs and check if the product process is running in the Unix machine
18:22
So we can use adapter to help the outside centralized Monitoring system to get the status of every component because we will ship this monitoring adapter Next to our main application and the adapter will know how to check if the main application is dead or alive It would know if it should check a file or check a URL
18:41
and so then the external the external part the Monitoring system would just use the same way to ask the adapter if the system is running or not This way you can also scale framework independent and you can keep the same monitoring system on a very very very Diverse zoo good now
19:04
Quickly going to the functionality after we know the patterns As we already discussed microservice usually has some business logic like the main thing that it usually does either serving some Crude operations on the database or doing some monitoring or something else and there are always
19:21
Communication functions which are repetitive across many different projects It makes sense now to take this communication functions and put up in a sidecar the patterns I already described to you because they are very repetitive. They do this failover and discovery and Grace will degradation thing. It looks much more elegant like that, but we still have one problem now, we
19:44
Offloaded all of the functionality of network functions to the sidecar. However Nonetheless, how should different? Application find each other you packed it into sidecar. But what exactly is the sidecar doing? How is the network communication and discovery happening?
20:01
You need to communicate with Not only with a database your components perhaps also need to communicate with each other and how exactly can this be done elegantly? You can of course put ambassador for each service, but you know, it will quickly get insane You can you can break things and lose your job. So you need a better way of
20:23
Establishing communications not just with a big Redis cluster, but also with the components Talking to each other like small dots and Here we can finally introduce the concept of a service mesh. It's a new step. It's yeah, it's a bit centralized
20:41
It's a centralized component that keeps keeps track of which service is doing what's and available where and so the idea is it's this that the communication between our little Our little services they do not communicate only over service mesh. They just use service mesh to find
21:01
Where are other components located? They still communicate directly. So this is like DNS and steroids. Basically, you can say I can just use DNS at the beginning but DNS depends on caching and in Such an architecture you cannot afford caching too much because if a system fails you need a very very quick
21:22
replacement for a failed system and that's exactly what service mesh does so you can think of it just as a Better DNS system and to be more precise what exactly is doing I have a list here like you normally have lots Tasks like routing access control. It's another big thing that I'm not discussing right now
21:40
But when you communicate within microservices you need to decide on the common way how if you encrypt the data how exactly? How do you decide if one service is allowed to ask another service of a particular operation? Then there is always a context of are you asking one microservice to another on behalf of user or you just need some meta
22:00
information all of these little things you have to do repetitively in every new microservice you bring in into your architecture and Not to do that you offload this repetitive tasks to the sidecar and The service mesh is just maintaining the list something like the access control list routing list task list and so on
22:22
Good Finally I want also to say a few words about the failover People say graceful degradation and short-circuiting there are many fancy ways of saying How would you elegantly tell your users? I'm down It's way more easy to bring the system down when you have many little
22:42
Services because it's a chain if one element of a chain fails I mean if instead of showing error 500 you are showing a message. I'm sorry. We're currently not working It's more elegant, but it's not good so Instead of instead of doing putting effort in this short-circuiting and failover
23:01
I would strongly suggest you to put your effort in better logging and better tracing There is an open trace standard that was developed Especially for this case when you have a big system, and it's it's hard to it's hard to debug it It's hard to trace where exactly did the request fail instead of putting your effort into excuses
23:21
I'm sorry we couldn't do it trace it log. It's make a have a system We have here sponsors. What was the data dog people like they do it. I use data dog for that I'm not advertising them, but also New Relic does it invest your time there build the reliable system Where you can easily trace every requests and every time you can say if something fails
23:43
It was fault of this component because without this you will pretty certainly eventually lose your job Thank you for attention, and I wish you a very nice view of your micro services
24:03
Thank you in turn for this refreshing talk, and we now have time for questions So we have microphones on the side or I can also move Okay so in the meantime
24:21
while they get inspired I wanted to go back to your recommendation on basically the frameworks and There has been quite some articles about Why and talks about why frameworks are can also constrain you a lot especially when you want to iterate over your products or
24:41
Let's say add new features because they basically fix the way the business logic is defined so Would you have also some recommendations there on maybe just smaller libraries, or how you how you would approach that if? Basically rewriting is not an option Yes of course well one trend is things like lambda functions on AWS
25:07
Many people do it many people do it successfully there is basically no framework That's just a function as long as it's simple it works however If you are not so much afraid of leaving that project, and you think that there should be another person replacing you eventually
25:23
I Frameworks are like like a common patterns that other developer knows If I leave a project, and they tell you it's just the the Django rest endpoints, and you can find right away It's a post handler. It's a get handler and you can just Continue working right away if you do a customized system obviously
25:42
It's harder for new people to get into it on another hand there is an overkill of using a framework Especially in microservices because even tornado It's quite minimalistic nonetheless it has also as Django does many batteries included like CSRF protection and so on which you normally would not need in microservices, so you're right
26:01
It's always a trade-off when you pick a framework You make things simpler for yourself and also for people who will be working on the project after you if you do it yourself You usually can tailor it better. You maybe can do it even more efficient But there will be difficulty in maintaining it when it grows and then don't forget the testing you would perhaps test
26:24
Things that be differently when you write microservices because it's not so much about Unit tests in microservices as about integration and acceptance tests you need to check the whole chain Not just one thing and here testing frameworks that act predictably is sometimes just simpler than testing
26:43
Custom code that is not so predictable for others not for you for others that test their component speaking to your component So yeah, but there is no Black and white answer. Thank you. Thank you
27:02
Yes As for follower said or discussing his blog post about microservices the reason is for this error propagation and synchronous communication And the problem is that when you add microservices to your system probability of going down actually increases because with with each
27:27
microservice that can go down So the solution is asynchronous communication between microservices it and it may be HTTP or anything else But did you try it to discuss it? What what can you tell about about switching to a synchronous communication
27:46
If you mean just switching on it's from synchronous HTTP to as synchronous HTTP It will not change much. The protocol is the same. It's just what I meant at least in my part was that the Framework the the web server behaves differently instead of blocking one worker doing one request
28:05
You have the IO loop that would handle that request as soon as it's not hitting any IO boundaries And if you hit IO boundaries Which you always do in microservices because you're always waiting for either a database or cache or another microservice It can serve other requests as well. That is what they meant with the asynchronous communication. I
28:26
Do not see the effect of this on the error tracing Because the protocol is the same at CL HTTP on as a client you should in the best case not even notice It is a synchronous or it's synchronous Regarding the error tracing yeah, and again
28:42
I didn't mean error tracing on the client level like HTTP error codes like error 500 or whatever else as I meant error tracing inside the application because you Your request is passing through so many systems. It's hard to to match by time for instance
29:01
That's a typical first step that developers do when something fails they try to find logs on that point of time But it is going so fast that you perhaps will have a hard time figuring out Because at exactly the same time in many many services many many things could have happened at the same time stamp So it's harder to do this chaotic time based debugging and you need more centralized tool that collects all logs
29:24
And that somehow is marking every request with some headers Which which are then passed around from one microservice to another so that you can trace it from the entry point till the exit point What exactly happened where that is what I meant with the error handling sure that's correct, but I was asking more about
29:41
dealing with Unavailable services for example, what do you do when you issue and a request to a microservice that should be there? But it's not there It's maybe it's not there for like 10 seconds and the question is will your system be failing for this 10 seconds? Or will your system be resilient and be able to recover? Of course this sorry to interrupt
30:03
We will have to stop here now So I would suggest you to continue that during the the small rig we have the rest fix and then once more Please and then we have a great