Stop trying to glue your services together; import lymph
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Part Number | 73 | |
| Number of Parts | 173 | |
| Author | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/20162 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language | ||
| Production Place | Bilbao, Euskadi, Spain |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
| |
| Keywords |
EuroPython 201573 / 173
5
6
7
9
21
27
30
32
36
37
41
43
44
45
47
51
52
54
55
58
63
66
67
68
69
72
74
75
77
79
82
89
92
93
96
97
98
99
101
104
108
111
112
119
121
122
123
131
134
137
138
139
150
160
165
167
173
00:00
Web serviceGoodness of fitWeb serviceWeb pageMereologyWater vaporAuditory maskingOperator (mathematics)Web 2.0JSONXMLUMLLecture/ConferenceComputer animation
01:06
MappingSystem callPoint cloudWeb service
01:46
Software frameworkLevel (video gaming)Multiplication signCASE <Informatik>Sheaf (mathematics)Touch typingWeb serviceSoftware frameworkSoftware developerCuboidExecution unit1 (number)Link (knot theory)Cellular automatonLecture/Conference
03:07
Software frameworkWeb serviceSoftware frameworkDecision theoryVariety (linguistics)Web serviceCodeComputer animationLecture/Conference
03:36
Link (knot theory)Web serviceSoftware developerFormal languageLecture/Conference
04:24
Software frameworkWeb serviceConfiguration spaceScaling (geometry)Remote procedure callEvent horizonBoilerplate (text)CodeAsynchronous communicationData storage deviceWeb serviceComputer fileConfiguration spaceEvent horizonScaling (geometry)Key (cryptography)AuthorizationOperator (mathematics)Software frameworkMereologyGoodness of fitDefault (computer science)Instance (computer science)Windows RegistryClient (computing)Physical systemMultilaterationData structureVirtual machineIntegrated development environmentLimit (category theory)Expected valueBoilerplate (text)Video gameGene clusterSoftware testingPattern languageLevel (video gaming)
06:30
Demo (music)CodeSlide ruleMereologyCodeDemo (music)Line (geometry)Web serviceElement (mathematics)Computer animationLecture/Conference
07:36
Web serviceInterface (computing)Computer fileConfiguration spaceInstance (computer science)Event horizonImplementationRun time (program lifecycle phase)Link (knot theory)Length
08:46
CuboidWeb serviceSource code
09:22
Gastropod shellCuboidLine (geometry)Instance (computer science)Computer fileTunisReading (process)Web serviceInterface (computing)Electronic mailing listRight angleMultiplication signServer (computing)Dependent and independent variablesBitWordFunctional (mathematics)Link (knot theory)Greedy algorithmConfiguration spaceExpected valueTDMAComputer animationLecture/Conference
11:57
Web serviceEvent horizonInterface (computing)Configuration spaceMultiplication signType theoryMedical imagingPattern languageReading (process)Letterpress printingComputer animation
13:17
Uniform resource nameAverageExpected valueEvent horizonCASE <Informatik>Web serviceInstance (computer science)Type theoryWindows RegistryCombinational logicAnnihilator (ring theory)Letterpress printingReading (process)DistanceLimit (category theory)RandomizationSource code
15:40
World Wide Web ConsortiumRaw image formatArmWeb serviceComputer fileString (computer science)BitInterface (computing)Dependent and independent variablesQuery languageMappingConfiguration spaceVector spaceWeb 2.0Instance (computer science)Functional (mathematics)CASE <Informatik>Reading (process)DistanceCellular automatonSet (mathematics)Greedy algorithmFlash memoryComputer animationLecture/Conference
17:18
InfinityUniform resource nameDependent and independent variablesInstance (computer science)Combinational logicQuery languageWeb serviceOrder (biology)Web 2.0String (computer science)Reading (process)Letterpress printingSinc functionServer (computing)
18:35
Server (computing)Instance (computer science)BitWeb serviceLink (knot theory)Gastropod shellDirectory serviceComputer fileSheaf (mathematics)Network socketServer (computing)Software developerConfiguration spaceWeb 2.0Lecture/ConferenceProgram flowchart
19:59
Uniform resource nameVarianceComputer clusterDecision tree learningAkkumulator <Informatik>Point (geometry)Infinity12 (number)Interior (topology)EmailFunction (mathematics)Web serviceTracing (software)Term (mathematics)Web 2.0Functional (mathematics)Instance (computer science)SequenceStatement (computer science)Letterpress printingVirtual machineRight angleInformationQuery languageExpected valueString (computer science)Dependent and independent variablesNumberEvent horizonReading (process)Multiplication signCASE <Informatik>Block (periodic table)FreezingSource codeJSON
23:24
Server (computing)TelecommunicationSupercomputerCurve fittingMereologyDemo (music)Slide ruleWeb serviceAsynchronous communicationInstance (computer science)DemosceneStatement (computer science)Natural numberPattern languageSynchronizationServer (computing)TelecommunicationQueue (abstract data type)BitEvent horizonGastropod shellFunction (mathematics)Web 2.0Uniform boundedness principleWindows RegistryDependent and independent variablesEigenvalues and eigenvectorsCASE <Informatik>Multiplication signLetterpress printingDefault (computer science)Reading (process)State observerLengthProgram flowchart
25:59
CuboidData managementSoftware testingSoftware frameworkExtension (kinesiology)Error messageWeb serviceMetric systemGastropod shellMessage passingEvent horizonDigital electronicsRaw image formatCuboidGoodness of fitWeb serviceSystem callLibrary (computing)Configuration spaceData storage deviceDefault (computer science)Vector spaceSoftware frameworkPlug-in (computing)CodeWritingEvent horizonSoftware testingMetric systemProcess (computing)Open sourceTrailMultiplication signInstance (computer science)Gastropod shellWindows RegistryWeb 2.0Computer fileReading (process)Extension (kinesiology)Remote procedure callInterface (computing)Message passingPhysical systemLevel (video gaming)DatabaseLink (knot theory)Product (business)PlanningCurvatureRight angleCellular automatonArrow of timeData managementGreen's functionCovering spaceDataflowConnected spaceVelocityComputer animation
31:17
Table (information)CommutatorComa BerenicesSource codeComputer animation
31:52
Web serviceComputer virusLecture/ConferenceComputer animation
33:03
Local ringSoftware developerRevision controlServer (computing)Product (business)Instance (computer science)Formal languageLecture/Conference
34:01
Endliche ModelltheorieRevision controlComputer animationLecture/Conference
34:47
Message passingWeb serviceCuboidRevision controlDifferent (Kate Ryan album)Interface (computing)Presentation of a groupRight angleComputer animationLecture/Conference
35:38
SoftwareLevel (video gaming)Product (business)Open sourceVideoconferencingPoint (geometry)Library (computing)Computer animationLecture/Conference
36:31
Right angleEvent horizonCategory of beingDecision theoryPosition operatorInstance (computer science)Computer animationLecture/Conference
37:17
Uniform boundedness principleMereologyFront and back endsSocial classEvent horizonWindows RegistryDefault (computer science)Computer animationLecture/ConferenceJSON
38:14
World Wide Web ConsortiumInstance (computer science)Web serviceComputer fileSocial classTheoryDataflowWater vaporConfiguration spaceInterface (computing)Lecture/Conference
39:04
RobotLecture/ConferenceXMLComputer animation
Transcript: English(auto-generated)
00:03
Good afternoon, my name is Max, and today I'm going to talk about why we think that you should stop trying to glue your services together and import Lymph instead. Who are we to get that out of the way first? We're Deliver Hero, we're an online food ordering service holding, so what online
00:23
food ordering is all about is basically the concept is simple, I guess there's no one around who's really unfamiliar with it. You get hungry and you go online to one of our web pages, you search for restaurants that deliver to where you are, you compile your order, you pay online and then you wait for the food
00:42
to be delivered, so basically it's like e-commerce but with grumpy customers by definition, and also the fulfillment part is interesting because food needs to be delivered quickly, that's something you need to take into account. We're operating in 34 countries and we're headquartered in Berlin, this is our
01:03
mascot, the hero, therefore Deliver Hero. So just a quick show of hands, who of you attended the talk by Matt about Namiko before lunch? That's a fairly good amount because there are a few things that we're not going to talk about but which Matt nicely introduced, so we're not going to talk about what services
01:25
really are or as opposed to monoliths or why you should go with a service-oriented approach, why you should not do it, how this helps you, neither are we going to talk about cloud stuff or Docker or why you should call
01:42
it microservices and not microservices or how micro is micro, but what we're going to talk about is Lymph today, so Lymph is a framework for writing Python services and to start with I'd like to justify a little why we wrote another framework because usually developers say like hey there's
02:03
something out there already, in this case there wasn't, so once we have that out of the way we're going to get our hands dirty with a live demo, fingers crossed, that that works alright and that's basically the main section of this talk and afterwards we're briefly looking under the hood of
02:21
Lymph, what other features are there which we don't touch today, briefly touch on things like Namiko and so on, give you a little outlook and then hopefully there's time for Q&A in the end, so I have to be fair and say if you want to go over things in your own time then there's this entire introduction, you
02:42
can find an article at import minus Lymph dot link, everything is written down there, you can go over things in your own time, there's even more detail, you will find the exact same examples or services that we're talking about today, you can try things for yourself, there's a vagrant box set up that you can use which we'll use later or just to debrief yourself on
03:05
what we talked about today. So why do we write another framework? That's where we said we want services in Python and not worry, so let's assume
03:26
that our decision to go with services was right, we were running with a big jungle monolith, basically a lot of spaghetti code of the legacy variety, the one that no one really likes, so therefore the idea of going with services became increasingly attractive to us and we wanted to stick with
03:44
Python because usually you say hey if you do services the idea is that you don't have to worry which language you're going to run with but as we like to do Python every developer should be able to be as productive as they are and if we would not have stuck with Python well then I couldn't be
04:03
here today and talk about it so that's good and we didn't want to worry, we didn't want to worry that means if you want to run your services and do services there was nothing really that was helping you a lot back then so we wanted services in Python and not worry, the first two things are easily
04:21
ticked off, the third one wasn't and therefore we came up with Lymph so we had certain expectations though. Running and testing your services should be as easy as possible. You should not have to worry about glue, that means I as an author or operator of a service you should not have to worry about how to
04:40
register your services, how to run it, how to configure it, you should not have to worry about any of these glue code at all. Configuration should be simple and flexible, you should get a lot out of your configuration files without having to write a lot of code to parse them and deal with them. Possibly you should take the same service, run it on your local machine, your lab
05:03
environment, staging life, possibly another country even, simply by configuring it differently. Scaling naturally should be easy so if you need more resources then you just throw more instances into your cluster yet the client code should be totally unaware of this. We wanted to have the
05:24
opportunity or the possibility to speak RPC rather than via HTTP interfaces be able to do to emit events so that we can do asynchronous communication but we also wanted to easily integrate or expose HTTP APIs.
05:42
And last but not least if you want to introduce a new service then there should be as little boilerplate as possible yet a fair amount of scaffolding helps you to nicely structure your stuff. So what we came up with was Lymph. You can find Lymph.io and we think that it or the
06:00
ideas that should satisfy all of these requirements. So I repeat myself here I think it's a framework for Python services and by default Lymph depends on RabbitMQ as an event system and Zookeeper for service registry. So just one more quick show of hands who knows what RabbitMQ is? Good. Zookeeper. Fair enough. So Zookeeper is a
06:23
distributed key value store and that's how we do service registry but you'll find about that later. So here comes the scary part. People say that I should not animate my slides, I should not show code on slides and neither should you ever, ever, ever do a live demo because
06:42
it will go horribly wrong. So I'm going to show you code and it's animated and we're going to do a live demo so there's nothing that could possibly go wrong. So to begin with and to jump into the thick of it, we're going to write services and we're going to
07:01
increasingly introduce new services to see how they interact with each other. We're going to run them, we're going to play around with them to explore the tooling that Lymph brings with itself. And we start with the most sophisticated hello world example you could think of. That's a greeting service and it's funny because Matt used
07:21
basically the same example. That was not planned but it's funny though. So this greeting service, you give it a name and what it's supposed to do, it's supposed to return a greeting for that name. So to begin with, we need two files. Sorry, that's the first line already. We need two files. We need the implementation of our
07:40
service naturally in a PY file and we need a configuration file in a YML file. Sorry, that's YAML. So to begin with, for our service, we start with importing Lymph and this is basically where this talk lives up to its claim. We import Lymph and we want to define a service called greeting and we do so by
08:02
inheriting from Lymph interface. And like I said, we want to expose one method as its interface. It's called greet. It takes a name and we can expose it easily by decorating it with Lymph RPC. And then what we do inside of this method is we simply print the name which we received saying
08:21
hi to. We're emitting an event to let the world know that we're just greeted someone and lastly, we're returning this greeting. The configuration file is rather straightforward as well. We have to tell Lymph which interfaces to run and where they are located on the Python path because what Lymph does is that it imports it at runtime to bring up an instance
08:43
of this service. So let's get our hands dirty and we'll do this within a prepared vagrant box which is readily accessible for everyone at import minus Lymph.link. It provisions via Ansible and what this vagrant box does is
09:02
it has ZooKeeper and RebnetMQ running inside and to make things even more accessible, there are prepared Tmux sessions which you could easily fire up which then start services and panels and which are nicely labeled using the toilet command. And this is what we're going to do now. So let's go
09:21
there and run our services and play around with it. You don't see the topmost line of my shell which is confusing but you should see everything from now on. So this is where in the box now it greets us very friendly with import Lymph and there is this Tmux session prepared and
09:42
we're fired up with mux start greeting. And what you see now is two panels. One is running an instance of the greeting service on the right hand side. You can see that we are running Lymph instance and we need to point it to the configuration file so Lymph knows which interface to run.
10:01
On the left hand side we simply have a shell and this is where we'll explore the tooling that Lymph comes with. So to begin with let's say we don't know anything about Lymph at all so it should tell us about the commands that are available. Lymph list does so. That's a whole lot amount of text. Don't worry. You don't have to read them one by one.
10:20
We'll explore things bit by bit. So let's say we have no clue whether there are any services running at all or which there are. Lymph discover should tell us so we can discover services and indeed it tells us there's one instance of the greeting service running as expected. Let's continue to play Dump and say we don't know anything about this service.
10:43
I want to get to know something about its interface. So Lymph inspect. Greeting should inform us about this services interface. And this is more than expected. Actually, so the topmost function that you see. Sorry, the topmost method. That's the greeting method, the one we've just implemented ourselves and below that you see
11:02
four built-in methods which you get by inheriting from Lymph interface. So let's exercise this. The service we can do so. By issuing Lymph request greeting. Greet will supply the.
11:22
Request body that needs to be valid. Jason talking and typing at the same time is hard. And I'll greet you guys the Euro Python. So what we'll expect to happen now is the. Request should hit the instance of the greeting service. It's supposed to print something.
11:40
And we should receive the greeting in the response. So fingers crossed this works. And it did. On the right hand side, you see that it said saying hi to Euro Python and we received the response on our end as expected. That's very nice on to the next service. So yeah, this is what we just did.
12:03
So. The greeting service is also emitting events every time we're greeting someone. This is something we haven't seen yet, and it's emitting events. There's no service that consumes them, so let's write the service. That consumes these events and as creative as we are, we're going to call it listen service and once more we need
12:22
two files, one where we implement the service and one where we configure it. So we start with importing Lymph. And we define the service by subclassing Lymph interface. We're calling it listen. And like I said, every time an event of type greeted occurs,
12:42
we want to consume it and we want this method to be invoked. Exactly invoked, sorry. It's called on greeted and it receives the event that has been emitted and all it does is. That it takes the name from the event body and it prints that somebody greeted.
13:00
That name. The configuration is just as straightforward as before. We have to tell Lymph that it's supposed to run one interface. The listen interface and we have to point it to where this is located. On the Python path so that can be imported. So let's. Run them in combination to see how they interact.
13:24
Therefore, I'm firing up the. Tmux session for that. And we're doing a leap of faith here. We're not only running one instance of each service, but in this case we're running two instances of the greeting service. And one of the listen service.
13:41
But. Let's assure ourselves whether they have registered correctly with our service registry Lymph Discover should tell us so as you can see indeed there are two instances of the greeting service and one of the listen service. So. The listen service is supposed to consume certain events.
14:02
Let's assert whether that really is the case, so we can event sorry emit an event of type greeted with Lymph. And we have to provide a body. Also, once more needs to be Jason and the name is Euro Python. So what we're expecting to see now is once we emit it.
14:23
The listen instance is supposed to consume it and it needs to print something. In fact, it consumed the you can see this. Down here it consumed the event that was emitted. Before when I when we were requesting the greeting instance.
14:40
So we were expecting to see it print again now. Very nice it printed as expected. So let's request. A. Greeting and see whether they're correctly working together. So we're expecting to see now to once we send
15:07
this request. We expected to be handled by one of the greeting instances. It should print. Should return to us and the listen service should print once more. And in fact, the second greeting instance handled it.
15:22
Now, if we repeatedly issue this request. The request should be randomly distributed over the greeting instances. And fingers crossed, yes, this worked topmost one handled it and the other one very good. So this seems to work. As expected.
15:42
But it wouldn't be 2015 if we were not talking about web services. So let's expose our the functionality that we have within our service cluster, which is the. Bleeding edge greeting service and we want to expose it via an HTTP interface.
16:00
What we need to do there is we're going to write a web service and once more we start out by implementing it in Python and configuring it afterwards. So this case we import from lymph web serve interfaces. The web service interface and will also need some vector tooling to deal with URL mappings and and since we
16:20
also want to return about response. Business as usual, we define our web service by inheriting from web service interface. We want to expose one URL. Which is slash greet supposed to be handled by the Greek method receives the request. And we expect the name to be in the query string.
16:43
And what we do there is when we receive the request. We pick the name from the query string. We print that we're about to greet someone. We're invoking the Greek method of one of the greeting instances. This is RPC basically sorry. And then the end we're turning the greeting in the response.
17:04
And we also need to configure it and as our web service is supposed to listen to a port. We have to include this in the in the configuration. That's the only bit that differs from the two configuration files we've looked at before. So let's run everything together.
17:29
So what we see here is one instance of each service running web greeting and listen and since all habits die hard. Let's make sure that they have all registered correctly. Lymph discover should tell us and indeed there's one
17:42
instance of each service. So let's exercise the web service now and see whether they are actually working combination as they should. So we're listening to Port 4080. And the name should be in the query string. That's euro Python once more.
18:03
So once we issue this request, we expect to receive the greeting in the in the response and all services, sorry, all instances should print something. In order to validate that they were actually being spoken to. Let's issue this request.
18:21
And in fact, all services, all instances printed something and we received the greeting in the response says, I euro Python over here. But there's one thing that you might see already is the more services you run, the more complicated it becomes to well develop
18:40
with them locally. You need more shells to run the instances, and if you want to run several instances of one service, you need to run them in several shelves. That's becoming rather painful, and it has become rather painful here already because we want to run three services. We need three shells. That's a bit of a pain.
19:00
But there is Lymph to the rescue. It comes with its own development server. It's the Lymph node command. And what we need to do to get its leverage is within the directory where we want to run our development server, there needs to be a configuration file called dot Lymph YML. And in there, we configure the services that we want to run
19:22
and we configure how many instances of each. So this is highlighting the important sections. So if you want to configure instances, you basically tell Lymph how to bring up that instance and how often. So we run two web service instances, three greeting service instances,
19:43
and four instances of the listen service. And within the last section, since we have two instances of our web service running and they're listening to port, we have to configure this one as a shared socket. So let's bring up our node.
20:06
You won't only see the node on the top right panel, but below that, you also see tail. With the Lymph tail, you can subscribe to all the locks of any service. So in this case, we subscribe to the web greeting and listen service
20:24
and it will print all lock statements that it receives from the instances. So let's make sure that everything registered correctly, because there's no output in the Lymph node window, sorry, panel. Lymph discover should tell us that we have indeed two,
20:42
three and four instances of the services running respectively. And let's hit our service cluster as before. With localhost, that's greet,
21:01
name goes into the query string once more, EuroPython. And what we expect to happen is once we issue this request, should be handled by the instances, we should see three print statements now in sequence in the node panel and below that, plenty of lock output.
21:21
So fingers crossed this works. Very nice. It did. So you see three print statements up here, almost reads like a little haiku about to read EuroPython, saying hi to EuroPython. Somebody greeted EuroPython. The response looks good as well. We can see the greeting has been returned as expected.
21:42
And we see plenty of almost confusing lock output below here. But now consider that possibly your instances might be distributed over any number of machines. And if you want to debug something or follow the locks and get information from that, it's hard to tell
22:04
which lock statements belong to each other. How can I relate them to the to a request? Possibly they well belong to the same request, but the lock statements come from several different machines. Lymph lets you overcome this problem with the trace ID.
22:21
So whenever a request enters the cluster and it does not have a trace ID assigned yet, Lymph assigns a trace ID to the request. And this trace ID is being handed forward via every RPC request and event that is being emitted. And then whenever we log, it's being locked with it. So you can see here we hit the web service
22:43
and returned a header called X trace ID. And that's where it included the trace ID. And let me allow to use I terms search highlight. No, sorry, search and highlight function.
23:01
So you can see the trace ID is appearing in the locks properly. And within your own time, maybe you can assure yourself that it actually locks correctly with the trace ID and we can correlate all the lock statements via the trace ID.
23:24
That's very good. I managed to successfully go through the demo part and nothing broke. So let's just briefly reason about the communication patterns which we've just observed. And I think I went a little bit too far with animating stuff,
23:41
but maybe that's ... Hopefully it's entertaining. So we started with having two, three, and four instances of these servers running respectively. And we issued an HTTP request. It was handled by one of our web instances. It printed something. And then we wanted to invoke the greet method
24:04
via RPC of one of the greeting services. So what happens behind the scenes is that we consult our service registry, which is Zookeeper by default, and we ask it for all the instances of the greeting service. And then we pick one at random to send the requested.
24:21
And in this case, it was the lowest one. The request is being sent over. It printed something, emitted the event to our event system, which is RabbitMQ by default, returned the response, and then we had nice output on the shell. And on a possibly entirely different timeline,
24:43
one of the listen instances consumed the event by getting it from the queue and printed naturally. So we see there's RPC available, which follows the request-reply pattern, and it's synchronous communication,
25:00
and we are also emitting events. That's the pop-up pattern, and that's asynchronous communication. So you've seen ... I've jumped slides here already. Exactly one instance of all the listen services will consume the event. However, there are situations where you'd like to inform every instance of a service
25:22
that something occurred, and all we need to do is ... On the lower left, you can see that we decorate the service ... Sorry, the method which is supposed to consume the events as usual, but we say that it's broadcast, and what happens instead is that when we emit the event,
25:41
we're publishing to four queues in this case, and then it's being consumed four times, and that, as a repercussion, naturally we would have seen four print statements. So these are the communication patterns which are available with Lymph.
26:00
But what else does Lymph come shipped with? So what's in the box is that what I mentioned already, Lymph manages your configuration files. You can get a lot out of your configuration with very little code. It provides a testing framework so that you can unit-test your services following the fashion of,
26:21
if I invoke this RPC method, is an event being emitted as expected, or run some together and exercise them. Its dependencies are basically pluggable, so you could exchange ZooKeeper for something else. You could do service registry with something else. You could not use RabbitMQ,
26:40
but something else like Kafka, for instance. There are service hooks, so when you ... service starts, you want to possibly set the stage for it, like you want to provision a database connection, and then once you shut it down, it's supposed to be cleaned up. There are service hooks for this. Lymph allows you to do futures, so usually classic RPC is blocking,
27:02
but possibly you're not interested in the reply from the service, or you're interested in later, so you can do this with a future. You can defer the call. Lymph collects a good amount of metrics. When it runs your service already, and then it exposes them, but you can also collect custom metrics,
27:21
so, for instance, if your service is talking to a third-party API, and every now and then this request times out, and you want to keep track of how often this happens, this is what you can easily do. You can also write your own plugins for Lymph. There are even more hooks that you can plug into and get custom code executed whenever something interesting happens.
27:40
Out of the box, there is a new relic and a Sentry plugin for Lymph. It's easily, sorry, the CLI interface is easily extendable. A colleague of ours wrote Lymph-top, which is basically like top, but for Lymph services, and you can handle, you can receive remote errors, you can get shells on remote services, and so on.
28:01
There's a whole lot more. So how Lymph works under the hood is that anything that is supposed to be sent over the wire is being serialized with message pack. Message pack is, well, that's their claim. Like JSON, but a little smaller and a little faster. RPC is being dealt with with zero MQ.
28:21
Like I said, service registry by default happens with via ZooKeeper, and the events are being, the event system is RabbitMQ. Every Lymph instance or every service instance is a Python process, and it handles requests and events within Greenlets, and this is what we do with GEvent. In everything that is web or HTTP,
28:43
we use vector tooling for that. So since some of you attended the Namiko talk already, as for things that are out there that are similar to what Lymph is, there's one thing that needs to be mentioned, of course, that's Namiko.
29:00
Namiko does a lot of things very, very similar, almost startlingly similar to how Lymph does things. Naturally, it does certain things differently, but it's very nice, and if you haven't attended the talk, I suggest that you have a look at Namiko. Also, there are other things out there which don't solve the big picture,
29:22
like Namiko or Lymph tried to solve it, but they supply solutions for niche problems, like zero RPC or other stuff. You would still have to provide a good amount of glue code yourself, and this is what Namiko and Lymph both try to avoid. So what we have in mind for the future of Lymph
29:42
is that we want to have this little ecosystem of libraries for writing special purpose services easily. So there's, we have Lymph storage in mind, Lymph monitor, which then collects all the metrics from other services and stores them wherever, or does with them whatever you want.
30:03
Lymph flow, which is basically the idea is to write business processing engines which deal with your business process and manage your entities, and whatever there is that is to come. So to sum things up, if you can remember this one thing
30:20
that Lymph is a framework for writing services in Python, I think I have been successful today. You can find out more at Lymph.io, and naturally, it's open source. Your contribution is very welcome. You can find the docs at Read the Docs. Everything is linked there at Lymph.io. Like I said, this introduction, it's all written down in more detail
30:41
following the same narrative as today at import minus Lymph.link. That's where you find all the examples. That's where the vagrant box is. That's basically where you can go and play around with Lymph. Last but not least, if you're a Spanish speaker and you'd like to hear this talk again
31:00
later this week in Spanish, then my colleague Castillo will give the same talk in Spanish. In Spanish, I had to learn this by heart. It's deja de pegarte con tus servitios, import Lymph. I don't know whether I made a good effort, but I see you're nodding. Very good. So that worked. And here comes the shameless plug that goes with every talk. So we're hiring.
31:22
If you're interested in working with us in Berlin and you want to work with Lymph, if you find that interesting, you possibly have seen this flyer in your attendee bag already, feel free to reach out at deliveryhero.com or see us at our table in the hall. We've brought goodies and gummy bears, most importantly.
31:44
Thank you. And thanks to the organizers, of course.
32:02
Questions? Or maybe you can just shout and then I'll repeat the questions to everyone.
32:20
Can you? Your first question,
33:01
well, theoretically it's possible to talk to Lymph services, but you would have to do it yourself. So you would have to basically re-implement exactly within any other language. And your last question, I didn't get it.
33:21
So your question was whether you can use Lymph node. The idea behind Lymph node is to be used in development. The idea is not to replace your production, how you should run it in production. The idea is to help you run stuff locally.
33:48
Yes, you can spawn anything with a Lymph node. Basically, you just supply a command. I don't know, for instance, to run your Redis server. You could include it there as well, and then it runs it for you. Hi. Just to correct, what versions of Python do you support?
34:07
And if you support Python 3, why didn't you use something like IO, HTTP, or modern things in place of Verxec?
34:21
So to answer your question, yes, both Python 2 and 3 are supported. I don't know with which Python 3 version it starts. I know that we have had a little trouble with this in the past, but we're supposed to support Python 3 as well. And your other question, IO, HTTP, I don't know about that, but to preclude your question already.
34:48
So about message versioning, do you support that or is that something somebody would have to do on top of your library? Message versioning? Yeah, for example, if you run two different versions
35:01
of a service in a cluster. You want to ... You're running two different versions of a service and there's nothing that deals with it out of the box right now. You would have to deal with it yourself, I guess.
35:21
I mean, it depends on whether the interface is backwards compatible, but you can't ... It would be another service then, if you want to run two different versions. Right now you would have to run two different services or just expand the interface. Yes. Thank you for a great presentation and it worked out.
35:41
It's amazing to see such a polished software, which promise a lot. But what's on your backlog? What's the issues you're working on? What's your road map for developing it further?
36:02
So the idea is to further let these special purpose libraries, I like to call them that, to let them further mature and then release them as open source at some point. But right now the idea is to simply make Lymph more stable. We're going to run it in production anyway,
36:20
so this is something that will naturally grow and mature within the future. Hi. I don't know if I got it right, but I understood that you're using 0MQ for handling the RPC calls and RabbitMQ for handling the events.
36:41
Yes. Have you considered perhaps using RabbitMQ for both and then getting rid of one extra dependency or did you experiment but didn't work or ... For instance, Namiko uses RPC, goes via 0MQ, sorry, via RabbitMQ there as well. For us it was a design decision not to do RPC
37:01
over something persistent as RabbitMQ. Actually, my question was going the other way around.
37:22
If it is also possible to replace RabbitMQ with 0MQ for the pub sub? Yeah, definitely. So is it, but you said it is pluggable, but is it already implemented or is it something else? I was expecting your question. Therefore, I prepared something. So this is the part of the .lib YML
37:43
which I actually didn't want to show because it's a little confusing. However, what you can see here is this is actually where things are pluggable, so you could provide another class which does registry or handles events. This is what it looks like by default,
38:02
but you could provide your own backends for either, and I see everyone's eyes narrow, so yes, it is confusing. And just one more quick question. I've seen in the YML files that were at least maybe in the simple examples just names of services and class path,
38:23
so would it be possible already just to have a decorator that defines the name and then just launch it with like getting rid of the YML file and just launch the instance just providing the path to the class? Well, in theory, yes, that's possible,
38:44
but the idea behind this is that you could group several interfaces together and run them as one service, and I think this is just more flexible because then you don't start to mix what's in your configuration, what's not. This way you have everything in your configuration
39:02
and that's where it is. Thank you. Cheers. Nice, cool, thanks guys.
Recommendations
Series of 3 media