Building Service interfaces with OpenAPI / Swagger
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Part Number | 130 | |
| Number of Parts | 169 | |
| Author | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/21226 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
EuroPython 2016130 / 169
1
5
6
7
10
11
12
13
18
20
24
26
29
30
31
32
33
36
39
41
44
48
51
52
53
59
60
62
68
69
71
79
82
83
84
85
90
91
98
99
101
102
106
110
113
114
115
118
122
123
124
125
132
133
135
136
137
140
143
144
145
147
148
149
151
153
154
155
156
158
162
163
166
167
169
00:00
Web serviceDesign by contractBitProduct (business)Library (computing)Conformal mapSoftware frameworkRevision controlValidity (statistics)Term (mathematics)Sinc functionStandard deviationSet (mathematics)Latent heatType theoryDependent and independent variablesAdditionIntegerWeb applicationCodeMereologyFocus (optics)Operator (mathematics)Object (grammar)Keyboard shortcutLocal ringInformationWordWebsiteNumbering schemeSystem callFormal languageElectronic mailing listService-oriented architectureIntrusion detection systemClient (computing)Nichtlineares GleichungssystemInternet service providerVirtual machineSoftware testingCASE <Informatik>Point (geometry)Parameter (computer programming)Query languageArray data structureOpen setEndliche ModelltheorieWater vaporDifferent (Kate Ryan album)Web 2.0Video gameWell-formed formulaLecture/ConferenceMeeting/Interview
05:55
Service-oriented architectureAsynchronous communicationLastteilungSystem callMobile appCodeDependent and independent variablesArray data structureElectronic mailing listSheaf (mathematics)Object (grammar)Heegaard splittingComputer fileResultantWeb serviceOnline helpNumberBitIntegrated development environmentMultiplication signOpen sourceCategory of beingIntegerField (computer science)String (computer science)Type theoryView (database)MereologyDefault (computer science)CASE <Informatik>Error messageReal numberMathematicsOperator (mathematics)Revision controlClient (computing)SoftwareInformationBookmark (World Wide Web)Text editorFreewareAdditionInternet forumPoisson-KlammerCartesian coordinate systemComputer programmingWhiteboardParameter (computer programming)2 (number)Form (programming)Library (computing)SynchronizationDifferent (Kate Ryan album)Product (business)Focus (optics)Event horizonLoop (music)Shift operatorCodierung <Programmierung>Intrusion detection systemQuicksortUniform resource locatorFront and back endsStructural loadStandard deviationSinc functionFlow separationComputer configuration
14:42
Point (geometry)Revision controlWeb serviceSoftwareInstance (computer science)CASE <Informatik>System callProduct (business)Block (periodic table)Code2 (number)Computer programmingField (computer science)Multiplication signMultiplicationType theoryDefault (computer science)Exception handlingSynchronizationResultantClient (computing)Service-oriented architectureLastteilungMereologyDependent and independent variablesIntegerError messageValidity (statistics)Graph (mathematics)BitScaling (geometry)Asynchronous Transfer Mode10 (number)Total S.A.Web browserProgramming paradigmLibrary (computing)Representation (politics)Endliche ModelltheorieObject (grammar)Office suiteGoodness of fitPhysical systemSphereOperator (mathematics)Open sourceStrategy gameInferenceNumberFrequencyComputer configurationDesign by contractQuicksortComputing platformStress (mechanics)Software developerMathematicsSoftware maintenancePerspective (visual)Mechanism designCache (computing)1 (number)Process (computing)Arithmetic meanParameter (computer programming)Message passingStructural loadPresentation of a groupComputer clusterSet (mathematics)Configuration spaceImplementationAdditionHand fanComputer animationDiagram
23:29
Computer fileState of matterPoint (geometry)Instance (computer science)Product (business)Revision controlClient (computing)CodePerspective (visual)Field (computer science)ImplementationMultiplication signReal numberCASE <Informatik>Data miningGraph (mathematics)Different (Kate Ryan album)BitPattern languageMathematicsRight angleSystem callParameter (computer programming)Dependent and independent variablesCombinational logicComputer configurationMultiplicationWeb serviceObject (grammar)Cache (computing)Arrow of timeHeegaard splitting2 (number)Error messageSign (mathematics)Lecture/ConferenceComputer animation
27:38
Boolean algebraAdditionRevision controlScripting languageGroup actionMathematicsCodeMoving averageExecution unitSoftware testingINTEGRALProduct (business)Web serviceReduction of orderOperator (mathematics)Software maintenanceState of matterMultiplication signSoftware developerSoftwareControl flowForestClient (computing)Flow separationData structureLevel (video gaming)FrequencyView (database)Point (geometry)Electronic mailing listForm (programming)Computer animation
30:41
Arithmetic meanService-oriented architectureCodeCartesian coordinate systemMultiplicationMathematicsProduct (business)Web serviceSoftware developerDifferent (Kate Ryan album)Scaling (geometry)Online helpOrder (biology)Endliche ModelltheorieImage resolutionSheaf (mathematics)PixelNumberLecture/ConferenceMeeting/Interview
32:32
Android (robot)Web servicePoint (geometry)Client (computing)Dependent and independent variablesComputer fileMultiplication signFlow separationData storage deviceFunctional (mathematics)Type theoryServer (computing)Open sourceSoftware testingRight angleINTEGRALLibrary (computing)outputInternet service providerElectronic mailing listForm (programming)CodeSoftwareFront and back endsAbsolute valueDigital electronicsSystem callSubsetLatent heatYouTubeCore dumpBlogRow (database)Real numberImplementationRoutingElectric generatorFile systemUnit testingDescriptive statisticsPointer (computer programming)AdditionOnline helpScripting languageCASE <Informatik>AuthorizationService-oriented architectureMereologyMobile appJava appletExterior algebraIntegrated development environmentPhysical systemRule of inferenceAreaPresentation of a groupInheritance (object-oriented programming)Goodness of fitRepresentational state transferFigurate numberObject (grammar)Numbering schemeQuicksortParameter (computer programming)Field (computer science)Moment <Mathematik>Process (computing)InformationState of matterComputer virusSquare numberEndliche ModelltheorieWordInsertion lossLecture/ConferenceMeeting/Interview
Transcript: English(auto-generated)
00:00
Welcome, everyone. We'll have a talk by Stefan Jensch from Yelp, who will talk about how they use OpenAPI to bind services, microservices, together. And then we'll have five minutes for questions and answers at the end. Please welcome Mike, Stefan.
00:24
Hello, everyone. I'm a technical lead at Yelp. Short bit about Yelp, we're in the business of connecting people with great local businesses. Emphasis on the word great. We have over 100 million business reviews written by our users on the website
00:42
that should help you make sure you find the business you are looking for. We are live in 32 countries and have over 90 million monthly active users. So, what is this talk going to be about? The talk is about OpenAPI or Swagger. Swagger is a set of tools
01:00
and a specification for dealing with service, a service-oriented infrastructure, making calls to a service. And I'm going to give you a short introduction of some of the available libraries. And then I wanna focus on the things tutorials typically don't talk about. So, issues you will face in production,
01:22
surprising behavior, especially when migrating from an older version of Swagger, Swagger 1.2, to the current one, which is 2.0. And the things that can go wrong. Some, I call them war stories. So, a bit more about Swagger. I'm going to use the term Swagger since that's what I'm used to. OpenAPI is basically a rebranding that was done recently.
01:45
It specifies the contract of your service. That is the main thing it does, which helps you a lot because it's human and machine-friendly. It helps you with testing. It helps you with describing your service and defining the contract, then making sure that contract is enforced.
02:02
The tools actually help you with that. It does automatic documentation for you. So, if you do several services, we have like hundreds in production at Yelp. The question is also like which services to exist? What API do they offer? What do they do? Swagger helps you with that. As I said, it ensures conformance to the spec.
02:22
It can actually like make sure your endpoints behave the way you define them. And it has a community driven set of tools, which are language agnostic. So, I'm obviously going to focus on the Python part of it, but there's like tools for many other languages. And there are tools written for Swagger,
02:43
the specification which you can use and take advantage of, even though they are not even written in Python. And as I said, they had solved several problems when faced with building your service-oriented architecture. The main library I'm going to talk about is Bravado.
03:02
It's the client library you can use to make calls to services that have a Swagger spec. In addition to helping with that and enabling you to write way less code and doing these calls in a nice, Pythonic way since you get like automatically generated object stubs. It also helps you enforce or verify the contract of your services
03:22
by making request and response validation. There's another library, SwaggerPy, which is for the old version of Swagger. We're not going to focus on that. I just want to mention that it's available. Bravado was born out of SwaggerPy, but it actually has quite a few surprising differences in behavior.
03:42
So, for those of you who are migrating from SwaggerPy to Bravado, I've got some tips of issues you might encounter and you might want to avoid when migrating. Then, since at Yelp we use Pyramid as part of our standard service stack, I'm going to talk a bit about Pyramid Swagger. Of course, there are like libraries
04:01
for other web app frameworks there like Connection for Flask or Django REST Swagger for Django. That one, that's why it's only in light gray, only supports Swagger 1.2 again. So, if you want to do Swagger 2.0 with Django, I fear you're on your own.
04:22
The good thing is that most of these libraries, at least Bravado, SwaggerPy, Pyramid Swagger, and all of those associated Python libraries you need for that, they are all Python 3 ready. So, Python 2, Python 3 both work. Now, let's take a quick look at an example Swagger spec.
04:41
This is done in YAML. You can also write them in JSON. This is the heart of everything. This is how you define your service contract. So, at first you say, yes, this is a Swagger 2 spec. Some information, this is used for documentation, not for anything else. Some information about the service itself,
05:01
where it lives, what scheme you use to talk to, like typically HTTP or HTTPS. It consumes and produces JSON. Can also be something else, of course. Now, let's look at one of the endpoints we define. In this case, we define a user's endpoint. It's an HTTP GET operation.
05:23
It returns, unsurprisingly, a list of users by user IDs. So, we define a parameter, user IDs, that has to be provided in the query, and which, this is why it's required, it's of type array, and it's an array of integers.
05:44
So, all of this is defined here. You can see the operation ID further up. We're going to use that in a bit. This is what identifies your endpoint when using Bravado. Now, what does this endpoint return? In our standard success case, HTTP status code 200,
06:02
we return a list of users, which is an array of user objects. We're referencing here the users. You can see this is a reference to another section in the file, but you can also reference external files. So, this is how you would split up your swagger spec if it's getting too big.
06:24
Just a default response I'm defining here, which applies in all other cases. So, in the non 200 response status code, it references an error definition object here, which I left out due to not make this too long.
06:41
So, let's get to the definition section real quick. We're defining a user object here. It has two required fields, ID and username. This is the user object we've referenced just a bit above. The properties are defined here, the required fields. It's an ID, integer, username of type string,
07:00
and we have an optional field business ID, which is also of type integer. It can be present, but it doesn't have to be present. So, specs like these, you can write them in your favorite editor, development environment. There's also a tool available. This is one of the tools I've mentioned, the Swagger editor.
07:21
It's a hosted solution, which you can just use to write your spec. It has nice syntax highlighting. It will immediately tell you if the spec is not correct, if it's not valid, and will point out what errors there are. It can also show you immediately a nice HTML documentation of your spec,
07:41
and all of this is, of course, free and open source. Now, we can also use a tool called Swagger UI to generate a documentation for our spec. This is what it looks like. This is for a service I co-developed at Yelp, the business owner app backend. You can see here some of the things,
08:01
how they are turned into documentation. Account and business, excuse me. Account and business are tags we gave to several endpoints, so these endpoints are then grouped by these tags. We can see a list of endpoints.
08:21
You can see the URL, the HTTP operation. In the bracket, you can see the operation ID, as I mentioned before. This is what we're going to use when calling the endpoint with Pravado. You can also see that there's an endpoint that is written in strike through. This is a deprecated endpoint, so typically one of the problems you have in service,
08:42
in a service-oriented architecture when you do services, microservices, you don't exactly know who is using your endpoint. You can't really remove them without being very careful, but you can deprecate them, and when you have to do backwards and compatible changes to your endpoint, you want to add or modify behavior.
09:02
You create a new version. Unfortunately, Swagger doesn't have built-in support for versions, so what we do is we just append the version at the end, which works just as well. So let's see how to use that. This is a small example of using Pravado
09:21
to make a call to the service as we have defined it in the Swagger spec. First, we create the client, the Pravado Swagger client. We just tell it where our service lives. Typically, this will be the host and port to your load balancer, I guess. At Yelp, we use SmartStack to do service discovery.
09:42
You might use some other form. You just point it to wherever your service lives, to its Swagger YAML or Swagger JSON. In this case, we're using the FIDO client. I haven't mentioned this yet. This is another library you can use. Instead of using the default synchronous client
10:01
to make requests, which is based on the great Python request library, you can use FIDO. Asynchronous communication is a huge topic here at EuroPython. I think it also was last year. My colleague is actually giving a talk on that tomorrow.
10:20
And if you have visited one of those talks, you know that it's quite a shift in how you program. You basically have to rewrite all of your application. By doing this, it won't enable full asynchronous communication, but it will do all of your service calls asynchronously, and it will hide all of the complexity for you,
10:42
so no event loop or tornado or twisted or whatever. It will just work, and you won't even notice. The only thing you need to do is you need to work with futures. In this case, we see we're calling the list users endpoint that we defined in the spec. We provide the user IDs.
11:01
That's the parameter, so we don't have to build any sort of URLs or do URL encoding. It's all handled for us, but what this call does, it doesn't return the result immediately. It returns a future that we store in a variable, and just as an example, we're doing a second service call here. We haven't defined that in the spec.
11:22
Just imagine we did. So we have two futures here, and then we call result on these future objects, which will block until the response is there. So why are we doing this? What does this give us? Let's take a look. There's a tool called Zipkin,
11:41
which you might have heard of that will help you see what service calls you do and how they behave, how long they are. I hope you can see this in the back. The names and numbers are not important. Our service is a pretty heavy service since it's the service that returns data to our mobile apps, so it does a lot of data collection.
12:02
The endpoints take quite a bit of time until they are done, and this is an example of doing many service calls with Bravado one after another with a synchronous client. So we see the first request is actually very fast, eight milliseconds. We have to wait until it returns. Then we do the second service call
12:21
and so on and so forth, and when all of these are done, we return the response. Now, you can gain huge performance improvements by doing this, by just collecting all of the futures as far as you can and then calling result on them. Obviously, this is not always possible since some of the network or some of the service calls
12:44
you do depend on the result of previous service calls, so this is what we see here. We still need to do the first two calls serially, like one after another, but then we have all the information we need to talk to all of these other service endpoints and services, fetch all of the data at once,
13:02
and the time it takes is the time for the longest of those calls, and now you can obviously see the difference. So this is the power of FIDO and Bravado. Very few changes to your application and you gain these benefits.
13:21
But let's focus now on the main part of the talk, my so-called war stories. Because all of this is nice, that's why I've done it pretty quickly. Like, there are tutorials online. They will tell you this, and that's typically where they stop. But when you use this in production, we'll encounter several issues
13:40
that you might not have thought of previously, so let's talk about them. The first of which is not really specific to Bravado, but you have to think of it any time you talk over the network. You have to deal with network issues. And one thing we have encountered are besides connection issues,
14:01
which you have to deal with as well, are timeout issues. So, you know, the load changes over day, and not every request will be as fast as you want it to be. You have hopefully set timeouts. Please always set timeouts. Bravado's default timeout is no timeout.
14:21
So please set it, because otherwise something in your stack will timeout after some ungodly amount of seconds. Please don't forget that. But let's say it does timeout. So what we've done here is we created a future just like we did before, and then we wrapped a call to result
14:42
within a retry decorator that will just call result multiple times in case the HTTP timeout error exception happens. This works, unfortunately, or fortunately, it works if you use the default synchronous HTTP client.
15:01
Why? Because in the synchronous case, nothing happens until you call result. That's the whole point of it. So when you call result, it initiates the network request to your service, waits for the response, returns to result. In the synchronous case, once we create the future, the network request is initiated.
15:21
When you call result, it just blocks if the response hasn't already arrived in the background. This also means that when you retry it like this, you will just add additional, in this case, two seconds, two seconds, two seconds, without ever redoing the network call and getting a result you want.
15:40
So with this programming model, what you need to do is you need to wrap both the future creation and the call to result in your retry decorator. This will work. Another issue. This unfortunately has, I might say, bitten or people have encountered it a lot,
16:01
both when upgrading from Swagger 1.2, so the Swagger Pi library, as well as newcomers to Bravado that just have a service already in production and they create a spec for it and they want to use Bravado. So what we see here is a JSON representation of the user object we defined in the spec. And as we can see,
16:21
there's a business ID field with the value of null. Unfortunately, the Swagger tools don't agree that this is correct. What they say is that field is optional. So either it's present and the value has the type integer or it's not there. But in this case, what you will get
16:41
is a validation error because you return the type none, the Python type none, the JSON type null, instead of type integer. In Swagger Pi land, so in the old Swagger version, what you could do is pass a parameter to your result call, telling it to ignore this case.
17:00
Unfortunately, this doesn't work anymore. There's two solutions. The one I prefer is to change your services to just filter out these fields. Just don't send them. It will make the response shorter. Everything is great. Sometimes you can't do that. In that case, pass the validate responses parameter
17:20
as a config option to your client. Now what I would suggest you do is that you create a separate client that you do endpoint calls that need this option, but please don't set it as the default like for every call you do because I consider the contract validation, like making sure everything behaves according to spec,
17:41
one of the really good features of this Swagger ecosystem. Now, another surprising issue. Creating the client, it's a dynamic operation, and it can take time. It will probably take time if you have a non-trivial Swagger spec for your service.
18:00
So you can see here, this is an example, a Yelp internal example, where I hit one of our bigger services, internal services, and I just measure the time it takes to instantiate the client. The thing is it will do like one or potentially multiple network requests in the background to fetch the spec. If it references external files,
18:20
it will fetch those as well, and all of this takes time. When you migrate from Swagger Pi, you won't even think of that because Swagger Pi had a built-in mechanism of caching that. So if I did, like if I handled multiple requests and it would just hit an internal cache
18:41
and recreate the client from the cache without you noticing even. So what we did when we upgraded an endpoint to Swagger 2.0, is we forgot about that, and then you get graphs like these. Unfortunately, I don't have the graph of where timings went up. This is the graph of where we saw the issue,
19:02
fixed it by caching the client ourselves, and then deploying that to production. You can see like our P50s, but the P50s you can probably not see. They went down a little bit, but especially the P95s and P99s, so like the slowest part of the requests, if you're not familiar with percentiles,
19:22
they became way faster by just caching that, and this is something you need to do yourself. There's no built-in solution for that in Bravado. Now let's talk about some of the issues of deployment at scale. I typically am not like a huge fan of talking about at scale,
19:40
especially like at what scale. I'm talking here about like not just running your one or two service instances, and you have like three service in total, but you maybe have like tens or hundreds of instances of that service, so when you deploy them to production, you have like some sort of deployment strategy
20:01
that ensure that you're not like dropping requests, returning errors like that. Basically, nobody notices you're deploying a new version of that service to production. There are ways to do that, like our platform as a service, open source offering that we have, the default strategy is crossover, so it will typically like just spin up
20:22
X number of service instances that you have defined you want in production for the new version, so like for a period of time, you have basically both of them running in production, and then after it verifies that everything works well, it will just shut down the old ones one by one until only the new version is in production.
20:42
Another deployment strategy that we use for our like old monolith at Yelp is just have instances running and then take them out of the load balancer, switch the code version, bring them up again, put them into the load balancer, so one by one or like slowly you flip them
21:01
until they are all running the new version of the code. So how does this affect you as a service developer? Like you're not an ops person, you write code, you deploy it, does it affect you? And it does actually. Let's get back to versioning a bit. I said versioning, you have to do that to do backwards and compatible changes,
21:22
and as a developer, you know multiple versions, that means more maintenance, you want to minimize the cost of maintenance, so you try to cut corners wherever possible, you try not to create too many multiple versions of the same endpoint. Now what if you want to add data
21:41
to a response of an endpoint? Typically this is fine. Bravado does enforce the contract of your service, but it just checks that the data that you send conforms to the spec. If you want to send additional data, it doesn't care about that, only if the type of the data is not correct
22:02
or if some data is missing that according to the spec should be sent. So you might be thinking, hey, I'm just adding data, that's not a problem. The point here is non-optional. So you're changing the spec to say, hey, in this response, it is required that you send this field. Of course we've done this.
22:23
The fact is this doesn't work really because of how you deploy your service. So let's say we have the service, we added the field to the spec and we did the implementation. We're deploying this to production. We have both versions running in production during deployment.
22:40
So a client wants to call our service. It fetches the spec first, gets hits by chance. A new version of our service gets the updated spec that says, well, this field has to be present in the response. All good. Now it does the service call, fetches the data.
23:01
As luck will have it, it will hit the old instance of our code running in production, which knows nothing about that new field and it doesn't return it. So instead of a nice response, all you get is an exception. And if you're unlucky, the way, depending on how you programmed it, this might result in a user-pacing error.
23:20
So how do you do this without creating an additional version? Well, like with many things, you have to split it up in multiple compatible steps. So the first thing is you add it as optional to the spec. So you add the field to the spec, but you don't say that it's required. And you add the implementation.
23:41
So you're already returning the field. You ship this. So you do a code push. You deploy this to production. And then in the second step, what you do is you just do a small change to the spec and mark it as required and all is well. If you don't do this, this is an example, like a screenshot out of a post-mortem that was sent at Yelp.
24:01
This is the error spike we saw when not splitting this up in these two steps when deploying to production. Now, another thing. Of course, it makes sense that, as I said, you cannot just remove data from the spec, right? From the response status, obviously not a backwards compatible change.
24:25
So you think, oh, but we can use the pattern we just used to do this other change to do this in a backwards compatible way, right? So what we do first is remove the field from the spec. So the spec knows nothing about this field anymore, but we don't change the implementation.
24:42
We still return the field. And only in the second step, we do a second code push and we ship the implementation and all is fine, right? No, it isn't. Don't do that. This will also cause or might cause issues in production.
25:00
And I got the post-mortem to prove it because, as I said, if you want to run this efficiently in production, you need to cache your client, which means it doesn't, on every request, refetch the Swagger spec and instantiate the Swagger client and everything. So you can't be sure that that client
25:21
in between the two code pushes will fetch an updated version of the spec. You don't know. And it doesn't, as you can see here. So what you need to do to do this in a backwards compatible way would be to do the two steps, like do the first step, like I said, change the spec, know of all of the clients,
25:41
all of the callers of this service, restart them so the cache is clear, it's invalidated, and then you can do the second code push and remove the field for real. A similar issue with adding a reference to a new spec file with the dollar sign ref. This should be probably a little bit obvious by now,
26:03
so I'm just going through it a little bit quicker. If you add a reference to a new file and you do it all in one step, so you add the reference and the file itself, you may hit the same issue that you get an updated spec that references to new file, but then when trying to fetch it,
26:21
you hit an instance that still runs the old version of your code that doesn't have this file. Same solution. Add the file first without modifying the spec, so you are sure every instance has the file, and then in the second step, you add the reference to it all as well. In this case, what you also can do
26:41
is you can let Pyramid Swagger combine the spec for you if you're using Pyramid and Pyramid Swagger. There's an option that was added recently that reads the spec, combines it for you, and creates one big spec, and it will then serve this one big spec to your callers, eliminating this issue.
27:02
The one thing this doesn't support is recursive references, so if you have your user object is self-referencing, it's referencing itself, this won't work. You need to have multiple spec files in that case. A colleague of mine is actually working on solving that, but right now, this is the state of things.
27:23
There's a pull request out. You can check it out if you're interested. Now, one of the last things I want to mention because I find it so surprising is changing the tag of an endpoint. I kind of glossed over this previously, but when you look at how we use the client,
27:42
we say client.user.listusers, and the .user, that's the tag we gave. It's actually like the first tag in the spec we gave to this endpoint, and list users is the operation ID. Now, operation ID, it has ID and the name. Obviously, you know you don't change an ID
28:00
or things will break, but that a tag change will break clients, that is unfortunate, I might say. Here's the post-mortem. All of this has happened in production at Yelp. The thing I find unfortunate here is that tags, typically you use them for documentation,
28:22
and they are used for documentation. So at Yelp, this was done to improve documentation, like you build more and more endpoints, you want probably to use finer grain tags to make them better, so that you can find them better in your documentation, and the developer just changed the tag,
28:41
and all of a sudden, all clients broke. So just be aware of this. In conclusion, I think the main conclusion here is when in doubt, version it. Several of the examples could have simply been prevented if additional versions of endpoints were created. We as developers, we like to be efficient,
29:02
we like to reduce maintenance costs as much as possible. Just be aware that there's also a cost when you don't version a potential cost, so be very careful. And I can tell you from experience, there's not a lot of automatic tools available, like scripts that will check for these changes.
29:22
So when you try to do this anyway, you need to rely on things like code reviewing and testing to catch these. And the thing is, most of these issues are very hard to catch when testing, like when writing unit tests or integration tests, or when testing manually on stage.
29:42
So when in doubt, version. Second one, this is more of a general service thing. You have to deal with the network. There's been a really great talk at PyCon this year that talks about some of the things you might want or probably need to do when talking over the network.
30:02
There's also a talk by my colleague that was done yesterday. I'm going to reference it in a bit. Also, this is something I don't see mentioned that often. We always talk about like, is this roll forward or roll backward compatible? Can we go from state A to B and we'll work everything in state B?
30:20
And what if we have to go back to state A? Will this break anything? But there's also like a transition period, like none of this is instantaneous. So think about during that time, because you will have both code versions running in production for a non-trivial amount of time. Then, just for those migrating,
30:41
be mindful of the differences between SwearPy and Pravado. I just mentioned this because as you've heard, but you haven't seen, the difference in API is actually not that big. So migrating your code is actually done really quickly. It's the behavioral changes that will bite you potentially.
31:01
So pay attention to those. And I just wanted to slip this in because there's still kind of a service hype or even microservice hype. We haven't had great success with microservices. We're more of a services company. I just wanted to mention something
31:21
that Martin Fowler and a lot of people that hopefully know what they're talking about will also tell you, services is not something you do because they're great. Services is something you do because you have to to solve other problems, like to solve problems at scale.
31:40
And scale doesn't mean traffic. If you're not able to reliably run your monolith or your one application in production, you're not going to reliably run dozens of services in production. It doesn't help you with that. If you're not able, if you're having issues with deployment, you're not able to reliably deploy
32:01
your monolith in production, you're not going to solve this by deploying multiple services with dependencies in production. All that services do is they help you scale the number of developers. So in order to help you with that and reduce the amount of blockage and issues
32:20
you will face when developing with multiple teams on the same code base, then maybe services make sense. I just wanted to mention this. Other talks, like I mentioned, yesterday we already saw protect your users with circuit breakers. This is another talk that deals with one of the things or best practices you should do
32:41
when developing with services. The whole premise, like Scott talked about it yesterday, if the call you do is a simple Python function call, you're good. This will succeed every time. Only if you go over the network, then you might do a bunch of additional things
33:01
to have a less worse behavior in case of failure. I urge you to check it out on YouTube once it becomes available. And if you want to know more about asynchronous networking, go see Loris' talk tomorrow. He tells you all about the nitty-gritty details, how to do it in Python 3,
33:22
how to do it with Python 2, and gives you a lot more detail and gives you some insight into what you have to do if you don't use something like Fido to do your network request. Go check us out. We have a pretty nice engineering blog. You can find us at the booth as well. I want to mention that we're doing a raffle right now,
33:42
so we're giving away a pretty nice drone. So if you want that, come find us. We're giving it away Friday, so you should be around on Friday. And that's it. Thank you very much.
34:01
Thank you, Stefan. We'll have a round of questions and answers. Please wait for the microphone so that we can hear you on the recording. Thank you. This was really great talk. Did the Fido client support Twisted and How?
34:21
Yes, internally it uses Twisted and Crochet actually. So as I said, it's not doing any magic, but it hides a lot of the complexities. If you do things like you also talk to your data store and you want to be that asynchronous as well, you have to do it yourself manually. Fido will not help you with that.
34:42
But if most of the time all you do is network requests, I think Fido is a pretty great tool to do that without having to deal with all of the peculiarities. Because I'm interested in if we are using Twisted, then if we can just yield their requests and it will work with all the Twisted machinery
35:03
already present in our code. Sure, absolutely. Like if you want to do that yourself, you have absolutely more power, more flexibility, absolutely. Thank you.
35:31
Yeah, thanks a lot. It was a great talk. Actually, I'm writing a client for an API right now and I'm realizing that I'm kind of reinventing the wheel
35:43
and reinventing, what's the library, bravado in a sense, because I didn't know about it. And I wanted to know also if bravado generated also all the mock requests, for example, that you would need if you were writing a client
36:04
at the same time of generating all those nice request functions for you. Yeah, that's a great question. Yeah, I didn't talk about testing at all. Unfortunately, right now, there is no open source solution for that. We have something that is being developed internally.
36:20
I hope, I cannot promise anything, but I hope we can open source this. We also have basically something written by my colleague, Loris, a mock server that will, when you write your mock, we still write our mocks manually right now for testing, but it will then serve these, read the swagger spec and find the appropriate mock and serve it
36:43
and all of it will actually be validated to make sure the mock conform to the swagger spec and everything. But I hope we can open source something that will actually look at the spec and then generate data of the correct form, like form and type automatically.
37:03
But sorry, nothing available right now. So just to understand the picture correctly, this swagger doesn't help you to write the backend. It only helps the client to validate data sent and received.
37:21
Yes, that is true. It doesn't, no, it doesn't really help you. Like you still need to write the backend yourself. It does provide you, like the pyramid swagger, for example, it does help you with serving the spec. It does help you with making sure your endpoints conform to the spec. But the implementation, you have to write yourself.
37:44
And exposing the swagger, the script in the files with the description, it's also up to you and you have to just export this via HTTP somehow, right? If you use pyramid swagger, which is what I'm most familiar with,
38:01
this is like one of the core features. So you just point it to the file on the file system and it will do everything for you. Like not only serve it, but potentially, as I mentioned, also like read multiple files and combine them to one file to increase efficiency and things like that. For pyramid, it takes care of that.
38:21
I think connection does the same thing for flask, but I haven't used it, so better read up on that. So about the hypothetical mocks,
38:41
are you going to do it like in process, for example, like responses works, or is it going to be like out of process, a separate server like Wiremock or Mountebank? Both are possible, like it depends on what type of testing you talk about.
39:02
What we have is like, obviously like we have the real service implementation and then we have this so-called mock server, which basically just you can spin it up just like a real service and instead of like doing real work, it will serve mock responses. This is really great, not only for integration testing,
39:20
but also like I said, we are like a backend for apps, like Android and iOS apps, and they can use this for their integration testing without having like everything at Yelp, spin that up and use it for testing. That is one thing. Then for the other part, we do have something where in a testing environment,
39:43
basically it mocks out, like you mock out the service call on the client side, and it will just generate a mock response for you. This is when you want to unit test your client without having, like we have an acceptance testing setup
40:01
that spins up a bunch of Docker containers. It's kind of sometimes a little bit brittle and it takes a lot of time, so you probably want to unit test as well, and this helps with that. The alternative is, and we still do, like most of our code is like that, we just write the response manually,
40:21
we mock it out and write the response manually. It works, but you should have some form of like acceptance testing, integration testing, preferably because we've had several issues where the mocks at some point just don't conform or are not what the service really returns anymore,
40:43
and that's when you hit issues. Okay, thanks. Just a quick question. How does Pyramid Swagger relate to a library like Cornus that also gives you convenience features
41:03
to create a REST API? Are they related at all? Chronos, do you mean? Cornus, the Mozilla. I'm sorry, I'm not familiar with that. Then does Pyramid Swagger provide, other than serving the spec to the bravado client,
41:20
does it provide route generation features? Yeah. All right. Yeah, basically you do that. We can go on detail over there. We can discuss details, but actually like Scott, I'm not sure if he's here. There he is. He's the main author of Pyramid Swagger,
41:41
so he'll be probably like the best person to talk to all of the details. Thanks.
42:04
About Swagger, you can have JSON input and output, and in the spec file you have a description that is really like JSON schema. Is it exactly the same thing or something different?
42:20
Yeah, great question. It's actually based on JSON schema, the Swagger specification. It's not a superset. It adds something, but it also takes things away, about the issue I talked about with the null value. JSON schema actually is able to deal with that
42:40
because you're allowed to give a list of allowed types for a field. Unfortunately, in Swagger this is not allowed, so it's kind of like, most of the time it's a superset, but sometimes not everything of JSON schema is supported in Swagger.
43:01
Any more questions? So thank you for your attention. If you want more information, you can contact Stephen also. Thank you.