Leveraging documentation power for better web APIs
This is a modal window.
Das Video konnte nicht geladen werden, da entweder ein Server- oder Netzwerkfehler auftrat oder das Format nicht unterstützt wird.
Formale Metadaten
Titel |
| |
Serientitel | ||
Teil | 73 | |
Anzahl der Teile | 169 | |
Autor | ||
Lizenz | CC-Namensnennung - keine kommerzielle Nutzung - Weitergabe unter gleichen Bedingungen 3.0 Unported: Sie dürfen das Werk bzw. den Inhalt zu jedem legalen und nicht-kommerziellen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen und das Werk bzw. diesen Inhalt auch in veränderter Form nur unter den Bedingungen dieser Lizenz weitergeben | |
Identifikatoren | 10.5446/21216 (DOI) | |
Herausgeber | ||
Erscheinungsjahr | ||
Sprache |
Inhaltliche Metadaten
Fachgebiet | ||
Genre | ||
Abstract |
|
EuroPython 201673 / 169
1
5
6
7
10
11
12
13
18
20
24
26
29
30
31
32
33
36
39
41
44
48
51
52
53
59
60
62
68
69
71
79
82
83
84
85
90
91
98
99
101
102
106
110
113
114
115
118
122
123
124
125
132
133
135
136
137
140
143
144
145
147
148
149
151
153
154
155
156
158
162
163
166
167
169
00:00
Dämon <Informatik>W3C-StandardLeistung <Physik>Kontextbezogenes SystemLeistung <Physik>VersionsverwaltungBetrag <Mathematik>Zentrische StreckungHardwareSoftwareComputervirusVorlesung/KonferenzComputeranimation
00:49
Kontextbezogenes SystemCoxeter-GruppeDifferenteSoftwareWeb ServicesBrowser
01:22
BrowserFramework <Informatik>Design by ContractInterface <Schaltung>UmwandlungsenthalpieDienst <Informatik>Design by ContractExploitParametersystemPunktMomentenproblemComputeranimation
02:19
Ein-AusgabeNichtlinearer OperatorTVD-VerfahrenValiditätInstantiierungDienst <Informatik>Vorlesung/KonferenzBesprechung/Interview
02:46
Chi-Quadrat-VerteilungKreisringMetropolitan area networkBildschirmmaskeEin-AusgabeNichtlinearer OperatorPhasenumwandlungDifferenzenrechnungExogene VariableFamilie <Mathematik>ParametersystemDifferenteSystemaufrufResultanteSichtenkonzeptXML
03:52
Chi-Quadrat-VerteilungKonvexe HülleFunktion <Mathematik>Umsetzung <Informatik>ParametersystemBildschirmmaskeDreiEin-AusgabeCodeFunktion <Mathematik>ValiditätCASE <Informatik>Mailing-ListeUmsetzung <Informatik>Nichtlinearer OperatorTypentheorieFokalpunktZeichenketteRechter WinkelXML
04:57
MultiplikationsoperatorTabelleEin-AusgabeClientBildschirmmaskeValiditätDateiformatBesprechung/Interview
05:22
Statistischer TestVerträglichkeit <Mathematik>Offene MengeDesign by ContractAdditionRegulärer GraphImplementierungDienst <Informatik>Statistischer TestUmwandlungsenthalpieFormale GrammatikComputeranimation
06:02
BildschirmmaskeStatistischer TestVerträglichkeit <Mathematik>Umsetzung <Informatik>Innerer PunktSyntaktische AnalyseEin-AusgabeAusnahmebehandlungFunktion <Mathematik>Automatische HandlungsplanungFunktionalEin-AusgabeTypentheorieAusnahmebehandlungInformationPhysikalischer EffektMapping <Computergraphik>QuaderMultiplikationsoperatorDefaultAggregatzustandElektronische UnterschriftFehlermeldungPunktWiderspruchsfreiheitBildschirmfensterMehrrechnersystemNichtlinearer OperatorVorlesung/KonferenzComputeranimation
07:52
Innerer PunktSyntaktische AnalyseUmsetzung <Informatik>AusnahmebehandlungFunktion <Mathematik>Ein-AusgabeTypentheorieNebenbedingungDatentypMailing-ListeBoolesche AlgebraZeichenketteGanze ZahlZahlenbereichSchwimmkörperObjekt <Kategorie>TypentheorieGrenzschichtablösungParametersystemArithmetisches MittelInnerer PunktTeilmengeDatenstrukturValiditätObjekt <Kategorie>Umsetzung <Informatik>Ein-AusgabeLeistungsbewertungDienst <Informatik>GamecontrollerRechter WinkelNebenbedingungDateiformatVorlesung/KonferenzComputeranimation
09:21
Boolesche AlgebraZeichenketteGanze ZahlInnerer PunktZahlenbereichSchwimmkörperObjekt <Kategorie>Umsetzung <Informatik>Funktion <Mathematik>DatentypNebenbedingungAusnahmebehandlungCodeImplementierungParametersystemGewicht <Ausgleichsrechnung>Ein-AusgabeDesign by ContractKlasse <Mathematik>DatenstrukturProgrammbibliothekValiditätUmsetzung <Informatik>AdditionFunktionalCASE <Informatik>TypentheorieNebenbedingungData DictionaryDienst <Informatik>AusnahmebehandlungGruppenoperationTemperaturstrahlungRationale ZahlMagnetbandlaufwerkBildschirmmaskeWiderspruchsfreiheitVorlesung/KonferenzComputeranimation
11:37
ResultanteFunktionalEreignishorizontDesign by ContractInhalt <Mathematik>AusnahmebehandlungSystemaufrufDienst <Informatik>TypentheorieVorlesung/Konferenz
12:14
BildschirmmaskeParametersystemSkriptspracheAppletTexteditorTemplateWeitverkehrsnetzKrümmungsmaßTexteditorSummierbarkeitAbstandCASE <Informatik>Nichtlinearer OperatorInhalt <Mathematik>DefaultBildgebendes VerfahrenTypentheorieSerielle SchnittstelleDeklarative ProgrammierspracheÄhnlichkeitsgeometrieSystemaufrufSichtenkonzeptTemplateFlächeninhaltBildschirmmaskeValiditätComputeranimation
13:20
AusnahmebehandlungAdditionInstantiierungSoftwaretestSystemaufrufStrategisches SpielGeradeKategorie <Mathematik>Umsetzung <Informatik>Ein-AusgabeFunktion <Mathematik>DatentypElektronische PublikationStatistische HypotheseBildschirmmaskeFlächeninhaltSoftwareentwicklerStatistische HypotheseMessage-PassingStrategisches SpielDesign by ContractGenerator <Informatik>Dienst <Informatik>Nichtlinearer OperatorRandomisierungAusnahmebehandlungInstantiierungCodeSoftwaretestTypentheorieEin-AusgabeProgrammiergerätTypprüfungStatistischer TestZeichenketteRelativitätstheorieWort <Informatik>MathematikCASE <Informatik>Inhalt <Mathematik>AbstandVorlesung/KonferenzComputeranimation
15:34
Mailing-ListeZeichenketteFunktionalVorlesung/KonferenzBesprechung/Interview
15:53
ImplementierungComputersicherheitWasserdampftafel
16:15
AusnahmebehandlungAdditionGeradeElektronische PublikationStatistische HypotheseStrategisches SpielInstantiierungSoftwaretestSystemaufrufKategorie <Mathematik>Umsetzung <Informatik>Funktion <Mathematik>DatentypEin-AusgabeWeb SiteExogene VariableVerträglichkeit <Mathematik>SichtenkonzeptVerschlingungFramework <Informatik>ZählenZellulares neuronales NetzBildschirmmaskeGraphfärbungFramework <Informatik>Virtuelle MaschineMAPExogene Variablesinc-FunktionDisjunktion <Logik>XML
16:52
MereologieExogene VariableSystemprogrammSchlussregelSichtenkonzeptInstantiierungTouchscreenVisualisierungVorlesung/KonferenzBesprechung/Interview
17:18
Verträglichkeit <Mathematik>SichtenkonzeptEin-AusgabeVerschlingungFramework <Informatik>Exogene VariableZellulares neuronales NetzMathematikZeitstempelGanze ZahlSichtenkonzeptFahne <Mathematik>Formale SemantikExogene VariableEntscheidungstheorieFunktionalPhysikalisches SystemComputeranimation
18:06
Verträglichkeit <Mathematik>SichtenkonzeptVerschlingungEin-AusgabeFramework <Informatik>Exogene VariableZählenDifferenzengleichungMAPParametersystemChi-Quadrat-VerteilungBildschirmmaskeVerschlingungSichtenkonzeptEin-AusgabeInformationExogene VariableNichtlinearer OperatorDateiformatFormation <Mathematik>MathematikStabDruckverlaufVorlesung/KonferenzBesprechung/InterviewFlussdiagramm
18:45
BildschirmmaskeNichtlinearer OperatorVerschlingungWeb-SeiteDatenfeldVorlesung/Konferenz
19:11
Offene MengeTypentheorieStatistische HypotheseCodegenerierungProgrammbibliothekClientSummierbarkeitTaskUmsetzung <Informatik>Statistischer TestBildschirmmaskeCodeImplementierungClientMessage-PassingDienst <Informatik>StandardabweichungOpen SourceStrategisches SpielStatistische HypotheseDokumentenserverFehlermeldungSoftwaretestTypentheoriePaarvergleichRuhmasseGenerator <Informatik>Design by ContractProgrammbibliothekMetropolitan area networkAbstandAdditionGemeinsamer SpeicherGruppenoperationInhalt <Mathematik>Coxeter-GruppeQuellcodeComputeranimation
22:00
CodeWort <Informatik>
22:31
Versionsverwaltung
Transkript: Englisch(automatisch erzeugt)
00:00
Welcome, we have Rudy here, showing us how to leverage documentation power for better APIs. Give him some clap.
00:35
Hello. So, hello, I'm Rudy, I'm working at Criteo, a company that delivers personalized ads
00:46
at scale. My team is building monitoring tools for our large software and hardware inventory. And sometimes, we decided to make monitoring as a service and start to experiment different
01:05
ideas around web service and browser APIs. So this presentation gives a glimpse of what we have done. And this is a joint work with my colleague, which is there.
01:28
So, what is a browser-able API? Obviously, it's composed of two worlds. The browser-able world, so, probably it's related to a browser, and, more importantly,
01:46
it's related to exploration and experimentation, and I will add to that more later. And there is a world API, we've got a formal talk, so an API is a kind of contract, it
02:04
can be very explicit, like if you're declaring a swagger specification file, or it can be implicit, like in the documentation. The contract defines the valid use of the service that is linked to the API, or the
02:23
operation you can do with this service, and also what are the valid inputs for the operation in the service. So why is it interesting? Because it brings a lot of features for your service.
02:44
So the first feature I will speak about is why we rely on the documentation to describe the API. Then I will speak about a basic feature that you have in browser-able API.
03:04
So the first basic feature is the exploreability, it means that you can go to the API and ask what the operations are possible in this API. So here you have a view of an API with different operations, and then you can just
03:22
click on an operation, and you see the documentation of that operation, the parameter types, and what are the possible results. And then, more than that, you can play with an operation, so you have to see the input of the operation, and call it, and so you can see what is the response and
03:46
explore the behavior of this operation. But still, typing the input, like a JSON input, can be difficult, so we probably can do something
04:01
better. Another really important feature of an API is to enforce the validation of the input on the input and the outputs. By the way, this talk focuses on JSON inputs.
04:21
You don't want an operation to be used in an adapted way, that's why you should validate all the inputs of your service, like if a list is transmitted instead of a string, nobody knows how the code will behave in that case. And even for valid JSON inputs, you may want to have a conversion between the JSON inputs
04:49
and between your Python types. Okay, this one is an obvious feature, like I said before, it can be really painful to
05:03
enter JSON data, because it's a verbose format. So, having forms makes you save time, and it brings client-side input validation on the table.
05:20
Another really interesting feature is that you have that kind of API contract, and you have your service implementation, and you want to be sure that they perfectly match, so that would be really cool to just ask for automatic tests for checking that they match.
05:41
And I will also present another feature that makes browser-able APIs more user-friendly, and last, just to mention it, you may want to generate, even if you use the doc, more formal specification contract, like in Swagger, because Swagger has a lot of amazing features
06:04
that you can later use, so, and this is the plan of the talk, in fact, so I'm starting now. So, imagine you want to add a new function to your API, what do you do?
06:22
Okay, it's easy. You document the functions, and you add the publish decorator on it, so this decorator will register the function as an operation in the API. The function will be left unmodified, so you can still use it as before, when the
06:45
decorator was not there. And behind the scenes, the function will be wrapped in the API, where all the data about HTTP will be handled.
07:02
So how is it done? Documentation will be parsed, so that we have information about input, output, and exceptions. This gives you type concerns about inputs, outputs, and possible zero, and also the
07:29
signature of the functions give you default value for inputs. So here we have type concerns, that's what's in the red boxes.
07:43
Type concerns, they are more than just type names. They can be parametrized for iterable and mapping types, like here we have a parametrized list, that is parametrized by mytb, or it can also contain several type names, like
08:05
the first parameter, it's either an int or a swing. And from that type concerns, we can find all the input should be validated and translated
08:22
from JSON data structure to Python object. For that, we convert that type concern to a JSON schema. So for those that, for people that don't know what is JSON and JSON formats, a JSON object
08:41
can be seen as a subset of the built-in Python types. So often the conversion between the two is free. But for your services, you still need to do, you may need to do additional validation. And this is done by a JSON schema.
09:03
So what is JSON schema? A JSON schema, it describes the structure of the expected JSON object, like its types or its minimum value, what it contains, like the name of the field, the one that
09:28
are optional, so it really describes what you are expecting as an input. And there are also situations where you want real conversion to happen.
09:43
So you have, like, a JSON dictionary and you want it to be converted to your own Python class. So for these two cases, when you want to have additional validation and custom conversion,
10:01
we introduce two classes that helps to attach the conversion and the JSON schema to your regular class. So from that, we are able to retrieve a JSON schema from any type constraint.
10:28
So what does the validation and the conversion happen? When the service is called and the inputs are received, there's first a validation. For most basic validation, we rely on the type constraint in the documentation.
10:46
And for more complex ones, we rely on this JSON schema. So there's a library called JSON schema that you can use to validate any of the
11:00
data structure with your JSON schema. After that validation, there is the conversion step. That is either implicit for most basic type or uncrafted using the from JSON cases.
11:24
And then the real function is called. And from that, there can be two outcomes. You can have an exception. And so what we do is that we check that the exception is respecting the contract. If it's not, the content of the exception is hidden because we don't want the user of the
11:49
service to see anything that is not accepted by the contract. And at last, if we get a result from that function call, we validate it based on its type.
12:11
So what else can we do with JSON schema? After all, it's a sundown. So you can use JSON editor. It's able to use a JSON schema to generate a form, which is also an editor.
12:27
It's able to validate the form and give you back the validate JSON value. So you just have to plug in a call to JSON editor in your HTML template.
12:41
And you have a cool feature for almost nothing. So here you have a view of what you had before. You had to type the JSON content in the play with me text area. And now using JSON editor, you just have a form.
13:03
So it's really easy to create an input. And note that there are similar features in Django REST framework, but you should also, in this case, declare things manually, like the serializer for the form.
13:23
Just like you have to write a JSON schema here. So what else can we do with JSON schema? I don't know if you know a package named hypothesis. It's a package that helps you to do property-based testing.
13:47
So it's composed of strategies that are able to generate random inputs for many usual types. And it's really a fantastic tool.
14:01
I encourage you to use it. So let's add the generation strategy for JSON schema. With that, we can generate tests to verify that all the operations of our service respect the contract described by the documentation.
14:23
So here, I've put two examples. So I've taken a valid code that respect the contract of the API, and I've modified it. So in the first case, I changed the code just before it was raising an exception.
14:45
I added a type error, like I had an internal string. And from the automatic test, we have this message where, first, the type error is discovered
15:00
and is shown to the developer with a proper stack trace. And second, the programmer is also warned that one of the exceptions that is described in the contract is never occur with this operation.
15:25
So it shows that what is described in the documentation could be outdated, for instance. And then I've put another example where I modified the valid code and make it return
15:45
a bad value, like a list of string for a function that is expected to return something else. And so what we see is that it's also detected automatically.
16:01
So we have something that is able to take your whole API and automatically check that what you have described in the documentation is on par with the current implementation, which is a really cool feature.
16:22
So what can I do next with JSON schema? I don't know. We stop it there. So we already have an easy way to enter JSON with forms. Can we have something useful to visualize the JSON response?
16:45
So taking inspiration from Django RAS framework that already does the coloring of the JSON response, we decided to go further and to add something that we called pretty view.
17:01
So a pretty view, it helps you to define custom visualization of the JSON response. So here, for instance, so you have the overview above and the pretty view in the lower part of the screen. And you can see that the JSON contains a timestamp.
17:26
And in the pretty view, the timestamp is converted directly to a human readable date. And there's also another example. You have an integer value in the JSON, and it has a particular semantic.
17:46
And the pretty view is able to show that semantic. So here it's the value zero, which is an OK flag. So it really makes the JSON response easier to read for humans.
18:12
And another thing that we added to the pretty view is the ability to add extra information.
18:21
Like here, the pretty view says something about active downtime that are not in the initial JSON response. And moreover, it adds a link near that extra information. And that link, which we called an input link, it's a link that is able to go to
18:42
another operation and pre-fill all the information in the form. So you can navigate between different features in the API. And you don't have to tip anything. So here, when you click on the red link, you will go to the operation page,
19:06
to another operation page, and every field will be already pre-filled. So what's next? We plan to open source all the things that we presented there.
19:29
So you can watch the Criteo GitHub repository. We will add the different contribution there.
19:41
We will also contribute to hypothesis with the JSON schema strategy. One thing that we didn't do, and could be very interesting, is to support NN natively, because it's a really interesting type of place.
20:04
And to integrate more with PyTests, the thing that I showed to you that gives you like a diagnostic for your API, like with the warning and the error about mismatch between the contract and the implementation.
20:23
Currently, you have to run it by hand. And it would be better if we integrated the standard testing package. Then there is the Swagger spec generation. So we already have a lot of cool features.
20:41
So why would we want that? Because Swagger has the ability to generate a client code library. That means you code your service in Python, and then you want people to have automatic access in Java. You just write the Swagger file,
21:02
and it will generate the client for you in Java. So that's really a good thing. So if you have to only keep a simple message from this presentation,
21:21
possible APIs, they are really interesting stuff, because they are very user-friendly, and you can discover the content and explore and test, and it's very good. The documentation and the types,
21:43
they can be really used to add a lot of features to your services. So here I give some examples with the features I present, but there are a lot more things to do.
22:01
And also the conclusion, Gson and Gson schema, they are very handy, they are standard, there are a lot of tools around them. So don't hesitate to use them. I think it will help you to easily have new features
22:22
in your code. And that's all, if you have questions. Questions?
22:44
Thank you for coming.