NDC London 2016 - The Rest of ReST
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 133 | |
| Author | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/48933 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
00:00
Representational state transferSoftware developerRepresentational state transferRight angleNumberBitPatch (Unix)State of matterComputer animationJSON
00:58
ArchitectureConstraint (mathematics)Representational state transferSoftware architectureRelational databaseSet (mathematics)HypothesisWeb 2.0Physical systemGradientSoftware frameworkSoftwareQuantum stateField (computer science)Latent heatPoint (geometry)Server (computing)Communications protocolScaling (geometry)MultilaterationComputer animation
02:25
Term (mathematics)SoftwareRepresentational state transferScale (map)Software developerTime evolutionConstraint (mathematics)Physical systemScaling (geometry)SoftwareExtension (kinesiology)Field (computer science)Multiplication signSet (mathematics)Web 2.0Representational state transferEvoluteChemical equationRelational databaseIndependence (probability theory)Communications protocolSoftware frameworkTerm (mathematics)Computer animation
03:56
Software developerServer (computing)Client (computing)Cache (computing)Physical systemInterface (computing)HypermediaConstraint (mathematics)Canonical ensembleProxy serverInternet service providerCartesian coordinate systemClient (computing)Physical systemHypermediaFront and back endsVirtual machineScaling (geometry)Connected spaceConsistencyData storage deviceState of matterInformationInternetworkingRepresentational state transferRelational databaseDifferent (Kate Ryan album)Server (computing)SoftwareSet (mathematics)Rule of inferenceMereologyDatabase transactionBitMultiplicationOperator (mathematics)CodeStructural loadComputer fileArithmetic meanMultiplication signUniformer RaumTrailInterface (computing)Uniform resource locatorDigital photographyCustomer relationship managementType theoryGoodness of fitRepresentation (politics)Scripting languageWeb browserFacebookDevice driverPhysical lawService (economics)Point (geometry)AreaPrincipal idealCopyright infringementVideo gameChemical equationCuboidEmailComputer animation
08:14
Server (computing)Software developerClient (computing)Cache (computing)Interface (computing)HypermediaConstraint (mathematics)Physical systemComputer fileDemo (music)DatabaseCodePhysical systemRelational databaseGoodness of fitRepresentational state transferProfil (magazine)SoftwareTwitterRing (mathematics)Multiplication signHome pageDemo (music)Database transactionBoss CorporationClient (computing)Type theoryMiniDiscDatabasePoint (geometry)Representation (politics)Coma BerenicesHypermediaRow (database)BuildingBand matrixSpeech synthesisNumberDependent and independent variablesData storage deviceEmailLevel (video gaming)Uniform resource locatorComputer fileSign (mathematics)Game theoryHydraulic jumpDensity of statesComputer animation
11:34
Software developerContent (media)Home pageConstraint (mathematics)Client (computing)Home pageProfil (magazine)ExistenceVector potentialMechanism designRepresentational state transferMereologySet (mathematics)NumberCartesian coordinate systemHypermediaToken ringQuicksortState of matterInformation securityPunched cardJSON
13:10
SupremumSoftware developerSupercomputerAdventure gameMusical ensembleHome pageHypermediaAdventure gameGreatest elementGraphics tabletHome pageRepresentation (politics)Computer configurationSet (mathematics)Standard deviationState of matterDifferent (Kate Ryan album)Scripting languageRelational databaseRepresentational state transferTerm (mathematics)Communications protocolInclusion mapSpreadsheetWeb pageOrder (biology)Musical ensemblePhysical systemWeb 2.0Sign (mathematics)HyperlinkPoint (geometry)PlastikkarteObject (grammar)Open setWordComputer animation
15:39
HypermediaSoftware developerStandard deviationFile formatVolumenvisualisierungComputer-generated imageryAvatar (2009 film)HyperlinkEmailLink (knot theory)Group actionForm (programming)Order (biology)Content (media)Data typeTwitterMessage passingStatisticsCodeLink (knot theory)Home pageCartesian coordinate systemProfil (magazine)Uniform resource locatorParsingGraph (mathematics)Multiplication signRepresentation (politics)Type theoryOrder (biology)Extension (kinesiology)Mobile appInstance (computer science)Band matrixContext awarenessMoment (mathematics)Message passingAdventure gameWeightHyperlinkPhysical systemTracing (software)Group actionProgrammschleifeVulnerability (computing)Term (mathematics)Slide ruleDegree (graph theory)NumberChannel capacitySimilarity (geometry)ResultantBuildingPoint (geometry)Field (computer science)FacebookKey (cryptography)Object (grammar)Web browserFlow separationHypermediaInclusion mapFitness functionTotal S.A.System callStandard deviationSubsetFile formatSpreadsheetBookmark (World Wide Web)CircleRevision controlThomas BayesGreatest elementQuicksortScripting languageCausalitySubject indexingMereologyCellular automatonSound effectAverageRight angleHydraulic jumpComputer animation
22:34
Content (media)Data typeLink (knot theory)TwitterSoftware developerThermal expansionDigital photographyNumberDatabase transactionShape (magazine)WordData conversionRight angleHome pageLine (geometry)Key (cryptography)Profil (magazine)DatabaseMultiplication signData loggerConnectivity (graph theory)MiniDiscExpressionLink (knot theory)CollisionSpacetimeMereologyData structureSystem callServer (computing)Computer animation
24:49
TwitterData typeContent (media)Software developerMessage passingWeightRepresentational state transferEmailWebsiteInformationRevision controlMathematicsPhysical systemMechanism designProfil (magazine)Line (geometry)QuicksortUniform resource locatorEinbettung <Mathematik>Thermal expansionInfinityMenu (computing)Element (mathematics)Information privacyServer (computing)Representational state transferSet (mathematics)ExistenceLink (knot theory)Pattern languageMultiplication signField (computer science)NumberComputing platformMoment (mathematics)EmailInstance (computer science)WeightWebsiteHome pageComputer animation
28:46
Revision controlOnline helpSoftware developerContent (media)State of matter40 (number)Strategy gameField (computer science)Object (grammar)Revision controlProfil (magazine)MathematicsPoint (geometry)Physical systemData conversionEntire functionImplementationPatch (Unix)InternetworkingGame controllerProgrammable read-only memoryLatent heatGoodness of fitSet (mathematics)Computer animation
30:02
File formatPatch (Unix)HypermediaSoftware developerHost Identity ProtocolContent (media)Data typeRepresentational state transferComputer-generated imageryType theorySet (mathematics)HypermediaIdentifiabilityPatch (Unix)MathematicsFile formatDescriptive statisticsCommunications protocolLatent heatEmailRevision controlServer (computing)Context awarenessRow (database)Incidence algebraCartesian coordinate systemBeat (acoustics)Representation (politics)Matching (graph theory)Level (video gaming)HierarchyContent (media)Order (biology)MultilaterationField (computer science)Medical imagingDifferenz <Mathematik>Physical systemDigital photographySoftware testingOperator (mathematics)Address spaceBit rateForm (programming)Arithmetic meanSystem callNatural numberStiff equationComputer animation
33:27
Software developerCodeContent (media)Data typeTwitterWeightRepresentational state transferEmailField (computer science)CodeRevision controlUniform resource locatorCategory of beingOffice suiteFlow separationComputer animation
34:15
WebsiteSoftware developerRepresentational state transferSoftwareScale (map)InternetworkingCartesian coordinate systemSoftwareScaling (geometry)MathematicsSet (mathematics)Revision controlBlogCodePoint (geometry)FrequencyPhysical lawRepresentational state transferGroup actionDifferent (Kate Ryan album)Special unitary groupDiscrepancy theoryNetwork topologyGoodness of fitComputer animationJSON
35:41
Software developerUniform resource locatorRevision controlHypermediaServer (computing)Client (computing)Different (Kate Ryan album)Game controllerFlow separationRepresentation (politics)Set (mathematics)Type theorySystem callSurvival analysisComputer animation
36:59
Software developerRevision controlUser profileEmailRevision controlRepresentation (politics)SoftwareEndliche ModelltheoriePresentation of a groupEquivalence relationBusiness modelSoftware engineeringDecision theoryDifferent (Kate Ryan album)Tape drivePoint (geometry)Domain nameUniform resource locator2 (number)Computer animation
38:22
Revision controlSoftware developerServer (computing)Branch (computer science)SoftwareRepresentational state transferRevision controlPoint (geometry)Product (business)Strategy gameDomain nameInteractive televisionWindowOffice suiteTerm (mathematics)Physical systemCuboidDifferent (Kate Ryan album)1 (number)Endliche ModelltheorieClient (computing)Category of beingData structureCodeUniform resource locatorRootSoftware bugFile formatMultiplication signService (economics)Branch (computer science)WeightBeta functionGame controllerAttribute grammarComputer fileAlpha (investment)Web 2.0Information securityType theoryComputer programmingSound effectHypermediaGoogolDampingControl flowMereologyBoss CorporationDisk read-and-write headGroup actionRoutingAdditionComputer animation
41:18
Software developerHyperlinkEmailMobile appState of matterAddress spaceBoss CorporationPhysical systemWebsiteQuicksortBitMechanism designControl flowPasswordInformation securityRepresentational state transferDisk read-and-write headCoefficient of determinationComa BerenicesTerm (mathematics)Metropolitan area networkThomas BayesForcing (mathematics)HyperlinkLoginComputer animation
42:27
Software developerSystem administratorCustomer relationship managementEndliche ModelltheorieTerm (mathematics)Communications protocolHyperlinkVoltmeterComa Berenices
43:28
Software developerHyperlinkMechanism designOrder (biology)Communications protocolAuthenticationLine (geometry)HyperlinkCodeTwitterWebsiteHome pageRow (database)AuthorizationFreewareReal numberComputer wormBuildingUniform boundedness principleServer (computing)WindowBeta functionKey (cryptography)Schmelze <Betrieb>Physical systemPasswordPoint (geometry)Pattern languageBitToken ringInformationCuboidSingle-precision floating-point formatLimit (category theory)DatabaseEmailComa BerenicesWeb pagePhysical lawProcess (computing)Multiplication signFreezingSystem callEndliche ModelltheorieDiagram
47:50
Crash (computing)Software developerPhysical systemLaptopDifferent (Kate Ryan album)Virtual machineUniform resource locatorInternet service providerVisualization (computer graphics)InternetworkingLimit (category theory)DebuggerDatabase transactionExistential quantificationMoment (mathematics)Address spaceComputer animation
48:41
Software developerLocal ringClique-widthGroup actionTwitterAddress spaceMessage passingComputer animationJSON
49:29
Software developerVolumenvisualisierungInternetworkingRight anglePhysical lawVirtual machineLocal ringVisualization (computer graphics)WebsiteInstance (computer science)Uniform resource locatorVideo game consoleJSONComputer animationSource code
50:34
Software developerHome pageFreewareEndliche ModelltheorieInternet service providerDebuggerService (economics)SoftwareRevision controlPay televisionPhysical systemVirtual machinePoint (geometry)Computer animation
52:19
Software developerCoding theoryReal-time operating systemService (economics)Real numberProxy serverFormal languageDatabase transactionScripting languagePhysical systemPoint (geometry)Uniform resource locatorLaptopInjektivitätRow (database)Multiplication signData storage deviceWeb crawlerControl flowGoodness of fitDemo (music)2 (number)Speech synthesisPhysical lawRight angleLevel (video gaming)BitSource codeComputer animation
55:15
Cache (computing)Cantor setControl flowContent (media)Software developerPRINCE2Control flowRepresentation (politics)ResultantPhysical systemGroup actionPoint (geometry)MultilaterationDatabase transactionHacker (term)Shooting methodSoftware bugSource codeComputer animation
56:02
Software developerComputer-generated imageryPatch (Unix)Closed setClient (computing)Server (computing)Slide ruleThermal expansionService (economics)Computer animation
56:50
Data typeContent (media)Software developerTwitterPhysical systemQuicksortGoodness of fitCuboidDecision theoryThermal expansionRepresentational state transferParsingUniform resource locatorMultiplication signDigital photographyHypermediaHyperlinkScaling (geometry)Presentation of a groupConfluence (abstract rewriting)HierarchyEinbettung <Mathematik>FeedbackRelational databaseRepresentation (politics)InformationDifferent (Kate Ryan album)Coma BerenicesSystem callMoment (mathematics)Link (knot theory)MereologyProduct (business)Field (computer science)Java appletScripting languageCommunications protocolCASE <Informatik>
Transcript: English(auto-generated)
00:06
Good afternoon, NDC. Could I ask you all to start, please, by standing up? Because we've just had lunch, everyone's a bit sleepy, so everyone on your feet, right. Now I would like anybody who has ever implemented an HTTP API to remain standing.
00:26
If you've never built an API that uses HTTP, sit down. If you've never used JSON, sit down. If you've never implemented HTTP put, sit down. If you've never implemented patch, sit down.
00:46
And finally, if you're not wearing a headset, microphone, sit down. Awesome, we're all in the right room. Hi, my name is Dylan Beatty. This talk today is called the rest of rest. Rest is, as I'm sure you all know, judging by the number of people who, A, have turned
01:03
up and B, stayed standing up, rest is a set of architectural constraints. Rest is not a framework, rest is not a module, rest is not a protocol. Rest is a set of guidelines. The reason I like rest is because I am sick to the back teeth of framework of the month
01:21
club. Every month, every two months, something new comes out. It's not Angular anymore, it's backbone. No, it's not backbone, it's knockout. No, we're doing this. Oh, no, it's Gulp, and Grunt, and Bower, and rest is a set of architectural principles about building scalable distributed software systems. It was postulated by a guy called Roy Fielding.
01:40
Roy Fielding was at CERN, he worked on the HTTP 1.1 specification, he built some of the Apache web server, he now works for Adobe. Rest was Ray Fielding's PhD thesis. It's probably the best known PhD thesis that has ever been published. I can't think of anyone else who, you know, anyone here knows post grads, they spend
02:01
four years, you know, beavering away and doing research and experiments, they write something. I don't know anyone else whose post grad dissertation is still being discussed by people who actually do it for a living 15 years later. So yeah, Roy laid out this set of architectural constraints about how to build software.
02:21
Now one of the very interesting things that emerged from this initial thing, rest is software design on the scale of decades, he said. Every detail is intended to promote software longevity and independent evolution and many of the constraints are directly opposed to short-term efficiency.
02:41
What this means in a nutshell is that if you are in a hurry and you are building your MVP and struggling with time to market, you are probably going to find a lot of things where there is a quick way of doing something and there is a restful way of doing something. And that's another theme that we're going to explore today is this balance between principles, purity, understanding, constraints, but also shipping working software and keeping
03:03
people happy. The other interesting thing about this is that rest was, Fielding's dissertation was published in 2000, it's now 2016, there have not been decades yet, you know. This is a guy, I would love to be able to get you all in here, build some software, keep you all in the room for 30 or 40 years while we see how that software evolves in public
03:24
once it's out on the web and then learn from what we've done. But there's a party tonight that I certainly know I want to get to, so we've only got an hour to get through this. But it's important to understand that there are principles in here, there is one spectacularly successful piece of software out there or software system out there which
03:41
embodies many restful principles which is the World Wide Web, and there are many, many others which to a lesser or greater extent embody some of those principles. So just a very quick recap on, as I said, rest is not a framework, it's not a protocol, it's a set of constraints. It's a set of things that they are not necessarily this is how you will build your system,
04:04
think of them as criteria for assessing a system, and like the pirate code, they're more guidelines than actual rules. It doesn't matter, you can build bad systems that are restful, you can build good systems that are not. Rest is not intrinsically going to guarantee that if you follow all of these constraints
04:21
your system is going to be wonderful. But it is important to just understand them because that's what we're here to talk about today. Client-server, rest is about software systems that work on more than one machine or more than one set of machines. The client is something that is receiving information, doing some work, the server is something which stores information and manages
04:42
and coordinates transactions between multiple clients. Stateless, statelessness is important. Statelessness is one of the reasons why restful systems can scale because you're not relying on any individual node to keep track of what all of your users are doing at any point in time. You're effectively offloading the management of state onto the users themselves.
05:04
So if you have 1,000 people hitting your system, you've got 1,000 places to store state. If you've got a million users hitting it, you have a million places to store state. It scales with the scale, if you like. And we're going to talk an awful lot about something called hypermedia as the engine of application state over the course of this talk.
05:23
It needs to be cacheable. When you request a resource, you go to a server, HTTP server. It doesn't have to be HTTP, but I'm going to stick to that for the purpose of today. You ask it for a bit of information, it's going to give you back something. And along with that, it should say, you can keep this. You can keep this till tomorrow, you can keep this till next week,
05:41
you can keep this for the rest of your life, or don't reuse this, never reuse this. And caching means if you have a resource that is not going to change, you can send it to someone once, you want to say, I don't know, the Mona Lisa. Never going to change. Great work of art, it's fixed. Not going to change. You've got a picture of the Mona Lisa, that's good, forever, you know. You want a differently-annotated painting, different resource, different URL.
06:03
That can be cached indefinitely. Your checkout receipt for your ACDC tickets, you don't want to cache that because if you go and buy some more tickets, you're going to need a different receipt. Layered system. This is the principle you want to be able to insert layers between. The simplest thing you've got is a client on the server,
06:21
and when you're debugging on local host, they're actually the same thing. But the client doesn't need to know who's on the other end of the connection. The client can talk to a caching proxy run by your internet provider. Your internet provider can talk to a caching proxy run by the backbone. They can talk to a front-end accelerator that's in front of a varnish box, that's in front of a load balancer,
06:41
that's in front of the server that's got the resources. And REST basically states, you know, this should just work. You can add and remove these layers, and no system should break because you've suddenly put a layer between it and what it was expecting to be the next hop. It should not expect anything to be the next hop. It shouldn't care. Code on demand.
07:01
Canonical example of this is servers that send JavaScript down to clients so that that client can become part of a system, deliver some behavior, which wasn't originally envisaged when that client was built. We all do this every day. Every time you go on Gmail, Facebook, anything that runs JavaScript in a browser, you're exploiting this ability that the server can send some code that says,
07:21
hey, there's a thing you didn't know how to do when you left the factory, but we're gonna tell you how to do it by giving you some JavaScript instructions you can run to do more stuff. Uniform interface is basically you get different representations, you get different resources. You could say, take a customer in an e-commerce system.
07:41
You could have the customer as a XML file. You could have the customer as JSON. You could have the customer as a photograph of the customer. Uniform interface is about making sure that even with these different media types and different ways of interacting with them, there is consistency about how you go about explaining and addressing and creating those operations.
08:01
And the last one, which as I said, we'll be talking a lot, hypermedia as the engine of application state. I'm gonna skip over that now. We're gonna go into that in some depth. The important thing to realize about this, when Roy Fielding wrote his dissertation, he said code on demand was optional, which means all the rest are not.
08:20
If you are not doing all of these other things, then well, if you are doing any of these things in a different way, you might have a very, very good system, but if you're being purist about it, it's not a restful system. And if you explain it as such, you're just gonna end up confusing people. And this is rest, so they're probably confused anyway. But it's helpful to bear this stuff in mind
08:43
and think about it as you're working through the constraints and looking and designing your systems. Right, let me, I'm gonna skip that because it doesn't work from here. Okay, this is where we are gonna start.
09:00
There's been some very good talks on the NDC agenda about principles of building HTTP APIs. I'm gonna assume everyone in the room is comfortable with this level, that you have an HTTP, oop, what did I do? Okay, you have an HTTP GET request. It's through a resource called Profiles.
09:21
It's speaking HTTP 1.1, and you are gonna get back a representation of a person because for the purpose of this demo, we are gonna build a social network. We're gonna build a social network for people who come to NDC and all their friends. It's gonna be called Friend DC, and because it's my social network, I am going to be the first person who registers on it.
09:41
So we built this, and it's brilliant, and I put it up and I start blathering about it on Twitter because we've got this now, Twitter's days are numbered, and somebody else goes, ooh, there's a new social network. I've gotta jump on that because I'm a nerd and there's gonna be a land grab, and I wanna make sure I can get my username. So Mark Rendell posts to Profiles HTTP 1.1,
10:02
and he says, hey, it's me, sign me up, and my system says back to him, okay, 201 created. Well done, we've made you a profile. Here are the details of your profile. This is where you can go to look at it. So it's got a location header in the response, and it says this is your ID that we've created for you, and that's the name that you asked for.
10:22
And because Mark is fairly influential on social media, when he tells everyone who does this, the crowd goes absolutely wild. And by the end of the afternoon, we have two million users signed up to our new system, and then Seb gets out of bed and goes, oh, did I miss something? And he becomes user number 201.
10:41
And at this point, we have a problem because suddenly when we go get slash profiles, we're sending back two million and one user accounts in a single transaction, and the phone rings, and it's the people in your network department who are saying your new API is completely clobbering our bandwidth and hammering our database, what's going on? And that's because every one of these two million users,
11:01
every time they log in, their little client is going, oh, get me the profiles, and they're getting all of the two million records returned in that single transaction. So we have a problem. Now, the obvious solution to this, does anyone remember in the days of MS-DOS when you would type dir and files would go flying past,
11:22
and you were trying to see which one ended in .exe or .com so you could see if there were any games on this disk, and there wouldn't. And then one day you found a DOS manual in Waterstones, and you typed dir slash p, and you had discovered something called resource pagination, which is the idea that instead of getting two million things all in one go, you can get them in little manageable chunks
11:41
one at a time. So we come up with this good idea, why don't we deliver resources in pages? So we'll have page one, page two, page three, page four, which seems like a good idea, and it works. It's not a bad idea at all, but you can picture a client engaging with this
12:02
by going, okay, get profiles two, get profiles page three, get profiles page four, get profiles page five. However, this, it works, but it's not restful. The reason why it's not restful is that we are assuming some knowledge on the part of the client that we are gonna keep using one, two, three, four, five,
12:23
the set of cardinal numbers to indicate pages in our profile collection, and that the client can safely do that for the rest of its existence. Now, potential pitfall of that, let's say we discover somebody is scraping us. They're going one, two, three, so we decide we're gonna change our pagination mechanism. We're gonna use a security token.
12:41
So for any given client, it can get page one, and we say, okay, well, you, if you want page two, you have to come back to us with this token that we've included and use the next one. And this is where we get onto this thing called hypermedia as the engine of application state, because what we need to do, when someone asks for a set of profiles, we need to say, okay, yes, we have some.
13:01
We've got quite a lot. Here's the first bunch, and we then give it some guidance as to how to go about interacting with the rest of that collection. The best example, or the first example of hypermedia resource representation I ever came across was these. Before iPads were invented, we had choose your own adventure game books,
13:21
where you'd start on page one, and it would lay out some wonderful hypothetical scenario that appealed to small children, and then at the bottom of the page, it would give you a set of options. Now this is a pretty good example of a representation of a piece of hypermedia, because what we've got here is something
13:41
which both includes the resource that we've asked for, but it also gives us everything that we need to know in order to interact with and navigate away from that resource. So our state here is we're at this point in our story. We've been at the NDC after party for four hours, we've watched the band, we've had some drinks,
14:01
party's starting to wind down, hotel bar's open, and Greg Young's trying to get people to go and play poker. So that's the state, that's where we're at, and in terms of the states that are available to us, the places we can go from here, we have three options. We can go back to the hotel and go to sleep, we can swing by the hotel bar, or we can go and play cards with Greg. Show of hands for who's gonna go back to the hotel
14:21
and go to sleep. I don't believe a word of it. A show of hands for the hotel bar, and a show of hands for anyone dumb enough to play poker with Greg Young. You're lucky he's not here. This is hypermedia.
14:40
This is very, very simple hypermedia. It's a resource that says here's some stuff, and here are some other places you can go and some things you can do with the stuff that you've just asked us for. Now, I mentioned earlier the World Wide Web is a great example of a RESTful hypermedia system that works very well, and one of the reasons it works well is that HTML includes support
15:03
for references and hypermedia. It's one of the very first things. First web page ever created was actually three or four different web pages that all linked to one another, and when people came along and went this is awesome, we can write pages that link to other pages, and the web just started growing and growing and growing and shows no signs of slowing down.
15:21
Now, the REST's not a protocol. You can do anything you like. You can do REST with Excel spreadsheets over FTP as long as you are obeying the constraints that are included in RESTful architecture design, but the defacto standard for building HTTP APIs based on REST is JSON,
15:41
and JSON does not include hypermedia because JSON was invented when someone went hang on, this JavaScript thing has literal objects and literal arrays, and we can turn our objects into those things and send them over the wire and then we'll have this nice thing that the browsers that are running the code can also get the data they need in a format that they speak,
16:01
which is much nicer than having to build XML parses in JavaScript or Excel spreadsheet parses in JavaScript or any other kind of parses, but in order to use this effectively, there have been several efforts, some of which I like more than others, but it's very much a matter of opinion about how you include hypermedia in your JSON objects,
16:22
so a very quick whirlwind tour. There are one, two, three, four, five of these at the moment, and somebody in here is invariably gonna put their hand up and say there's a sixth. No? First time I gave this talk, there were three of them. So there's JSON-LD, JSON-API, collection plus trace on siren and HAL,
16:40
and they differ in their strengths and weaknesses. JSON-LD's biggest advantage, if you're into that, is that it's a W3 standard. It looks kind of like this. By the way, there are gonna be some slides here where we've got so much code on the page. The point of the slide is to show you there's a lot of code. Do not obsess too much about what the code says.
17:00
The salient points will be highlighted, expanded, and explained. JSON-LD looks kind of like this. So it's a JSON representation. It uses the app character. Do I have a? Yes, I do, cool. So we've got the app character as a prefix indicating, so we've got this defines some context around a certain type of media, and here we have an example of, this is data, this is data,
17:21
and this is a representation or a link to a representation of another resource. Collection JSON. If anyone was in Jimmy Bogard's talk earlier today about using React and .NET to build systems, he was doing some demonstrations of this. It's a more verbose syntax, but it has this basic idea.
17:41
You get back a representation, has a version, says, hey, this is where you find this example. It contains a collection of links and a collection of items. So that's, items is, if you like, the top half of our choose your own adventure page, and the links is the bottom half where it tells you where else we can go from here. JSON API, very similar idea,
18:01
links is a collection at the top, data is a collection, and then, you know, it's JSON. It's a very flexible format, so it's about being disciplined in the way you construct it. Siren is interesting because Siren is the only one of these which, as well as telling you what other resources you can retrieve, you can get, if you like, from here.
18:21
It also tells you what actions are available in terms of updating, modifying, posting the data. So this is a Siren example of, this is an order in some kind of e-commerce system, and down here we have a collection on here. So we've got the data collection at the top, and we have a collection of links at the bottom, but we also have a collection called actions.
18:42
And action is saying, okay, there is an action available from here called add item. If you wanna put a new thing in this shopping basket, you can do it by calling this method, post, to this API with this submission type and including the following fields defining what it is that you are adding to your order.
19:02
Finally, HAL, which is the one I'm gonna use for most of the demonstrations today because it's my personal favorite. HAL is lightweight. HAL does not include the richness of Siren for defining how you interact with objects, but I find HAL readable and relatively lightweight, and it's a good fit for the kind of stuff we're talking about today. So HAL basically gives you a,
19:22
it's a content type is HAL plus JSON, so it's a prefix extension on the beginning of the JSON data type, and bag of links, and then in this instance, this example, a bag of items. Now the key thing to notice here is that the links, we've got a self link. Self is a convention.
19:41
If you've got something, you should always include something that says where this thing came from, and that may in some instances be a canonical URL. If you do something like a search and a search returns a single result, the single result is not gonna say, oh, you get to me by searching for marmalade recipes. The link is gonna include I am recipe number 27, or I am recipe slash marmalade slash 12.
20:03
So that doesn't necessarily have to be the URL that you've got it from, but it's the URL you should use if you want to link to it or get it again next time. Next and last are the key to our hypermedia-driven pagination system here. So what we've got, there is no previous link because we're on page one,
20:20
and we've got a thing here saying to get the next page in this, get a page two, and if you wanna jump right to the end of collection, get a page 214. You can extend these things. Something that we've started doing on systems we're building now is we include a count, an index, and a total. So you can see how many items on this page, how many items in the whole collection, and what's the page number, if you like,
20:43
of the subset that we've returned on this resource. Okay, so we've solved the problem of having two million people coming back from our hypermedia API. Now we need to start doing something with it because at the moment we've just got this great big graph of two million people and it tells you who's friends with who.
21:01
But there's really not much more to do with it. There's nothing to see there. So we're gonna add updates. So now when you go in and you say get profiles number one, which is me, you're gonna get an update that says I'm on my way to Excel to speak at NDC. And you've got a message saying, yay, I got the front seat on the DLR.
21:21
And then you got a message saying, hey everyone, I just met Scott Hanselman, which was cool. But now we have a problem because now when you go to get profile number one, you get me and you get my friend Mark. But Mark's also posted some updates. He's just installed DNX beta seven on his fridge.
21:40
And you get Ben and you get Seb. And Ben has got updates and Seb has got updates. But Ben and Seb and Mark have also got friends. And some of them are friends with each other. So we start getting circular loops. And some of them are friends with other people. And it doesn't take very many degrees of separation on an average social network. If Facebook implemented this as naively
22:00
as we have in this demo, when you looked into Facebook, it would bring back, I don't know what, 20% of the world's population because that's your friends and their friends and their friends' friends and their friends' friends' friends. So now we have another problem. We've solved the whole problem of getting two million people back, but actually we're still getting two million people. It's just we've actually made it worse because now we're getting them back in loops and circles and things.
22:23
And then this starts to happen. We get another phone call. Your API is using all our bandwidth again because every time anyone tries to log in, it's completely swamping the capacity of the systems that are hosting our application. So we are now going to look at doing something
22:41
called resource expansion. The principle behind this is the idea that instead of including all of this stuff in line, when you get profile number one, me, instead of friends and everything being included in line, we're gonna enrich this links collection. We're gonna say, by the way, the little underscore there on the beginning of links,
23:01
that's a howl convention, that's part of the howl spec, avoids collisions with keywords you might want to use. If you're doing a database of golf courses, you're gonna need the word links for something else. So links with an underscore is technical and links with that an underscore is funny shoes and hitting things with sticks. Self is me. This is where you go to get my friends.
23:21
We're not gonna include it in line. We're going to allow you to navigate to it. This is where you can get my photos. This is where you can find all of my updates. So by doing this, you have broken down what was a very, very big connected graph of data into lots of little individual pockets of data.
23:40
So, yeah, get me, cool, all right. Now retrieve friends, yes. Now let's get the updates. Now let's get photos. Oh, look, there's a interesting photo. Let's get that one. Let's get photo one, two, three, four. Oh, look, photo one, two, three, four has some comments associated with it. Let's get those. Okay, let's get photo one, three, four, five.
24:01
Oh, that one's got some comments as well. Let's retrieve those as well. And so on and so on and so on and so on. And then this conversation happens because someone says, I'm using your API, but you've broken it down into such tiny pieces that just to draw one profile page now, I need to make 50 different calls. And someone else says,
24:21
what the hell's happened to our log files? We used to rotate these at the end of the month, and now suddenly we're getting to two o'clock in the afternoon and the servers are running out of disk space because we're logging so many HTTP transactions. What's going on? Now this is where we, as I said, we're gonna get onto something we call resource expansion. The idea of resource expansion, resource expansion is hard.
24:41
It's hard to get right because there are so many different, it's so subjective based on the shape and the structure of the data that you're exposing. But in a nutshell, what it is, when you make that request, you can say, get me profile number one, by the way, expand update. It's kind of the equivalent, say to someone, oh, go into Starbucks
25:02
and pick up a menu, bring me a menu. And they come back with a menu and you're like, okay, I'll have a coffee. Oh, and can you get a dessert menu? And they go and they come back out and you're like, resource expansion is go and get a menu. If there's coffee on the menu, get me a coffee. If there's a dessert menu and it's got muffins on it, get me a muffin. It's basically saying, I want this thing and I would also, if available,
25:21
I would like you to also include these associated resources with it. The convention for doing this is this question mark expand equals, in this example, expand equals updates. So we've got profile coming back. Here's the links collection we talked about. What this says, the updates here matches the updates here, which is how your server knows
25:41
which element of those links we want to expand in line. And your server is gonna go away and say, okay, so we've got the links collection, which is magic. We've got normal data. We've got another collection here now called embedded. And embedded is gonna include a collection called updates. So what we would have got by navigating to this URL, we can now instruct our API to go away
26:02
and bring that back in line. This is a very, very powerful pattern because this means you can say, we have 25 different, very small, very simple resources. The following relationship exists between them. And you, the person who is consuming our API, can decide what do you need to get back in one hit?
26:22
What can you save to later? What are you gonna pull back every time you get to the end of an infinite scrolling page? And so it builds a much more extensible and discoverable platform, which comes back to this idea, we wanna build an API that five, 10, 20 years from now, people are still gonna be able to build innovative new things that talk to it
26:41
without us having to keep reinventing the wheel to cope with their requirements. Okay, so social network, we got updates, but social networks aren't really fun until you start getting all sorts of detailed personal data on there. It's much easier to stalk people when you know how tall they are and what they look like. The data protection people at work
27:01
can go and have a field day with all this stuff. So we're going to enrich our social network profiles by saying, okay, so here's a bunch of stuff. Here's height, here's weight, here's location. Here's status, what am I doing? I'm talking about rest at, no, I'm not, I'm talking about rest at NDC London. There's my hometown, email, website, birth date, all sorts of rich information. Now I wanna go in and I want to update my status.
27:25
So anyway, well, okay, there's me. That's what my profile looks like. I want to change it. I wanna change one detail of it. At the moment, the naive mechanism for doing this in the HTTP spec is HTTP put. So I wanna change my status to,
27:41
I am talking about HTTP put. Bang. But I got a conflict. The reason why I'm getting a conflict, put is a very inelegant mechanism for updating a small piece of a big set of data because put basically says take what you had before, throw it away, replace it with this. It's idempotent because if a put fails
28:03
or we don't know if it succeeded, you can try the same thing again and get the same outcome. So the other nice thing about it, putting a resource, it doesn't matter whether there was one there before. You can put something that didn't exist. You can put something that existed previously. However, any reasonably mature system will have some sort of version management
28:21
change tracking system. And in this instance, what's probably happened is that whilst I've been talking, my phone, which is in my pocket, has detected that I've moved and it's updated my latitude and longitude. And then when I've gone to put this, it's gone, hang on, you got a profile from us, you changed it, you sent it back, but the one you've asked us to update has got a conflict because it no longer matches
28:41
the one that was on the system. So we get a conflict back from this. So these are the conversations that you'll find happening then. It's like, I just wanna update status. Why do I have to put the entire customer profile? You know, by this point, we could be up to 40, 50, 60 data fields. Having to retrieve the whole lot,
29:00
change one detail, put the whole lot, is inelegant, it's inefficient. And the foreign name conflict, the more data you're including in one resource, the more likely it is that someone else will have modified a different piece of it. So there are a couple of strategies that you can use to work around this. One of them is you just keep breaking things down into small objects.
29:20
So instead of status being a field on a social network profile, we'll say, okay, status is a resource. My status here is the thing that lives on the internet in its own right. It's got an addressable URI. You can get it and you can put it. So instead of having to put the whole profile, you can just put the status. This works. The problem with this is you end up with very, very verbose controllers
29:43
or whatever server technology you're using because every little piece of detail on any of your objects ends up with its own implementation. So one way of doing it, it works. It's a good way of getting up and running quickly. There are more elegant ways of doing it. This is where we get onto my favorite and most misunderstood verb
30:01
in the HTTP specification, patch. So HTTP spec says patch method requests a set of changes described in the request be applied to the resource identified. The set of changes is represented in a format called a patch document identified by a media type. This is the example from the specification.
30:22
So that's your verb, it's your resource, protocol version, HTTP headers, description of changes. Any description of changes. There is no guidance from the people who write the RFCs on how it is you should go about doing this. If you want to, you can download your document.
30:43
You can make the change you want. You can save it. You can run the old and the new versions through Unix diff. You can tell your server to understand application slash X Unix diff and you can upload a patch file, an old fashioned Unix patch file. That works. There is nothing, this does not violate any specification.
31:03
You are totally free to do this. There is another way of doing it. There is a specification for something called JSON patch. JSON patch effectively, it exploits the fact that most JSON documents have a hierarchical structure that you can engage with at various levels.
31:21
It digs into this and it says, okay, we're gonna define a set of steps. So what we want you to do is, one, so we're patching this resource. We've got an if match header which is the HTTP protocol header saying, check that the version you're about to clobber is the same version that you think you're clobbering. So if this if match header doesn't match,
31:42
it'll send back a conflict. It'll say you need to get a fresh one, do your changes again. So they're gonna say, okay, so in order, so this is a array, it's a sequence, a ordered collection. We want you to perform the test operation on the path last modified and check that it is still equal to this. And if that passes, we want you to replace the thing at slash status
32:01
with talking about JSON patch at NDC London. This works. This is quite nice. I mean, in practice, patch is one of those things, the incidents where we have implemented it have been sufficiently different. We've just done something that worked in the context. Just purely for the sake of
32:21
showing you how vague the specification is, we could take a server that understands a content type of image slash Nic Cage PNG, and we could upload this as a legitimate representation of a requested change to a resource, and the server could return to it to accept it. And this seems daft,
32:41
but if you imagine building a system where you've got people out in the field who are processing handwritten order forms, say, you could build an API where you actually say, I want you to change of address form, take a photo on your phone, upload it. That's an HTTP patch to a specified resource. The server takes a representation, which is a photograph,
33:01
and it returns accepted. We got it. We'll deal with it. Not right now because we're gonna need to print it out and send it downstairs where someone's gonna go and update the record. But there is nothing in the specification which says that you can't use Nicolas Cage as your representation of your resource changes. Because Nic Cage could beat Chuck Norris.
33:25
We'll talk later. So then this happens. Got your API, version one, great, all fantastic. Everything is smashing and wonderful and working, and someone says, oh yeah, we've been, we just had this name field, but now we found out that for tax reasons,
33:42
we need to be able to do customers by surname, so we need to split the name into four names and surname. And someone else says, oh, we got hometown, but now we wanna start doing geolocation, so actually we need to replace the hometown with an ISO 3166 country code. So you think, hey, no problem, Friday after, yeah, I can fit that in before I go home.
34:00
So you jump into Visual Studio, and you start banging around, and you split name into four name and surname as separate fields, and you turn hometown into a little JavaScript collection here with a city and a country, and the country's a lookup property, and it's brilliant, and just as you're about to leave the office, the phone starts ringing, and people say, hey, you broke the internet. Nothing works, the website stopped working,
34:21
your API's broken our application. This is a breaking change. Breaking changes happen, and it's very interesting thinking about how to deal with the conflict between you're trying to design something, or you're trying to embody a set of principles for building software on a scale of decades,
34:42
but you don't know what you're gonna be doing two or three weeks from now because you're agile. So how do you manage that discrepancy? The easiest thing to do is never break anything, but rest of software design on a scale of decades. Things are gonna change for any kind of, you're gonna have good ideas.
35:00
You're gonna realize some of your good ideas were bad ideas. The law is going to change. You're going to start hosting in a different country where the legal requirements that apply to your infrastructure are different. Now, there's a couple of, there's three different ways of doing what we call API versioning. The point of API versioning is you can say, okay, well, all the stuff that worked yesterday is now gonna talk to this API, and we're gonna ship some new stuff.
35:21
So version two of our API will give you ISO country codes and forename, surname are separate, but version one will still work for some sunset period while we find the people who are relying on it and help them make the transition. Troy Hunt, who's, I believe, currently top of the leaderboard at the Speakers for NDC London, wrote a fantastic blog post last year
35:41
in which he basically explained API versioning done wrong in three different ways. So we're gonna go through those three different ways. The three ways of doing it in a nutshell, one of them is that you put your version two at a different web address to your version one. So you have your first API. This tends to be quite a good one if you didn't realize when you built version one that you were ever gonna need to version it
36:01
because you can have the existing one on slash API and the new one goes on slash V2 slash API. That's one way of doing it. You just deploy a separate set of controllers. Another way of doing it is you can actually say within the negotiation, okay, this client is now accepting only
36:23
version two representation. So we're gonna put a vendor prefix on the beginning of this thing. When you start doing this kind of stuff and you start using how, you can end up with quite a lot of pluses in the prefix on your media type. But what we're basically saying here, we've got a client coming in and our server will go, okay,
36:40
well clients that are speaking version two are gonna include this accept header which says I want version two of these resources because I'm down with the cool kids. I've been upgraded. If this header's not present, then the server can assume that it's a naive client that's still running what has turned out to be version one even though we didn't call it that at the time. The third way of doing it
37:00
is that you just put a header in, a custom header that says, by the way, hello, this is me and I'm speaking version two. The decision about which one of these to use, like an awful lot of things in software engineering, it looks like a technical decision. It actually boils down to a decision
37:21
about how you are modeling the things that happen in your business. So if you have a customer and that customer exists in version one of your API and in version two of your API, are they the same resource? We have this idea. You have real things in the world and then you have a business which models them and then you have some entities in your software
37:42
which reflect that business model and then you have resources and you have representation. So there are lots of different ways of thinking about the same thing. I'm not gonna try and answer that question because I don't understand your business domain and even if I did, these are the kind of things you can talk over well into your second bottle of whiskey. But the point is,
38:01
if you think that version two of Sebastian is a different resource to version one of Sebastian, then they are different resources which belong on different URLs because a URL identifies a resource. If you believe they are the same thing, then it's the equivalent of going, get me a JPEG as opposed to get me a PNG. There are different representations of the same underlying resource.
38:24
And if you're in a hurry and you're being pragmatic and you're fed up of people drinking whiskey and arguing about rest, you can always go, we'll just do this. Lots of software does this. That software works. It does things. People use it. It's fine.
38:41
So these are the, if you like, the three different approaches. Two of them are RESTful and applied properly, represent different scenarios in terms of the interactions that you're doing, the way you're modeling your domain. The third one's not REST but if it works for your business and your domain, that's absolutely fine. Now it's very seldom in this kind of thing
39:00
that you actually come across anything where I believe there is one unambiguously correct answer. Back in the days of boxed product software, Windows 3, 3.1, 3.11, there was quite a common strategy with revision control systems when you ship version one and now you've got a couple of people in one office
39:21
who are waiting for the bug reports to come in so they can do version 1.1 and you've got a couple of people who are gonna start all the new stuff for version two. But of course, version two's gonna start with version one because you don't start from scratch. So you would branch your code base and you'd say, okay, so we're probably, the last time I did this was on subversion which is how long ago this kind of stuff happened.
39:40
But you'd have this idea of having basically two separate heads. One's your version 1.1 bug fix release and one is your version two alpha or beta preview. And if you fix a bug, you fix it in one branch and then you'd use the tools to push that fix across to the other one. API versioning is much more subtle because it has the wonderful property that all of the versions of this are running on the same system
40:01
and unlike boxed software, you're completely within control to expose the code for version two but not let customers actually use it because they haven't paid for it. So in terms of how you manage your version one and version two, the two ways of doing it, do you branch your code base or do you just have one code base that covers all of your versions? You have one code base, you have many versions.
40:22
You keep a single, if it's a service with versions, you have one version of the code for that service. If you're Google, you have a massive, monolithic gigabytes of code in one great big thing but all the different versions, you basically explain versioning as another thing in your code like format types or file paths or security keys.
40:41
You just bake it in. It's very, very easy to do. This is an example based on Troy's blog post. So we've got a root. This is API slash v2. This is with the thing in the URL and this example actually demonstrates how you can do both of them on the same root. So version root here, this is a .NET web API as a custom attribute that is gonna do some negotiation based on the accept header
41:03
and look for clients that are speaking version two properly and this thing here is just gonna expose it to everyone else who's speaking version two improperly by using the structure of the URL and then you write your program here. So it's great.
41:20
It's all fantastic. Everything's nice, working, hypermedia, it's RESTful, no state and then you get one or possibly both of these. So one of them is your boss comes in and says yeah, the CFO's teenage son is on a summer break doing an internship so we've hired him to build us a mobile app. We're gonna need access to all the data and the other thing that happens
41:41
is that someone finds out that linkbait.com which is some dreadful website where you give it your email address and password and it tells you what Batman character you are is asking your users for their login details. So you now have a problem. You have data, confidential data, belongs to your customers in your system
42:01
that they want to share. They want the mobile app and they want to know what Batman character they are but the mechanism by which these things are being shared is keeping you awake at night because of the security flaws that are inherent in it. This by the way is getting off the topic of REST a little bit but all the big REST stuff I've ever worked on has ended up hitting this head on
42:21
so I'm going to include it here. And one of the reasons for this is there is a sort of convention of thinking about IT security in terms of, imagine you walk into a bank. You go into a bank, there's a couple of people who aren't allowed in the bank at all. You've got a guy on the door going, no, we don't think you should come in here, sir. And most people are allowed to go in and get as far as the front desk
42:41
but if you try and jump over the front desk, the alarms are gonna sound and the people who are allowed in the front desk, not all of them are allowed in the back and the people who are allowed in the back, some of them can go downstairs into the vault. So you have this idea of roles. You have the people who are outside on the street, you have the customers, you have the tellers, you have the administrators and then you've got the bank manager.
43:01
And you have this quite old fashioned idea that the bank manager can do anything anywhere in this bank because it's their bank and everyone else has this very global idea of you can see checking accounts for all the customers but you can't go in the vault for any customer. This doesn't work.
43:20
It doesn't work with APIs where you want to share data with one customer but not with another. The reason it doesn't work is that you need to try and get a model which is far more akin to the idea of safe deposit boxes. The protocol that exists to facilitate this with HTTP is something called OAuth2. Now the lovely thing about OAuth2,
43:40
in our example here where we've got linkbait.com, we can create a mechanism where our customers can log into Linkbait and it'll say, oh, you need to go and authenticate yourself with Friend DC in order to get your data. The customer can do that. Linkbait never gets into their account. We just share a token. The way that the protocol itself actually works
44:01
is along these lines. So a request goes, this is our user over here, and they go to Linkbait and Linkbait says to them, hey, go over to us.com and log in there. User bounces over here. We send them a login page. They log in with a username and password. Thing to note, the username and password do not go here.
44:23
Linkbait never sees those credentials because those credentials are sensitive because once they've got the password, that user is in their account and they can see absolutely every detail of anything that they want. We send the user back and we redirect them to Linkbait with a access code.
44:41
It's a one-time code, only reusable once. We've got a record of what it is and we've basically said to them, hey, go back to Linkbait. Tell them to tell us this. Linkbait gives us that code. We check the code. We go back to Linkbait. Yes, we know who this person is. We gave that code out a few minutes ago.
45:00
That's all good. Here is the token that you are now allowed to use to retrieve data for that customer. Linkbait goes to our API, includes that. Now the interesting thing to note here, when properly implemented, that code is not a database ID. That code is an encrypted payload which actually contains identifying information
45:22
about the user. So we don't need a single point of failure. We can have multiple servers which all store, we've got our photography on one system. We've got all our statuses on another system. We've got all of our friends, social network stuff on another one. That token that's been issued can be decrypted and used to dig into that data
45:41
without having to make reference to a central point of authority. We return data to Linkbait. Linkbait draws the page. Linkbait says to the end user, here's your webpage. The interesting thing about this, the other thing to bear in mind about it, there's quite a counterintuitive thing. These tokens that we give out
46:01
at this step in the process, we give out a token Linkbait.com can use but it has an expiry date on it which is very, very short. These tokens typically last five or 10 minutes. Imagine someone goes into the bank, they say, I wanna get the safe deposit box and we have a big freezer where we have molds shaped like keys and they're full of ice. So we give them a key made out of ice.
46:21
It's like, there you go, have a key. It's gonna melt in 10 minutes. There is a limit to how much harm you can do. So we don't need to worry about what happens to that key after that 10 minutes because we know that it's gonna expire. And at the end of 10 minutes, if they wanna go back into the deposit box again, they have to come back to the desk, use what's called a refresh token in OAuth 2 and say, hey, can I get another key please?
46:41
The last one expired, the last one melted. It does open a very small attack window. If someone compromised your system and managed to steal these keys, they could wreak havoc for 10 minutes. But after 10 minutes, they're locked out. And the user can go onto our system when they discover that link beta doing horrible, nefarious things. User can jump onto our system and go,
47:01
don't give link bait anymore. That refresh token has been revoked. If they come back knocking, they can't have it. And at that point, the user goes to link bait and it'll send them back to our site to authenticate themselves again. Very, very powerful, flexible pattern. It includes a facility for something called scopes. So a user can actually say, you've probably seen this. You ever tried a website where it says,
47:21
log in with Twitter? You click it and it says, Twitter would like your name, your email. And then it's like, would like to post tweets for you. And you're like, no, not to get free pub wifi, you're not, I'm sorry. That's not what this is about. So this is a very powerful granular pattern for restricting access to these things. So now I'm gonna just move on and talk a little bit
47:42
about a couple of real world tools that I have found to be absolutely invaluable in building and working with these kinds of APIs. So classic example, you've got something that works on your machine. Big distributed systems, there is a limit to how much work you can do on them when they are running on one laptop. You get some stuff deployed.
48:01
You've got RESTful transactions flowing backwards and forwards between your system and your payment provider. You've got half a dozen different things, different companies, different APIs, different countries all coordinating and interacting and something's not working quite right. And you can't just, well, it's easy to say, you can't just step into the Visual Studio debugger
48:21
and start hacking around with stuff that's live on the internet. But that's exactly what we're gonna do. Grab your phones or anyone who's got their laptop online. I'd like you all to punch in this web address but don't go to it yet.
48:40
Let me tweet that address, hold on a moment. Actually, no, I can't tweet that address from here.
49:20
So hit that address, go to it,
49:21
and you should get a message saying, probably tunnel not found. Tunnel not found.
49:42
Tunnel not found? Grand, right. There is a tool called ngrok which I love dearly. ngrok allows you to set up a temporary secure tunnel from the internet to your local machine. So what I've got here is I'm going to fire up a little, it's a little console executable. Ding, ding, ding, ding, ding.
50:08
So what I'm doing right now is I'm running, ngrok has a website out there somewhere and ngrok is now tunneling traffic that goes to ndc.ngrok.io, which is what's behind that URL redirect thing.
50:22
And it's tunneling that to port 57417 on my local machine. Which means that all of you right now, who are, if you go to that page and refresh it, you are all hitting my local Visual Studio instance. And just to prove it, refresh the page.
51:21
One of you in this room is gonna hit that break point. And it is Wing. Wing, we have a winner. This is a really, really powerful thing because it allows you, for example, if you've got something like a payment provider out there
51:42
that's sending HTTP hooks to your system, you can temporarily point them at this thing and this thing will tunnel the traffic to your local machine. So you can actually take real traffic flowing into a public system and just connect a debugger to it for the purpose of tracking down whatever it is that you're trying to do. ngrok, there's a free version available. If you wanna do cool stuff like have ndc.ngrok.io,
52:03
like I've used for this as a subscription model for it too. Very, very useful tool to know about. The second tool that I am going to demonstrate is a, anyone here familiar with a thing called RunScope? RunScope's a software as a service that started out as an API monitoring thing
52:25
and then they had the bright idea, well, we're logging all this traffic that's being monitored through our service. Why don't we offer the ability to go in and look at and inspect that traffic in real time? So, where's my, there it is.
52:54
So every single request that any of you in the room have sent through has actually, the shortened URL I gave you
53:01
was going to a RunScope proxy on top of an ngrok tunnel on top of my laptop. So I now have a record in here of all of the transactions that have flown through
53:23
and this thing is live real time and it'll store stuff for about 72 hours. If you're being very, very, very agile and occasionally maybe shipping something that hasn't been quite as rigorously tested as you thought it had to be, you can dig yourself out of a hole by going in here and replaying traffic that one of your systems accidentally dropped.
53:40
I of course have never done this and I'm sure none of you ever would either but it's good to know. But this basically gives us a, so we had seven new requests in the time I've been chatting. So, hello, I like your beard. I like your beard too, that's very nice. Who else have we got in here? We've got Trump, we've got Matt speaking English. We've got Script Injection speaking American,
54:01
why am I not surprised? We've got Simon speaking Klingon and English. Polyglot, very good work Simon. And for any one of these requests, so let's dig into, so this is Simon saying hello in Klingon. We can see the request that went through there. So this gives us all the details, user agent, the referrer that's come through,
54:21
the language that's come out the other end and it also gives us the ability, if we want to, to actually go in and replay any one of these transactions. And if I jump back over here and take that debugger off. Right, I'm gonna ask you all nicely not to press refresh for a second. So, put the break point back on there.
54:43
We jump back over, oop, and we're at again and that is, no it's nameless crawl. The joy of doing live demos with audience participation. Right, so let's try replaying that one, Script Injection attack.
55:12
Okay, Visual Studio has decided it's not breaking anymore. But this is an incredibly powerful tool that you can use because it does give you
55:21
a great insight into transactions that are flowing through real systems. You can see JSON representations, you can replay things, you can log them, you can see transactions, you can, so here we can see the same thing being replayed, we can diff it, we got the results coming back from the, ooh, that's why it didn't hit the break point because someone's trying to be, there we go.
55:41
Someone's trying to be nasty. Robin, Robin? No, no, no, I know, I know, I know. You're just really lucky, you hit the break point. Well done, congratulations, I'll buy you a beer later. So yeah, two very, very useful tools, ngrok and RunScope, which have saved me
56:01
probably literally days of hacking and debugging and troubleshooting and trying to get things working. And on that note, I'm going to draw to a close and ask if there are any questions. Yes, so the question is, at the beginning
56:29
when I was showing resource expansion, can you just expand a single level? You can do anything you like because effectively you control the server and you tell the clients what to do, so you can have an expansion
56:41
which says, let me rattle back to the slide. So in this example, you could have expand equals updates, comma, photos, comma, photos.comments, comma,
57:00
photos.comments.friends, it's up to you. It becomes more difficult to build the, I mean this thing ends up as a little chunk of text and somewhere in your system you need to build a parser for it. If you restrict it to a single field, the parser is very, very simple. If you have the idea of hierarchies, you then have to make decisions about, well if someone says expand photos.comments.commenters,
57:25
you have to decide, well does that mean that you just want the commenters or do you want everything that you go past on the way down that stack and do you want to expand sideways as you traverse that hierarchy and so on? Resource expansion is just a sort of HTTP thing.
57:43
HAL is, so resource expansion is a sort of conceptually the idea of if you have something which has related resources, you can fetch them now. HAL is one way of embedding related resources in JavaScript. You can do resource expansion with Siren, you can do resource expansion with JSON collection
58:01
or JSON API, lots of these things. Resource expansion is a company called Stormpath have a very good presentation about resource expansion predominantly, which goes into a lot of detail about this. Atlassian's products, if you use JIRA or Confluence or any of those, they have some pretty good
58:22
resource expansion ideas in their APIs. It's not a standard and it's not a protocol, it's a concept. Yes, it does. It could be, because what you could potentially do
58:44
is to include information in the link representation explaining what expansions are available. So we could have a thing of, you know, links, so self, we could bundle these up. So at the moment, we're using a shorthand which is just name and then a URL. We could say self is actually a collection which includes a URL and then includes
59:02
an array of supported expansion paths on that resource. And that way, we would be getting the benefit of resource expansion without violating the constraint about hypermedia. So once you start thinking about the different concepts and ways that they can fit together, it's possible to do, like I said at the beginning, you can do almost anything restfully.
59:21
In most cases, you can also do it a quick and dirty way that'll solve the problem you've got right now. You need to decide what the time scale of the problem that you're dealing with is. If you think it's gonna be decades, rest is probably a good idea, but we'll need another six years before we know for sure. Any other questions?
59:41
Grand, well, thank you all very much for coming. Please leave feedback in the little box by the door on your way out. Enjoy the rest of the day and I'll see you all at the party this evening. Thank you. Thank you.
Recommendations
Series of 133 media
Series of 96 media
Series of 6 media
Series of 17 media