Building a scalable tiling service using Amazon API Gateway
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 351 | |
| Author | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/69089 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language | ||
| Production Year | 2022 |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
| |
| Keywords |
FOSS4G Firenze 2022155 / 351
1
7
13
22
25
31
33
36
39
41
43
44
46
52
53
55
58
59
60
76
80
93
98
104
108
127
128
133
135
141
142
143
150
151
168
173
176
178
190
196
200
201
202
204
211
219
225
226
236
242
251
258
263
270
284
285
292
00:00
TessellationGateway (telecommunications)BuildingService (economics)TessellationGateway (telecommunications)TesselationPoint cloudAddress spaceComputer animation
00:17
Scale (map)Software developerBit rateService-oriented architectureModule (mathematics)Vector spaceGateway (telecommunications)Dependent and independent variablesVelocityReplication (computing)Velocity <Framework, Informatik>LogicTessellationCASE <Informatik>Level (video gaming)Mechanism designAuthenticationFront and back endsGateway (telecommunications)Uniform resource locatorSoftware developerIntegrated development environmentPoint (geometry)ArchitectureService (economics)Run time (program lifecycle phase)Module (mathematics)Demo (music)View (database)TesselationProxy serverFunctional (mathematics)BitLocal ringInformation privacyAddress spaceTouchscreenDiagramBookmark (World Wide Web)Point cloudVelocityCategory of beingTemplate (C++)Projective planeCuboidDialectClient (computing)Formal languageService-oriented architectureCodeSoftwareLink (knot theory)Different (Kate Ryan album)HTTP cookieReplication (computing)Lambda calculusVector spaceComputer animation
Transcript: English(auto-generated)
00:01
Thanks everyone for being here. My name is Thomas from Address Cloud. Hands up if you love vector tiles. Nice. Hands up if you use AWS. Cool. You're my target audience. We're going to talk about building a very simple tile service using API Gateway. So why did we build it? Address Cloud, we provide geocoding, property intel, and risk services to insurers.
00:24
We work with a network of commercial data partners, and we need to protect their data and our data. Unfortunately, it's not open data in this in this case, and we didn't deliver that data to our customers very quickly. We do thousands and thousands of requests every day.
00:40
But more importantly, the reason we built this is because I watched our developers who are here try to do some local development using our vector tiles that we've built that were deployed using CloudFront, and CloudFront has only two methods for authentication. Signed URLs and cookies, and that meant that actually developing locally is a really painful experience.
01:01
And so watching these guys do this, it's like, oh man, we need to find a better way to do this. So what we've built, this is hopefully the simplest architecture diagram that you're going to see whilst you're at FOSS4G, but we've got Amazon API Gateway, which is essentially a slippy map tiles proxy that's proxying onto some tiles in a bucket. And the really nice thing about API Gateway is that it's
01:23
completely serverless, as is the back end. So we can scale to do many thousands of requests a second, basically, however many requests you would ever need to deal with this. We can cope with that. So you pass in your request, your client passes in your request. API Gateway's there, it's protecting your bucket, it's protecting whatever's going on in that bucket.
01:41
You can have that custom authentication mechanism, and it serves the tiles back to the user. I think I've kind of covered most of these points already, but a couple of extra bits that it does that's really nice is you've got inbuilt caching available, so we can be caching those tiles on the fly. There's a live demo if you hit the GitHub that's shown on the screen, you can play around with a little map of Florence.
02:02
We've got API Gateways deployed at the edge, so it's really close geographically to your users, so you get quite low latency. And then just to recap, there's no compute layer in this architecture. So we've got, it's like 100% uber serverless, I'm calling it, because there is no, there's no lambda function, there's no code, there's no logic. It's just like API Gateways there, as a service, waiting for these requests and passing them on to S3.
02:25
In the spirit of FOSS, we thought, well, other people might be interested in this, or maybe not, but there's a few of you, we've determined here today that might be. We've published this as a Terraform module, we use Terraform for all of our infrastructure at Address Cloud, so you can either grab that module and run it, or you can at least use it as a point of reference
02:43
to understand how we've built this architecture and use your favorite cloud SDK. And there's quite a few examples and instructions on GitHub to show how that works. Couple of bonus features that come out of the box of the API Gateway. So API Gateway supports velocity template language, which is an Apache project,
03:02
which means that we can actually do really interesting things like rewriting our tile JSON on the fly. So if we've got different environments, we might have a tile set that needs to go to a dev and a staging and a prod environment, for example. Now, we don't know what the URL of that tile set is going to be, where those tiles are going to reside until runtime.
03:21
So we can actually inject that into our tile JSON using API Gateway on the fly, and do some pretty fun things there. Also, we can do custom S3 policies. This is all supported by the module already. So we can do S3 replication across different regions around the world and support that multi-regional approach, I think.
03:41
Yeah, there's some resources. We've talked about vector tiles with API Gateway in this talk. There's a great tutorial from my colleague Mark online from Phosphogee a few years ago about using CloudFront, if you can use CloudFront or if you've got public data. And we've also done some stuff with Rio Tyler that I've talked about at Phosphogee in the UK. So links are there for those available.
04:02
Thanks very much.