We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Building a scalable tiling service using Amazon API Gateway

00:00

Formal Metadata

Title
Building a scalable tiling service using Amazon API Gateway
Title of Series
Number of Parts
351
Author
License
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language
Production Year2022

Content Metadata

Subject Area
Genre
Abstract
Recent advancements in both raster and vector tile generation mean that TileJSON services can now serve tiles from on-the-fly sources as well as pre-built caches. Currently, Addresscloud uses CloudFront backed by S3 buckets to serve tile caches for its customer-facing applications. Whilst this configuration worked well for pre-built tile caches, it does not readily support on-the-fly generation and is limited by CloudFront's requirement for cookies or signed URLs for private tilesets. In this presentation we will look at the use of Amazon's API Gateway to provide a scalable interface for multiple TileJSON sources. This approach benefits from providing on-the-fly generation tile in a serverless manner and supporting multiple authorization configurations. The presentation will demonstrate the integration of API Gateway with three tile sources: (1) a Lambda function using rio-tiler for on-the-fly generation of raster tiles from a Cloud Optimised GeoTiff. (2) a Lambda function using Amazon Aurora's HTTP API for MVT generation from PostGIS. (3) a proxy interface to a pre-built cache of tile objects stored in an S3 bucket. The presentation will include publication of source code under an open license, which will be available to the community as a reference architecture. This presentation is of interest to anyone developing tiling services in the cloud.
Keywords
202
Thumbnail
1:16:05
226
242
TessellationGateway (telecommunications)BuildingService (economics)TessellationGateway (telecommunications)TesselationPoint cloudAddress spaceComputer animation
Scale (map)Software developerBit rateService-oriented architectureModule (mathematics)Vector spaceGateway (telecommunications)Dependent and independent variablesVelocityReplication (computing)Velocity <Framework, Informatik>LogicTessellationCASE <Informatik>Level (video gaming)Mechanism designAuthenticationFront and back endsGateway (telecommunications)Uniform resource locatorSoftware developerIntegrated development environmentPoint (geometry)ArchitectureService (economics)Run time (program lifecycle phase)Module (mathematics)Demo (music)View (database)TesselationProxy serverFunctional (mathematics)BitLocal ringInformation privacyAddress spaceTouchscreenDiagramBookmark (World Wide Web)Point cloudVelocityCategory of beingTemplate (C++)Projective planeCuboidDialectClient (computing)Formal languageService-oriented architectureCodeSoftwareLink (knot theory)Different (Kate Ryan album)HTTP cookieReplication (computing)Lambda calculusVector spaceComputer animation
Transcript: English(auto-generated)
Thanks everyone for being here. My name is Thomas from Address Cloud. Hands up if you love vector tiles. Nice. Hands up if you use AWS. Cool. You're my target audience. We're going to talk about building a very simple tile service using API Gateway. So why did we build it? Address Cloud, we provide geocoding, property intel, and risk services to insurers.
We work with a network of commercial data partners, and we need to protect their data and our data. Unfortunately, it's not open data in this in this case, and we didn't deliver that data to our customers very quickly. We do thousands and thousands of requests every day.
But more importantly, the reason we built this is because I watched our developers who are here try to do some local development using our vector tiles that we've built that were deployed using CloudFront, and CloudFront has only two methods for authentication. Signed URLs and cookies, and that meant that actually developing locally is a really painful experience.
And so watching these guys do this, it's like, oh man, we need to find a better way to do this. So what we've built, this is hopefully the simplest architecture diagram that you're going to see whilst you're at FOSS4G, but we've got Amazon API Gateway, which is essentially a slippy map tiles proxy that's proxying onto some tiles in a bucket. And the really nice thing about API Gateway is that it's
completely serverless, as is the back end. So we can scale to do many thousands of requests a second, basically, however many requests you would ever need to deal with this. We can cope with that. So you pass in your request, your client passes in your request. API Gateway's there, it's protecting your bucket, it's protecting whatever's going on in that bucket.
You can have that custom authentication mechanism, and it serves the tiles back to the user. I think I've kind of covered most of these points already, but a couple of extra bits that it does that's really nice is you've got inbuilt caching available, so we can be caching those tiles on the fly. There's a live demo if you hit the GitHub that's shown on the screen, you can play around with a little map of Florence.
We've got API Gateways deployed at the edge, so it's really close geographically to your users, so you get quite low latency. And then just to recap, there's no compute layer in this architecture. So we've got, it's like 100% uber serverless, I'm calling it, because there is no, there's no lambda function, there's no code, there's no logic. It's just like API Gateways there, as a service, waiting for these requests and passing them on to S3.
In the spirit of FOSS, we thought, well, other people might be interested in this, or maybe not, but there's a few of you, we've determined here today that might be. We've published this as a Terraform module, we use Terraform for all of our infrastructure at Address Cloud, so you can either grab that module and run it, or you can at least use it as a point of reference
to understand how we've built this architecture and use your favorite cloud SDK. And there's quite a few examples and instructions on GitHub to show how that works. Couple of bonus features that come out of the box of the API Gateway. So API Gateway supports velocity template language, which is an Apache project,
which means that we can actually do really interesting things like rewriting our tile JSON on the fly. So if we've got different environments, we might have a tile set that needs to go to a dev and a staging and a prod environment, for example. Now, we don't know what the URL of that tile set is going to be, where those tiles are going to reside until runtime.
So we can actually inject that into our tile JSON using API Gateway on the fly, and do some pretty fun things there. Also, we can do custom S3 policies. This is all supported by the module already. So we can do S3 replication across different regions around the world and support that multi-regional approach, I think.
Yeah, there's some resources. We've talked about vector tiles with API Gateway in this talk. There's a great tutorial from my colleague Mark online from Phosphogee a few years ago about using CloudFront, if you can use CloudFront or if you've got public data. And we've also done some stuff with Rio Tyler that I've talked about at Phosphogee in the UK. So links are there for those available.
Thanks very much.