TileServer: Hosting Map Tiles And MBTiles
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 95 | |
| Author | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/15587 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language | ||
| Production Place | Nottingham |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
00:00
MappingSoftware developerMatching (graph theory)Product (business)MappingServer (computing)TesselationOpen sourceXML
00:21
Moving averageEmulationMach's principlePhysical lawSphereUniform resource locatorServer (computing)Tesselation
00:39
AerodynamicsData storage deviceProcess (computing)Heat transferBefehlsprozessorMappingPoint cloudDistribution (mathematics)Different (Kate Ryan album)Reference dataScaling (geometry)AlgorithmSoftwareServer (computing)Asynchronous Transfer ModeMobile WebComputer-generated imageryPrice indexMetadataGeometryCodeOrder (biology)MappingNumberSound effectProcess (computing)Medical imagingOpen setPresentation of a groupStudent's t-testUtility softwareSlide ruleData storage deviceStructural loadFigurate numberMultiplication signComputerRow (database)Exterior algebraMatching (graph theory)Client (computing)Entropie <Informationstheorie>Server (computing)MathematicsComputer fileContent (media)TesselationPoint cloudState observerProjective planeOvalLevel (video gaming)Cartesian coordinate systemData transmissionCoordinate systemUniform resource locatorDirection (geometry)WebsiteEvent horizonResultantRevision controlLibrary (computing)Parameter (computer programming)Cycle (graph theory)File viewerData structureElectronic visual displayTexture mappingDirectory serviceSoftwareZoom lensWrapper (data mining)Interactive televisionMiniDiscProbability density functionSoftware developerWorld Wide Web ConsortiumMobile appVector spaceFlow separationComputer animation
06:17
ComputerPublic domainMappingAddress spaceComputer filePoint cloudWindows ServerWindows AzureWorld Wide Web ConsortiumCAN busInternet service providerOpen setUser interfaceScripting languageJava appletSoftwareReverse engineeringElectronic mailing listServer (computing)ImplementationDrop (liquid)Plug-in (computing)Android (robot)Client (computing)GeometryCommunications protocolMetadataUser profileClient (computing)Order (biology)Internet service providerComputer programUser interfaceComputer fileArchaeological field surveyProjective planeFile viewerLevel (video gaming)Server (computing)Standard deviationMatching (graph theory)Directory serviceDistribution (mathematics)Special unitary groupNetwork topologyInformationReading (process)MappingTesselationCartesian coordinate systemBuildingMetadata4 (number)PhysicalismRevision controlPlanningLatent heatInterface (computing)Musical ensembleFormal languageEndliche ModelltheorieWeightBitDifferent (Kate Ryan album)ImplementationSpeech synthesisBootingState of matterOpen setMetropolitan area networkDefault (computer science)Open sourceWorld Wide Web ConsortiumOnline service providerCloud computingMobile WebSoftware maintenanceNormal (geometry)Electronic mailing listFreewareZoom lensProduct (business)WebsiteStudent's t-testWindowMathematical optimizationComputer animation
11:54
ImplementationPoint cloudData storage deviceGoogolMappingCodeCache (computing)TesselationMatching (graph theory)Texture mappingBitFormal languageImplementationHypothesisStudent's t-testUser interfaceServer (computing)Projective planeInterface (computing)WritingComputer fileInternet service providerComputer animationXML
12:54
Multiplication signMereologyOrder (biology)Product (business)Server (computing)Dependent and independent variablesTesselation
13:17
File viewerSpecial unitary groupDependent and independent variablesCAN busUniform resource nameOrder (biology)MappingInternet service providerClient (computing)Server (computing)TesselationVisualization (computer graphics)Uniform resource locatorComputer animation
14:05
SharewareMappingMappingServer (computing)Texture mappingCartesian coordinate systemTesselationProcess (computing)Product (business)Projective planeJSONXML
14:29
Google EarthRaster graphicsMetropolitan area networkNewton's law of universal gravitationQuadrilateralComputerPhysical systemSimulationStructural loadInterface (computing)Process (computing)Computer fileStructural loadPhysical systemTesselationCoordinate systemUser interface
15:04
Lie groupoutputMagneto-optical driveZoom lensBefehlsprozessorMedical imagingSet (mathematics)Computer configurationMiniDiscRhombusCartesian coordinate systemTesselationProcess (computing)BitWeightLevel (video gaming)Computer fileZoom lensMultiplicationXML
15:37
Discrete element methodMetropolitan area networkWorld Wide Web ConsortiumServer (computing)MappingPhysical lawMUDSoftware bugFibonacci numberRAIDAreaWebsite1 (number)User interfaceFunctional (mathematics)Computer fileElectronic mailing listWordTesselationMatching (graph theory)Basis <Mathematik>File viewerSingle-precision floating-point formatCartesian coordinate systemProjective planeMultiplicationServer (computing)Computer animation
16:42
Pointer (computer programming)Wide area networkExecution unitFirst-order logicArmValue-added networkInterior (topology)ComputerMappingOpen setAxonometric projectionLine (geometry)Computer fileCommunications protocolDynamical systemEmailSpeech synthesisDatabaseAlgebraUser interfaceMetropolitan area networkMatching (graph theory)Medical imagingBuildingCompilerWater vaporServer (computing)Process (computing)State observerUtility softwareMultiplication signClient (computing)Projective planeCartesian coordinate systemWorld Wide Web ConsortiumFile viewerTesselationMetadataCache (computing)MappingSoftwareRaw image formatElectronic program guideTexture mappingStapeldateiInterface (computing)Direction (geometry)Uniform resource locatorComputer animation
19:25
Gamma functionSphereServer (computing)Matching (graph theory)Roundness (object)AdditionOpen setArchaeological field surveyTesselationExecution unitCASE <Informatik>Reading (process)MiniDiscSpacetimeOrder (biology)Complete metric spaceMobile WebCartesian coordinate systemOpen sourceSphereMappingGoogolAndroid (robot)
20:21
Model-driven engineeringDialectArtificial neural networkWide area networkMappingCartesian coordinate systemMetropolitan area networkMatching (graph theory)Flow separationWorld Wide Web ConsortiumMobile appClient (computing)Computer animation
20:42
Port scannerMetropolitan area networkComputerUniform resource nameWireless Markup LanguageSpecial unitary groupHidden Markov modelAddressing modePointer (computer programming)Convex hullPersonal identification numberAlgebraic varietyCartesian coordinate systemTexture mappingMappingDigital photographySoftware
21:11
Maxima and minimaInformation securityComputer data loggingImage resolutionSoftware bugMIDIDigital photographyComputer virusVisualization (computer graphics)PlastikkarteResultantComputer animation
21:39
Fermat's Last TheoremHydraulic jumpRamsey theoryState of matterEmailWechselseitige InformationHand fanOpen sourceComputerTesselationSoftware bugMappingTraffic reportingServer (computing)Internet service providerLink (knot theory)BitAverageDressing (medical)View (database)Physical systemPole (complex analysis)Pulse (signal processing)Multiplication signGroup actionComputer animation
Transcript: English(auto-generated)
00:01
I'm from a small Swiss company. We work on contracted-based open-source development. We have a couple of products. And what I'm going to show here is a cooperation with NOAA from USA on publishing of the pre-rendered maps. So it's a new open-source tile server project.
00:20
And let's start. As you probably know, whenever you look at a pannable map, in fact, you are seeing tiles, small PNG JPEG files with predefined geographic location. And these tiles are normally rendered dynamically from the data on a server.
00:44
So you install a special software map server, GeoServer, and this server is accessing geotaves or the original raw data and providing them to the client through the WMS server. The more modern approach, faster, is pre-rendering or seeding of these data.
01:07
So the tiles, the small images, are in fact faster available because you store them already somewhere. And with the seeding approach, in fact, you run a software which is grabbing these tiles in advance
01:23
and storing them somewhere on the way between the client and the server. So it's faster than just rendering them dynamically. With the pre-rendered approach, which is now standardized as WNTS, you can also just get the raw data on your desktop computer or on your server
01:46
but during the time of the update and process all these data and create a tile set in advance. And this is the approach on which we are speaking during this presentation.
02:03
With this pre-rendering approach, traditionally, already for years, there is digital-to-tiles utility, which was my student project for Google. And you can also pre-render vector data with TileMill, you probably know, and there are other approaches.
02:22
We will show the map tiler, which is kind of like advanced version of digital-to-tiles. There are several advantages and disadvantages of the pre-rendering approach. You can read it yourself on the slide. The biggest advantage is usually if you have high-quality hosting with agreements
02:46
or if you host your website in the cloud, you can simply put the map data in the same location as your web content, like HTML files and PDF or whatever appears else on the website
03:01
and still have the zoomable map interaction through the JavaScript libraries like OpenLayers without a need to install any other software. And this can be quite a good thing because if you install a software, you have to maintain it. It can stop the run, you have to upgrade the versions.
03:21
If you simply have there the data, it's much more reliable. Also, without the dynamic aspect, it's then even faster than if there is a software in between for the serving because the web servers are normally optimized for really, really fast delivery of the content which is on the disk.
03:44
And this is the advantage of using this pre-rendering approach for sharing the data. Of course, it does not fit to all the data which you have. If you have data which are regularly updated and you need to change what is visible to the people,
04:07
then this approach is probably not the best one because you would have to do the processing and handling a lot of data transfers on the server. And on the other side, if you have something like a base map which you create once
04:22
and then it's on the server for years without any change and you have a large number of visitors accessing the maps, then this pre-rendering approach is really the way to go, especially if you don't have a huge amount of data. Then you can even use really cheap hostings and you save yourself a lot of headaches with the hosting.
04:47
It's also recently launched quite a lot the pre-rendered tiles with the mobile applications where the mobile developers are using it also for offline caching, offline storing on the mobile devices so you have it with you somewhere on the way and as well as hosting the tiles on the server.
05:06
The most straightforward way how to see the tiles is in fact the directory structure where you have the pyramid with the zoom levels and X and Y coordinates for the storing of the tiles. So if you run GL2Tiles or MapTailor, you typically get this directory structure with Z, X and Y and the tile
05:28
and this is directly mapping what you need to share for the open layers or other JavaScript viewers to the world. So if you simply open Google Maps API, it is able to load tiles and display these.
05:47
The alternative to the folder storage is some kind of like package wrapper where the tiles are in fact the same tiles more or less are just put into SQLite or people are using GeoTIFF as well as the storage for the tiles because it's also just a container,
06:05
something where you can put those pre-rendered images internally. There are again advantages, disadvantages. With the package you already need some kind of software which is serving it. On the other side it's really easier to transfer the data from one server to another
06:24
because you don't have to handle potentially millions of small files and that's good. So traditionally the access on the server is done just directly through the Z, X and Y and there are also, like if you need more, if you write a viewer for Z and XY,
06:46
you are kind of like missing the metadata, what is the bounding box, where should I zoom, what are the available zoom levels and this is being solved by TileJSON recently quite well in the web world. Of course there are standards which is solving this problem pretty well
07:05
but TileJSON is more designed for the JavaScript viewers and other viewers are using it as well. These pre-rendered tiles can be hosted practically anywhere
07:21
so you just can copy them on any cloud service. On Amazon S3 you can just put them on whatever server you have in the company. It can be a Windows server. Just copy the files there and if the directory is exposed to the web, suddenly your maps are available as a resource on the website and it's really fast.
07:47
You can even host in free hostings, free PHP hostings or Google Drive, Dropbox are such crazy things. If you are a student and you don't have any budget at all, it's even possible to have the maps online with these free services.
08:02
If you are serious, of course, you will set up something like NGINX server with optimized web server for serving the tiles really fast or use some kind of cloud service or CDN service for distribution of the maps.
08:22
Here we come to the WMTS. The XYZ is nice but we have a standard now and the question is how to expose the maps which are pre-rendered into the tiles through this Open GIS Consortium standard.
08:43
We work with other people on the open source project which is doing exactly this. We were pushed to use PHP, not that it would be the best language from the world, not at all, but it's the easiest one to install.
09:00
The idea of the project is that it's really easy to use. You simply put the tiles into the folder on a web hosting server. You put next to it the PHP files which are in the project and simple .htaccess file on Apache.
09:24
Once these tiles are in the same folder as these PHP files from the project, you get an online service which is officially following the WMTS specifications. This means you can open the maps in traditional GIS desktop clients like UGIS, ArcGIS desktop and others,
09:49
but also reuse the tiles in modern viewers and mobile devices. The first reason why to work on the tiles server was the easy to use usage for normal people
10:05
who are beginners or who don't want to install and host, maintain special server software or who do not have server where they can install it. The other cool stuff which we did is reverse engineering of the S3 WMTS implementation.
10:22
According to my knowledge, this is the first open source project which is really exposing WMTS in a way that S3 products can open it. We had to study the way how S3 implemented WMTS standard. They are always finding a bit different ways than exactly straightforward implementation of the standard.
10:46
We were following their way in providing the WMTS the way that it's compatible with their clients. That's some interesting and cool stuff on this project as well.
11:01
Then we come up with one more idea on how to implement the tiles server. That's the idea that, in fact, very often if you are implementing tiles server, you need some kind of user interface and you have to implement it in this language,
11:21
that language and that language if you have maps hosted on various technologies. We, in fact, come up with the idea of having just an exposure of a list of metadata about the tile grids and do all the user interface in JavaScript.
11:42
The JavaScript is loading this small list of information about the maps, about the tile grids, and building the whole interface which looks like this in the end for previewing the tiles and giving people a way how to use the tiles and how to use the map layers.
12:08
There are now alternative implementations being implemented for the tiles server. The same ideas are now being ported from the PHP language to other languages. There are students in Switzerland working on some master thesis on rewriting the tiles server into Python.
12:28
We are shaping a bit the MapCache project to have the C++ implementation, which gives you very, very fast hosting for the MBTiles as well.
12:42
We have Amazon S3 implementation, so no server software at all, just pushing the tiles to the cloud, but you still get the user interface and the access as it is presented here. In fact, the tiles server PHP has been used in production by NOAA for the Hurricane Sandy response imagery,
13:05
so they replaced ArcIMS server with these pre-rendered tiles and simple PHP, and the reason for doing this was that during these events, there were too many people accessing the server,
13:21
and the server crashed simply because there were too many clients accessing the same data sources, and they replaced the service for providing the maps and tell simply all the people who were keen to use the imagery, just after they were flying with the airplanes, they just put the tiles, processed tiles on the server
13:43
and provided all the others the access to WMTS. Most of the people never recognized that it's not ArcIMS anymore because it was just different URL. They were loading the data into their GIS clients and doing further analysis, overlaying data visualizations and so on.
14:05
So the project has been in fact used on production already last year. If we go through the process of how the tiles server can be used, how easy it is in fact to put the maps in this way on a server,
14:23
the first step is to render the tiles. The easiest is probably to use the map tile, which is a desktop application. You just choose in the first step what kind of tiles you want to render. You drop in the geodata, the geotiff is supported, everything what GDAL can read is supported.
14:42
So MrSID, ECW, geotiffs, a lot of data formats. And you can just specify the coordinate system and georeference is loaded from the file if it's available inside of the file. If not, you can you can specify manually choose the user interface as shown here.
15:05
You get a preview where the image is located and then you can choose if you want to render into folder or into a package. There are advanced options as well for setting zoom levels and so on. Then you wait a bit and it's in a processing process.
15:21
It's processing the data. It's able to combine also multiple files into seamless layers and has another advantage. And once you are done, you end up with these crouch tiles already prepared on your disk and you can preview these in the application. So if you render into a folder, you end up with something like this already prepared viewer.
15:43
If you just open the Google Maps viewer, leaflet viewer, you are getting the zoomable application. You can start to enrich it with other functionality, put markers on top of it or use it as a single layer from multiple layers and so on. But the raw data are prepared.
16:00
The second step is the upload of these data into a hosting anywhere together with the tile server PHP. So you go to the project website, just download the zip file, unpack it and into the same folder where you unpack the file, you just put the folder with the tiles. And once you visit the website, it gives you this user interface already, which is completely JavaScript based with the list of the tiles.
16:31
And by clicking on one of the maps, you get the preview in Google Maps and ready to use viewer. And for multiple viewers, by clicking on the show source, you can just copy and paste
16:48
it and host it somewhere else as HTML is directly loading the layers into the viewer. You can use the maps on the desktop as well.
17:03
So we have step by step guides for various desktop clients which are directly supported. So there is even a way showing step by step where you should click and what you should type so that the maps appear inside of the GIS software.
17:25
All of this without any installation of the software. I mean, it's three PHP files on a hosting and the tiles uploaded. And the performance is really, really good. I think it's comparable to map cache and it's significantly faster than the dynamic approaches for
17:48
doing this, because in fact, the tiles are served as pre-rendered files by the web server. They are not served through PHP. So all the headers are correct.
18:00
The HTTP tells you this tile has modification dates back when you were rendering it. And in case, simply it runs fast because all the HTTP protocol handling is done by the Apache or other web server. And this is highly optimized already.
18:21
And all what the PHP stuff is doing is providing the metadata and the XML files which are loaded once in the beginning by the clients. And then the raw image data are accessed through the web server directly. If you are interested in the project, this is the main URL. If you are interested in the MapTiler, you can get it at maptiler.com.
18:46
There is also, like if you compare the gdalt2 tiles and MapTiler, we worked quite hard on improving the performance. And the whole utility has been rewritten into C, C++.
19:01
And it's also available for using in processing through batches. So as a command line application next to the user interface which you saw. So it's both easy to use but also usable in automated processes. It's able to render very large datasets.
19:24
We were, let's have a look at some examples. We rendered for example the Ordnance Survey open data complete dataset. So we have the spherical Mercatrol tiles pre-rendered. It's about 15 gigabytes of data which you simply put on a server. And you have full replacement of open space API which you control and you are not restricted by anything.
19:47
All what you need is simple hosting with 15 gigabytes of space on the server. And you can zoom in and use Google Maps API, Leaflet, the mobile Android SDK for the maps.
20:03
Either Google one or open source one or iPhone access to the data. And this can be used as a background maps for additional resources. It's in fact used by police in the UK in some of their internal applications.
20:23
Another application is on the old maps together with National Library of Scotland in UK. We made base layer based on seamless historical maps for the whole United Kingdom. And there are several clients, several iPhone native mobile applications and web applications,
20:48
research projects who are using this instead of the Google Maps base map and putting their own historical data on top of this base map. So you see how fast the maps are loading. Another commercial application in UK, just to show a few.
21:05
Google is using the same approach in our software or was using it on the aerial photos of the fires in Australia. The researchers are using it, for example, for the magnetic visualizations of the Earth,
21:24
commercial companies for showing various visualizations. These are the thermometry imagery for the roofs. How are the houses well isolated or not?
21:44
And the old maps. So this is the base links. If you want to use the projects, you are welcome. We also seek contributors, so it's on GitHub. You can forward the tile server and provide patches, report bugs and hopefully you find it useful.
22:04
Thank you for your attention.