We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

COMTiles: a case study of a Cloud optimized tile archive format for deploying Planet-Scale Tilesets in the Cloud

00:00

Formal Metadata

Title
COMTiles: a case study of a Cloud optimized tile archive format for deploying Planet-Scale Tilesets in the Cloud
Alternative Title
"COMTiles: Cloud-Optimized Format for Planet-Scale Tilesets in the Cloud"
Title of Series
Number of Parts
266
Author
License
CC Attribution 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language

Content Metadata

Subject Area
Genre
Abstract
COMTiles is a novel geospatial data format designed to efficiently store and manage map tiles in a cloud-native environment. It addresses the limitations of existing formats like Mapbox MBTiles and OGC GeoPackage, which are not optimized for cloud deployment. COMTiles introduces a streamable index that stores tile offsets and sizes, enabling efficient retrieval from cloud object storage using HTTP range requests. The format leverages metadata based on the OGC "Two Dimensional Tile Matrix Set" specification to support various tile coordinate systems. One key feature of COMTiles is its simplified deployment workflow. Users can upload a single COMTiles file to a cloud object storage service like AWS S3, eliminating the need for complex tile server setups. This ease of deployment reduces hosting costs significantly, making it accessible even to non-GIS experts. In tests, a planet-scale OpenStreetMap tileset with 90 GB in size was deployed on Cloudflare R2 storage, incurring hosting costs of only $1.35 per month. COMTiles also offers a better user experience by reducing latency and improving tile access efficiency. Its batching approach minimizes the number of HTTP requests, especially when displaying maps in fullscreen mode. In performance tests against another cloud-optimized tile archive solution (PMTiles), COMTiles demonstrated faster decoding, reduced data transfer, and faster initial map load times. While PMTiles had a smaller index size, the additional storage cost of COMTiles proved to be negligible, given the affordability of cloud storage. Overall, COMTiles streamlines large tileset deployment, reduces storage costs, and maintains a user-friendly experience, making it a promising format for managing and deploying map tiles in cloud-native environments.
Graphical user interfaceCASE <Informatik>ResultantObservational studyComputer animation
Observational studyCASE <Informatik>File formatMultiplication signFile archiverScaling (geometry)Mathematical optimizationResultantSet (mathematics)Automatic differentiationLevel (video gaming)Tessellation
Coordinate systemSystem programmingPrice indexRootRevision controlEmailScale (map)Metric systemData storage deviceGraph (mathematics)NumberApproximationReduction of orderVector spaceTesselationExtension (kinesiology)Sparse matrixSheaf (mathematics)NumberObject (grammar)Random accessTesselationDifferent (Kate Ryan album)Data storage deviceBitvektorMassSubject indexingSchwerpunktsystemMetric systemTessellationSheaf (mathematics)Term (mathematics)DivisorSpacetimeScaling (geometry)Observational studyReduction of orderGraphical user interfaceFile archiverCoordinate systemCurveDirected graphComputer fileSkeleton (computer programming)File formatField (computer science)Performance appraisalCASE <Informatik>Level (video gaming)Sound effectSlide ruleMereologyAdditionData compressionData structureLatent heatResultantSoftware testingPhysical systemBitContent (media)Total S.A.Group actionCodeCodierung <Programmierung>Graphics tabletPattern languageHeat transferMetadataInteractive televisionSinc functionVector spaceComputer animation
Price indexReduction of orderTesselationExtension (kinesiology)Sheaf (mathematics)Vector spaceComputer animation
Transcript: English(auto-generated)
Hello everyone, my name is Markus Remmel and today I want to present the results of a case study called COMTILES.
This case study deals with the optimization of cloud-optimized tile archive formats for large or planet-scale tilesets and should significantly reduce the access charges and simplify their workflow for deploying large tilesets in the cloud, known from sleeping map ads.
For deploying large geospatial data in the cloud, different cloud-optimized formats have been invented in recent years. Also, for storing massive tilesets, different cloud-optimized formats have been created, like for example PMTiles, which is the most widely used and leading one in this field.
In this case study, I tried to first improve these cloud-optimized container formats on the following topics shown on this slide by choosing a different file layout. The first requirement or goal is to support different tile-coordinate reference systems. Then also, the transferred amount of data and the number of requests should be minimized, compared to other formats.
Also, every tile in the archive should be requested with at most one additional request. As performance is crucial, fast decoding of the index should be achieved. To fulfill these requirements, a larger size of the total index has to be accepted, based
on the assumption that cloud storage is in general cheap and additional storage costs are negligible. To fulfill the specified requirements, I have experimented with different file layouts and then ended up with the following file structure.
First of all, there is a metadata section, which is based on an OGC specification, which enables the usage of different tile-coordinate systems. For specifying the offset and the size of specific tiles in the data section, an index is used. To minimize the number of transferred data and the number of requests, the index is basically divided into two parts.
The first part is a compressed group pyramid, which contains an overview of the most frequent access tiles, and in general is initially loaded. The second part is called index fragments and is used for lazy loading portions of the index.
Fragments offer random access to portions of the index as they have equal size. To evaluate the efficiency of the layout, I compared COM tiles with PM tiles based on three different metrics, which significantly impacted the user experience and cloud access charges.
For the transferred amounts of data from an object storage and the number of requests to a cloud object storage, two different realistic map interaction patterns have been selected. The first one is more zoom-focused, while the second one is more focused on a panning-based navigation pattern.
In both test cases, COM tiles outperform PM tiles by about factor 3 less downloaded amounts of data. In terms of the number of requests, both approaches showed no significant difference.
Another evaluation was comparing the decoding performance of the index. In that case, COM tiles outperform PM tiles by approximately 19 for a country-scale tileset and by approximately factor 63 for a planet-scale tileset. The performance advantages are basically based on the fact that COM tiles only uses
a lightweight bitpacking encoding, while PM tiles is using an advanced heavy byte compression. This results in a by approximately factor of 10 smaller size of the total index of a PM tiles archive compared to COM tiles. But as said before, since cloud storage costs are cheap, we are assuming that additional about 800 MB on necklace label.
To further reduce the access charges approach where individual tile requests are batched together were evaluated. This is possible since all tiles are ordered on a space filling curve.
In the test, this could reduce the number of requests by approximately 77%. Future work may include further reduction of the index size while continuing to meet the specified requirements. One approach is to specify the presence of an empty fragment and encode this based on a bit vector encoding.
This could reduce the total index size by about 50%. Thank you for your attention and I wish you all a nice FOSFETi conference.