We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

COMTiles: a case study of a Cloud optimized tile archive format for deploying Planet-Scale Tilesets in the Cloud

00:00

Formale Metadaten

Titel
COMTiles: a case study of a Cloud optimized tile archive format for deploying Planet-Scale Tilesets in the Cloud
Alternativer Titel
"COMTiles: Cloud-Optimized Format for Planet-Scale Tilesets in the Cloud"
Serientitel
Anzahl der Teile
266
Autor
Lizenz
CC-Namensnennung 3.0 Deutschland:
Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen.
Identifikatoren
Herausgeber
Erscheinungsjahr
Sprache

Inhaltliche Metadaten

Fachgebiet
Genre
Abstract
COMTiles is a novel geospatial data format designed to efficiently store and manage map tiles in a cloud-native environment. It addresses the limitations of existing formats like Mapbox MBTiles and OGC GeoPackage, which are not optimized for cloud deployment. COMTiles introduces a streamable index that stores tile offsets and sizes, enabling efficient retrieval from cloud object storage using HTTP range requests. The format leverages metadata based on the OGC "Two Dimensional Tile Matrix Set" specification to support various tile coordinate systems. One key feature of COMTiles is its simplified deployment workflow. Users can upload a single COMTiles file to a cloud object storage service like AWS S3, eliminating the need for complex tile server setups. This ease of deployment reduces hosting costs significantly, making it accessible even to non-GIS experts. In tests, a planet-scale OpenStreetMap tileset with 90 GB in size was deployed on Cloudflare R2 storage, incurring hosting costs of only $1.35 per month. COMTiles also offers a better user experience by reducing latency and improving tile access efficiency. Its batching approach minimizes the number of HTTP requests, especially when displaying maps in fullscreen mode. In performance tests against another cloud-optimized tile archive solution (PMTiles), COMTiles demonstrated faster decoding, reduced data transfer, and faster initial map load times. While PMTiles had a smaller index size, the additional storage cost of COMTiles proved to be negligible, given the affordability of cloud storage. Overall, COMTiles streamlines large tileset deployment, reduces storage costs, and maintains a user-friendly experience, making it a promising format for managing and deploying map tiles in cloud-native environments.
Graphische BenutzeroberflächeResultanteCASE <Informatik>BeobachtungsstudieComputeranimation
Globale OptimierungResultanteZentrische StreckungCASE <Informatik>SchnittmengeBeobachtungsstudieDateiformatPunktwolkeMultiplikationsoperatorPackprogrammMAPAutomatische DifferentiationTesselation
KoordinatenSystemprogrammierungIndexberechnungVersionsverwaltungWurzel <Mathematik>E-MailMaßstabMetrisches SystemPunktwolkeInformationsspeicherungZahlenbereichApproximationOrdnungsreduktionVektorraumMaßerweiterungTesselationSchwach besetzte MatrixGarbentheorieDatenstrukturCodierungDatenkompressionWahlfreier ZugriffMAPSoftwaretestAutomatische IndexierungOrdnungsreduktionTotal <Mathematik>BitCOMGarbentheorieGruppenoperationInhalt <Mathematik>KoordinatenLeistungsbewertungMereologieMetrisches SystemPhysikalisches SystemRechenschieberResultanteTermWärmeübergangZahlenbereichZentrische StreckungTeilbarkeitRuhmasseCASE <Informatik>DatenfeldGraphische BenutzeroberflächeAdditionCodierung <Programmierung>InformationsspeicherungInteraktives FernsehenBeobachtungsstudieUmwandlungsenthalpieMetadatenDateiformatSoundverarbeitungQuellcodeElektronische PublikationPunktwolkeDifferenteObjekt <Kategorie>KurvenanpassungBitvektorTesselationGraphiktablettMinkowski-MetrikSkeleton <Programmierung>PackprogrammMusterspracheVektorraumsinc-FunktionComputeranimation
OrdnungsreduktionIndexberechnungMaßerweiterungTesselationGarbentheorieVektorraumComputeranimation
Transkript: Englisch(automatisch erzeugt)
Hello everyone, my name is Markus Remmel and today I want to present the results of a case study called COMTILES.
This case study deals with the optimization of cloud-optimized tile archive formats for large or planet-scale tilesets and should significantly reduce the access charges and simplify their workflow for deploying large tilesets in the cloud, known from sleeping map ads.
For deploying large geospatial data in the cloud, different cloud-optimized formats have been invented in recent years. Also, for storing massive tilesets, different cloud-optimized formats have been created, like for example PMTiles, which is the most widely used and leading one in this field.
In this case study, I tried to first improve these cloud-optimized container formats on the following topics shown on this slide by choosing a different file layout. The first requirement or goal is to support different tile-coordinate reference systems. Then also, the transferred amount of data and the number of requests should be minimized, compared to other formats.
Also, every tile in the archive should be requested with at most one additional request. As performance is crucial, fast decoding of the index should be achieved. To fulfill these requirements, a larger size of the total index has to be accepted, based
on the assumption that cloud storage is in general cheap and additional storage costs are negligible. To fulfill the specified requirements, I have experimented with different file layouts and then ended up with the following file structure.
First of all, there is a metadata section, which is based on an OGC specification, which enables the usage of different tile-coordinate systems. For specifying the offset and the size of specific tiles in the data section, an index is used. To minimize the number of transferred data and the number of requests, the index is basically divided into two parts.
The first part is a compressed group pyramid, which contains an overview of the most frequent access tiles, and in general is initially loaded. The second part is called index fragments and is used for lazy loading portions of the index.
Fragments offer random access to portions of the index as they have equal size. To evaluate the efficiency of the layout, I compared COM tiles with PM tiles based on three different metrics, which significantly impacted the user experience and cloud access charges.
For the transferred amounts of data from an object storage and the number of requests to a cloud object storage, two different realistic map interaction patterns have been selected. The first one is more zoom-focused, while the second one is more focused on a panning-based navigation pattern.
In both test cases, COM tiles outperform PM tiles by about factor 3 less downloaded amounts of data. In terms of the number of requests, both approaches showed no significant difference.
Another evaluation was comparing the decoding performance of the index. In that case, COM tiles outperform PM tiles by approximately 19 for a country-scale tileset and by approximately factor 63 for a planet-scale tileset. The performance advantages are basically based on the fact that COM tiles only uses
a lightweight bitpacking encoding, while PM tiles is using an advanced heavy byte compression. This results in a by approximately factor of 10 smaller size of the total index of a PM tiles archive compared to COM tiles. But as said before, since cloud storage costs are cheap, we are assuming that additional about 800 MB on necklace label.
To further reduce the access charges approach where individual tile requests are batched together were evaluated. This is possible since all tiles are ordered on a space filling curve.
In the test, this could reduce the number of requests by approximately 77%. Future work may include further reduction of the index size while continuing to meet the specified requirements. One approach is to specify the presence of an empty fragment and encode this based on a bit vector encoding.
This could reduce the total index size by about 50%. Thank you for your attention and I wish you all a nice FOSFETi conference.