Using SQLite to take maps offline on mobile devices

Video in TIB AV-Portal: Using SQLite to take maps offline on mobile devices

Formal Metadata

Using SQLite to take maps offline on mobile devices
Title of Series
Part Number
Number of Parts
CC Attribution 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Release Date

Content Metadata

Subject Area
Nowadays, the internet allows us to access maps on mobile and tablet devices in real-time when we need them. Very often we might be without an internet connection. How to get access to maps in such situations? The answer is to take maps offline. What are technical challenges to get maps offline? Why is SQLite ideal database for offline map storage? How to store all vectors and rasters into a single SQLite database on a mobile device? How to render such maps using hardware with limited capabilities? What are storage and bandwidth requirements? How to solve offline map editing and synchronization? These are only some of the questions this talk will focus on.
Keywords GIS Cloud
Presentation of a group Icosahedron Mereology Product (business)
Group action Texture mapping Source code Zoom lens Explosion Mathematics Befehlsprozessor Forest Vector space Core dump Process (computing) Volumenvisualisierung Physical system Point cloud Graphics tablet Source code Collaborationism Texture mapping Mapping File format Computer file Data storage device Tablet computer Process (computing) Vector space Raster graphics Different (Kate Ryan album) Self-organization Whiteboard Game controller MUD Mapping Service (economics) Open source Computer file Field (computer science) Intranet Read-only memory Software Computer hardware Artistic rendering Energy level Computing platform Mathematical optimization Computer architecture Mobile Web Scaling (geometry) Tesselation Limit (category theory) Computer hardware Computing platform Point cloud Collision 5 (number)
Axiom of choice Complex (psychology) Android (robot) Source code Execution unit Range (statistics) File format Database Price index Client (computing) Shape (magazine) Tracing (software) Medical imaging Vector space Set (mathematics) Damping Volumenvisualisierung Multiplication Physical system Compact space Texture mapping Mapping Channel capacity Relational database Building Structural load Bit Portable communications device Virtual machine Message passing Raster graphics Geometry Open source Sequel Computer file Mobile Web Data storage device Architecture Goodness of fit Computer hardware Subject indexing Energy level Metropolitan area network Form (programming) Computer architecture World Wide Web Consortium Mobile app Default (computer science) Overhead (computing) Standard deviation Key (cryptography) Information Tesselation Server (computing) Weight Code Client (computing) Core dump Limit (category theory) Cartesian coordinate system Web browser Software Query language Function (mathematics) Video game Musical ensemble Window Building INTEGRAL Multiplication sign Instance (computer science) Mereology Geometry Order (biology) Graphical user interface Strategy game Semiconductor memory Network socket Query language Electronic visual display Process (computing) Library (computing) Area Source code Service (economics) Electric generator File format Computer file Complex (psychology) Data storage device Entire function Band matrix Vector space 4 (number) Volumenvisualisierung Configuration space Normal (geometry) Right angle Energy level Freeware Classical physics Functional (mathematics) Server (computing) Statistics Mobile app Mapping Service (economics) Overhead (computing) Link (knot theory) Web browser Binary file Attribute grammar Flow separation Band matrix Software Mathematical optimization Mobile Web Socket-Schnittstelle Raw image format Planning Database Single-precision floating-point format Subject indexing Database normalization Computer hardware Key (cryptography)
Pulse (signal processing) Subject indexing Geometry Virtual reality Table (information) Network topology Price index Database Table (information) Metadata Geometry
Rectangle Group action Multiplication sign Numbering scheme Database Price index E-text Geometry Order (biology) Strategy game Semiconductor memory Vector space Query language Row (database) Search engine (computing) Volumenvisualisierung Extension (kinesiology) Texture mapping Touchscreen Mapping Interior (topology) Sampling (statistics) Attribute grammar Numbering scheme Hand fan Vector space Volumenvisualisierung Reading (process) Resultant Geometry Server (computing) Mapping Sequel Token ring Maxima and minima Lattice (order) Attribute grammar Frequency Goodness of fit Read-only memory Network topology Operator (mathematics) Artistic rendering Energy level Ranking Mathematical optimization Form (programming) Scale (map) Zoom lens Scaling (geometry) Key (cryptography) Tesselation Server (computing) Database Transmitter Subject indexing Word Personal digital assistant Search engine (computing) Musical ensemble Table (information)
Point (geometry) Pixel Mobile Web Source code File format Device driver Price index Online help Insertion loss Mereology Attribute grammar Geometry Strategy game Synchronization Vector space Volumenvisualisierung Source code Texture mapping Mapping Point (geometry) Data storage device Core dump Database Device driver Vector space Raster graphics Revision control Synchronization Data conversion Geometry Spacetime Directed graph
Robot Texture mapping Vector space Chain Attribute grammar
Source code Manufacturing execution system Synchronization Row (database) Euler angles Special unitary group Information security Formal grammar
Raw image format Mapping Texture mapping Mapping Computer file Programmable read-only memory Demo (music) Polygon Electronic mailing list Maxima and minima Mass Data storage device Cartesian coordinate system Special unitary group Hand fan File viewer Endliche Modelltheorie Wide area network Vacuum
Search engine (computing) Maxima and minima Mass
Point (geometry) Content delivery network Complex (psychology) Slide rule Functional (mathematics) Computer file Sequel Parity (mathematics) Multiplication sign Computer Product (business) Semiconductor memory Bayesian network Extension (kinesiology) Metropolitan area network Mathematical optimization Physical system Stability theory Form (programming) Area Electronic data interchange Texture mapping Mapping File format Cellular automaton Correlation and dependence Database Entire function Connected space Degree (graph theory) Subject indexing Vector space Query language Personal digital assistant Right angle Library (computing)
chair welcomed the money this guy is
a lower among coming from the learned have you ever written everything goes well because this is normal winter passed or parts of I personally have did not telling us about that as you I it's used in mobile devices and after that we have presentation of to of the or OpenStreetMap move with the new product announced last we have what was that if you do not parentheses how to do the work it's cross with delivery general you see so close presentation please thank you that everybody and enough
cofounder and CEO G explode I'll be talking about sickle like today and how we can use it to make your maps offline on mobile and tablet devices something about years glove so we're company doing platform for collaborative mapping is like Google Docs for GS where we want to basically enables the entire workflow for collaborating around maps from field collection to create the maps in the cloud and then publishing intranet with others of course as many of the services we are built on being open source so the core technologies we use in our system are reflected you know of course this scale boat yes of course and the collide with special lights that I will actually focus on in today's stock so be patient a lot about maps and speed of maps because we try to raise the bar in the you x when you work with the GIS hermeticism we believe that is that actually Bavaria in adoption of such systems and if you want to scale as a collaborative platform for the entire organizations then of course maps has to be very very fast but and this is why we also built and math engine which is for now coupled with our system and if it was was that can render those maps for a quick clicker quickly as a vector and a roster tiles we had some control formats that used initially to basically down to the data and quickly read it about recently we integrate physical items are actually very satisfied with the performance and I'll show you actually how we did it and what's what's inside something on
that rendering in general so as you all know maps can be sometimes heavy to store protease process and render it can be a complex data that you want to visualize but sometimes the data is not optimal the user the 1 that is producing a map is not making it a lot about zoom levels what features should be shown an Indian etc cetera so that's not optimal of course but also Salters itself has not been focused much on performance and optimization because mostly it was coming from desktop and it is for the enterprise so that he was that you can see something not that it looks OK or its skills very fast and also we have lots of data sets a mapping so we have data sources from files databases vectors thrusters various formats and also services so like WFS WMS something that actually coming from the outside something that can control and that often can be slow can be a bottleneck in actually taking the whole speed of your map of when we're talking about so devices with limited hardware mobile phones and tablets and then of course then it's it's a it's a bigger challenge to really make maps fast regarding the data size you want to upload your mobile device so like I said before so mobile technologies or action mapping technologies have not been designed for mobile forests and then it's very hard to board then if this architecture although technology are using it's not thinking about mobile-phone in such limited it's limited limited resources so some engines are hard to portable well and I'll talk you today how you can actually use it like for that so 1
of the some then guidelines that he was many rection designing a range and so on so that we never actually random data directly from the source so that can be like to external database or WFS something that we cannot control and if you are a real-time map that has been published awards millions of users so if it depends on our servers that maybe doesn't have enough capacity to handle that float then your map will go down it will be very slow 1 common solution how actually solve this problem is you see the map tiles so you can generate the map for offline create all those pyramids and what about health and then you follows the tiles the online adequately put them on a mobile device and is grateful for performance because you're not depending on the original data source but it takes lot of time it takes a lot of time to generate those style so what's a band with lots of resources so it's not actually the the the the great weight how you could make that up portable or collocation puts on mobile phones so we were thinking what about dumping all the data original data into a format that is common so that we can 1 1 form of that you're working with that is very fast and that we could easily basically integration is so that you don't have to do all the formats and and the the blue older drivers but that we only have 1 form of that is very quickly that we could rely on so how did you that formats that can be a database that can be a file or no sequel so database of course is 1 of the oldest choice traces how we organize our maps and data in general tho was risk relative pose yes of course is the standard in free and open source and we're using it a lot so it's a great way how you could organize your 2 maps and data and there is that's a lot of functionality in and you can set it up with good performance especially if you're building a storage services which is more dependent so have lots of distributed servers an application of the API request from the need to have a relational database right so much so that y maps and also good to be rendered from a client server database so it takes that there is an overhead right so what you what we have to do we have to store a map we want to read the map and render it we don't need all this functionality that classical client server database is giving to us so it adds a lot of all of that we can make you don't have to actually use also your depending you need to connect over a socket your database which is also an overhead you have to have all that service up and running it can be locally but are usually you set it up as you as an independent server and it's not portable for mobile so that's actually the key thing why databases are not good for plans related to this is not good for mobile devices you cannot install yes on your iPhone or Android right so it's it's impossible and this is why light is so great right so a simple life is actually the most widely deployed database in the world today so it's basically there billions of copies of the database today in the wild it's found by default In Andrade and I on Mac and Windows than on all browsers sky basically all applications in online of that is that applications or death publications uses it to store the data query the data because it's it's so it's so great these are some of the key benefits or features physical brings you so the key thing that that this database is embeddable so there is no configuration and installation not installing any kind of service that will actually serve you data a database which is the classical way how come you're thinking about it is actually the entire database engine it's it is being built and linked together with so the database engine itself becomes a part of your app the whole and the base of the API of supply for example and the engine itself and its door had it's always like for 400 kilobytes of that that it does not interact so it's like it can be even even down to 250 depending how you configure this makes it very portable so if I create a sequel like database on my desktop or on the server I just think this 1 file which is actually the database with the file I take it on a mobile device and economic data there and then add link with sit politely and I cannot read the data and actually have full that this girl at my hands and it's fast so it's really optimized for speed especially when when you're using it on Linux on Unix where OS itself actually carry catches up makes a lot of optimizations and with your father memories there's a cash but it's very aggressive it's catching so basically when you're dealing with a database with file you're actually dealing with memory so you're actually always writing and reading from memory and see quite itself it takes care that doesn't use much memory at 1 hand it's very optimizing away if you have a low memory to work and there's no complexity go complexity so there is no client service that such architecture from what the couple the database redirect you can do that easily with so what was it like so there are some strategies so you can store maps is a quite a 1 a legal norms is actually that you start tiles so you start vector arrested tiles the whole pyramid like any tiles from the man book-style format which is good for smaller maps more areas it's very easy to render a mobile phone you basically have all the images or predefined at thousand in the database you just read the display them and that's it but it's bad that if you want to store all details if you want to have all the dis the tools of your geometry because the the tiles where you don't have to be you can go on and planet unit right there is a limit to the levels that you can go and if you add more as wealth of course then you have mortality generate and making a bigger bigger filed the database so it's actually limited level Senate philosophy format because you're not getting oxygen also in the engaged in sexual you're not you're getting geometries you're getting the maps that you're not getting it a bit information you cannot query the data what I'll be talking to you about today is actually helped you can have a raw data for the original data is a quite both vectors and rosters but also focused more most vectors that due to time so there some some colleagues built on the to that you can use spatial light and light all talk about them because so with having possibilites map the top as wrong is very always storage and bandwidth requirements because you're dumping raw vectors there is no redundancy you have full put precision of your geometry you have all attributes at your hands on Matt's can be rendered on-the-fly even in that scenarios and it's possible to query the data so you can have the full functionality of spatial database at your hand me that kind of strategy but there can be a challenge how to process and render such maps on limited hardware because if you are dumping 100 thousand hectares million vectors in a form it can be a challenge to render them and I'll talk about some strategies about that as well so I mentioned spatial IT sexually what GIS system pause there and that's what specialized especially used to the polite so so he adds the spatial of functionality of like geometry but also all the as the functions that can user can queries especially the dates I make really basically almost exact queries you can make in in approaches and you can make it on a on a mobile phone also stores geometry in binary format which is very compact it's it's great to to to read like in the 19 c of so that you can in a single pass actually reading terror geometry so it's a format from form of spatial light which is quite similar to the well-known binary form so here's 1 example of like like that of the data from the OpenStreetMap like road network what 1 country it was originally 97 megabytes in size so will uh sh PDB and all other files when you converted to sit like it takes around 93 megabytes so basically more or less the same when read indexes which you want a simple light goes up to 160 so it's a bit larger it indexes that are then us file but not much and then use it bits and that's important for transmitting when you're shipping it on a mobile phone he just 50 megabytes so getting like 93 thousand features so the whole network road network or for 1 country into 250 megabytes that you can ship on a mobile phone so how you can load vector statistical light it's very easy to use due dull that these or G R 1 command to dump the data from that's a shape file or any
other vector into light of good practices that it is able indexes of 1st and you can also do some other optimization skewed make it because I and then once you've done that you can then create an index with the query from a special like that would create actually spatially indexed in your physical edited so
spatially tables especially at some tables in your database role whatsoever in that example is actually the raw data from all my edits geometry of pulse that it creates indexes like
I mentioned so it's an actually an artery a spatial index which is really great to to query on and it ends up some this you're used from produce like geometrical and spatial references and some others like some metadata you could use welcome to query the data very easy you basically select
from of the table but the thing about it but in a few the key you can also use the artery indexed to select from off from certain extent and that extent can be mapped out sense and this is a very very fast operation that you could do so you basically intersect your geometries that these bonds your geometries reaches the during the index intersect at your map extent and that's the sequel can do very fast also see quite has
built in full-text search engine which comes very but it's a lot easier to actually index all the words and tokens in your attributes and you can then use that to group of 4 data initiatives that from phone there are various versions like full-text 3 recently 5 which is still experimental but all we tested it and it's actually quite stable and some things about this is that it has good performance is less memory and you can ordered by rank so that it can actually ranked results and you get the best match and top which is great on the phone because you don't have to then scroll so how to render maps from the 1 strategy would be that you get actually a reading those roles when you get the geometry just put it on a screen with certain Member engine you could do that but you need to do optimization because again the old all the data that was and you can catch so it's a good it's a good idea to catch even on the local device not for transmitting files on as a way to to transmit data on mobile phone but actually when you're on the phone you can catch in it's good practice actually to cash because if you are constantly hitting the database and every time dependent 0 that's not good for performance in in on the phone so the idea is that actually you generating vector tiles on the phone and so you have the full period scheme that doubt can be cast and the challenge is how we can How can actually form the pulsar in that case so can deform the leader of the tiles so that then the phone is renderer but also the server in that case so that can be a challenge but yes it can be a server there some strategies used and this is how I can describe it here so 1st we're actually querying for each style so we quickly are getting the least of all geometries we could possibly rendering it then we can remove features that actually too small to render so fans and out and this I be at the level of C all possible features right I see only what those that have bigger so we remove everything everything else we make another queries equal like to get actually geometries and it gets rid of sample geometries according to this wealth and scale we are currently at and then across the tile of the maps extent and then they're getting the better talent rendered on the device very very quickly and this is a form of music oral formant where we
store that says so that we can of course is always a licensed all the maps of points incoming as as in pixel space as geometry and also you can store attributes and future ideas have everything starting vector talent uh as well
regarding roster so that a couple of strategies you could always use Russell to is built by the same guy will special like I didn't try it and you know as well I tried that debate so you could down the data intuitive for example and then build a gene all just duty if you could do it with some text but I tried it but it didn't work at 1st so I guess I need to be more careful with stripping down all the drivers and parts of it there is synchronisation so 1 way would be that you don't always the entire database on the device which is easy but it be called that you could actually incrementally at the sink it our readers will be the original source you could actually send up inserts and updates and the deletes so that you can sink your mobile phones once you have that once you have done the initial so now show you 1 map with them all with the help like 2 minutes so this is
yes although creating map without the basement and then
let it lets the later so it's like fossil data and the lifestyle is slightly to certain attribute like this 1 so this is a chain 5 can mess it's all vector so the of
the director and I have all possible security 50 thousand
volunteers right and now when I went to down there it's interesting polite
this is how we will do it so this is the
same but it's probably due to the caller the so this is online so it's it's about base but we have
a model that met fewer reach you can log into an axis maps so here that here is a list of my maps this is the same
map is the same manner rendered
on the mobile phone also additional file can then you
can download the map if someone is
preparing than downloading and that then there is the map so it's like 50 megabytes even less so 100 thousand polygons in 50 megawatts and it works
offline so I can share in the former couldn't make a lot them all that but it really works and you can show visual features that you
can search in the full text search engine and you can get all attributes many at that women you're tapping on the feature and you know that it's thank you and
you have resistance the that for
we would they would create a database for each the it is that meaning even you can attach multiple databases in a single it's a connection you know you're opening a file and if you want to reuse them so you have 1 map recent those datasets are then is great but then you have a different matter to showing some datasets and you want to have redundant data the form so if you have 2 maps showing the same datasets you will only not once is 1 of the things that will use if you just a single file could you for example so the file from 3 and and just have the library can remotely with this needs to be of the same thing with the same place it could only remotely thinking or have but not learning from our is the PIC it's like a zip file a package that you prepare everything collecting files that you could pull it from a CDN arrested without a problem and the right things book the then you have heard formation sorry Bayesian points times yeah so because you didn't do that but it's possible to do it so the data is there you could edit the sequel lights using an update on the device and you could just refresh the map so yeah you could just as well is something that you're working on the direct there's some shirts for the questions so could you please just raise hand for pretty much of the world where you seem to alleviate some of these products is like having tried to prove it's like all those 3 the date of you inside of the like tried to make it use yeah yeah actually we did lot memory made metabase that so when we add on the map we're now education like finished basements rosters we can do that as well but now we actually creating vector-based maps so that we can actually see the base map from on the area working with because you know they should not always work in the entire world world or rarely right to have all the details for all the streets so each year if you're working locally could sneak the base map and actually download in the same way just the basement you get all the roads streets and search them as well so yeah we did some aspects and like on the queries you're doing getting very complex and you need to do a lot of optimization which is rejection those degrees have shown next to the 1 we there are treated as if key it you can actually 1st filter out features that you looking for in your metal extent and then you just get all features the 2nd query so you there are no joint stock complexity that basically just pulling the data based on the bonds and you're creating my proposal of before said like vector tires which map of cells and this is like well see and I have you of but in our case styles are generated by the device so you can zoom in as much as you want because we can always generate the next slide all right all right you will not now hi I'm monitoring if you also considered you edges yeah well yeah we definitely looked into it also we are exploring this at the engine is not coupled with the entire system but we're thinking to open it up and then to play with some other formats as well so that they could better the whole thing 1st Lucullus yet another function yeah OK and so on and so on and and I'm sure and this is not just 1 more question that the full text search the showed is that happening on the device so is that going so no it's always fun on the right of what he sees that something proprietary ways something I know it is actually a participle light you build it it's comes built built in that you said pops up a couple queries stability index and you can do that on the surrendered on parity now so the 40 49 megabytes I have shown that actually built in full-text search as well who got thinking that you will come there we have actually was down there is so if you want to have to them all the later have a question feel free to to stop by I'm not agree this is less than 1 man rendering engine used to think that the maps the we built around so it's built basic of signal lights sensors and that's it for everything else is the season was and no 1 USA experience I have it you laptop 5 7 5 the whole thing is created are all system which I will be used as pure light and it was some more this what do you think is


  260 ms - page object


AV-Portal 3.20.2 (36f6df173ce4850b467c9cb7af359cf1cdaed247)