Bestand wählen
Merken

Tileserver on a diet using node.js

Zitierlink des Filmsegments
Embed Code

Automatisierte Medienanalyse

Beta
Erkannte Entitäten
Sprachtranskript
some of and not so all around and my
name is bigger I worked for a Swedish company called him that and the is to real estate search site in Sweden like true lens you are in the United States so I would like to do a story about how we went from using the mapping and service like GeoServer the which are capable of doing mean different things to using a small tiny custom-developed tell server that's on the do those things that we need so therefore the name the song tell servant
so mounts are a vital by part of outside because the cation these very important when you buy a house or an apartment and therefore earlier this year we started the project where we wanted to replace the old maps and to completely rethink the map such experience that we have quite a few challenges along the road in this project and 1 of them is that we have quite a lot of data and we have about our about 500 thousand properties in our database and which had recording which represents properties and this data
is constantly changing the so every every single minute real estate agents send updates to our database so when the house gets old they did to the problem in our database when the new house is laid out for sale they publish it and very often they just a big
attributes of the properties and all these which was server all this data and the quite high traffic load and we serve about 100 million pages that contain map per month the be so while most of the search is that how users can result in just a a couple of hundred search results and in this case it's of course no problem to show to render this data directly in the browser and modern the there are quite a few searches that result in thousands and thousands of search results and in this case even in the fast brothers Google Chrome are not capable of friend data while maintaining good use experience the the so 1 way to solve this
problem is of the city's clustering so clustering is but was less feeding group the nearby properties so instead of for example rendering 5 different markers and you can group them around and just 1 with the digit Python it indicated there but there is more data out there and use as users and this 1 marker would began 5 individual markers so we did a few tests in this solution and it turned out to have quite poor use experience because in many cases to do it with users users would need to zoning all the way down to the last some level in order to see the individual markets and be able to click on them so we thought that we can do better than that in so it's surrendering
the browser is not an option and obviously you can use rest and you can request data styles that are prevented from the server in that way and there is no limit for how much data you can show in a browser because it's just images right so our data is stored in post yes database so he's a switchable was to set up the W service with your server we do that and and pretty soon noticed some of considerable Don sites so 1 of them was an quite slow running time in some cases where lots lots of points were about to be grown entirely it can take you can take up to 10 seconds which of course is not acceptable another the the challenge was and that our search queries are quite complex so when users search they can search for a number of rooms and reading area keywords and it would be quite hard to transfer these such parameters to gesturing following from of filters because it would be assumed that neural so the TAs will become very very long and given if we would somehow find a way to handle such parameters it was still generate a lot of float database because every time a subset of tiles were generated at the base choir and with high traffic will move we have it to be challenging to keep our database up and running during the peak traffic hours and another thing was that the 1 to tell much to be responsive and interactive with things like core effect on the markers and dynamic HokusaI's size depending not only on the zoom level but the amount of data on the map the so we decided to take care of mixed approach the and whenever there are less than 500 points on the map in the viewport and we would render them in a browser because even older browsers are capable of doing now and this is the rumors uses unseen allowed the pants in the map we would make it we requested server then check how many people inside a new view of if the number is more than 500 and we would request data as styles and it's less will continue running in the browser the so let me show you how it
works and outside the so let's let's make a
search for for all property types in Stockholm region the as you can see
this search and give us more than 5 thousand search results and rendering the in the browser would not work in this situation so all this search results are rendered as Stiles assuming we continue to draw points with trials which was the well but and now that we are on the same level where we see in the current view but on a still less than 500 points they become a bigger points and suddenly they become interactive so we get the whole that and we can click on them to show their attributes the if we continue as a man Marcus began
in bigger and so clickable beauty so that's kind
of a prose that we had in mind and in order for
it to you to work well and for that
transition between foreground and styles and rendering in the browser to be the most the we need it out has to be really fast and with other for that we would need to create a custom tell server where we can tweak can't you never tiny
detail and for that we picked the we use chose to do it and not just because without that it's a synchronous nation would fit well in this situation the so in so we'll 1st version and it shows a popular rendering library called map think and Martinique is a superstuff library but there are and there's a protocol not and which gives the bindings uh phonology s so it you can use the map we demonstrate applications there is also a project on GitHub uh with sample code from mapping which is really great starting point if you want to play around with no nothing at so we 1st version with sparsity as data source because it was the easiest way to start and as expected the performance was quite poor because every time a request was making call to the database so is it the base was slowing things down habits readable was to switch to enamored at a strong so
we use red is a lot in our main Application behind behind him and other C so because of that we chose to store search results as since strings in register so
for every unique search was unique such parameters that how users would make we would search stores search results assess screening radius and generate unique such key that we would then included in the tower quest the and that would also would know where to fetch data for each child the which shows since the by the way because and that's the form that node mapping supports and where you can use all types of geometry from the point of plants importance so everything was working quite well but we pretty soon notice some memory issues this is the screens that had lots lots of points in them for example the 2000 mn for this flea searches osmotic allocated quite a lot of memory up to 500 megabytes and this is just a string represents 1 in 1st and we have a vertically lots and lots of different unique searches and the memory consumption of photos so will become unpredictable we have good and so this is where we decided to to skip Martinique and do the rendering on on our own and for that we use no GS library recalled not canvas the so while it was really easy we map to render the test because it was just a matter of providing the data synaptic doodle of job now we have to handle a lot of stuff on our own but it really turned out to be pretty straightforward and it was just matter that a matter of filtering out the points for the bounding box of the specific style translate geographical coordinates of each point to pixel coordinates a the right I can for corresponding provided by and draw it on the canvas the and the results are not to be rude where the great so the data that before too about 500 megabytes memory not upon the 5 and that was the difference is something that we can have a good handle to now that we didn't use but we can them or we could switch from Sicily to Jason because in that way and we would need to
translate since to JavaScript objects because Jason is already just because it's the Zora going from database to be mayor source the really huge improvement
but sometimes some to some tiles that had a lot of points in them the with you quite slow we set up a goal for the most cells to render in between 50 to 100 ms not found that and would a some call profiling by measuring the king is the execution time of different parts of the code and British so notice that the actual drawing on the canvas took time so the response to drawing and and the faster the trousers rendered the so well tiles are on the 256 pixels wide it was obvious that when you draw thousands of points on it a lot of points are drawn on top of each other which is completely meaningless so it is quite a good optimization here and so when I would draw a point and on the canvas the we would remember it's pixel coordinates and when the next point is about to be drawn in the same coordinate it was seem skip it so endows like you see here where it seems to be a thousands and thousands of points only about 10 % of all the points needed to be drawn actually which of course gives a huge performance boost another performance optimisation we
did was to utilize multiple cost In knowledge you can for processes and start as many instances of application of the recipe you variables on mission so like in this example if you have for cost in your machine you can run ran for instances of Tulsa which of course an improved a lot hominid Tulsa can render simultaneously on 1 machine and the it's
always the promise of 2 optimizations would not be possible if there wasn't uh is a way to match the performance and we use the application called gender which is really great for that I encourage you to check it out In this application you can use number of a sentence users number of requests and then can just send thousands and question in that you can see how many tiles your application is capable of producing per 2nd per 2nd and what's the average running time this so now we have good performance and low memory usage we could do some must resort innovations so houses and apartments or more popular types than vacation houses and other so when all of them are about to be grown we want to be shown on the top of each other and as you can see here the houses are the green dots and the belongs are apartments and there's some red and blue ones which are less popular property that so we could really easily before dropping just deciding on the most popular proper diet and render them in the right order so the most popular had run on top of all of the other the not and
as a condition that would it was written at the house so we have an Android and iPhone apps as well and there we have quite a similar performance issues and because even perhaps up up-sell have some for time when when when they need to render a lot of data so we wanted to be to use 0 tell server not as well but has more and more and more smart phones and tablets have high resolution space In order for things look look sharp increase we need to support written process and it was far as well really easy to do and was just a matter of using double sites so images the so when we're done with this with all the performance of innovations and then visual once the end result turned out to be really great uh muscle the tiles rendered under 100 ms the gold we've set up and that considering that during the peak load we have about 30 thousand time tells from I think was a really great enough results so I would say that great and uh surface cracks is really not so hot and the codebase base proto survey just a couple of hundred rows of cold and well in most cases using existing software like GeoServer monster of course works through the Great in some in some cases when you have scenarios like ours where you have need to have full control and into tweaking a tiny detail and integrate with existing infrastructure an existing creating custom fell server can difference that be an option I and I really encourage you to consider it your scenarios thank you very much for listening of legends and presents a
few and it I know when using dose and you click on arrested tiles on a point you know what sort of interaction is easier yeah well you don't have an interaction because the we that we quickly we show files on that in this situations where you have also lots of data and you can't really distinguish points from each other anyway so with that in these situations users are not really interested in in clicking on individual points
so they were and we need to do is I mean that it to see whether . 6 and then would render was in this metadata this whole direction and the
story and the acid thousands of stored in the back of the the memory yeah it's it's it's a race which is an enormous store that
the try any other solutions like ongoing but you want in the yellow 1 it never to be really really fast because we assume as you write to the disk here you the was forms the speaking the might to this one's pretty quiet you have branches have the uh the Tulsa
rot somewhere and get over something quick and try to look
and not yet we're we're definitely considering outsourcing is because we've used a bunch of open source stuff an but as of now it's really has a lot of connections to our infrastructure so it's not so so generalized as we would like it to be and but we're definitely considering
did you did you ever considered clustering the points rather than displaying them all individually and why did you decided display the individual points yeah I mentioned that before we
we did some testing clustering and but the problem was that you the kind of data that we have properties and Verizon density very much so in in in the big cities properties are very close to each other while in the countryside that went far
French others so it's it's really a hard to find a clustering algorithm that would work in the whole country and in some cases it really was an anonymous peer experience where you have to just keep something in just to see the individual task although I can show here say
some some some In some cases you has just 2 points closer and course here there you could render them separate as well so in this case in order to see 1 of them all of those at once you would need to sum in a bit more and we thought it was frustrating experience
this some questions thank you must feel free to a few
the example means of state of the consequences of a long
and
Mapping <Computergraphik>
Dienst <Informatik>
Web Site
Reelle Zahl
Knotenmenge
Computeranimation
Mapping <Computergraphik>
Datensatz
Kategorie <Mathematik>
Datenhaltung
Einfache Genauigkeit
Projektive Ebene
Computeranimation
Softwaretest
Resultante
Kategorie <Mathematik>
Güte der Anpassung
Familie <Mathematik>
Gruppenkeim
Google Chrome
Sichtenkonzept
Computeranimation
Homepage
Übergang
Homepage
Mapping <Computergraphik>
Last
Digitalisierer
Server
Ordnung <Mathematik>
Attributierte Grammatik
Bitmap-Graphik
Web Site
Punkt
Browser
Vektorraum
Zahlenbereich
Computeranimation
Homepage
Datenhaltung
Übergang
Typentheorie
Mixed Reality
Inverser Limes
Bildgebendes Verfahren
Soundverarbeitung
Parametersystem
Filter <Stochastik>
Diskretes System
Datenhaltung
Zwei
Rechenzeit
Abfrage
Zoom
Konfiguration <Informatik>
Mapping <Computergraphik>
Teilmenge
Dienst <Informatik>
Flächeninhalt
Parkettierung
Parametersystem
Schwimmkörper
Server
Kategorie <Mathematik>
Speicherabzug
Resultante
Inklusion <Mathematik>
Punkt
Kategorie <Mathematik>
Browser
Computeranimation
Übergang
Ausdruck <Logik>
Datentyp
Hypercube
Schlussfolgern
Radius
Metropolitan area network
Attributierte Grammatik
Typentheorie
Spieltheorie
Ordnung <Mathematik>
Computeranimation
Homepage
Server
Browser
Gruppenoperation
Server
Computeranimation
Resultante
Schnelltaste
Server
Punkt
Protokoll <Datenverarbeitungssystem>
Datenhaltung
sinc-Funktion
Stichprobe
Versionsverwaltung
Systemaufruf
Kartesische Koordinaten
Quellcode
Schwach besetzte Matrix
ROM <Informatik>
Code
Computeranimation
Mapping <Computergraphik>
Rendering
Stichprobenumfang
Programmbibliothek
Projektive Ebene
Speicher <Informatik>
Zeichenkette
Resultante
Subtraktion
Punkt
Quader
ROM <Informatik>
Räumliche Anordnung
Computeranimation
Bildschirmmaske
Digitale Photographie
Prozess <Informatik>
Datentyp
Translation <Mathematik>
Programmbibliothek
Punkt
Turm <Mathematik>
Speicher <Informatik>
Touchscreen
Softwaretest
Radius
Parametersystem
Datenhaltung
Eindeutigkeit
Digitalfilter
Quellcode
Objekt <Kategorie>
Mapping <Computergraphik>
Generator <Informatik>
Rechter Winkel
Festspeicher
Bildschirmsymbol
Schlüsselverwaltung
Pixel
Zeichenkette
Subtraktion
Server
Punkt
Prozess <Physik>
Minimierung
Zellularer Automat
Kartesische Koordinaten
Computeranimation
Virtuelle Maschine
Multiplikation
Variable
Globale Optimierung
Endogene Variable
Speicherabzug
Drucksondierung
Eindringerkennung
Pixel
Kanal <Bildverarbeitung>
Softwarewerkzeug
Profil <Aerodynamik>
Rechenzeit
Systemaufruf
Parkettierung
Mereologie
Zentraleinheit
Instantiierung
Resultante
Web Site
Prozess <Physik>
Minimierung
Zahlenbereich
Kartesische Koordinaten
Sondierung
Raum-Zeit
Computeranimation
Eins
Monster-Gruppe
Lastprofil
Software
Flächentheorie
Datentyp
Bildgebendes Verfahren
Bildauflösung
App <Programm>
Cracker <Computerkriminalität>
Kategorie <Mathematik>
Güte der Anpassung
Rechenzeit
Ähnlichkeitsgeometrie
Rendering
Konfiguration <Informatik>
Integral
Skalarprodukt
Geschlecht <Mathematik>
Parkettierung
Meter
Festspeicher
Konditionszahl
Tablet PC
Server
Gamecontroller
Ordnung <Mathematik>
Smartphone
Metadaten
Punkt
Parkettierung
Besprechung/Interview
Interaktives Fernsehen
Elektronische Publikation
Quick-Sort
Richtung
Bildschirmmaske
Mini-Disc
Atomarität <Informatik>
Festspeicher
Besprechung/Interview
Verzweigendes Programm
Einfach zusammenhängender Raum
Open Source
Besprechung/Interview
Softwaretest
Punkt
Kategorie <Mathematik>
Datensichtgerät
Besprechung/Interview
Dichte <Physik>
Task
Algorithmus
Intranet
E-Mail
Twitter <Softwareplattform>
Computeranimation
Bit
Punkt
Ordnung <Mathematik>
Computeranimation
Besprechung/Interview
Computeranimation
Aggregatzustand

Metadaten

Formale Metadaten

Titel Tileserver on a diet using node.js
Serientitel FOSS4G 2014 Portland
Autor Tihonov, Igor
Lizenz CC-Namensnennung 3.0 Deutschland:
Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen.
DOI 10.5446/31590
Herausgeber FOSS4G, Open Source Geospatial Foundation (OSGeo)
Erscheinungsjahr 2014
Sprache Englisch
Produzent FOSS4G
Open Source Geospatial Foundation (OSGeo)
Produktionsjahr 2014
Produktionsort Portland, Oregon, United States of America

Inhaltliche Metadaten

Fachgebiet Informatik
Abstract Imagine you were to present large amounts of constantly changing, live data to the users on a web map. Imagine it was on a website with high traffic volume(83 millon page views per month) and high requirement on quick response time. What software would you use to solve this challenge?This presentation will cover the journey that Hemnet, a leading real estate property portal in Sweden, took while remaking a vital part of the website. A journey, during which a number of existing map servers, such as Geoserver, were put on test, but were not fast and flexible enough. A journey, that ended with creating a custom tileserver with technologies like Mapnik to make it as fast and efficient as possible. During the presentation we will cover the challenges we had and how we faced them with different technologies available. We'll take a look at how we did performance tests and how we rolled everything out to the masses.
Schlagwörter tileserver
web maps
mapnik
node.js

Ähnliche Filme

Loading...
Feedback