Taking dynamic web mapping to 1:100000 scale

Video in TIB AV-Portal: Taking dynamic web mapping to 1:100000 scale

Formal Metadata

Taking dynamic web mapping to 1:100000 scale
Title of Series
CC Attribution - NonCommercial - ShareAlike 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license.
Release Date
Production Year
Production Place
Seoul, South Korea

Content Metadata

Subject Area
CartoDB is growing to be one of the biggest mapping platform for the masses, being powered by a fully open-source stack, with PostgreSQL, PostGIS, Mapnik and Leaflet at its core. Our aim is to democratize map and geographical data visualization, making it easy for non-GIS people to create simple maps using the CartoDB Editor, but still keeping all the power and flexibility of the underlying components available to advanced users, with a variety of building blocks ranging from the frontend with CartoDB.js and Torque to the backend with the Map, SQL and Import API, parts of what we call the CartoDB Platform. Serving dozens of millions of map tiles daily has its own set of problems, but when they are being created by hundreds of thousands of users (which have their own database and can alter everything from styling, to the data sources and the SQL queries applied) everything turns out to be a big source of challenges, both development and operationally speaking. This talk will go through our general architecture, some of the decisions we’ve had to take, the things we’ve learned and the problems we’ve had to tackle through the way of getting CartoDB to scale at our level of growth, and how we're giving back to the community what we've discovered though the process.
Enterprise architecture Context awareness Scaling (geometry) Texture mapping Divisor Computer file Connectivity (graph theory) Multiplication sign Debugger 1 (number) Bit Database Parameter (computer programming) Coma Berenices Mereology Quicksort Associative property Computing platform Physical system
Complex (psychology) Greatest element Multiplication sign Mehrplatzsystem View (database) Workstation <Musikinstrument> Source code 1 (number) Client (computing) Mereology Bookmark (World Wide Web) Subset Response time (technology) Different (Kate Ryan album) Core dump Cuboid Information security File format Moment (mathematics) Sampling (statistics) Maxima and minima Bit Entire function Category of being Type theory Wave Befehlsprozessor Website Summierbarkeit Figurate number Metric system Spacetime Geometry Point (geometry) Game controller Server (computing) Service (economics) Open source Observational study Gene cluster Control flow Online help Heat transfer Scalability Number Term (mathematics) Operator (mathematics) Reduction of order Energy level Associative property Computing platform Posterior probability Task (computing) Dependent and independent variables Scaling (geometry) Interface (computing) Forcing (mathematics) Planning Database Line (geometry) Cursor (computers) Limit (category theory) Cartesian coordinate system Symbol table Software Personal digital assistant Query language Statement (computer science) Point cloud Library (computing)
Point (geometry) Pixel 1 (number) Heat transfer Distance Dimensional analysis Number Latent heat Fluid Hypermedia Business model Cuboid Position operator Mapping File format Tesselation Projective plane Coordinate system Sound effect Database Line (geometry) Affine space Flow separation Video game Right angle Quicksort Window Geometry
Point (geometry) Game controller Multiplication sign Set (mathematics) Solid geometry Theory 2 (number) Number Sign (mathematics) Estimator Mathematics Different (Kate Ryan album) Single-precision floating-point format Metropolitan area network Social class Graph (mathematics) File format Keyboard shortcut Volume (thermodynamics) Category of being Befehlsprozessor Process (computing) Software Query language Summierbarkeit Family Spacetime
Point (geometry) Service (economics) Source code Computer simulation Mereology Symbol table Medical imaging Word Network socket Single-precision floating-point format Cuboid Summierbarkeit Table (information)
Area Texture mapping View (database) Resonator Bit Replication (computing) Equivalence relation Data management Estimator Goodness of fit Integrated development environment Different (Kate Ryan album) Business model Quicksort
then setting off but it was also OK so hello everyone i'm a economist in I work at the academy team in the system spots Mrs. the bombings Spain and the date of the stock is like asking upon their to it to a factor 2 1 2 1 a 100 thousand the scale because when we sent awareness and then the abstract for this talk we where our we're is supposed to have my now 100 thousand users but it's it's it's a little bits and now it actually should be 1 that uh that with to 130 thousand there is the scale because with reality of bit my expected will release is always give so yeah in that was wrong to 130 k users created maps using the the Kennedy online platform plus the on the users of or enterprise solutions and brand and it cannot be is scanned it has been kind of file find way to scale and it's quite challenging on times but yeah it's been a wild ride at the end so just being dates the date component the main components of a cavity be a stack I 3 the main 1 is this 1 which is a kind of EHR which is the fancy user interfaces that you can use which is actually made on Robie rails and you can use to it you can use it to manage to create associations simple data I kind the sort of database related things the but yeah it the 1st thing you do you see when you sign up for an account academy . com that's the attorneys like the front ends but the all of this is resting behind whether color-coded from the carried from a which is composed of several parts the 1st of it is them of CPI which is really simple in the sense that you just throw SQL and CSS to lead is in an assembly bilities argument here our guided allele corridor JS which is like this simple simple library and do it dies out of it you can get in you can get the tires seeking that talk size which is the ones that we use for a and maybe even geo-temporal serious associations and work into a lot of mock more from its here in fact next week next week restaurants in
a worse to serve and number traffic times a strong the same Tyler that you can use to battle Mabel your favorites a rendering and and and
this skill API this skill again so as a subset that a city as interface to access the data because when you create academia come to think about that not up this awful it posses quell database that you can embody there to and of all the yeah everything sits on top of those 2 API and was built in the bottom and everything in this a stock is free in the sense that you can set up your own really to give haven't studied planning univariate it's a bit complex because it has a lot of moving parts no because it renders the competition was that we actually do simpler for our own health because someone who has a mind in all of this so yeah you know uh but the Gators cavity which is really them up which have which which is the sum of DAS and is built on top of Martinique dark and a lot of knowledge and non libraries this KPI is moralists of have prompted a was almost made by us in moving the whole which is a non country noticed clearly again if the at the various is being passed well must past posteriors and unknown entire inside the main liberal half appliances Sailorman is steadily JS this space on the flat with some in a pooling to me to be able to previously stations in the sense that you just throw it on a scale was assessed username ABA Kyiv and makes everything for you in the front and quite easily this was the thing about that they can be a and the thing that and where and the most familiar with it that was posted for you and you with with a with have hosted on we're hosting it for at 130 K users and you had to analyze right through the clouds and there with a gun in 3 years we have from from some dedicated servers in fact when we went to be was a study it hide it wasn't the idea that everything lines 3 to 4 they get a service a pattern and if so what we have a scaled the meiji 17 service and easy to in cloud for Gould Gao platform clients mitigating sensors and premises sensors like to many computers running too many things related to that it be but our basic facts something like this will have different servers running the different applications which is like an obvious thing uh with some routing 11 on top of it which is up here which is in units and and was quantum posterior some from the the beginning of this kitten and this is the part where it was gone down or case can this is gonna be just have a bunch of those everywhere and do the run of if if the run-up of Sibiu just keep buying mark but different answers not quite simple the gas and the way that we have got to be set artists they tended to the the scalability aspects of it to be a bit more complex because sometimes you you have to it with your search do have the leg you're feeling a lot of crickets in a box and they give to open and moving the cursor in a the problem and the advantages of the freedom of categories that the user behavior it's unpredictable in the sense that big users can run any to equate with set limits we apply a CPU needs and a statement by amounts allowed then run an indefinite queries they can use any unoptimized dataset if it doesn't make sense for approaches case for sample even just ovular 200 megabytes phi of points and try to get that I 0 0 which is like that that I contained in them all and it will render them will take a lot of time and will render them it doesn't make sense because I'm at that you won't see any of you is in any doubt the ideas of the bonds that will do it so that's suppress wave of freedom and where it wrong itself trying to reduce freedom We're lending to to deal with it and the money to it and this is nothing this is 1 of our beautiful and the things and not with the help of many metrics to try to be shown that we keep track so far off everything I'm in the will to keep everything control and show that these everything Ricketson consistently in blue and that we can scale in terms of their this is traffic and demand using things an idealist again verified where all the typical metrics that we can end up going to a single user and ending July 1 and 2 standard generated to see this is his response of his response times the number of times etc. for any any any user and try to leave if it wanted to help in our not have talking about a if there's a prominent and managed to fixing the line today at this level forgot from for all the all the users and that is a special will do it so it's quantile Indian because we've everything in bring the some self Florence and sometimes they're like a stupid problems for example there was discusses have hundreds of forces for their receipts depends on that on type on the payment get more sources of his and you're paying customers here free it we don't do we keep um we we give quite that it will perform a sustainable than the the uh within the guarantee of your views but some things like and this is this simple and for the fact that backs tasks that we computed for example operating policy analyst ideas just on other CSI symbols EIS operate to Weber's that some simple then again you have to run at 130 thousand there were 4 1 the 100 and 30 k times the not quite and also it quite quickly because sometimes it's security releases that we have to get like out of the there are a of good with some urgency to not keep up a In the operator for various so we use the dataset really just going through each salaries and operator operator Peterborough the break which that is seriously to have to do things like simple things not they're useful for us I wanna make about us at some point would probably did do it like the digital it which is that dust that's uh I go straight which also a a bunch of them eighties that although this time a Almaas quote clusters and runs at an arbitrary explode core and then that's a simple but the problem is that you have to run it always 50 per cent the where 130 K users sometimes that gasses with 500 users and on its and you have to run it quickly but you don't want to stress there is the max so we had like they did you know this is not enough to further back in the overseas high enough thank it way and so it doesn't in government so things alot of control so it was found in simple like this 1 that that have quite of fatalities to have a half cost there has to operate possessing 50 databases in 1 . 2 minutes which is like compared to like 10 hours that the work to at some point it's quite improvement I don't think that we face up on a set a user-defined bottlenecks in in JET yes indeed typical of associates stock that under the obvious ones and you don't expect that to to find them but they're there and that sometimes the observed so I'm trying to be back some use in which they whose styles where abnormally slow lack really slow that 20 seconds slow and this was the did over the ties and a simple little look like in fancy just some and when we started so in the morning was signed dividing we've we figured out that a we had implemented gets in the that map which is that the random for the for dies that's the geometry so it a when he goes to the nearest less for the unity money is the pain in the by that he wants to paint subtly any anything and all of those guys that's available in the the body often the user and this but it was 25 megabytes of and if you some that with a bunch of they do so us coming our way the activity because it was the popular website it we guess it it was like to use both labeled moments when have thing we know we should look for where and why why things are going slow aloof and I would have that is the of these interlocking daily sense only funds on like this word like doesn't want the alright per 2nd of the arteries and like that into the bottleneck where we use the peak of the of the network interface and in making their uses all defocus of that the figures were tougher judges it in an absurd amount of so that about a bit more come from like there the follows geometry transfer format is by the Martinique and pose Yates inside that is the 1 that money money who requests for is WKB well binary when I married a has a thing where all the jet is a very
simple model a simple roof format in the sense that the distance geometry and coordinates but the dimension coordinate ascent in an 8 byte floats OK that sounds like a leading of forward killed then which they in the numbers of how precise where the continents been just it the recession was likely like this number it's like to apply and 17 to 10 to the minus 10 centimeters which his son and outfits it gets to be sort on precision for vast media map of a city which is like do Debbie we proceed to who at so that the final way to solve this from this is that of this problem and the 1st this is still in and I mean we haven't had you having got to be the only ones who have suffered the I guess we're we're all of them we a notice at the knowing and only ever bothered to put the database annotation and affinities are the and I was from this model next and was studied the body and try to find a reasonable and and if a fluid transfer format for geometries where we don't have to turn for the 4 position because we if we only had likes of pixel position that and the position to that line for the wonders of a tile like a to do the noble you never wanted to know there's some is some atomic position for that and was eliminated from for the whole geometry but that is so with an with another feature marketing project in Bogyay still point 2 which is this t 2 box to the which is way faster and then at the geometry was but that's another thing the thing we implemented is it anyway nobody formants which is something we work on this specification with several parties of it there would to some of the
burst it's there are that is the value Katie effective in in some they in most aspects this inevitability and what it does they that but it's much of it in the sense that this is the 1 who's the window the window of rights for the rest of your life I know that
and my format is that the encoding so it doesn't 1 point and the next point will not be the whole did the whole coordinates but this difference with the previous point to a more efficient this especially if you want justice after that Watson on applied to from a big 1 point and then 1 minute is 1 of the like 2 peaks of next and and violent but by way in and by precision bind a there any going by it and I said goal a which a user solid that's required space for that and yet but this is the value of money of 1 of our outsiders on in the job that men and what I think of it like to Denver San of what it was before and this is stupid thing in this really stupid thing like that like the volatility was doing this and there was really on what's being about forever because we have to put where it's a CPU or I O-bound that's on the deposits just brother had to replicate classes just because of the worker but Britain did assignments and estimate if it is a network and yet this stupid thing like that we and that fixing things the so that when a man is a piece of it on those CAS to when do we have for graph publicly quits has that although we do want them to go to the back to back to Martinique and amp prolactin and category of doing notice it which is a good way which is a good thing and know that in the so far brothers was caching passion is very important for us because a and 11 percentage bad because sent for that that the users upload doesn't it change I mean if a newspaper accounts and uploads of dataset set of something that was shown and puts his book but so this website there's a in the most probably this that they will never touch it than our theory than Newton's is good query but it had it can because very well at the same time with have users that users really PI to change things every 2nd every number of seconds and we also have to be able to respond to that change it's gonna affect double-sided here will have to pass by the sum of the problem that you would have to be able to be really good at that humiliation this is the this is my that's what this commercial have would have seen this again uh something there is it because it was right right now is is bodies and then that's it sits on top of the of the time when this lady I rendering and that in this Gallego 7 so it they reduce some of the volume of traffic and we will have a city in which is slightly what's really allows to do this it it's out of control on where where the few which release and the course and the traffic is coming from and everything it all started to embody it can't very quickly like in in a 2nd you can embodied ladies and our custody necessarily make me a I was nearly synchronously so value no of their assimilation is 1 of the hardest thing single signs of a phonemic things so there was no way that we're
doing right now is is that there is built by configured to connect to manage the money if things with says like a hack by books and it's said it in this cold is acclimating endemic ushers comedy when used in certain of the the the it is who works a fair enough but wanted them to do something advanced my they advanced so it is an end uh because in the this was still felt even if it works is still feel as the sum of like a hack to but a begin words but we want to do some more things like million a CDN and we're not connecting this example for parts of something that's democracy so would have 11 services which we're calling the motivation so this up this 4 so a which is going to be connecting from their Istria and from other sources that we want to implement the were weightless hockey things of doing things and actually it's above the red there red historical because it was symbols implement it a TCP socket and some Rockaway's this commonsense erase things I mean they my sister is the 1 that's going to be they responsible of asynchronously interesting mating everywhere with vastly or body she's already not gasses and those who allows us to do some cool things like image in the bounding box of the dataset because right now only allow the whole table in this discipline has changed and cigarettes single simulations send get is something that we don't open socialism and yet that was all those that uh promotes some of the high so wanted argue today so thank you and viewing questions if the to that and the the points in the I couldn't
whom you to go and now I am in that carries its ability let longitude
it is there it is the center be equivalent as delicately as possible and so you can use it for the same thing gets a bit in the partial and where estimates at the edge of environment by transferring collections resonance and dual to the very Katie where do you stop was breasts clustering the using these specific technologies and a we have tried and up manager but some sort of thus the position and in for a personal replication we're running out our own solution to lot automatically a in dynamic replicate so set up the the set obligation automatic failover of things work and from that we're we're all having will with all solution the things we have done have with the red manager and it's not that it's good it's good enough but wanted to be more than model Monte can work in that aspect the that along with the invalidate the cations are you going to also of the view them or do you wait for the use of lost for the area you not we don't actually know it's on them and actually we don't we we don't need progression this it's really needed it but yeah where were fast enough to not having to lecturing being generated by different hobbies maps are generated on the fly we online catches from the use of living things things the the if you got


  326 ms - page object


AV-Portal 3.20.2 (36f6df173ce4850b467c9cb7af359cf1cdaed247)