We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Speeding up the RapiD map editor with WebGL and PixiJS

00:00

Formal Metadata

Title
Speeding up the RapiD map editor with WebGL and PixiJS
Title of Series
Number of Parts
351
Author
Contributors
License
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language
Production Year2022

Content Metadata

Subject Area
Genre
Abstract
RapiD is an advanced Open Source map editor for OpenStreetMap built by the MapWithAI team at Meta. RapiD makes it simple to work with openly licensed geodata and AI-detected road, building, and landform features. For years the RapiD editor was based on a SVG rendering engine built with D3.js. As our users map the world in increasing levels of detail, and as more open data sources become available, our rendering tech has struggled to keep up with the massive amounts of data that we’re asking it to display in a browser-based JavaScript application. Our team recently converted this legacy rendering engine to instead use WebGL technology by leveraging the popular Open Source PixiJS game engine. The conversion from SVG to WebGL yielded a considerable performance boost, and the new WebGL-based renderer is up to the task of working with massive world-scale datasets and handling the increasing data density of OpenStreetMap. In this talk we share our progress on bringing new datasets into RapiD, tell the story of how we built a modern map editor on top of an Open Source game engine, and share our roadmap for the future of mapping.
Keywords
202
Thumbnail
1:16:05
226
242
BitText editorRevision controlBuildingSet (mathematics)Self-organizationMeta elementLevel (video gaming)MereologyFront and back endsINTEGRALShape (magazine)Open setCanonical ensembleComputer animationPanel painting
Type theoryPoint (geometry)TouchscreenAddress spaceLevel (video gaming)BuildingVector spaceMultiplication signVertex (graph theory)PredictabilityAreaQuicksortComputer animation
Reading (process)Address spaceDisintegrationBuildingDifferent (Kate Ryan album)Point (geometry)Shape (magazine)Level (video gaming)Set (mathematics)Order (biology)Multiplication signFocus (optics)Computer animation
Personal area networkFrame problemBitZoom lensDemo (music)VolumenvisualisierungRight angle
Object-oriented programmingDemo (music)Lattice (order)Web pageDisk read-and-write headLevel (video gaming)DemosceneFrame problem2 (number)
Software frameworkSlide ruleDemosceneVideo gameFlash memoryOpen sourceVolumenvisualisierungFitness functionWeb browserMultiplication sign
Web browserLiniengruppeLocal GroupOpen sourceBuildingPrototypePower (physics)BuildingElectronic mailing listVisualization (computer graphics)Event horizonMappingWeb browserOpen setCodeRectangleSlide ruleMathematicsNormal (geometry)Software frameworkLevel (video gaming)Object modelQuicksortDynamical systemClassical physicsObject (grammar)Computer animation
Line (geometry)Cartesian coordinate systemWeb browserMultiplication signMathematicsWebsiteContent (media)Web applicationProgram flowchart
Graph (mathematics)Level (video gaming)Default (computer science)Data bufferTexture mappingElement (mathematics)Open sourceTimestampLine (geometry)Point (geometry)BuildingGame theoryFrame problemScene graphBuffer solutionCodeTerm (mathematics)Web browserDiscrete groupDemoscenePixelLevel (video gaming)Semiconductor memoryData conversionGraph coloringMobile appMultiplication signVolumenvisualisierungSystem callStapeldateiTranslation (relic)Drop (liquid)Video gameWordView (database)Mountain passLoop (music)VideoconferencingComputer animation
Multiplication signGraph (mathematics)Pairwise comparisonPoint (geometry)Task (computing)Different (Kate Ryan album)CASE <Informatik>Graphical user interfaceDemosceneFrame problemComputer animation
Product requirements documentCovering space40 (number)Google Street ViewArtistic renderingSpacetimeRight angleBuildingNear-ringLevel (video gaming)DemosceneMereologyNetwork topologyOpen setView (database)Computer animation
State transition systemRaster graphicsCone penetration testProgrammable logic arrayBit rateMetra potential methodSlide ruleRevision controlBitZoom lensComputer animation
Software as a serviceReading (process)Graphics processing unitBit error rateState of matterAbsolute valueThermal expansionSima (architecture)Slide ruleGraphical user interfaceFrame problemCodeRight angleComputer animation
Event horizonTask (computing)Function (mathematics)System callDemosceneElement (mathematics)Graphical user interfaceProfil (magazine)System callMobile appLevel (video gaming)Rule of inferenceRight angleTracing (software)Stack (abstract data type)Clique-widthMultiplication signSemiconductor memoryFrame problemScene graph2 (number)Personal area networkLine (geometry)VolumenvisualisierungGraph coloringComputer animation
Web browserDemosceneCodeBitValidity (statistics)Multiplication signGraph (mathematics)Process (computing)Graphical user interfaceSpacetimeObject-oriented programmingMobile appGraphics processing unitUser interfaceVolumenvisualisierungTask (computing)Goodness of fitGraph (mathematics)
Cross-site scriptingVolumenvisualisierungDemosceneGame controllerMathematicsMemory managementComputer animation
Personal identification numberComputer animation
MappingInflection pointComputer animation
Line (geometry)Dot productMedical imagingAlgorithmView (database)Clique-widthDebuggerProcess (computing)MultiplicationRight angleLimit (category theory)Computer animation
BuildingDemosceneMappingFitness functionAddress spaceOpen setComputer animation
ClefRectangleSweep line algorithmComputer reservations systemWebsiteSlide ruleShape (magazine)QuicksortBuildingRectangleComputer animation
RectangleOperations support systemPolygonTexture mappingFreewareGoodness of fitSquare numberComputer animation
Codierung <Programmierung>RectanglePolygonZoom lensSpeech synthesisElektronisches MarketingZoom lensTheory of relativityQuicksortPole (complex analysis)Projective planeDecision theoryNichtlineares GleichungssystemWeb 2.0Computer animation
Zoom lensPairwise comparisonBuildingZoom lensComputer animation
Zoom lensMaxima and minimaZoom lensDecision theoryPoint (geometry)Computer animation
QuicksortBuildingLevel (video gaming)VideoconferencingPoint (geometry)Shader <Informatik>Computer animation
FingerprintRewritingSystem callGroup actionSemiconductor memoryTexture mappingMultiplication signProgrammer (hardware)Computer scienceComputer animation
BitTouch typingVolumenvisualisierungZoom lensComputer animation
Bit rateType theoryCore dumpComputing platformFunction (mathematics)WaveSoftware testingLocal GroupRight angleZoom lensLevel (video gaming)Software bugComputing platformFrame problemAlpha (investment)Web browserCodeControl flowSystem callInternational Date LineGroup actionTesselationSoftware testingTwitterOpen sourceFacebookForm (programming)MultilaterationComputer animationSource codeJSON
Meta elementDaylight saving timeComputer animation
Transcript: English(auto-generated)
that works on Rapid. So if you were here for the previous talk, you saw a little bit of our editor. So we're gonna introduce you to a few other things that we've been up to lately, namely the v2 version of Rapid that's gonna be coming out later this year. So you had a pretty good introduction to what Rapid is in the previous block,
but for the folks who haven't been here, it's basically the canonical OSM editor known as ID, but with some extra datasets added on top of it. So datasets from all kinds of non-governmental organizations, some government entities, and as well as global datasets from meta.
Even Google actually submitted 50 million buildings across Africa later last year. And the data is served up and conflated by a meta-hosted backend, which is part of our kind of value add that we're not just shipping you shapes that already exist on the map. We very conveniently hide those for you so you don't have to be looking at two versions
of the same building. And we also have the integration with Esri's ArcGIS open data API. So really quickly, this is what you'll get if you look at Rapid, which is pretty identical to what you get at ID. This is just the OSM data that you see drawn in handy vectors on the screen.
As you move into Rapid, you will actually get these very nice AI predictions from Microsoft that are pretty high-fi and they're getting better all of the time. So if you saw this data maybe a few years ago when it first came out, I think you'll be pleasantly surprised by the level of improvement that Microsoft has put in. And finally, we're also looking at an area where there are Esri address points.
So right away, if you're a new mapper, this is gonna save you tons of time. There's probably a few hundred vertices in all these buildings that you would have to sort of laboriously add and then square and then add the address points, type a whole bunch of data in, where all of that is pretty much just done for you. And we also have a dataset picker.
Chris just covered this, so I don't wanna belabor the point, but there's a lot of data in here, I think on the order of hundreds of millions of different points and shapes for you to add to the map and potentially edit. And there's more of these being added all of the time. So what else we have been up to is really the focus of today's talk, and that is something new,
which is Rapid v2 that Brian and I have been rewriting all of this year to use WebGL as opposed to our older rendering stack. If you were to say use Rapid 1.0 over Florence today, you would have to zoom very far in before you got a very decent performance.
Right now, if you were at zoom 16, you're looking at like two frames a second as you pan the map around. So we have a little bit of a demo here. On the right, you see Old Rapid just clicking the map and attempting to pan. And this is not a particularly busy scene, but it still takes two frames a second. After a few performance improvements
and converting the engine over to WebGL, we get some pretty substantive performance gains and we aren't done yet. So we're pretty happy about this. So I'll hand things over to my colleague and he'll take you through how we did all this. All right, thank you. All right, so like the slide says, right, Rapid is slow.
We know that there are faster technologies for rendering complex scenes in the web browser. And this is something that we've kind of wanted to improve for a long time. So last year, we really started exploring this idea of replacing the renderer with something better.
So we looked into a whole bunch of WebGL rendering frameworks and we settled on one that's called Pixy.js. Pixy is a popular open source framework, which is really geared towards doing 2D graphics and video games in the browser. So it's kind of a replacement for Flash or SVG, which makes it a really good fit for what we're doing.
And I gotta say, it's been like a real joy to work with. So let's start, we'll talk about the old code. This is the code that we replaced. We used D3.js, which I think a lot of you know this, but anyway, D3 is a visualization engine, but it's very dependent on binding your data to the browser's DOM.
The DOM, when we say DOM, that is the document object model, which is kind of how the browser organizes everything that it's showing to you. So this slide shows sort of the classic D3 example, where you've got some data up top, like a list of cities, and then your D3 code will turn that into an SVG document, which has some rectangles,
and then you see it in the browser as a chart. And D3 is also really great about handling if your data changes, so it can be very dynamic. You change the data, and then the rectangles change. It's a really powerful visualization framework. And it works for maps too, like a lot of you know.
This is traditionally how Rapid and ID before it did all of its rendering, so you start with some data, like the nodes and ways that we would receive from the OpenStreetMap API, or maybe some GeoJSON that contains building footprints. And then your D3 code is going to run through it, and it's gonna turn it all into, like here it's turning it into paths, and you see it in the browser as a map.
And you can do things with this SVG, like you can style them magenta like we did here. You can listen for events so that the user hovers over the buildings or clicks on the buildings. You can respond to those events just as normal browser events would. But it starts to break down, as we've seen performance wise,
once you push more and more data into it. So the DOM API, this is again how the browser manages everything that it shows you, can only handle so much, and it's kind of notoriously slow to be changing the content of the website while you are on the website. So you generally don't want a web application
to be changing the DOM if you can help it. Because when your application changes what the browser is showing, the browser has to recalculate everything like where it is and what it's styled like, and that can be very time consuming. So we're not gonna do that anymore. This is what our new rendering pipeline in Rapid looks like.
Our familiar map data still flows into it from up top, but now we're generating a pixie scene graph in memory. Pixie then turns all of that into WebGL draw calls. And it's actually very, very good at what it does. For example, it will actually batch together similar things.
So like all these buildings that are magenta, it's not gonna draw them one by one. What pixie will do is it'll just load up a buffer with pixel data, and it'll just say, hey WebGL, set the color to magenta and just draw whatever's in the buffer. So you end up with a very efficient pipeline, and it gets drawn onto a very efficient WebGL canvas instead of into one of those chunky SVG documents
like we were doing before. As we've refined the renderer over the past few months, I put up some words here. This is work that's still ongoing. We realized that it made sense to really think about these two steps in the pipeline as discrete pieces. And so that's why I wrote on there app-col-draw.
If you're familiar with any really old school renderers like Panda3D or even SGI's performer, you might know these terms. So app is work that we do to prepare what belongs in the scene. Col is when you're removing things from the scene that has gone out of view and the user just can't see them and draw is the conversion or translation
of that scene into WebGL. So the idea here is what we're doing is we've set up like a game loop in the browser. It's almost like a video game, like it really is, where we have a request animation frame callback. And that means that on every frame of browser time, our code gets called and it has an opportunity
to do the next tiny little chunk of work so that we're keeping the browser happy, we're not overloading it. And it still retains, it can still be interactive and it doesn't drop frames like it used to. This is a side-by-side comparison. Again, the difference in performance has been just really impressive, especially as we work with scenes
that have added more and more data to them. So in most cases, we're getting around a 10 times improvement in speed, give or take. Once you get past a certain point, obviously Chrome is not gonna go any faster than 60 frames per second, so you can only speed it up so much. But even basic editing tasks like this
where you're drawing points and lines, it becomes so much more responsive than it used to be. So just to kind of like demonstrate how much better the new rendering pipeline is, this is kind of a really busy scene. Somebody on Slack a couple months ago shared this. It's someplace like near Miami, Florida,
but it has like 45,000 OSM features in one little view. People are mapping all the buildings, all the roads, all the parking spaces and trees, right? And look, OSM has really grown up a lot, and so some parts of the world have become really micromapped at a level of detail that was really unthinkable
even when ID launched 10 years ago. And with Rapid, it's not just about the OpenStreetMap data, but we wanna layer more data on top of this data, right? We wanna layer on the AI-detected buildings, the roads, you know, whatever other data that we get through our partnership with Esri, whatever other QA layers that people wanna see,
or even OSM notes, and you know, on top of that Mapillary imagery, Street View, and so it's just a lot of data when it all comes together. So here's that place from the previous slide. This is what it was performing like in Rapid version one. I actually did have to zoom in a little bit
just to have something happen, otherwise it would just be completely frozen. This is zoom 18. But I guess we're getting around four frames per second here, give or take, with occasional pauses where it just stops. And this is what it looks like, kind of a new Rapid. It just like slides around very smoothly. We're getting around, like Chrome is reporting
around 35, 40 frames per second. So that's about that 10x improvement, right, going from four to 40. The frame rate that's being displayed up in the corner is Chrome's frame rate, and our code actually uses a few tricks to render less frequently than that. So we're gonna get into that, the details.
This is a little bit detailed, but right, this is what we're seeing here is called a flame chart. I ran both of these scenes in Chrome's profiler so you could really see exactly where it's spending its time. And I zoomed in, so what we're looking at right here is just one second's worth of data. So we've got two frames, right, that's not very impressive
and most of that work here is being done just by like, just updating the DOM and cleaning up that memory because those DOM elements really get churned up a whole lot. And then up in the corner, like if you see that purple, it says recalculate style. That's Chrome trying to run all of our thousands
of CSS rules against everything in the scene just to make the map styled correctly, just to decide the widths of the lines, the colors, things like that. So this is the new profiling trace of new WebGL. We're getting around, like I said, 40 frames per second. And remember when I said we use some tricks to render like actually less frequently than that?
It's kind of a bit obvious on this chart. You'll see like there's kind of stacks where the work is being done. Those are our app and call passes over the scene graph. And all those tiny little slivers in between are where we let the user pan the map, but we defer the redrawing until one of these other times.
Oops, can I go back? Yeah, good. The scene graph's also actually really interesting because you'll see there's a lot of space where our app is doing nothing. So Chrome is almost like idle. That leaves more space for us to perform other tasks in the browser like validation or user interface code.
So we're really only spending a few milliseconds of time to set the graph up. And then at the bottom, those green bars, you can see that's the GPU doing its job. And this scene, we're actually kind of GPU limited, which is a thing that has never happened to us before, but it's exciting because that means that now
there are other ways that we can speed the scene up even more than what I just showed you, either by just drawing the data a little bit smarter or maybe filtering some of it out to draw less data. So in summary, this new renderer really represents a change in how we think about data in Rapid. Before where the DOM and the styling
were really like locked together, now that's no longer true. There's no SVG, there's no CSS, and the drawing of the scene happens much, much faster. And we have more control over when these things happen. Like when we draw, when we free memory, it's just better all around. So thanks to this new render,
the rewrite also gives us opportunities to improve some things in Rapid that I was never really happy with. So we're gonna talk about those now. We took a fresh look at label placement. Before, the labels really had to be placed exactly next to the pin, but now we try a whole bunch of placements
around the label pin. So this is especially great for situations like this where they're all lined up. It will actually stagger the labels so that you'll see more of them. Here's another, this is kind of a before and after. It shows like before, you only get a few labels, but now you get a whole lot more labels. And this is really useful if you're mapping POIs, right?
Like businesses and shops. And we'd really like to see more people mapping more of this stuff. So it improves the user experience a whole lot. We also improved the line labeling algorithm. The old line labeling algorithm was kind of limited.
It would just pick a few spots and you would either get a label or not. And now Rapid can test a whole lot more spots along the line and even place multiple labels on a single line, like if it stretches out very far. So this image is kind of like a debugger's view of that process. Here, the red dots are places where a label would just cover something.
So we don't wanna put a label there. Yellow are places where it's just not enough width for the label that we wanna place. Magenta are places where we think the line might be a little too bendy to make the label look right. So we avoid those. And then green are the spots where a label would look fine.
So the great thing about all this labeling work that I just talked about is that now Rapid can put labels on other things too. We could never do this before. We were just limited to labeling OSM features, but now we can, this is a screenshot showing one of the Esri building layers and one of the Esri address layers. And we're actually putting labels on all of the address points
so you can see what they are. And this is great too, because all of those labels are placed coherently, right? They avoid the labels from open street maps features and everything just kind of like fits in the scene really well. Okay. This next slide shows another cool little performance trick
that we are doing. As we zoom out, we can actually replace building shapes with sort of these standalone, like little rectangles. In this animation, I had to turn them blue so that you could actually see what's happening. But in the real Rapid, they stay red, so the user doesn't really notice. But it's cool because Pixi can draw
a whole lot of textured sprites nearly for free. So this is a good way of, as you zoom out, we're not overloading the GPU because it can just draw hundreds of thousands of squares, like almost in an instant. Speaking of zooming out, a lot of us know the Web Marketer projection
sort of stretches everything out at the poles. So this idea of what a low zoom is can be kind of relative. So going forward, we're gonna be making more of our rendering decisions based on a thing we're calling effective zoom. That's kind of like saying, like what the zoom would be if we were at the equator for this feature. And I'll give you an example here.
So this is a building in Ghana and a building in Norway. They're actually the same size building, but because of the stretchiness of Mercator, the one in Norway looks a whole lot bigger. So it'd be like, if you were editing in Norway, zoom 14 might be like what zoom 16 is closer to the equator.
So going forward, it was always like kind of unfair that if you've used ID or Rapid before, you might hit a point where it says zoom in to edit and all the data is gone. We're not gonna do that anymore. Here we're actually showing editable features at zoom 15, which is a thing we couldn't do before.
We're still rethinking what it actually means to edit at different zooms, using this idea of an effective zoom to guide those decisions. So users might need to zoom in to actually see more detail or to specifically work with the features that have been simplified. But we wanna move away from this idea that all editing has to happen only at 16 and above.
And finally, we feel like we're really just getting started, right? This is a little silly video of something I made last week. This is, we hacked it together. This is a WebGL shader to just kind of make the buildings look all squishy. Like we're probably not really gonna do this unless we do like a Halloween release or something.
But like the point of this is that we have new opportunities to sort of style the map in ways that like we couldn't do before either to like draw attention to certain features or just make them stand out or look special. So we're excited to play with this. All right, so I'm gonna turn it back to Ben
who's gonna talk about some of the challenges we faced with this rewrite. So really quickly, we'll go over some challenges we had while implementing all this stuff and then a short call to action. So one of the things in computer science that you do not like to be told as a programmer is that you might have to work with an infinitely large dataset. That means you have to worry
about how many textures you're using, your memory usage. So we had to spend considerable time figuring out how to use texture atlases so that we weren't literally overflowing the GPU with texture references. Yep. Oh, you can speak more into there. Sure. Also, Brian touched on this a bit. Now that we have a faster renderer,
we can't just render everything at every zoom because we're just going to be burying the user in data. One of the first things I did when we got this render stood up is zoom over to Manhattan, where we have way too many features. And if we just label everything, you get the following picture, which obviously we can't do that, right?
So we have to maybe work closer with some cartographers to figure out what we show at given zooms. Another challenge is just WebGL itself. We're not entirely sure that support is evil even across every platform. There was a horrible bug in Safari a few months ago that caused the speed to rapid to crawl at like one frame per second at most, but other browsers and other OSes worked fine.
Finally, it makes our code more complex, but we would also like to be able to just ask the browser to do things later. We can do that with request idle callbacks to say maybe load some tile data, but don't do it right now. Wait until you have an idle frame. It's great for performance because it doesn't stop your rendering, but it also makes the code more complex.
So very quickly, next steps. We have an alpha test. This just got released a couple of days ago. You can go and use the alpha. You can use an improved bug nub in the lower right where we have a nice bug ingestion form up on GitHub and try it out. Let us know what you think of it. Let us know if anything breaks.
As this is an alpha, we do expect there to be bugs and we are hoping that you will help us find them. A few places where you can come find us. We're in the OSMUS Slack. You can find us at Map with AI on Twitter or on our Facebook group. And finally, we also have a birds of a feather session, two of them going on over in room 12 now. So if you have questions about Mapillary,
about daylight or about rapid, come to room 12. We'll be happy to chat more with you. Thanks.