Building a New Rendering Backend for MapLibre Native: Industry Collaboration in FOSS
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 156 | |
| Author | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/68434 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
| |
| Keywords |
FOSS4G Europe 2024 Tartu17 / 156
6
33
35
53
55
59
61
67
70
87
97
99
102
103
104
105
107
111
121
122
123
124
125
126
127
128
134
144
150
151
155
00:00
Streaming mediaArtistic renderingFront and back endsBuildingCollaborationismComputer fontArtificial intelligenceSoftware developerSoftware maintenanceSoftwareSelf-organizationPresentation of a groupLevel (video gaming)Open sourceMappingBitSoftwareSpeech synthesisProcess (computing)FreewareMultiplication signSoftware maintenanceComputer fontExpected valueArtistic renderingWeb-DesignerFront and back endsTrailLogistic distributionCASE <Informatik>Right angleArtificial neural networkDegree (graph theory)CollaborationismBuildingProjective planeLibrary (computing)Observational studySource codeClosed setSelf-organizationLecture/ConferenceComputer animation
05:41
SoftwareClient (computing)Formal languageRun time (program lifecycle phase)TesselationCoprocessorCASE <Informatik>MereologyLibrary (computing)Right angleCodeDatabase transactionDesign by contractLevel (video gaming)Open sourceFreewareReal-time operating systemWordQuicksortoutputClient (computing)Software developerStructural loadNumberFunctional (mathematics)OnlinecommunitySoftwareWeb browserEnterprise architectureWhiteboardDivisorLattice (order)Internet service providerMappingVideo gameGraphics processing unitWeb 2.0Medical imagingProjective planeSoftware maintenanceSlide ruleSelf-organizationComputing platformAndroid (robot)BuildingCore dumpVector spaceMaterialization (paranormal)Open setSystem callLecture/Conference
11:18
MereologyMobile appCore dumpPosition operatorProjective planeGame theoryData storage deviceEnterprise architectureOpen setJSON
11:58
Medical imagingMathematicsTesselationSoftwareComputer-assisted translationFreewareCASE <Informatik>Internet service providerSoftware developerSoftware maintenanceArtistic renderingNumberProjective planeOpen sourceMappingFront and back endsPoint (geometry)outputMobile appSlide ruleBing MapsCoordinate systemTerm (mathematics)CodeMereologyDifferent (Kate Ryan album)Computing platformCollaborationismModul <Datentyp>Level (video gaming)Meta elementBitEnterprise architectureWindowUsabilityXMLComputer animationLecture/Conference
19:16
Human migrationGoodness of fitPresentation of a groupOpen sourceDifferent (Kate Ryan album)Stability theoryType theoryTouch typingCodeProduct (business)Spherical capCASE <Informatik>Multiplication signRevision controlSoftwareHookingQuicksortWeb 2.0Power (physics)Level (video gaming)Mobile appTesselationStatisticsNewsletterSoftware bugBlock (periodic table)Game controllerSoftware developerMultilaterationState of matterClosed setProcess (computing)Service (economics)Arithmetic progressionSource codePoint (geometry)Interface (computing)Lecture/Conference
26:20
Special unitary groupLeast squaresComputer-assisted translationWhiteboardLecture/ConferenceComputer animation
Transcript: English(auto-generated)
00:00
Thank you for being here today at my presentation. So my name is Bart Lowers, and I'm here to present Building a New Rendering Backend for Map LibreNative, Industry Collaboration in Free and Open Source Software. Now, you may have noticed that this talk is in the community and foundation track of this conference,
00:21
and that is no mistake. This will not be a technical talk. Instead, I will talk about how we are doing free and open source software in the case of Map LibreNative. I think that Map LibreNative is quite an interesting case study, because it may defy some of the expectations you have
00:44
about how free and open source software is done, or even how it ought to be done. So not only will I be talking about Map Libre and Map Libre Native, I also hope that in this talk, you will gain some insights that,
01:02
although every free and open source project is unique, get some insights that you can bring to your own projects. Right. So here's a short outline of my talk. First, I will properly introduce myself.
01:20
Then I'll talk a little bit about Map Libre. And then I'll introduce this new rendering backend, this project, the first major milestone that we've achieved with Map LibreNative, I would say. We'll go into detail in that. Then we will talk about the most important thing in free and open source software. You can already think about what that is, maybe.
01:42
And I'll shortly touch on what's next for Map LibreNative, and then we have some time for a short discussion. All right, let's get started. So about me, I have a bachelor degree in artificial intelligence. So you could say I was doing some reinforcement learning
02:01
before that it got cool. And when I was absorbed by the workforce, I became a web developer for a couple of years. And in 2022, I was working for a German B2B marketing company. And I felt like it was time for something new.
02:22
And I thought to myself that I would do something that I would enjoy. And I would not stop looking until I found something like that. And so as a kid, I was fascinated by maps. And I read that when deciding on your career path, you should look at what fascinated you as a kid.
02:42
So I developed some geospatial aspirations. And there was a subsidiary of Volkswagen that, in the logistics sector, I applied there. They made me do some stuff with Leaflet during the interview and I was hooked and I loved it.
03:01
But fortunately, the job itself was not what I expected and I wasn't happy there. So after three months, I quit and I just started looking for something else. And I was very lucky because in this time, I was following open source mapping libraries, open source geospatial libraries, and I came across MapLibre and they
03:22
were looking for a maintainer. And thanks to my studies, I happened to have just the right qualifications to do this job. So since the start of 2023, I've been almost full-time maintainer for MapLibre Native. So very happy. This is me at FOSFORD GNL last year.
03:42
I really like my job. I like the work I do. And at FOSFORD GNL, it was quite interesting. So there was the keynote speech there. And the keynote speaker asked the audience, so who here is working for a company?
04:01
So about half of the people raised their hands. And then he asked, so who's in education? A couple of people raised their hands. OK, who's working for government? So that was basically the rest of the hands. But I didn't raise my hand yet. And this was at a FOS4G conference. So this is where we get the first FOS insight of my talk,
04:25
which is you can actually pay people in a free and open source software. And yeah. So let's talk about MapLibre now. Now, talking about MapLibre can be a little bit confusing because what do we mean about MapLibre?
04:43
Do we mean the software? Do we mean the MapLibre organization? Or do we mean the MapLibre community? Now, I'm going to talk about all three of those. But I'm going to start with the software first because that is the most interesting thing. And that is the reason why the other two exist.
05:01
So there's MapLibre GL.js and there's MapLibre Native. And both are mapping toolkits that use real-time rendering. And real-time rendered maps are the kinds of maps that end users have come to expect with a very
05:21
smooth experience with the ability to dynamically style the map, dynamically change the map. And you can use MapLibre with your own data sources. And it's good to mention that MapLibre started out as a fork when Mapbox closed sourced their software.
05:42
So the official reason they gave for deciding not to continue with free and open source client libraries is that they expected that by open sourcing their client software, they would be sharing the load with a community and they would receive lots and lots of contributions so they wouldn't have to hire as many developers.
06:02
So it would be cost savings for them. And they said they closed sourced it because that didn't materialize. Maybe also a factor was that what Mapbox was making their money with was the data to render these maps. So the way real-time rendered maps work
06:22
is you don't send images to the client and then stitch them together. Instead, you send vector data to the client. And then you render it at 60 frames per seconds or maybe even more, just like you would do with a video game. So you need a graphics card for that, which almost all devices have these days.
06:41
In any case, that's what they earned their money with. And other tile providers started to pop up. And they were just compatible with their software. So maybe that was also a factor there. So this brings us to the next free and open source software insight, which is a corporate free and open source
07:02
project may change its license tomorrow. Now, this may sound a little bit scary. But we're going right into free and open source software insight number three, which is you can fork a free and open source software project, which is to copy the code and continue its development. Now, you can go to GitHub right now and fork Map Libra
07:24
and give it your own name, call it MapBart or whatever your name is. You cannot call it Map Libra, by the way, because we trademarked that name, so sorry about that. But of course, there's more to forking a project. And this is, I think, where the Map Libra organization comes
07:41
in. So the Map Libra organization was created to support the continued development of Map Libra software. Interesting fact, it is not a legal entity. So maybe it will be someday. But right now, we use a platform or a company called Open Collective, which is our so-called fiscal host.
08:02
So I have a contract with them. And I send my invoices there. And they are managing our money at a 10% transaction fee, basically. Just interesting fact. Nowadays, it's also an umbrella organization for other mapping libraries.
08:21
But making tiles is specifically out of scope for the Map Libra project. Now, we have some governance. We have a charter that defines that we have a board. We have voting members. We do have a board, which, by the way, is great. So it consists of five people, one of them, Luke,
08:41
who's in the audience here. And I think that's a great thing, because when you have a free and open source software project, it's, especially for enterprise users, very calming to know that there's not one person that can take this project and take it in some weird direction, but that there's consensus building.
09:01
Also, two of the people on the board have a company that have a vested interest and rely on Map Libra software. So that's also nice. All right. Now, let's talk about the Map Libra community. Now, community is a word that gets used a lot in free and open source software. But what is it, actually?
09:22
What is a community? What is the Map Libra community? OK, so probably active code contributors. Those are part of the community, right? Yes, yes, of course. They are part of the community. What about the people that attend the monthly meetings of Map Libra?
09:40
Yeah, of course. They're part of the community. What about people who are active on our Slack channels? Yeah, definitely part of our community. Or users who discuss new functionality on GitHub, part of the community. Users who just ask questions on GitHub, they're part of the community. What about users?
10:01
What about people that take Map Libra software, they put it into their project, we never hear from them. Are they users? Are they part of the community? I mean, it's not a very communal thing. So, I don't know. What about if they only do the above once?
10:22
So they ask a question on GitHub once, and we never hear from them again. We don't even learn their name. Are they part of the Map Libra community? It's interesting to think about these things. What do we actually mean with community? And we're getting back to this one.
10:40
Okay, so this slide again. So it's good to repeat. I was hired to be a maintainer for Map Libra Native, and Map Libra Native is unlike Map Libra GL.js, which is sometimes also called Map Libra Web, runs in the web browser. Map Libra Native, our main or core platforms
11:01
are iOS and Android, although we also support various other platforms. And on iOS, we had sort of a problem. So I mentioned that for real-time rendering, you need to use the graphical processor. And typically, you use OpenGL ES for this.
11:24
But Apple announced in 2014 that they would be deprecating OpenGL ES. Now, there's like a bazillion apps in the Apple App Store or games mostly that use OpenGL ES, so they didn't remove it outright. But this was not a nice position to be in,
11:42
especially enterprise users where they wanted to use Map Libra Native, but they didn't want to use this project where they would really rely on it as a core part of their business. So Apple said, okay, OpenGL ES, we're not gonna do that anymore. Use this instead. Use Metal.
12:01
This is a new API to talk to the GPU. Now, this is, oh, there's one image that's not loading here. But this is basically, the image that was here was Steve Gifford asking on our GitHub,
12:23
what's the status? Are you interested in doing a Metal port? And Oliver, our wonderful coordinator at Map Libra, said that, yes, we're very interested in a Metal port. Is this something you could do, right? Like in free and open source software, is this something you ask that?
12:41
People ask a feature, and they ask, oh, is this something you can implement? And Steve gets back to Oliver and says, yeah, sure, but I cannot really just do this. This is like a six-month project, and you need a lot of money for that. And he says, the hardest part is hurting those cats, and that's going to be almost impossible.
13:01
So hurting the cats means getting companies that can pay for this stuff, and get them aligned, and get them to pay for this, because it's a large project. And he says it's almost impossible. So this is free and open source software insight number four.
13:21
You will be constrained by what someone is willing to pay. Of course, there are projects you can do that you can rely on volunteers, or you can rely on people that really have their heart in free and open source software, and they live for it. But something like this, where you have this super large project,
13:42
you need professionals, basically, and they cost money. But hurting the cats turned out to be possible. The need turned out to be big enough that we could do it, and this is how the Metal project was born. It started out with design proposals. So if you want to make a big change to our software,
14:04
you're often invited to first write a design proposal so that everyone can comment on it, we can get everyone aligned, and we can discuss the design before we actually start with it. This was the first step. We had the renderer modularization proposal. It sounds complicated, and it is,
14:21
but basically OpenGL ES was very interwoven with MapLeaver Native, and a lot of assumptions were made about using this rendering API, so we needed to modularize the renderer first to make sure that we can use a different rendering backend, and then the Metal backend proposal.
14:40
This is basically what features of Metal do we want to use, and how are we going to use it exactly. Then a maintainer was hired, which is me, so they realized that they needed someone to manage this project to keep the team that will be working on this accountable to help support the project in terms of CI and stuff.
15:06
So this project was led by Steve Gifford, and the engineering was done by the Metal team, and it's interesting. It's a collaboration between AWS and META. Development kickoff was in early 23, and we pre-released it at the end of that year,
15:22
and at the beginning of this year, we released it, so it was a success, and MapLeaver Native is now ready for the future on iOS. So for free software insight number four, this one is a bit basic. It enables people to work together, META and Amazon. Pretty interesting that companies like this
15:42
can work together in this way. Also pretty unique, I think, but I think this is the more interesting insight, which is, so we have wonderful maintainers for Windows, for Node.js, for Qt, but I think it's fair to say that MapLeaver Native doesn't have a lot of active developers,
16:00
and this project has been the most active development. So this insight is the most active community members may be professionals working for or hired by corporates, and organic contributions from volunteers may not be what is driving your project forward the most, which is definitely the case for MapLeaver Native,
16:22
and you can ignore this at your own peril. Now, what I mean by you can ignore this at your own peril is you need to realize what kind of free and open source project do you have at your hands, and then optimize for that. Because if you pretend it's just a community of volunteers,
16:42
and you're gonna be able to push the project forward with just volunteers, you may not be able to do that. So let's talk about the most important thing in free and open source software. Can anyone make any guesses? And can be anyone that has seen my slides online.
17:02
Okay, so I think the most important thing is the license, because without a free and open source license, it's not free and open source software. But the second most important thing in free and open source software is, I can give you a hint, it's the most important thing in any software project. Huh?
17:21
No? No? Huh? What people? What? Users! Yes, bingo. Yeah, I think that's the most important thing. Now, of course, we were very fortunate to already have a lot, a lot, a lot of users
17:41
that were using this code base at the inception of our fork. And I think this is the thing that it will be very hard, it can be very hard for like a grassroots, free and open source project to get off the ground, to get enough enterprises, for example,
18:01
interested in using the software. But of course, they thought Mapbox is this billion dollar company, it's just not, they trusted that Mapbox would be not pulling the rug. And of course, yeah, so this is a sponsor slide and I show this slide after I mention users,
18:21
because our sponsors are not just people that happen to give us money, but they are big users. Amazon is rolling out Map Libre Native to over a million devices for their delivery people, for navigation. MATA is shifting some of its apps to use Map Libre Native too. Maybe you will see it in Instagram
18:41
or WhatsApp at some point. Then we have Mirun, our first Japanese geospatial company commodes for outdoor activities. We have some tile providers. Microsoft is using Map Libre, GL.js for Bing maps and it's also starting to use Map Libre Native now.
19:02
Maptain is an indoor mapping company and MapMe is a platform for creating maps in a user-friendly way. So I have one more minute to talk about what's next for Map Libre Native. The development of another rendering backhand has actually been kicked off last month, which is Vulkan.
19:23
What's interesting about Vulkan is that it's used in the, heavily used in the automotive industries for various reasons I won't go into. So you might have, at some point, a car where Map Libre Native is running and I think that's pretty cool. So thank you very much for your time and now we have some time for discussion.
19:50
Thank you very much. We have time for some questions. Yeah, if no one has a question, I have a question for the audience. That's always possible. So I was wondering, why do you think
20:01
that Amazon is using Map Libre Native? What is their main reason? Yes, you sort of raised your hand. You're on the hook now. I'm sorry for repeating the question. So why is Amazon a big corporation? They have a lot of money.
20:21
Why are they using Map Libre Native? Good question. Yeah, you can just shoot, just shoot. I've seen a couple of presentations where, like open source, people were saying that if you have a really good product,
20:40
Jeff will, Jeff you, and they will just like package it and sell to their customers. But one thing which Jeff can't do is pretty UIs. So maybe you have a very good product and it provides very nice interface, so.
21:02
Could be it, but that's not it. Really? We have another person that wants to take a guess. Amazon does a lot of like vertical integration and tries to avoid using outside services. So maybe that's why they don't want to rely on Mapbox and prefers the Map Libre Native product.
21:21
I think you're close. So the reason is that they're rolling out this piece of software and they want to be able to fix things when they break. They want to be in control of their own software, basically. And this is what you have with free and open source software.
21:41
Because we work based on consensus and we're not gonna block any pull requests because it doesn't align with our business goals. We're gonna merge pull requests based on if users are interested in something. And so this is it.
22:01
It's that they're able to get critical bug fixes in basically, yeah. All right. So it's not the cost. Oh, getting involved. We have a Slack channel. Also follow us on LinkedIn. We have a monthly newsletter you might be interested in to keep some tabs. And of course you can find us on GitHub.
22:22
And yeah, here's another overview. So if there's another question, maybe I think we have time for another one. Yes, we have time. I also have a question if nobody has a question. Please go ahead if you have a question. Okay, my question is, after forking in the beginning,
22:41
did you also have a lot of customers switching then from Mapbox to Map Libra? And what is your main competing power actually if you compare the two? So this is a really interesting question. So in the case of Map Libra GLJS, I think people were switching pretty quickly. But in the case of native, things move a lot slower.
23:04
For example, we have a company Grab, which is, I'm not sure if it's five or 50 billion market cap. So they're very active in Southeast Asia primarily. It's like the Uber of Southeast Asia.
23:21
And they have their own fork of Map Libra or of Mapbox GL native still. And there's some other projects that are on their own fork. And they're only going to switch if the cost of switching is worth it. And we have a very stable metal port right now. So we have a development that they're interested in.
23:45
And only then can you make the business case to switch to Map Libra. If it's just the same software but just packaged with a different name, no one in their right mind would switch and take this risk. So we have the metal port. For example, if we have good support for Map Libra tiles
24:04
and that offers great benefits, I think we're going to see even more companies that are making the switch. But quite frankly, we have a lot of users where we simply don't know that they are users. I go to conferences and I speak to people and I look at their app.
24:20
Like I was at the state of the map last year and there was a company called GeoVilo, which is like routing for bikes. And I looked at their app and it's like, oh yeah, you're using Map Libra native. I've never heard of you. And we have that a lot. We simply don't really know who's using us. So if you're using us, go to GitHub discussions, leave a comment so we have some insight
24:41
into who is really using us. Thank you. There's another question there. Let's take it. No, I just want to add about Map Libra GLJS migration. So quite a few users
25:00
and you can observe that in a download statistics on NPM are stuck at one X version of Map Libra web. And simply because the cost of migration is too high, not because it's difficult to migrate, but because even spending five minutes on the code when you don't have to,
25:20
people like if it's works, don't touch it type of thing. So yes, there is quite a few of these and that's why you can see a big, difference between the one X downloads of Mapbox code versus later versions. And also you see a progressive growth of Map Libra because as more and more people slowly migrate,
25:43
you see a constant growth. We have achieved, I think we peaked at 450,000, hundred thousand, not thousand, 450, hundred thousand downloads per week on NPM.
26:00
So we are growing and lots of companies are switching and lots of users and users, organizations are switching. But it's a very slow process simply because, someone has to actually go into the old code and migrate. Thank you, you want to respond? I'm with Yuri.
26:21
Yuri is an under board member by the way. Okay, any final questions? Okay, thank you.