The life of open source spatial
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
Title |
| |
Subtitle |
| |
Alternative Title |
| |
Title of Series | ||
Number of Parts | 52 | |
Author | ||
License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
Identifiers | 10.5446/44696 (DOI) | |
Publisher | ||
Release Date | ||
Language |
Content Metadata
Subject Area | ||
Genre | ||
Abstract |
|
FOSS4G SotM Oceania 201934 / 52
9
11
12
14
15
17
20
23
26
28
30
32
34
39
44
00:00
Open sourceSoftware developerSoftware developerProjective plane
00:24
QuantumCodeComputer iconQuantumMultiplication sign
00:49
Functional (mathematics)Projective planeComputer fileDatabaseCodeLine (geometry)Entire functionWindowTable (information)
01:29
Line (geometry)File viewerVolumeSoftware protection dongleElectronic program guideCNNSoftwareOpen sourceSoftware developerStability theoryCodeAutomationExecution unitSoftware testingTotal S.A.Gastropod shellSource codeRight angleTerm (mathematics)Software developerUnit testingProjective planeFile formatCodeMultiplication signTranslation (relic)Formal languageLine (geometry)Pattern languageStability theoryElectronic mailing listVideo gameMathematicsEmailQR codeRange (statistics)File viewerCycle (graph theory)Computer fileTwitterOpen sourceGoogolInstallation artArc (geometry)SequelGroup actionComputer animation
04:38
Negative numberVapor barrierOpen sourceCore dumpRight angleTelecommunicationFreewareCore dumpInstallation artMathematicsSystem callFood energyTelecommunicationProjective planeMultiplication signData conversionWindowLevel (video gaming)Expected valueCodeMereologyRootPoint (geometry)Enterprise architectureBit rateTwitterMixed realityNetwork-attached storageSoftware developerVirtual machineOpen sourceFilm editingCodeWebsiteVariable (mathematics)Functional (mathematics)Vapor barrierControl flowView (database)outputSoftware bugAndroid (robot)Different (Kate Ryan album)Video gameSoftware maintenanceBuildingQueue (abstract data type)Lie groupRight angleMeeting/InterviewComputer animation
10:02
PredictabilityChaos (cosmogony)Associative propertyExpected valuePoint (geometry)BitPredictabilityProjective planeSoftware developerMereologyOpen sourceStability theoryData miningMultiplication signCycle (graph theory)Software bugQuicksortWhiteboardFunction (mathematics)Data structureInstallation artSelf-organizationResultantStreaming mediaOperator (mathematics)SoftwareStandard deviationData conversionComputer programmingQueue (abstract data type)Process (computing)MathematicsEstimatorRoutingChaos (cosmogony)DataflowControl flowVideo gameAxiom of choiceTranslation (relic)Form (programming)Website
15:21
Axiom of choiceSoftware developerInclusion mapOpen sourceMereologyOpen setPhysical systemGeometryServer (computing)Projective planeBitMixed realityAreaMultiplication signPerformance appraisalSoftware bugSoftware frameworkFormal languageUniform resource locatorStability theoryQuicksortInstallation artHydraulic jumpTranslation (relic)Channel capacityWindowMomentumIntegrated development environmentAxiom of choiceSoftware developerElectronic mailing listLogic gateMathematicsPoint (geometry)Operating systemJava appletProcess (computing)Game theoryEmailInclusion mapTerm (mathematics)1 (number)GenderNP-hardCircleLine (geometry)Mobile appQueue (abstract data type)Right angleSuite (music)Level (video gaming)Computer animation
Transcript: English(auto-generated)
00:03
Yes, I'm one of the Australian QGIS developers like Dave said I've been involved since 2011 So I joined the project It's sort of in the middle of its growth, I guess. So this is some of that some of this stuff in here is personal opinion Some of it's just plain facts. So we'll go through it from the start
00:22
so Back in the beginning. We've got a fish. Maybe not that far back Um 2002 so Gary Sherman created the start of quantum GIS at the time It wasn't actually called QGIS, although the code references QGIS. So official name is quantum GIS This is the first logo that you would see if you started it up in the title bar
00:43
I went back in the code to find the rendered little animator icon If you check out the the first commit that was ever made in the project's history, which is this commit up here That's the entire code base like most projects I start with small beginnings and then you don't just dump a 10 million line file into a project and go done
01:04
This is also the readme. I'm sure it doesn't look like this now. It's a lot bigger And you can also say it's very humble I guess doesn't have any functionality basically just enough to list the tables in a in a post just database And to build it is three steps. It's definitely more than that now
01:25
Maybe I'm less maybe on Linux is three steps on Windows. It's definitely not So this 2,700 lines in the source file doesn't include a readme. That's just pure pure code And it was started if you know what the history of QGIS is just as a post just viewer Gary's
01:43
Past experience was Esri and he moved to Linux and created QGIS to be able to read post just layers because there was no viewer at the time So Started as a hobby project there's like most things do some of these throws code out there just to see what happens and Announced onto a mailing list basically that was there
02:02
however, most things tend to change over time whether on purpose or Not purposely. This is the trend like Google Trends essentially of the achievements in the projects life going up to 3-4 so it's quite outdated, but you can see the group the quite rapid growth after after the two releases
02:25
So this kind of change Brings more people more more developers more community members And it's pretty much inevitable if you've got a project that actually does something well, which is what Gary created originally So where are we where are we basically now?
02:42
One of the biggest open-source GIS projects there is a few others As the I can't pronounce it so I'm not going to there's another one That's quite probably just as large in terms of code, but I definitely was prevalent as what QGIS is It's definitely not no longer just a post just viewer as you know, it opens almost every format You can possibly poke a stick at thanks to
03:02
Gdell As well as sequel and Postgres and WMS and all those fancy and even arc Um It's also got a very very extremely large community compared to my other projects and that includes
03:20
Actual users and developers basically so as Niall said yesterday's lightning talk was 170 contributors to the project plus all the translators and Documentation writers, so it's quite large It also has a lot more stable and frequent release cycles now than what we used to have in the past back in the early Days, it was kind of whenever felt like cutting a release was when we cut the release right now
03:41
So now it's a lot more structured and what follows a lot more a frequent pattern rather than just sporadic and better code stability and unit testing and automated builds and installer packages and documentation and translation and fancier website Things like that
04:00
It's also quite large code base so it's almost a million lines of C++, which is hairy and Also 500,000 lines of XML, but we don't write that XML. That's UI files so they're generated, but still it's in the code base regardless There's also a Python and Perl and shell and bash and every other language you can also poke a stick at so
04:25
there's experience a wide range of experience in the project in terms of growth in terms of Developer growth if you touch the QGIS code base you may be doing Python C++ and Perl all in one day Which is probably true of most projects So that all basically means we're not pretty much in Kansas really anymore compared to what we used to be
04:44
And of course all that does come at a cost to the project It's not free as soon as you got a large code base Nothing is not nothing is free even a small code base. Nothing's free But the more things you add the more cost there is and that's not necessarily cost as in dollars that's cost in time and energy and resources and
05:03
stamina to stay motivated basically So things that cost us, you know code changes now require more thought because the project is larger So you have to think about your changes more. There's diversity issues as you would have seen like in talk last night yesterday with my Santa hat
05:21
Changes require thought rather than just sporadic cowboy coding and who cares what happens basically Documentation definitely doesn't write itself. It's an issue We've currently got and we still have and most projects will always have it even enterprise Paid solutions have that problem You Have to maintain documentation. It's just a fact of life
05:40
There's also now a bigger barrier to entry So as the projects grown code base has got larger, which means more tooling and we have to rely on a lot more tooling onboarding people comes harder, so Those things just naturally grow out of a large project And as a mix of funding and volunteer early on was pretty much volunteer 99% of my work was volunteer
06:01
Most of Nas works volunteered most of the developers works volunteer and it's grown into a more sustained project with funding Let's go talk about installers for a minute because you can't just do a source dump when you're this large Back in the early days was just a source dump There was no install a package if you want to just build it yourself on Linux probably no Windows package
06:21
Definitely no Mac package. You're on your own pretty much. It doesn't fly when you're a large project. There's expectations around Being able to install it and if you can't install it people just won't use it and then your project will probably Die Hopefully not die, but go slowly away So making install package is pretty easy. All right, you just do it, but it's really not it's really really hard
06:48
Especially because of the nano variables that are involved in in making install package that actually works Reliably has to be automated so you have to do it manually every time And there's also the problem of works on my machine, but then doesn't work on someone else's machine
07:03
And we've actually had communication issues as the user expectations have grown through the project as a project's grown People expect to be able to install it on on OS X on Windows on iOS on Android so communication around what we Provide from an installing package point of view
07:21
Hasn't always been super clear and that's purely a communication issue An example of this is a Twitter exchange that I was part of later on but someone was saying that the Mac packages Sucked and we should be ashamed of the project to have crappy Mac packages, which is not necessarily a lie, right?
07:41
So a few people got involved As Twitter's conversations usually go they went to crap pretty quick it Involved people saying well if you want better Mac packages, you should just install like shut up and sponsor it yourself turns out there already was a sponsored installer package that the Q just project had actually funded you'll see in a minute how much and
08:02
It was pretty much ready to go The difference is we hadn't communicated that was ready to go It's sitting in a repository waiting for that final cut over to the website to say we're done no one had been prepared to make the call for that for Various reasons so After that conversation and an I throw dug into the root of the real issue not the issue that that's not funded
08:23
It's the fact that it was funded and just not published we Just made the call to publish it and now it's better hopefully But there's still issues of course because installs are hard, but communication is key there ultimately Which also comes into this people rely on QGIS for the daily work
08:42
They expect reinstall to work the next time they run it if I install three four and then I run three six install I don't expect it to just suddenly break everything that I had installed So It makes things trickier when you make installs you have to make sure they work when you do bug fixes you have to make sure that they don't break existing functionality and Then instability also is that point just you have people you now expect the project to be
09:06
More stable for every release if we're making it breaking things constantly then People will not use it. We only have a reputation like we have now Purely because of the increased stability and that comes at the cost of slower rate of change potentially or more harder changes
09:23
but more thought-out Support is also important. Like we said yesterday and the talk we gave start There's a myth around support But as the project has grown that demand and that cry is definitely Always been that always will be as most people who use open source are aware
09:41
You can't ignore that you have to provide some level of support whether that's commercial whether the project itself takes on support if you've got your own open source project may decide to just Do support for people and make the project do the support? But you have to communicate again where the support is. Otherwise people will just assume it's like a most like most open source projects
10:03
you just on your own and There's no support Talking to a few people as well about The project and it's not a negative. It's a it's a good thing Ultimately is that predictability of the as we've grown we've learnt that predictability is a good thing If you're growing if you are growing a large open source project predictability will help you a lot people expect
10:26
People know when the installers are coming you can go on the QGIS website You'll see a little ticker at the top that says next release in 10 hours Like you know when the release is coming Give or take some house depending on the packaging But you we know when the release is coming we know when the next LTR is
10:42
So my advice if you are growing an open source project is to definitely get in some sort of predictability when we used to have Sporadic releases and chaos. No one would ever know what the release cycle was It was just whenever we felt as developers was good which leads to scope creep like most software projects
11:03
And as we've grown coming back to stability and also coming to predict predictability We have to avoid those breaking changes we made major breaking changes in QGIS 3 with the conversion to PI QT and QT 5 and Python 3 and a few other little API things in there g-dog operator things like that that
11:24
Quite substantially hurt the community. I felt at that time because people weren't expecting a API change, right? We communicated it probably communicate well enough and then cause a bit of pain to people are really only starting to migrate I've got a project that I've maintained for a long time I started upgrade mine because I didn't want to deal with the changes
11:42
So as the project grows if you're growing your own open source project If you're on an open source project, if you want that stability in their predictability, you pretty much have to avoid breaking changes It's just that standard software practice kind of thing. Anyway, you're gonna end up with gross API's but sometimes
12:01
That is life Unless you're forced to like we had no choice So what happens basically when the project goes from being a hobby project and turns into a serious project that people expect to deliver serious Results people use it for research Publications for academia all that kind of stuff. It's not really a hobby anymore
12:20
like people make money out of QGIS in their jobs and as developers we make money and as Researchers you make money or as journalists you make money And there's people who use the outputs for humanitarian reasons So we have to produce results that are expect like predictable and Not just sort of willy-nilly whatever we feel like
12:40
as part of that as the projects grown we kind of The project grew to the point where users demanded that more stability and more predictability and more structured approach I think it's inevitable that most projects go down this route I don't know any other communities even in the program world like with Python and rust all those they tend to all
13:02
Form some sort of organizational structure. Even if you're an open-source project you have an organizational structure if you don't admit it So a board was established with voting members. They rotate through like we saw this morning to keep Burn out low if you're not feeling like you're comfortable in that role you can bail out
13:22
You also have quite substantial income compared to a lot of other projects this is 2018's income Hundred and fifty thousand euros We are quite privileged to have that amount of money that's come over time from sponsorship donations and things like that, but we're also Lose a lot of money. So we have money in the bank, but we spent a lot of money 2018 expenses
13:45
There's a massive chunk here for bug fixing. So the project itself actually Infests money back into the project to bug fix because we realize that people expect stability So therefore we spent spend money on stability on stability itself There's other things like translations and documentations like that and grant projects to fund
14:04
work So the expected income for this year is a hundred ninety four thousand euro It's an estimate it's not a fixed amount But you also see seventy six thousand of that is going out to it's not seventy six euro. That's seventy six thousand euro
14:22
It's going out to oh, we could spend seven dollars on bug fixing. That's That won't get you much Probably nothing but seven seventy six thousand is bug fixing just bug fixing on the project including GDAL So the QGIS project itself helps Would like to maintain our upstreams. There is no point QGIS just being off its own world and treating other projects
14:45
Themselves and let them deal with the problems. We realize that QGIS itself is relying on GDAL. It's relying on Proj Those projects are fundamental to what we do. So the bug fixing flows down We have developers that work on GDAL, Proj and QGIS in their day jobs So we might as well just let them fix across the project. It helps everyone if you're using GDAL by itself
15:04
You get those bug fixes. That's the whole point of being a community And then upstream bug fixes to QT. QT is actually sponsored by an organization, but it's an open source project. So There's bugs in there that affect us. So we might as well just spend 7,000 euro on that
15:22
Talk about money some more The line between volunteering and paid gets quite blurry at times When do you volunteer time versus when do you hunt for money? If you're relying on for a day job You have to hunt for money obviously Whereas I may have a job that pays quite well so that I can just volunteer
15:41
But that bug fixing money also you have to work out who gets paid for that basically User perception Come back to start about having installers and being more professional being more professional Don't ever ever under any circumstances shame users for their choices I've seen it in the past being shaming users for
16:01
Particular operating system if you're a hobby and you're only one user one developer. It's your project fine But when you're a large project like we are that you have no choice you have to users will come to you with with windows and And Whether they should get a release for that for Windows 95 probably not right
16:21
But that's Microsoft's problem. I'm out not ours, but don't we don't shame them for it You don't want to shame people for it all their choice of operating system running on a Mac But not you're not any better for running Linux if someone runs Mac, right? Developer and user experience will definitely matter the build yourself is fine for developers We're used to it, but as a user you can't just say build it yourself fix it bug yourself and go away
16:43
Like there's going to be differing experiences there So be prepared to have those little bit of clashes and you have to just deal with them, but be nice This is my final points use other projects for inspiration. I heavily look for Inspiration with pot in the Python community and in the rust community and those kinds of communities. There's the ones I use
17:04
And other open source like leaflet and open lanes inside They just see how they are doing it to see how they're maintaining the stability and Not doing API changes and how they're funding and things like that Also be willing to change your project There's no point being the big person in in the game and then just sticking like that forever
17:22
Everything will die. If you don't change we've seen we've seen companies come and go like that they Dug in and then someone else comes in to squash them basically So be willing to change your everything about your project if you have to to get better a
17:41
Sort of conferency stuff that we dealt with all today is that the community itself matters a lot QGIS has only grown so large because of the community we have we have a Ridiculously diverse in terms of location. We've got myself and all in Australia. We've got people in Cambodia Developers Egypt developer in Japan, there's developers in America and a lot of them in the European area
18:01
But that brings as you're saying yesterday some cultural issues that we have to deal with Clashes, but you have to deal with it basically and also inclusion basically, so don't Don't gatekeep people just because they're new to the project. I was new at one stage and if I got gatekeep I would never have joined the project
18:22
There's people who I've joined because I didn't get keep them They got a bad experience and then they eventually found myself and Nile and a few others and then eventually got brought into the project They wouldn't be there if we can't keep them out And and at least be safe don't Don't use sexist terms and gendered terms for no reason just
18:43
In your mailing list and things like that, but keep it as neutral as you can and make people feel safe in your environment And also coming back to everything needs money You probably should if you can if you have the capacity doesn't have to be money sponsorship. It can be This kind of thing. Ultimately, this is some level of sponsorship. You should sponsor the projects that you care about
19:05
Whether that's gdell or QGIS or geo server or Like the people who maintain the specs things like that Throw money at a problem or throw time or resources at a problem. Basically if they're not there The projects will pretty much fade out. So
19:24
Thanks Nathan time for questions See Martin's hand Thanks for a great talk and for all the work most most of all on the project. I just yeah
19:50
I think it would be useful for us as a community to See back to the dire days of the early 2000s in the arc few days When huge I started and jump and you dig were around as well
20:05
Which were at that time in my eyes sort of almost leading this the open GIS is there were somehow ahead I have the I had the impression and then QGIS knows it what happened to them that they did not become so globally successful SQGIS, what is the lesson to be learned for other projects?
20:24
Or maybe it's just my bias team. No, no I used you I used you dig and open jump when I first started QGIS now where they were part of my evaluation On where I go We can we can attribute some of QGIS a success to the underlying framework We use in the fact that we use QT which gives us a very quick leg up
20:40
Compared to having to write native C++, which is really hard Um Java Like open open jump was based on Java. So they've still got that kind of leg up already I suspect it was probably the people made me It's not nothing against those other projects either. I wasn't involved in them, but I know that when I joined QGIS myself personally
21:01
Tim and Martin were very welcoming to take contribution So I think the gatekeeping thing is of high risk for projects to lose Momentum if you gatekeep too hard, then you'll basically just kill your own kill your own project basically So that's probably for me personally That's why I think it grew there's a lot more welcoming than some of the others possibly
21:23
Technical reasons also speed and things like that and it just had more capabilities I think some of the stuff we could get for free. We didn't have to work on The other projects may have to work harder to get so I think there's a mix of things there Yeah, it would be good to go back and actually try and break that down more I suppose
21:41
But I know that you dig it survived for quite a substantial amount of time. It's probably it's still around I think it's probably just a community thing more than anything else for Those who are thinking about it and getting involved. Would you say that your involvement has increased and
22:02
With your reputation increasing has enabled you to get higher paying jobs. So it it sort of comes full circle Volunteering versus paid. Yes, so QGIS itself was an enabler for my development path. I don't work on QGIS full-time paid full-time I got other I
22:22
got other priorities but QGIS itself was was the gate to me being a developer now and enabling that that So I look at QGIS now differently to how I looked at when I was a starter That was more of a user But as soon as that as soon as that gate opened
22:41
It basically opened me to develop a community and then everything that came with that So I may not be doing what I do now if I wasn't for QGIS basically, so Yeah, contributing definitely helps regardless of what it is. Even if you contribute to a translation you learn how translations work and how other languages work and things like that, so