From proprietary GIS to OpenSource : overview of a software stack
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 295 | |
| Author | ||
| Contributors | ||
| License | CC Attribution 3.0 Germany: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/43488 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
| |
| Keywords |
00:00
PC CardHuman migrationProduct (business)BitSpeech synthesisPairwise comparisonOpen sourceService (economics)Physical systemMultiplication signInternet service providerQuicksortForcing (mathematics)Projective planeCASE <Informatik>Position operatorLecture/ConferenceMeeting/Interview
01:40
Open sourceHuman migrationSoftwareStack (abstract data type)Characteristic polynomialComponent-based software engineeringDisintegrationThermische ZustandsgleichungProcess (computing)Physical systemDifferent (Kate Ryan album)Goodness of fitLatent heatCategory of beingLibrary (computing)Projective planeCASE <Informatik>ImplementationSoftwareFile formatOpen sourceCharacteristic polynomialINTEGRALFunction (mathematics)BitBuffer solutionDerivation (linguistics)Key (cryptography)Multiplication signProcess (computing)FeedbackRun-time systemClient (computing)Core dumpRevision controlConnectivity (graph theory)Component-based software engineeringOpen setGeometryJava appletVirtual machineAlgorithmLevel (video gaming)Single-precision floating-point formatClassical physicsData conversionSoftware developerComputational geometryPoint (geometry)Line (geometry)Basis <Mathematik>Confidence intervalCAN busCode refactoringOntologySpeciesMilitary baseNoise (electronics)ArmView (database)PolygonEvent horizonPurchasingForcing (mathematics)NumberElectric generatorWordExpert systemXML
08:11
GeometryThermische ZustandsgleichungMathematical analysisParallelverarbeitungSeries (mathematics)ScalabilityData storage deviceProcess (computing)Software frameworkRaster graphicsFile formatVector spaceDifferent (Kate Ryan album)Basis <Mathematik>Time seriesProjective planeDatabaseCASE <Informatik>Process (computing)Default (computer science)Computer configurationCartesian coordinate systemMathematicsMathematical analysisGoodness of fitGeometryOpen sourceElectronic data processingExtension (kinesiology)Graphical user interfaceLatent heatMultiplicationScalabilityRevision controlObject (grammar)DistanceCombinational logicMultiplication signSoftware developerSoftware frameworkSoftwareData storage deviceScaling (geometry)Library (computing)Generic programmingPoint (geometry)Data managementPoint cloudMachine visionQuery languageBus (computing)Axiom of choiceSummierbarkeitNetwork topologySequelComputer animation
11:59
Film editingGoodness of fitLevel (video gaming)GUI widget
12:27
Level (video gaming)Term (mathematics)Goodness of fitMappingAuditory maskingData analysisLatent heatLevel (video gaming)Symbol tableArtistic renderingDifferent (Kate Ryan album)
13:00
Server (computing)Function (mathematics)Configuration spaceRevision controlCodeData structureClient (computing)Texture mappingGroup actionSanitary sewerMathematicsMobile WebArchaeological field surveySynchronizationField (computer science)Coordinate systemMathematical analysisProcess (computing)Computer networkData managementData storage devicePersonal digital assistantSoftware developerHuman migrationOpen setSmoothingIntegrated development environmentStack (abstract data type)Design by contractMereologyComputer-aided designDisintegrationGoodness of fitData analysisWorkstation <Musikinstrument>Group actionElement (mathematics)WebsiteSequelSoftware testingHuman migrationData managementDifferent (Kate Ryan album)Attribute grammarForcing (mathematics)Product (business)PropagatorWeb serviceSummierbarkeitMetreModule (mathematics)40 (number)Web 2.0Process (computing)Software frameworkCodeProjective planePresentation of a groupServer (computing)CASE <Informatik>Degree (graph theory)Axiom of choiceExploit (computer security)Shared memoryTable (information)Directory serviceDivisorSoftwareRevision controlSoftware industryINTEGRALData storage deviceFunction (mathematics)Visualization (computer graphics)Dependent and independent variablesEquivalence relationInformation securityCodecPurchasingBus (computing)Service (economics)Mathematical analysisField (computer science)MappingClient (computing)Internet forumCartesian coordinate systemFreewareInterface (computing)Software developerData structureConfiguration spaceBitGeometryWave packetImplementationSpecial unitary groupSurjective functionState of matterMetropolitan area networkRight angleCategory of beingStandard deviationComponent-based software engineeringScaling (geometry)Software bugGeostatisticsArtistic renderingWechselseitige InformationLevel (video gaming)Archaeological field surveyProbability density functionGUI widgetCanadian Mathematical SocietyComputer programmingWeb applicationWeb browserComputer animation
22:06
Multiplication signArithmetic meanOracleBoss CorporationCartesian coordinate systemContext awarenessDatabaseOperator (mathematics)Software developerObservational studyHuman migrationDampingForcing (mathematics)Different (Kate Ryan album)CASE <Informatik>40 (number)Projective planeSoftwarePresentation of a groupParameter (computer programming)Wave packetData modelOpen sourceDesign by contractEndliche ModelltheorieClient (computing)Computer animationLecture/Conference
Transcript: English(auto-generated)
00:07
Our next speaker is Vincent Picabay, and he's going to be talking about what you can find in the open source GIS stack. Are you ready?
00:21
Yes. Thanks. Hi, everyone. I'm going to do a very beginner talk, so if you expect something technical or whatever, you may leave as well. I won't take offense. So it's mainly an overview of what's available on the market, and especially when you think about migrating. So just as for me, I'm from Aslandia, and I'm actually the CEO.
00:44
We are a service provider, and we are actually positioned as a QGIS editor, or co-editor of course, because we are not the only one developing QGIS, but we are a strong contributor to QGIS, and we are an open source pure player. I've been in the 4G industry with Aslandia for 10 years, so I've seen a lot of migration.
01:05
I think time has changed, so I'll show you a bit of what we can offer now in comparison to proprietary system. I won't name the big corporation which is selling a lot of products in the GIS world,
01:21
but how many of you use S3 systems? Oh, I told you. Okay, so like 10 to 15%, 20% maybe. So I'm speaking to you and to others who haven't any system yet. I talk about migration.
01:41
We heard a bit of that before, and I will show you a proposal for a software stack answering the needs of a GIS system in general. So first thing, when we go to big companies, big clients, and we say we have an open source system for you to use for your GIS, and they say, oh, open source, no, we don't
02:05
want that, too many software is too complicated, the quality is too low, it's bad, there is no support, no one to complain to, the features are poor, and you have intellectual property risk associated, so please get away.
02:25
And that's usually what we were used to hear for more or less 10 to 20 years now with open source, but hopefully time has changed and now the feedback is pretty different.
02:41
So we are now proposing a software stack where we want some characteristics of the stack because effectively we have a lot of different GIS software in open source, so what we want is a coherent stack with the same foundation for the software, we use the same technologies
03:01
globally and something which is kind of simple, because the less components we have in our stack, also the better support we can provide, because you cannot know every single software piece by heart, you cannot have core developer for every single piece of software, so the less components you have, the better support you can provide
03:23
to a specific client. Of course we work as a community, we work together, so you can always find support, but from a client side, from a client point of view, they are used to have just one interlock, one person to talk to, and they want this person to be able to ensure support
03:43
as best as they can. Also we want to reuse and to mutualize the software we use, so why is that? It's because it's less expensive, so if you use the software for one specific use case and the same software for another one, then you can avoid some license fees and
04:04
you're more cost efficient. We also want better integration and compatibility between the different components we use, so we don't want to have to convert from one file format to another one, just to
04:21
use another software which we'll use to output some data, which we'll use in another software and another format that's complicated for the user, for the end user, so we want integration and compatibility between the software pieces. And for this stack we will use PostGIS and QGIS as key project, because nowadays that's
04:45
what is on the market, mature and all. So a stack is pretty simple, and I'll show you the different components we use, and we expect this stack to be able to fulfill most of your needs in GIS.
05:01
I'm talking here about global GIS needs, not specific needs like, I don't know, very high-end machine learning or very high-end extra performance map generation, that kind of thing. We're talking about multi-purpose classic GIS use cases for mostly big companies.
05:24
So let's start with libraries. Libraries are what is at the heart of the software, it's the basis of the software where the hard work is usually done, and that's what we want to use and materialize among the
05:42
different software components we want to use later. And we are very glad to have very good quality libraries in the open source software environment in GIS, namely Gios, Gios is for 2D geometry processing, so it handles for example intersections between polygons, points, it handles, let's do a buffer on
06:05
this geometry, so this geometrical computation is done inside Gios. You have to know that Gios also is a derivative from a library called GTS, which is a Java library, so usually algorithms are developed first by Martin Davis mostly on GTS, and
06:24
then implemented in GOS in C++, which is used everywhere else in open source GIS software. Then we have Proj, Proj is now version 6, which is a very new thing, Proj has its
06:41
name, and there's Proj, everything related to geodetics, so the very complex stuff you don't want to re-implement at all in any other software. You want to have it implemented once at one point, and well implemented, and robust, and solid, and of confidence
07:01
as well. So that's a really, really important piece of software in the GIS world, and what's really good is that thanks to even Ruo, and Walt Butler, and the Barnrays, recently they had some funding to refactor Proj, and implement new features inside Proj,
07:23
which will make it like top-of-the-art projection library. So thanks to them, even Ruo did a really great job on this very difficult piece of software, and thanks to everyone who contributed to the funding as well. So Proj is one very important piece of software,
07:43
and on top of Proj you know probably GDAL, and OGR, which is the day-to-day library of every FOS4G user for GIS format conversion. It's a library, it's also a common line
08:00
of tools, but it's at the basis of a lot, lot, lot of different GIS software, be it proprietary or open source. That's a very important library too, and that's a library allowing us to handle hundreds of formats of different formats, raster and vector. So that's the basis for the stack, and it's very important not to forget these libraries,
08:25
because usually you have a lot of developers for QGIS for example, but developers of Proj and GDAL are not, there are not a lot of them, and it's very important for this library to be very robust. So we use mutualization, that's also less funding to get if you
08:43
don't re-implement format management or projection management in other software. Data storage analysis, PostGIS. Who doesn't use PostGIS here? Oh, that's a zero. Ok, great, so you are now PostGIS, so that's PostGIS. Of course, that's a database, GIS database
09:05
of choice, be it for open source or even proprietary people come to PostGIS now by default. In fact, PostGIS is great, but PostgreSQL is even greater. I mean, PostgreSQL is probably one of the best pieces of open source software in the world.
09:26
It's really a really, really good database and multi-purpose database. So with PostGIS and PostgreSQL you can do 2G analysis with geometry of course, you can do 3D analysis as well with SF-SQL extension, you can do point cloud analysis with PG point cloud
09:45
extension as well, and now with latest version you have parallel processing, so you can use multiple threads, multiple processes to handle your queries automatically, and with latest PostgreSQL version you have very good scalability options to deal with, for
10:05
example with Citus database extension or that kind of thing, you can extend to multiple nodes and processing on distributed processing queries, that kind of thing. You also have a lot of PostgreSQL extension which can be used in combination with PostGIS, for
10:21
example time series with time scale DB which is a PostgreSQL extension and allows you to handle a very huge amount of time series data. So this piece of software, PostgreSQL and PostgreSQL is also a gem in the world of the open source software. So of course
10:40
we use it. Then QGIS desktop, desktop application, so that's a generic GIS use case. We use QGIS, QGIS is now everywhere, you have seen it all over the place here at First4G, it's a general purpose GIS, so that's desktop GIS, but also it's got a data processing framework, so you can agree to design your own processes and to process
11:10
the data, but it's also important to note that it's a framework for application development as well. So you can use QGIS, totally change the application, change the
11:20
GUI and adapt it to your specific needs, for example here we have the wastewater network of Paris and they're managing all the underground wastewater network where you can walk in. Everything is done with PostGIS and QGIS and everything is LLS, so it's not only geometries but you define the object according to their distance
11:44
to a reference point. So whenever you move one pipe, everything attached to this pipe will move automatically with it, so that's a very good use case of a specific application and you can change everything in QGIS to adapt to that. QGIS is a very
12:02
good cartography tool, that's some example of what you can do with QGIS as almost art you would say. This is a very nice map with QGIS Composer, you can see all kind of very detailed and very well designed piece of graphical elements.
12:27
This is another one, so in terms of cartography QGIS doesn't have anything lacking almost compared to proprietary GIS and that's very important. We are currently implementing
12:41
something which is called selective masking, where you will be able to automatically mask different levels of symbol for one specific layer, so it's very tailored towards cartography and it's very precise mapping for very specific rendering. You can do
13:02
some very good data analysis as well like this one which are very neat. That was QGIS desktop, but whenever you want to go to the web then you have QGIS server. QGIS server is a certified server, if you've seen the presentation from Eric LeMond earlier,
13:21
it does take your data, use your QGIS project for symbology and all and outputs web services and OGC web services. It's a certified server from OGC, it's a reference implementation for OGC WMS and it does more than OGC standardized web services, for example you can output
13:42
directly your PDF which you designed in the composer onto a web application. It's very easy to configure it and that's where integration is important, that you design your project in QGIS desktop and then you will just copy your QGIS project onto a web directory and you can serve your data directly as WMS. And very important, it's
14:06
been totally refactored for version 3, before that don't use QGIS server, it's not industry grade, but from version 3 and now on it's very good quality, it works pretty well, you can use it in production, there is no problem with it.
14:24
Then you have a server with web services, cartographic web services, but you want to see it in an application on the web. There are plenty of applications, I'll come back to that, but our choice of preference goes to QWCE2, which is QGIS web client 2.
14:42
It's a web mapping application, so it's dedicated for QGIS, it's something which works for QGIS server and use the QGIS features at the best. It's based on open layers for the mapping framework, JavaScript mapping framework, and
15:01
very important, it's a community based project, which means everyone is invited to contribute to that and to improve the software. It's got a modern structure and code, so it's very easy to extend, you can add new services, you can add new widgets in the interface with programming of course, but it's kind of easy to use. So it's an application,
15:24
but also you can use it as a framework to develop your own application, or to integrate that into your existing SDI infrastructure. Then you have GeoNode. GeoNode is a geo data portal, or geo CMS. It's originally
15:41
based on geo server, it allows you to share geo data, so you can browse, search, import, manage, create, and share your data, your different layers, and you can also manage users, you can create groups, you can manage the access right to the data,
16:01
and it's also something you can easily adapt and extend, and nowadays it kind of supports QGIS server. I talk a bit more about that, but that's a very good thing, because we want something more integrated, so we want to use QGIS server with our data portal
16:21
or geo CMS portal. And also QGIS desktop is able to connect to GeoNode, get the data from GeoNode, and then you can open the layers and all. So GeoNode is like this, it's an interface, you can find your layers, and you can view the maps, you can search, you can get
16:43
categories, properties, groups, etc, etc. It's a web application. Then you want to go to the field and get some data from the field. So you've got mobile QGIS, which is QField, that's an Android application developed mainly by
17:02
OpenGIS. It's reused QGIS engine, so again, virtualization, we have the same rendering engine, the same project, so it's very easy to go from QGIS desktop to QField and back. You survey on the field, you synchronize your data back, and you can work seamlessly between both of them. So at the interface on Android, you
17:24
can acquire data, you can use your GPS, you can edit data, you can do lots of different things, edit attributes, and then you get your data back inside QGIS easily. Yeah, I'll try to finish fast then. So what do we have as feature for this
17:43
stack? We can do cartography, you can do 2D analysis and 3D analysis with PostgreSQL and Pochis, we can have data processing pipelines like this in QGIS processing, we can do imagery analysis thanks to QGIS processing modules, we can do data visualization with some plugins, we can have web
18:02
publication with QWC2, we can do network analysis and management in PostgreSQL as well. And it's a smooth workflow. You go from the field, you survey with QField, you have unified data storage with PostGIS, you use desktop GIS to define symbology, to define cartographic exports, you put
18:22
that on the web with QGIS server and QWC2, and you can have a portal on top of that for data sharing with Journaud. You have the same rendering in all applications, you will use the symbology, you will use the composer, you will use the same software elements, and that's very important. That's a feature table, maybe these business cases remind you of something, you can
18:45
find them on some documentation of sales for a very big software company, and you have the equivalent in FOSS4G, so for desktop production mapping developers you have QGIS, for application development on the web you have QWC2, for mobile you have QField, 3D analysis, network analysis,
19:05
pipeline referencing, spatial analysis, workflow manager, you can do that in PostGIS, geostatistical analysis and business analysis with QGIS and PUBGIS, interoperability comes with Google and OGR and QGIS server for OGC web services,
19:22
data review can do with QGIS processing, so you got kind of an equivalent of all features and that's for free, so you don't have to pay extra licenses to get extra modules like you have in some other world. Why migrate now? We talked a bit about that already, so I'll be fast, better TCO,
19:43
it's not less expensive necessarily, but you have better value for the money, you can do training, so you avoid vendor locking for knowledge management as well, what you learn to your user, it's not specific to an application, they won't have it in their next job, and it's important for the sales department as well, you can adapt the stack,
20:04
you have more features, you can have performance and scaling without adding extra dollars, you have a good reliability now, you can do mutualization. OGC standards make transition and migration easy, because you can migrate one element
20:21
after the other, and actually you do have support, there are a lot of established companies now providing support for this software stack, you have global support for all the software components at first, at once, and you have support from level 1 to level 3 and 4, if you want to go really deep inside Postgres SQL for example, you can get your software fixed.
20:45
Some migration, I'll stop there because I'm already 20 minutes, Ministry of Environment, French one, they did a large QGIS migration and POSGIS migration years ago, they found bug fixes, enhancements, Orange Telecom is a big application for network infrastructure management for the world,
21:06
network in France, they migrated to POSGIS, they found bug fixes, they found enhancements, and they're very happy and eager to talk about their use case, so you'll probably hear of them later on as well. Veolia, we kind of kicked out S3 from Veolia for a very big application,
21:24
it was kind of easy comparing the prices they wanted for licenses. Do not forget there are a lot of other tools, GeoServer, Nexus, Mapfish, Listmap, JS Cloud, you have the choice in the 4.4G, my stack is just a proposal from currency, mutualization and all.
21:44
What next, that's my last slide, we want to improve an application builder, that's something which does not exist in the 4.4G world yet, we want to improve QGIS geonaut support, it's kind of basic for now, better CAD support, better 3D support and better BIM integration as well.
22:07
So, any question?
22:25
Yes, Juan. Thank you for your presentation. If 4.4G as you mentioned now and described is very good, and also from cost perspective, why we still see proprietary companies getting contracts and getting also projects,
22:47
big projects from different industries, telecom, oil and gas and this, why still there is a hesitance to go to 4.4G? That's a big question.
23:00
So the question was why do not everybody migrate right now? Legacy, mainly. So we did some study for example, how can I migrate from Oracle Spatial to PostgreSQL and PostGIS? So it was a question of a big client. My answer was do not migrate.
23:21
Why? Because you had like 20 or 30 different applications relying on the Oracle database and if you wanted to do migration, it would cost like millions, three millions of developments for changing the different application, testing, training for the user, etc.
23:41
So, better was do not touch your existing application, it's actually working, but for every new application you are going to develop, use 4G. And then you decommission your old application from time to time once you replace them. It's sometimes refactoring as too high cost compared to restarting from scratch.
24:04
So that's one of the explanation is legacy. Other explanation is someone told no one ever got fired to use Oracle for choosing Oracle. So usually it's fear. And fear, uncertainty and doubt is some marketing arguments or method used by proprietary software
24:30
to try to continue the vendor locking situation. So that's another explanation. This is not true any longer. I mean people got fired because they use Oracle now instead of PostgreSQL.
24:43
So that's it. Another question? Do we have time?
25:01
Yeah, the question is for the wastewater application I just showed. Is it based on a proprietary data model in PostgreSQL? The answer is proprietary in the sense of it's not open source. No, I mean it can be open source. The thing is for Paris, the wastewater network is very special because you can walk in it.
25:25
It's 2,000 kilometers of walkable path under the ground. So it's not something you find in many different places. So the data model is specific to this context. So it's not much use for other contexts.
25:51
Yeah, some have proprietary data model and data model migration can be complicated as well. That's for sure.
26:03
Other question? Okay, thank you.