Usage and contribution of FOSS at GISCO
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 351 | |
| Author | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/69152 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language | ||
| Production Year | 2022 |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
| |
| Keywords |
00:00
Mixed realityGoodness of fitSoftwareOpen sourceCoordinate systemIntegrated development environmentDependent and independent variablesPresentation of a groupBitPoint (geometry)Representation (politics)Office suiteStatisticsFreewareMereologyMeeting/Interview
01:55
InformationInternet service providerState of matterPermanentStatisticsMaß <Mathematik>Distribution (mathematics)Address spaceComputer networkWeb serviceBuildingSystem programmingSoftware frameworkImplementationObservational studySoftwareComputer programmingInterface (computing)Function (mathematics)Analytic setType theoryRoutingSelf-organizationRow (database)MappingDistanceDatabaseInterface (computing)State of matterComplete metric spaceVariety (linguistics)InformationWeb serviceMatrix (mathematics)Nominal numberInternet service providerInstance (computer science)Personal area networkMathematical analysisMixed realitySoftware frameworkTesselationDistribution (mathematics)Pairwise comparisonSet (mathematics)Uniform resource locatorAddress spaceDialectOffice suiteOpen sourcePresentation of a groupTraffic reportingLevel (video gaming)StatisticsContext awarenessElectronic GovernmentCartesian coordinate systemAlgebraic varietyDecision theoryPermanentRouter (computing)System administratorUser interfaceChainSoftware developerDirection (geometry)Data structureGeometryBoundary value problemModulare ProgrammierungLocal ringVisualization (computer graphics)Information securityOpen setMereologySoftwareDifferent (Kate Ryan album)Observational studyMedical imagingBitShared memoryStapeldateiDomain nameMathematicsImage resolutionDegree (graph theory)OntologyGreatest elementWebsiteFunction (mathematics)1 (number)DebuggerNatural numberComputer hardwareRootCASE <Informatik>Computer animation
11:34
StapeldateiAddress spaceWikiWeb serviceFormal verificationData storage deviceSystem programmingUniform resource locatorData conversionCodierung <Programmierung>Basis <Mathematik>Default (computer science)SpacetimeComputer-generated imageryVector spaceLatent heatObject (grammar)Structural loadVulnerability (computing)Mathematical analysisBoundary value problemAlgebraic varietyNatural numberStatisticsTesselationRevision controlLibrary (computing)Java appletSurjective functionClient (computing)ArmWeb serviceAddress spaceSet (mathematics)DatabaseInstance (computer science)DemosceneLevel (video gaming)MappingInclusion mapCore dumpRootWave packetInformationInterface (computing)Software testingPresentation of a groupCASE <Informatik>Visualization (computer graphics)Process (computing)Open sourceTemplate (C++)Multiplication signFreewareFile formatModulare ProgrammierungStandard deviationVulnerability (computing)Repository (publishing)BitForm (programming)Front and back endsProxy serverProduct (business)Field (computer science)Formal languageOpen setStudent's t-testLattice (order)Extension (kinesiology)Analytic setResultantRow (database)Library (computing)LoginWeb pageScaling (geometry)Point (geometry)View (database)Validity (statistics)Position operatorSystem administratorStatisticsAuthorizationDesign by contractKonferenz Europäischer StatistikerHorizonVariety (linguistics)Uniform resource locatorOffice suiteSoftwareDifferent (Kate Ryan album)Medical imagingRange (statistics)CodeMathematicsInstallation artDomain nameMathematical analysis2 (number)Computer fileFeedbackGeometryVector spaceDifferential geometryData conversionComputer animation
21:14
MathematicsFrustrationComputer animation
Transcript: English(auto-generated)
00:00
Thank You Marco, good morning everybody Welcome to the session welcome to our presentation titled usage and Contribution and of freedom open source software at disco and I need to correct Marco. Sorry disco stands for geographic information
00:21
So it was coordination of the Commission So so the Commission is good in Creating acronyms, and that's the reason this acronym exists since over 20 years, you know So if you're asking why are we called disco? Why are we placed in your start? Don't ask me legacy Legacy Act since 20 years. We are there
00:44
Probably back then someone said well, yes, they belong to the statistical office I'm standing here on behalf of my colleagues from the just go team We're a small team of 12 people and I just want to talk to you a little bit about what and how we're doing and
01:03
free and open source Just one point in the beginning We have been moving our GIS activities from an closed S3 shop, which we have been
01:20
Nine years ago to into a mixed environment We got accused by an S3 representative you are only doing open source Our response was that's not true. We just moved into an mixed environment So for certain parts, we're still using as this of fine
01:41
This this is there's a dedication or there's a reason for being in a mixed environment For certain things we doing it purely free and open for certain things not Depends and this is our advantage now So talking about just go How many of you have heard about just go in the past? I know I have seen member states speaking. Yes, I know
02:06
Someone else. No, okay, then I need to introduce it Sorry for the people who already know me who know what we're doing. It will be boring but for the rest We're permanent GS service Like we're a service provider for your start like we're doing analytics
02:26
like root distance matrixes We service provider for the Commission So we are buying data from the national mapping agencies via Eurogeographics from commercial data providers
02:41
We get data from open street map NASA natural earth We get that together ship it out and we coordinate in partnership with it also with members It's just to give you an example on what we're doing there I mean if you go to your start website and look at you in your regions in Europe statistics visualized we provide their
03:03
development support the GIS data for applications or software development like that If we're talking service provider for the EC During Corona we have we went to actually to all the member states organization and located where the hospitals are
03:22
We didn't have to date a database a complete database of Europe yet and we have compiled a pan-european data set of healthcare locations in the member states and then we also do this kind of work with the member states where we Funnel dedicated grants to statistical offices as well as national mapping agencies
03:44
Where we're trying to bring together? statistics and geospatial information So to move the whole world in that respect forward Just talked yesterday to a colleague from Slovenia where we funded the project
04:00
Probably eight years six years at six years ago. Don't don't hassle me for that one Yes, it's still running and you were instrumental moving us forward in that direction just to give you a couple of examples To make it more practical for you, what are we talking about? What are you doing for GS?
04:20
So we are the guys actually responsible in the Commission For the country data set which countries do we recognize at least from the geospatial side? This is For you, maybe is simple, but I mean in the Commission you have to always draw the correct boundaries
04:42
So do we recognize Kosovo or not? And we're working together with a publications officer in the council for political agreements and we sold all these kind of Political issues. We also disseminated regions and Laos population distribution addresses geographical names all these kind of things transport networks
05:01
You find it here at the bottom. There's a URL This is a kind of data sets you which you find in public dissemination We have all that in a our reference database where it's a little bit more Detailed because due to license agreements We cannot share all this data with the highest resolutions
05:22
Yeah, just a nice screenshot here done by our colleague Joe Davis Yes, really nicely then that The Greek islands or the Greek state Based on Copernicus data just to give you an example Copernicus data. You can also get from the aza and
05:43
other dissemination channels, we just have Brought them together in 10 by 10 degrees tiles. So people can more easily work with this kind of data so in General I would like to Conclude that part from just aside. What are we doing? We trying to localize analyze visualize
06:04
jazz data Bringing in different software packages trying to coordinate inside listening to our users Responding to that one bringing them together Coming to a common understanding and then rolling out solutions to them as well
06:21
Um Now before I'm going to the main part I just want to introduce two concepts here bringing us a little bit outside is a Our GI our IT focused on me
06:42
We're working also with the United Nations, this is UNG GM and there we have a global statistical geospatial framework and You know as a public servant My duty is to support decision-making. This is a little green bar down at the bottom here But for that one, we need to have do analysis
07:03
And then we need to have an output for that one, so we need to have harmonized and standardized information We need to have interoperability Comparability and I don't think I need to convince here the already convinced But to bring it into practice This is really hard to talk with member states to talk with different organization that they are all harmonized
07:26
That they're all working at the same level Just to put it into context So you see where we are playing because we want to support our decision-makers and we plays a whole chain from Using a fundamental geospatial structure and geocoding providing services for that one
07:43
even up to disseminating The nuts geometries so people can visualize it and can make maps or analytics part of that one So we are at the interface of this workflow The second thing which I would like to introduce here and maybe this is completely
08:03
Discovered from the rest of the presentation is API's we all aware of API's I assume so but I mean this is a big mind to change for Quite a many of people which are used to monolithic applications. They're not used to API's
08:21
They're not used to have an endpoint which they can Do they want to click and click off a button, you know with a user interface? That's wow. We can do something. No. No, we want to have an API plus a human front end to that one We learned it the hard way and with that one we can contribute to a wide variety of things open government
08:44
to modernize public administration Systems can talk to each other From European interoperability framework and we can also contribute to the provide only once Or the once only principle So data are shipped in to a commission service and is used everywhere and not that everyone needs to import it
09:05
This is a mindset change it requires significant Efforts for that one. You can read further and the GSC b6 API's for Digital government study reports on that one if you are interested, so how do we do that?
09:24
From just go We provide corporate level services and for that one, we have a couple of concepts which we learned over the last couple of years and An API is not enough We always need to add or we we try to add and human friendly interface on top of that one
09:44
We learned the hardware a performance performance performance and security security security Sorry Thomas was earlier here on stage. This is our requirements from security. I'm coming from research I mean before that, you know as a researcher you coat your
10:02
Arts running it's working Yeah, fine If you're on a professional software You focus on performance and security and you will see later on we have done on security what we have done and how we contribute back actually to the community and We document everything and also for the public ones. We have also JSON documentation all these kind of things
10:25
Just I want to give you now a couple of examples what we are doing for corporate level services So I can show you where and how we use free and open source software So we provide back on maps geocoding booting ID service image dissemination
10:40
So here for example images on the backside Dissemination example and I will go into this different aspects in a couple so we find geocoding find a location service internal internally to the Commission we have For such as you type around 500,000 requests
11:02
Today, we have detailed record search geographical name search a batch geocoding interface On top of a nominative instance, and we have our European register of artists, which we have even developed in-house For that one we use postgres nominatum for tone I think Sarah has the southman has a talk
11:22
I think now later on and we're really grateful for the work which are doing there Maintaining this one and we have also contributed back with some fixes to this kind of software Because sometimes you get the edge cases to us Which which then needs to be addressed because sometimes these processes are
11:44
Used quite heavily and you will be alerted on all the kind of edge cases where the datasets are not correct Where it's just across the border Just one example here one day a colleague comes to us and says look I need to I have here all the rest my students of Europe and we want to do an analysis
12:06
Well, we talk half an hour. We show them all the API's a couple of months later he presents in one of our meetings which we have internally in the Commission about jazz and he says Initially, we had estimated three years for that one. We are done after three months
12:23
Thank you for you services just due to this one because they could could use it and rapidly accelerate the results We've built ourself an endpoint which is called artists API or UP and register of artists Where we have compiled all the national mapping on national data from artists
12:45
based on no chairs postgres swagger Which we identified as one of the root causes in the Commission Because have you ever entered an artist yourself?
13:02
Yes, no, maybe any one of you how many times are you entering it in a freeform text field and If you get that sorry, we get it in production. You get five million records of freeform taxes Not separated
13:21
50 different spellings of the same old All these kind of issues you need to sort it and for that one We built a service so you can validate against the authoritative data information and then you can we can say yes This road that does not exist officially so we cannot accept it and this goes in all our contracts and cordis and
13:44
a Rise in 2020 You know all this and we tackle with that one's issue of non validated data entry It costs you 1% of the cost if you fix it at data entry it cost you 10% of the cost If you fix it afterwards
14:00
But it cost you a hundred percent if your emergency vehicle cannot reach the location given in the address Or is that the wrong location? You know, this is the point why we have developed it and and shipped it out to to our colleagues second example ID service Just a simple identify, but we have really made it much more simple
14:22
You just put in coordinates and you can convert work between different data sets Just to give an example what we're doing here again Postgres in the in the back end. We also ship out feature and vector services We Use PG feature serve PG test for that one again based on on top of a Postgres instance
14:44
To ship out our data sets to our customers And here again, I want to spend two minutes on how to be contribute Back to the community. We have done Vulnerability tests on a range of software packages which are running in production with us. We have done it for
15:05
Map proxy we have done it for open source geo network and we even have contributed fixes back then with geocat So we're a contractor back then with us we have Evaluated PG features here and PG tie serves. It was just minor issues there And this is a kind of feedback where we are contributing back to the free and open source
15:25
community If you have further questions to me, I mean we are not doing that in the public and the open We do it behind the scenes. We usually contact The core contributor asking them if they are interested
15:40
If we have resources we can also contribute fixes of identified issues But I mean this is a kind of contribution which we are trying to do from our side to that one last service I would just move on Is visualization? We provide
16:01
wide variety of background maps Based on map neck my proxy Postures Global extent this is our most used product on a given day. We have five million users around even including
16:21
Multilingual you languages and we adapt the open street map to the position of the European Commission Because open street map as you are aware maps the situation on the ground We need to reflect that one and this requires some resources the last five minutes I want to
16:41
Show a case to use some of the other things which we are doing in the visualization domain where we also Contribute back to the open source community even the GS so we have a library called Eurostat map Based on
17:01
Visualizations you see a couple of examples here GitHub URL is at the bottom We use it ourself for a product called image which stands for interactive map generator Which is a tool which we ship out to our policy DG's so you have the policy officers and he has an exosheet was a country code or nuts and was a value and
17:24
Within five minutes the uploads of access file or ten minutes. They can get a predefined map to Map ready for inclusion in a PowerPoint presentation in a web page In in a press the template and a document a for format
17:40
Ready to be shipped out and we don't need to do this stand up maps Let's say but we control the cartographic language We control the map design and we're not having 50 different maps in a document Have seen that in in other publications So we try to do that at least from a cartographic language point of view and we estimate not estimate
18:04
We know from monitoring we we create 250 maps a map map maps a month with that tool Which is not coming down to us for that one. You're welcome. You can even use it yourself It's available at just go minus service dot ec dot oppa backslash image
18:21
You just need to any you looking and you can go in and do it yourself We're in the process to publish it also to the open source repository of the Commission For that one. We need to do one two more things before we can do so, but it's on the legal side on the technical side
18:40
so More is here on grid with and darling cartogram So we do quite a bit of visualization also based on free and open source and Bring it out. So you can reuse it. You're welcome to do so. I Want to complete conclude?
19:01
The overview was a couple of analytics tools. We have the region simplify tool here, which we're using to generalize our data sets across scales so from 100,000 K data sets to 20 million 60 million something like that Good maker or give you Libraries to change or to look at differences between data sets
19:24
Which people are welcome to use for their productions it I've talked mainly today about What are we doing at services, but we have also other phosphor G found fun facts here Some people some facts are left out
19:43
as of Monday, we had over 500 installations of Let's say the OSG or sweet. So we have QGIS cross saga and you know, I Need to pun now my colleagues from GSC because they are in a separate IT ecosystem So I don't know how many QGIS installations are down there, but I mean in the main Commission
20:03
Let's put it that way because we are differently like There's a research arm of the Commission in the GSE. So we have over 500 We have managed to get Postgres as a database recently and we also do some training on free and open source software So coming up with a summary I hope I made my story here to you. We are service
20:24
Based on your start to the Commission. We serve a wide variety of services based on free and open source software Apis plus human-friendly interface based on OTC API standards is key to success
20:40
We serve around five million users a day We contribute back with vulnerability fixes with back fixing if we have sometimes we also develop our own solutions if we don't find anything on the market and put it on our own open license, I Want to conclude my talk because I see already I'm I'm off
21:01
With one sentence here change is hard. Don't get frustrated, especially if you're working in public administration. I You