geOrchestra, a free, modular and secure SDI
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 183 | |
| Author | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Germany: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/32045 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language | ||
| Producer | ||
| Production Year | 2015 | |
| Production Place | Seoul, South Korea |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSS4G Seoul 201596 / 183
7
8
47
53
54
65
73
74
79
82
84
92
102
103
105
124
126
127
130
141
142
143
156
161
162
170
176
178
181
183
00:00
Web serviceMedical imagingSoftwareSystem callPresentation of a groupSpring (hydrology)Information securityConcentricModule (mathematics)InformationDigital electronicsInformation privacyJava appletDialectMultiplication signExecution unitInstance (computer science)Server (computing)State of matterSinc functionSoftware maintenanceConstraint (mathematics)Physical systemSystem administratorService (economics)1 (number)Water vaporHeat transferMappingOpen sourceRankingDistanceKeyboard shortcutPredictabilityStapeldateiBitForcing (mathematics)Right angleComputer chessSoftware testingSpeech synthesisLabour Party (Malta)FreewareFlow separationSurgeryModul <Datentyp>AuthenticationProxy serverCore dumpClient (computing)2 (number)Web 2.0Standard deviationProduct (business)SharewareRepresentational state transferMetadataSoftware bugUniform resource locatorComputer animationDiagram
06:26
Event horizonWater vaporDialectIterationDirection (geometry)
06:56
FreewareOrder (biology)State of matterInstance (computer science)AdditionExtension (kinesiology)Goodness of fitSoftware architectureReading (process)Ocean currentElectronic mailing listSoftware developerEmailOpen sourceDialectComputer animation
08:18
Information securityProxy serverModule (mathematics)Different (Kate Ryan album)Standard deviationMoment (mathematics)Open sourceContext awarenessStability theoryMiddlewarePhysicsInstance (computer science)CuboidProjective planeArmSource codeSystem administratorService (economics)Video game consoleAnalytic setMusical ensembleData managementForcing (mathematics)Process (computing)EmailSingle-precision floating-point formatSign (mathematics)NumberRadical (chemistry)Single sign-onFormal languageMedical imagingWeb serviceComponent-based software engineeringSoftwareRevision controlAsynchronous Transfer ModeAttribute grammarFront and back endsMathematicsForm (programming)TwitterPredictabilityImage resolutionDatabaseRepresentational state transferText editorPeer-to-peerRoutingServer (computing)View (database)Multiplication signWeb 2.0Computing platformFrequencySelf-organizationOpen setPlanningGraphical user interfaceBitObject (grammar)Adaptive behaviorDistribution (mathematics)Reading (process)WordPhysical systemAdditionTheoremTerm (mathematics)SharewareFreewareMereologyGeometryUniform resource locatorFile viewerFunction (mathematics)Operator (mathematics)Product (business)Computer hardwareCore dumpScaling (geometry)Discrete element methodVariable (mathematics)Cartesian coordinate systemStructural loadDebuggerDiagram
17:26
Flow separationService (economics)Order (biology)Scaling (geometry)Cartesian coordinate systemBitArchitectureRight angleBackupMultiplication signChemical equationComponent-based software engineeringInferenceMoment (mathematics)State transition systemLatent heatUniverse (mathematics)Cloud computingInstance (computer science)Serial portSoftwareModule (mathematics)Stability theoryQuicksortInformation securityOpen setMotherboardFreewarePhysical systemSystem administratorComputer fileFile formatWeb applicationShape (magazine)Slide ruleBoss CorporationData storage devicePredictabilityStandard deviationProxy serverComputer configurationBuffer solutionRemote procedure callData conversionInternet service providerNeuroinformatikSystem callPrisoner's dilemmaBranch (computer science)Revision controlWeb 2.0MathematicsDefault (computer science)Function (mathematics)Software developerConfiguration spaceDrag (physics)Process (computing)Distribution (mathematics)Streamlines, streaklines, and pathlinesInstallation artAnalytic continuationProcedural programmingVideoconferencingSelf-organizationWater vaporForm (programming)Different (Kate Ryan album)Point cloudReplication (computing)SharewareVirtualizationSoftware bugRange (statistics)Ocean current1 (number)Server (computing)File viewerStructural loadVirtual machineLastteilungElasticity (physics)Computer hardwareXMLComputer animation
26:22
Computer animation
Transcript: English(auto-generated)
00:03
Okay, today I'll talk to you about G-Orchestra, which is a free modular and secure SDI. My name is François Renerbiest, and Florent also participated in this presentation, in writing this presentation, but he could not attend today.
00:23
So, we are both working at Camp2Camp, which is a Swiss and French company, founded in the beginning of the year 2000. Oops, so it doesn't work.
00:43
Never mind. So, you may ask what does an SDI stand for. So, SDI stands for Spatial Data Infrastructure. So, it means spatial data, it means, of course, we are talking about geodata, and infrastructure means that with geodata, we can store them, we can share them,
01:04
we can view them, compose maps, download data, extract data, portions of the whole geodata. An infrastructure should also allow you to discover data through metadata, and describe data also through metadata.
01:25
So, all this should be allowed by a good spatial data infrastructure. What are the benefits of using an SDI? So, for users, of course, as we said, it eases geodata search and access, and it should work with all OGC client software.
01:45
Of course, since we are talking about distributing data, the standards here come from OGC. For administrators, and particularly in Europe,
02:02
where the INSPIRE constraint is strong, people can turn this constraint into an opportunity by using a good SDI, because with an SDI, there's no more data duplication, since, as you know, data is distributed through web services,
02:26
and of course, there's less maintenance work, because all the software is centralized on a unique server. So, these are the benefits of using an SDI. Now, let's talk about G-Orchestra precisely now.
02:42
What is G-Orchestra? First of all, it's a Java software, which is based on Spring, and it is made of modules. At the core of G-Orchestra is the security proxy module,
03:02
and there are other modules which you can use with the security proxy, behind the security proxy. The most famous of them are GeoServer, GeoNetwork, and we've got also other modules that we've been developing for G-Orchestra.
03:22
Authentication is handled through an external component, CAS. It can be your already existing CAS, or it can be the one provided by G-Orchestra. So, you see the main advantage here is that it's modular.
03:41
It means you can use the modules you want to compose your own SDI. For instance, you want GeoServer, GeoNetwork, and a viewer, but no extractions, or things like that. What is G-Orchestra? So, it's a free software, free as in speech.
04:00
The license is GPL. It's modular, as I already said. There are more than 10 available modules at the time. I'll introduce the modules later on. Of course, it's interoperable, because it natively speaks OGC services, and it uses internally and externally REST APIs.
04:24
It's also very secure, because there are many reasons, but the most important one is that we support full HTTPS, and we also do continuous delivery, which means that our customers, or people using this software,
04:43
can update their software every week, if they want, or every day, and benefit from the latest bug fixes. So, this is a URL of the demo. You can go later, see it if you want.
05:00
So, now a bit of history. Where do we come from? It all started in 2008, when at Camp2Camp we were developing Brittany. I don't know if... Brittany is a French region, and we were developing Brittany's own SDI. They ordered us an SDI, and very quickly, we were thinking about creating something more generic
05:23
that could be used by more people than just Britons. So, 2010, first production deployment, and other customers, as we had predicted, asked us about the same kind of SDI software.
05:42
So, we could reuse the software that we did for Brittany, and improve it, build upon the first call. And G-Orchestra really began with our second customer, with Aquitaine region, and continuously improved.
06:04
For instance, in 2012, Bolivia adopted G-Orchestra as its state SDI, and more regions, more cities have been using it since then.
06:22
At the same time, research labs and industry also used G-Orchestra software. So, let's speak about community now. This picture was taken earlier this year during our annual event, where the whole community gathers and speaks about evolutions,
06:45
and which directions should we adopt. The community is very diverse, as you see. Regions, cities, research companies, all of them were represented at least by one people in this community. But the regional extent, G-Orchestra is mostly present in France,
07:07
mostly deployed in France, but also around the world. We've got Nicaragua, Senegal, India, and also we have a strong presence in Bolivia, as you see here, seven instances in Bolivia,
07:23
one of them being FAO, and the other one being GeoBolivsar, which is a state instance. Again, with the community, the community gathers on IRC, Channel G-Orchestra, and Freenode.
07:42
We've got two mailing lists, a general one, and another one which is more related to development. Source and issues are public, they are hosted on GitHub, and the following URL. And as I said, we hold an annual meeting,
08:00
and this year was a third edition in Strasbourg, France. Okay, let's dive into the subject more deeply by talking about software architecture. So on this schema, you might recognize the first one I showed you,
08:23
the security proxy at the core, the CAS, which handles authentication, and the other modules. Here is the proxy, here is the CAS, these are the other modules, and the launch part is an alternate setup
08:44
which is made to handle more load on GeoServer. So how does it work? Well, as I said, we've got a single sign-on component, which is CAS. It means that once you're authenticated on the software,
09:04
you are authenticated on all its components. You are authenticated once on GeoNetwork, for GeoServer, and for the viewer, and you get access to private layers, for instance. So this CAS,
09:23
so it authenticates the user against an LDAP database. Where is the LDAP user database? The security proxy, so as I said, the proxy here, it handles the user session.
09:41
Once you log in, the security proxy creates a session, and it keeps it during a certain time. And this is the web server. Every request that comes from the web server
10:03
goes through the security proxy, and the security proxy routes the request to the proxied modules. And by the way, it adds security headers. Security headers in the HTTP request
10:26
tell about which user is connected, and which roles he has. So the modules read these security headers, and they grant or they deny access to resources accordingly.
10:45
Okay, so about the modules. As I said, we're standing on the shoulders of giants in the form of GeoNetwork. We have both versions, the two, and the third, which is the latest one.
11:02
We have GeoServer, we can use the latest version of GeoServer. Each time GeoServer releases a new version, we can take it, and you can take it, and use it with GeoOrchestra. Optionally, we can improve GeoServer's layer and service security
11:21
by using GeoFence. GeoFence is a free software component made by GeoSolution that allows you, for instance, to say, well, people from this group only see this portion of the layer, and people of this other group see the whole layer
11:44
and all its attributes, for instance. It's very, very powerful in terms of restriction. And also, CAS is a software that we've been taking that was already existing.
12:01
In addition, we've been developing an advanced geo data viewer and editor that I will show you later. An extractor, which is a module which allows you to download geo data extracts, to say I want data in this bounding box, in this output projection, and the job runs, and at the end you receive an email.
12:23
You can download your geo data at this URL for 10 days, and it will be deleted later. We also have got users and groups management console, and analytics is a module which allows an administrator
12:41
to monitor the usage of OGC services from the platform. So other modules, but less important. So here is the viewer, the viewer UI, which presents JERKestra instances on top of a DEM.
13:04
So it's a bit old, I know, but we have plans to replace it with an OpenLayers 3 viewer and made with AngularJS. This one was made with GeoXT, XGS, and OpenLayers 2,
13:24
but it still works very great. This viewer is also a nice front end for all our modules. Here you see that within the viewer, you can switch a layer into addition mode,
13:42
and you can change the attributes, and it's synchronized into the database through WFST. And the same viewer allows you also to, it interfaces with the extractor web services so that you can just draw the portion of the layers you want to get,
14:05
and say I want a shapefile if it's a vector, I want a GOT if it's a raster, and for this resolution, and you get your email when it's finished.
14:20
So now let's talk about JERKestra in production. What does it mean in production? You're installing it on a server, so we'll speak about hardware, operational system, middleware and provisioning, how we install it. Scaling is an important question too,
14:42
because when you have a huge amount of people arriving, you have to handle the load. And we'll also talk about monitoring the system. So first about hardware. At Cam2Camp, we've been deploying JERKestra on small to medium-sized machines,
15:03
either on dedicated hardware ranging from two to 32 core CPUs, and eight to 128 gigabytes of RAM. And we also use OpenStack instances. This is mainly for our demo or dev instances.
15:24
About the OS, JERKestra has been run-time tested on Debian 6 to 8. It works very great. It's a reference platform. And some customers also reported that it works okay on Redat and CentOS boxes.
15:46
So JERKestra relies on components provided by your Linux distribution. This is what we call middleware. And these are the four components that we use to support JERKestra.
16:06
Apache can be, of course, replaced by Nginx as a web server. Tomcat, which hosts the applications. PostgreSQL with PostGIS as a DB.
16:22
And OpenLDAP as a user and group database. All this middleware can be installed in the context of JERKestra through two, through either manually, of course, but this is not very practical.
16:41
Or we can do this automatically through two software called Poopet or Ansible. For Poopet, we have a recipe which we keep private at come-to-come for the moment. It's not yet open source, but the Ansible recipe is open source. So I show two different deployment scenarios.
17:04
One with Poopet, where you say, on this machine, I want to install a standard JERKestra. Or with Ansible, people write some YAML files, which contains the variables describing your deployment. And just with one command, if you have SSH access to your server,
17:23
you can install automatically a JERKestra instance on your server with Ansible or Poopet. Yes, a difference is that Poopet, for the moment, goes up to the middleware, but Ansible goes a bit further.
17:41
So it installs the middleware, but also the applications. A bit about scaling. So we've got a modular architecture. This means it's easier to scale, because you can distribute the applications on several machines.
18:02
But when there are too many people on your service, you have to scale the GeoServer component. The GeoServer is responsible for the OGC services. So with Poopet, this is how we do. We say we want to load balance GeoServer, and it creates two worker instances of GeoServer.
18:25
To go further again, we would need to scale the security proxy. So we have two options. Either we share sessions between Tomcat instances. This can be done.
18:41
Or we can remove the sessions from the proxy, make it a stateless proxy. This is the solution that we think we will be investigating in the future. Okay, so when you start having a system in production, you cannot forget monitoring. Monitoring is about getting your system always up and running.
19:05
There are several pieces of software that we are using or considering using. I've listed them in this slide. In bold, these are those we are using currently. The other ones are those we are considering in the future.
19:21
And so we are using a range of solutions which allows us to check from the base system to the OGC services. This is very important to have. I won't go through this because I'm short. So what's next in GeoCastra?
19:42
So as I already said, we will have a new viewer based on OpenLayers 3 and August GS, maybe next year when it's ready. Many customers are currently buying us custom modules tailored for specific needs. For instance, modules for handling data related to city planning, cadastre, urbanism.
20:12
We're also working on Red Hat and Debian packages, because this is very, very important for software distribution.
20:21
It will allow us to streamline the installation process so that from the bare OS to the OGC services with Poopet or with Ansible, we can be up and running in five minutes. You may also have heard about Docker, a virtualization system that we're currently using for development or demo purposes.
20:47
The big question right now is, is it ready for production? So we're still trying to answer this question. We do not have a definite answer for now. And scaling all the components would be really nice.
21:05
For now, as I said, we'll only scale GeoServer, but we'd like also to extend scaling to auto-scaling, which means that according to the load, your components would, without human intervention,
21:21
replicate across different nodes in a cloud architecture, for instance. That would be really nice. This is something that we'll be investigating. So as a conclusion, what we learned at Camp2Camp with SDIs, we learned that infrastructure is key,
21:42
because once you set up OGC services and you tell your users to use these services, you cannot tolerate any downtime. So you always need load balancing, monitoring, and this is provided by your IT.
22:03
So for all these reasons, also for backup reasons, you need to talk to your IT guys or to us if you have such concerns. Okay, thank you very much. If you want to know more about GeoChestra, you can go to this URL, and I'm also available to answer your questions.
22:24
Thank you very much. So it's two questions.
22:42
The first one is, do you usually have this on cloud servers like Amazon AWS, or do you usually have in-house servers? This is not in-house.
23:00
Usually we host this at a different provider. We do not provide the hardware. But this is not yet cloud computing. But this is something that we have in mind for the future. Yes, of course, elastic computing would be great to automatic scale. And the second question is how we store the geospatial data.
23:25
Is it converted to PostGIS, or is it saved as files in the server? This is as you want. Because as this is a GeoServer, you benefit from all its connectors. You can connect to PostGIS DB, MySQL DB, shapefiles,
23:44
WMSW, remote WMS, or WFS, and so on. For instance, a user has a shapefile. Does it offer the chance of converting to PostGIS and storing there, or it goes in the same format as the user had?
24:04
At the moment, the user has to send his file to an administrator, and the administrator references it in GeoServer configuration. But there are people in the georchestra community which have developed
24:21
Drag your file to upload to the server using own cloud, for instance. And once the file is uploaded, it gets automatically published in some workspace in GeoServer. But the problem with this approach is that it's not structured.
24:41
Usually, with SDIs, you want to structure your data, put this layer in this workspace. And users might make mistakes, so that's the reason why, at the moment, it goes through human intervention before the data gets visible to everyone.
25:09
I have a question on the continuous delivery. Is there an update procedure for existing geo-orchestra installations? Yes, you can always. We've got stable branches in georchestra,
25:24
and at a given time, we support three versions. And for each version, we have bug fixes in the release branches. So it means that at any time, you can redeploy your georchestra instance
25:46
using the same branch, the same version that you've been deploying at first. So you don't need to change anything. You just need to redeploy your web apps.
26:02
So what about if there are some changes in the database? Of course, when you switch versions, then you need to apply manually. We have a change log where we say everything that has changed and that has to be done before upgrading your web apps from one version to another.