Running QGIS Server in production
This is a modal window.
Das Video konnte nicht geladen werden, da entweder ein Server- oder Netzwerkfehler auftrat oder das Format nicht unterstützt wird.
Formale Metadaten
| Titel |
| |
| Serientitel | ||
| Anzahl der Teile | 295 | |
| Autor | ||
| Mitwirkende | ||
| Lizenz | CC-Namensnennung 3.0 Deutschland: Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen. | |
| Identifikatoren | 10.5446/43391 (DOI) | |
| Herausgeber | ||
| Erscheinungsjahr | ||
| Sprache |
Inhaltliche Metadaten
| Fachgebiet | ||
| Genre | ||
| Abstract |
| |
| Schlagwörter |
00:00
Produkt <Mathematik>ZahlenbereichServerKonfiguration <Informatik>SoftwareentwicklerBenutzerbeteiligungOpen Source
00:36
SoftwareServerSpezialrechnerServerOpen SourceProjektive EbeneQuantengravitationBenutzerbeteiligungZahlenbereichHochdruckKartesische KoordinatenMAPSpeicherabzugSoftwareentwicklerVerkehrsinformationParametersystemMapping <Computergraphik>TesselationUmwandlungsenthalpieKonfiguration <Informatik>StandardabweichungWort <Informatik>ElementargeometrieMehrwertnetzSoundverarbeitungSystemaufrufBestimmtheitsmaßMomentenproblemCASE <Informatik>HypermediaProtokoll <Datenverarbeitungssystem>
03:19
MathematikServerVersionsverwaltungThreadMultiplikationSchnelltasteDienst <Informatik>SystemplattformExplosion <Stochastik>Web-SeiteThermodynamischer ProzessRenderingSchnelltasteVersionsverwaltungSystemplattformVerkehrsinformationThreadServerProgrammfehlerQuantengravitation
04:42
SpezialrechnerServerProtokoll <Datenverarbeitungssystem>StandardabweichungWeb-SeiteProgrammCGI <Gateway>Prozess <Physik>Overhead <Kommunikationstechnik>Divergente ReiheThermodynamischer ProzessAuswahlaxiomDatenparallelitätFrequenzServerNotepad-ComputerProzess <Physik>BenutzerbeteiligungProgrammierungFigurierte ZahlCASE <Informatik>Generator <Informatik>ZahlenbereichParametersystemDigitale PhotographieDynamisches SystemKartesische KoordinatenMultiplikationsoperatorWeb-SeiteTypentheorieLeckHalbleiterspeicherInhalt <Mathematik>ComputerspielBeobachtungsstudieAuswahlaxiomExogene VariableDifferenteWeb SiteInternetworkingBeanspruchungProtokoll <Datenverarbeitungssystem>Zellularer AutomatBitMapping <Computergraphik>Divergente ReiheGatewayBimodulMAPMultiplikationLokales MinimumGüte der AnpassungParallele SchnittstelleMechanismus-Design-TheorieSpezialrechnerArithmetisches MittelDreiecksfreier GraphOverhead <Kommunikationstechnik>StandardabweichungDatenparallelitätProgrammierparadigmaClientInterface <Schaltung>Thermodynamischer Prozess
11:03
AuswahlaxiomThermodynamischer ProzessDatenparallelitätBenutzerfreundlichkeitZahlenbereichMultiplikationsoperatorProzess <Physik>HalbleiterspeicherKartesische KoordinatenCASE <Informatik>ATMModul <Datentyp>Konfigurationsraum
11:27
Thermodynamischer ProzessProxy ServerPASS <Programm>BimodulPhysikalisches SystemSpezialrechnerProgrammWrapper <Programmierung>AuswahlaxiomServerPerfekte GruppeVektorrechnungInformationWeb logSocketTelekommunikationBenutzerbeteiligungProgrammierungServerBimodulProzess <Physik>Proxy ServerBitVirtuelle MaschineBildgebendes VerfahrenInstallation <Informatik>DifferenteThermodynamischer ProzessZusammenhängender GraphWrapper <Programmierung>SpezialrechnerPhysikalisches SystemKartesische KoordinatenGüte der AnpassungSocketRichtungKonfiguration <Informatik>InternetworkingWort <Informatik>Web logDreiecksfreier GraphKonfigurationsraumMechanismus-Design-TheorieCASE <Informatik>ComputerspielTopologieBootenDatensatzMailing-ListeDistributionenraumAuswahlaxiomMessage-PassingBeanspruchungBeobachtungsstudieRechenwerkMultiplikationsoperatorNatürliche SpracheWiederkehrender ZustandComputeranimation
17:24
SocketKanal <Bildverarbeitung>Snake <Bildverarbeitung>MereologieBildgebendes VerfahrenServerBenutzerbeteiligungMultiplikationsoperatorFokalpunktComputeranimation
18:20
Vollständiger VerbandImplementierungMereologieNeuroinformatikServerBildgebendes Verfahren
Transkript: Englisch(automatisch erzeugt)
00:07
Today, I'm going to report on our experience with running QG server in production. So I will go through a number of techniques that you have to running QG server in production.
00:25
And I will talk about the drawbacks and the advantages of each option you have based on my experience. So my name is Eric. I work at Oslandia, which is an open source company located in France.
00:43
So we provide service on QGs, POSGs. We also do development on 3D web, 3D stuff. And we also do data science activities. So I'm not a core QG developer, but I've been setting up a QG server for a number
01:02
of customers that we have. So I have some experience with setting it up, and this is really a report about this. So this is the agenda for the talk. First of all, I will introduce QG server, for those of you who don't know QG server. And then I will get into more technical details, talking about CGI and
01:24
fast CGI, which are standards for the web. And then I will explain the options you have to running QG server using those standards. And before concluding, I will say a word about Docker and
01:40
how to use Docker for QG server execution. So a quick introduction to QG server. And by the way, this is a logo that colleagues of mine created. It's not official logo at this moment, but it may become official in the future. But I just wanted you to see it, because I think it's quite nice.
02:04
So yeah, QG server, QG server is a map engine, it's an OGC server. So it kind of competes with a map server and geo server. And it supports a number of OGC protocols, including WMS, WFS, WFST as well, is supported, and also WMTS for tiles.
02:27
One of the particularities of QG server is that it includes lots of vendor specific parameters and requests, which makes it very flexible for developing applications. So for example, there is this get print request that you can use for
02:44
printing maps to PDFs, so very convenient for your applications. So what QG server really is, it's about publishing your QG projects to the web. So what you do is you design and create your maps using QG's desktop.
03:04
And then you save your map as a QG project, and then you can reuse this QG project with QG server. So you can publish your QG projects using QG server to the web.
03:21
So actually, QG server changed a lot between version two and version three of QG's. There have been many, many improvements between those two versions. Some of the highlights include multi-thread rendering, as done in QG's desktop, WFS 1.1 support.
03:43
And now there is WFS 3 that have been merged very recently. You can implement your own services using the Python bindings support. There is many things, and tons of bug fixes as well,
04:03
that were made in this release. And we've also worked on the OGC compliancy platform, which is a platform that we use. It's a server that we use to assess
04:21
that QG server remains compliant, OGC compliant. So for example, this is a report page for the WMS 1.3.0 compliancy. So each night, we have processes running to assess the compliancy.
04:44
So that was the introduction to QG server. Now I'm going to get into more technical details. Because to know how to run QG server, you have to understand a bit of those two standards.
05:01
So CGI and fast CGI standards. Because yeah, QG server is actually a CGI, fast CGI program. So let's start with CGI. I'm sure many of you know this standard. It's a very, very old standard.
05:20
It stands for Common Gateway Interface. So it was invented a long time ago, where the web was essentially HTML static pages. So it was invented for web servers to execute programs that generate web pages dynamically. So basically, you have an HTTP request
05:42
that comes in. It's handled by the web server. The web server figures that this request needs to be handled by a program. So it will pass the request to the program. Actually, it's going to start a process for that program. Then it will pass the request to the program.
06:01
The program is going to handle the request, create some contents, a response passes it back to the web server. And then the web server will finish the process. And then it will wrap the response into an HTTP response and sends it back to the requesting client.
06:21
So that's the way it works. And the paradigm of this standard is that you have one process per request. So the CGI process is created at the start of the request and torn down at the end of the request. And this may incur a very high overhead based on your application workload.
06:45
So CGI, again, was invented a long time ago for dynamic content. It's very, very, very simple. Very simple to implement, very simple to understand, but it's definitely not appropriate for high-end applications.
07:01
And it's not appropriate for QG server type of workloads. This is why fast CGI was invented. Fast CGI, obviously, is more recent than CGI, but it's still a very, very old protocol. So in this case, with fast CGI,
07:21
instead of creating a new process each time for each request, your processes are persistent, and they can handle a series of requests as opposed to handling only one request. So for that reason, fast CGI is much more
07:41
efficient than CGI. And you want to make sure you use fast CGI for QG server. This is very critical to get good performance out of QG server. And it might seem obvious, but there is some compatibility between CGI and fast CGI.
08:04
So maybe you think that QG server runs using fast CGI, but actually it's not, because you didn't configure, let's say, Apache the proper way, and you think you use the correct interface, but you actually don't. So now let's look at how we can execute QG server
08:24
as a fast CGI program into more detail. So you have to have a web server first, because as a fast CGI program, QG server does not know anything about HTTP.
08:40
It's unknown to him, to it. No HTTP support. It's just a fast CGI application. So you have to have a web server, and a web server that supports fast CGI. And Apache and Nginx are very popular web servers,
09:01
very flexible. You can do many, many things with them, including fast CGI, but also other stuff. So we are going to focus on those in the rest of the talk. And there is actually a big difference between the two servers, for as far as fast CGI is concerned,
09:23
is that with Apache and with the FCGID module, Apache will manage the application processes for you, meaning that it will take care of creating the process, tearing, stopping the process, et cetera. All the life cycle of your application
09:43
will be handled by Apache and this module. While if you use Nginx, this is not the case. So you will have to have some mechanism for postponing the application processes, and I will go back to this in a minute.
10:01
So let's look into more detail at Apache with this FCGID module. It's a technique that people have been using for a long, long time. Yeah, I think people running Map Server, for example, they use this technique a lot, this mechanism.
10:21
And based on my experience, it's still a very, very good choice. It has many advantages. It's very easy to set up. You can find many good examples on the internet. As I said already, the FCGID module will manage the fast CGI processes for you.
10:40
So very, very easy also for that reason. And you can have also multiple application processes for concurrent requests. So it handles parallelism for you, which is quite good. And it's also a very robust solution because you have parameters like this one, max request per process.
11:03
So you can configure it to say, after a certain number of requests, I want you to stop the process and recreate it the next time. And this is very, very good for buggy applications where you have memory leaks, for example, and in that case you want to restart the process
11:20
from time to time. And this can do it. So I don't want to describe this into detail, but this is what the configuration looks like using this module, Apache module. And there is another module that you can use. Apache, again, it's the proxy FCGI module.
11:44
But this one is very different because this one will not manage the fast CGI processes for you. So you have to do it by yourself. And this is exactly the same case as with NGINX. With NGINX, there is only one module that you can use.
12:03
And it doesn't manage the fast CGI processes for you. This is the configuration using NGINX. I want to go first and talk about, so we said with NGINX,
12:20
you have to manage the fast CGI processes so how do you do that? How do you start the processes? How do you handle all the life cycle of your fast CGI application? So there is very easy solution. That's a solution you find when you read documentation,
12:40
when you Google fast CGI plus NGINX, for example, you will see this solution. It's very easy because it's packaged for a lot of Linux distros. There is a system disservice within the package, so very easy to set up, but don't use it.
13:02
Because it's very, very slow. Actually, it's fast CGI, but it will run your applications as a CGI program. So one process per request, very slow. Don't use it. There is a better choice, which is this Pwn FCGI package application
13:26
that you can use. It's actually a thin wrapper around your fast CGI program and it will take care of creating the socket, the communication socket, between your web server and your fast CGI program.
13:42
But it's also more work on your side. It's also packaged in your distribution, but you will have to write some service for it to be able to start it, for it to be started at each reboot, et cetera. So you will need to write a systemd service
14:00
for a bit more work on your side. It's also a very good tool for if you want to run QG server in Docker, and I will go back to this at the end of the talk. Another good choice, which we found,
14:22
is to rely on systemd directly. systemd has all the mechanisms needed for this. So there is the socket kind of service that you can use to create a socket and it can wakes up your process.
14:41
So there is some very good mechanism that you can leverage here. So in this case, you don't use pwn-fcgi as a wrapper to your process, but the wrapper is systemd itself. So it's one component less, because in this case, you also need to write a systemd service. So one component less.
15:01
So it's quite a good solution and we have good experience with it and we think it's a good option. We actually have a full blog post explaining how to set this up. If you want to check, if you're interested, you can check it out.
15:22
So to sum it up on the options you have, so Apache and NGINX are actually a very, very good solution. They both work very well, so it's no surprise here. Apache fcgi-d is very easy, flexible and robust solution. So if you're an Apache user
15:42
and you already use Apache for other stuff, this is definitely a good option. If you use NGINX, don't use fcgi-wrap for managing your processes. Use pwn-fcgi or systemd directly.
16:01
A word about Docker. So there are actually many Docker images for QG server on the internet. Some are good, some are not as good. They're often based on Debian because it's very easy to install QG server on Debian-based systems.
16:23
So Docker is very useful, I think, for QG server because, as I said, it's very easy to install it on Debian. It's not as easy to install it on other Linux distros, like, for example, CentOS. So if you use Docker,
16:41
then you can use Docker on your CentOS machine and this makes it easy to install and deploy QG server. So at Oscillondia, we have a Docker QG image. Yes, it's based on Debian. It uses the pwn-fcgi wrapper that I talked about
17:03
and it exposes a fast CGI TCP socket. So there is a big difference between other images that you can find on the internet is that it doesn't include any web server. It's just fast CGI exposed socket, fast CGI process running into this container
17:23
and so, yeah, no web server. So this is quite flexible for you because if you already have a web server running for other workloads, you can reuse it. So it's not, it's an Oscillondia image at this time and there is this QG enhancement proposal
17:45
discussing having official Docker images for QGs and we'd like to push our image. We'd like to, yeah, we are talking about using this image as part of this QEP. So we'd like, we'd really much like to make it official.
18:04
I don't know how I'm running that. Yeah, this is the end of the presentation. Thank you. Thank you, Eric. Is there any questions for him?
18:27
No question. Yeah, there is one over there. It's actually not a question but a comment. We discussed it at the contributor meeting
18:40
to have an official Docker image for QG server. So your QEP would be most welcome and there is other implementation. We want to try to get the best of all the approaches out there. Okay.
19:00
Anyone else? Okay, thank you. Thank you.