Improving QGIS plugin developer experience
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 266 | |
| Author | ||
| License | CC Attribution 3.0 Germany: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/66445 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
00:00
Software developerPlug-in (computing)SoftwarePhysical systemSystem programmingClient (computing)Archaeological field surveyPlug-in (computing)Library (computing)Directory serviceProfil (magazine)ImplementationSingle-precision floating-point formatMedical imagingGraphics tabletCodeReading (process)Scripting languageBitSoftware developerRevision controlIntegrated development environmentDistribution (mathematics)Multiplication signRepository (publishing)Subject indexingPhysical systemModule (mathematics)Similarity (geometry)Musical ensembleDivisorProduct (business)SpacetimeSocial classComplex (psychology)Dependent and independent variablesDebuggerField (computer science)SoftwareMobile appComputer animation
03:43
Data managementSoftware developerLibrary (computing)Different (Kate Ryan album)Data managementPlug-in (computing)Line (geometry)Computer animation
04:06
Asynchronous Transfer ModeRepository (publishing)Electric currentPlug-in (computing)Integrated development environmentDebuggerServer (computing)CodeCodeAeroelasticityOcean currentIntegrated development environmentMaxima and minimaNormal (geometry)Set (mathematics)Run time (program lifecycle phase)Revision controlPlug-in (computing)Module (mathematics)Server (computing)DataflowVariable (mathematics)Library (computing)Proxy serverAdditionWeb pageComponent-based software engineeringInstallation artNamespaceDistribution (mathematics)Thread (computing)Dot productComputer filePlanningBinary codeComplex (psychology)Medical imagingRepository (publishing)Asynchronous Transfer ModeDebuggerMereologyDirect numerical simulationComputer programmingOperator (mathematics)Buffer solutionComputer animation
09:52
Component-based software engineeringCodeComplex (psychology)Library (computing)Binary filePlug-in (computing)Revision controlRepository (publishing)SoftwareGraphical user interfaceInstallation artConfiguration spaceDistribution (mathematics)Binary codeRun time (program lifecycle phase)CodeWrapper (data mining)Integrated development environmentModule (mathematics)Plug-in (computing)Library (computing)GeometryRepository (publishing)Revision controlSoftwareRun time (program lifecycle phase)Installation artRemote procedure callDebuggerUtility softwareInstance (computer science)EmailNamespaceDifferent (Kate Ryan album)Configuration spaceBootstrap aggregatingPointer (computer programming)LogicComputer fileRootImplementationGraphical user interfaceNormal (geometry)MathematicsComplex (psychology)MultiplicationOcean currentSource codeData managementComponent-based software engineeringKeyboard shortcutMusical ensembleDynamical systemGraphics tabletMetadataINTEGRALData structureCase moddingMultiplication signBootingCartesian coordinate systemRepetitionStatement (computer science)Form (programming)PlanningComputer animation
15:38
XMLUMLComputer animation
Transcript: English(auto-generated)
00:08
Hello, welcome to this talk on improving future plug-in developer experience. A little background about me, my name is Sander Komi and I'm a software developer at
00:21
China Institute of Technology in England. It's a governmental company responsible for its development of future plug-in developers. And about our future plug-in developer, we do our top traffic data production system which is currently in development. And we have a few similar systems in use already, which are basically plain QGES with future
00:49
provided by custom QGES plugins. And basically based with our developer experience were firstly the dependencies.
01:00
We use deep software tools, which I believe many of you use. It's a bit difficult to organize together with dependencies that are provided from deep install and also with that, if you control the repository where you have the support
01:21
tool, you also have to use the related supports because the software will be in the main space of your plug-in package. And also, there's the debugger setup, when you want to debug future plug-in.
01:42
We also have often, we duplicate the code in our plug-in package before the class factor method so that we set up the debugger there. And also, for the development environment, so that you get the QGES running with your plug-in and upgrade for development, for us it was a collection of field scripts to
02:06
make the QGES launch with the plug-in back. Or maybe there was one implementation where the plug-in was actually similar to the profile directory, so that also was a bit complex for us.
02:25
And also with the dependencies that are not key sub-modules, if you want to use, for example, any library from the package index, you must have the dependencies available at runtime. So in the environments that you do not control, it's a bit difficult.
02:46
So you have to prompt the user or make the user install it in the region. And for distribution, in the environment that we do control, it's another step to make it in the base image or install it in a separate step
03:05
for the deployment pipeline. With all those approaches, if you can have only a single version of the library in a single QGES session, you can't really make sure that it's the version that you want.
03:21
But if the user has a library installed in the QGES environment, and some other plug-in imports that before your plug-in does, it will not be that version that you, for example, wanted in your own plug-in and maybe added that to the app via some code in your plug-in.
03:43
So with some of these issues, we created a library called QGES Plug-in DevTools, which is essentially a command-line tool to improve the developer experience and make the dependencies management of all of these things. It's available in our GitHub and also installable from different package engines.
04:05
And what it is, it's essentially three commands. There's a start command, which has the development mode. Then there's the build command, which is a plug-in to the creation,
04:22
and a publish command, that is the way you push it to the QGES plug-in development. So it has all the three main features for it. And what it does, the start command, it's a command to launch the QGES
04:42
with the current repository for your environment, ready for development. So the plug-in code is stored from the environment. You have to have the plug-in installed or available to import from the environment you run this tool from. And the dependencies are also available from the current environment.
05:01
For example, we use virtual end for our development, so the dependencies can be installed in that, and it's available in the QGES after it's launched. And also, the debugger server code is moved to this library, so we don't have to duplicate that in each one of our plugins.
05:22
So this handles that, and we don't have any non-front-time essential code in our plug-in code. And also, it works with any double installs. If you have a library that you are developing together with your QGES plug-in, it also creates some hooks to the QGES reload plug-in commands
05:42
to also reload the DNS library as well. And what the forge command does, it basically creates your plug-in to the file. It does this by bundling the runtime dependencies into the plug-in zip file.
06:03
And it's also compatible with the other build-on-package tool. For example, we use a set of tools for our build-on-package, and the version can be sourced from the installed distribution method. And if you compare this to the build-on-package tool,
06:22
you might compare it with the build-on-package operating standard minimal with command. The next command is the buffer command, which is a very simple comparable to line,
06:42
which just pushes your package to QGES plug-in. And we created this in addition to the library. We used it previously because we had issues in our environment with proxies. It's the XML-MVC library.
07:01
It's not that well supported. So this one uses requests via JSON-MVC, so that works with the standard proxy environment. And how does this improve our developer experience? We get the dependencies from the environment.
07:21
So, in a normal workflow, we can just take any dependency that's well formed with the package, and use that with the install and do the import from there, and we work on the development part. And also we can use absolute imports,
07:42
so that we always know what the package is, because if you have relatively important, and you have very nested namespace, the sub-module will be found in the library. We have 60 nodes. It's not that obvious where the module is imported from.
08:01
And also the debugger is set up in the tool, and the debugger environment is set up like you start there.
08:22
Right, with the command that you get there. You get the kujasa. And the improvements in our dependencies, we can use any well-formed written package,
08:46
so that the library can be installed in the virtual end, and we have it available. So this works for us, mostly for sharing common code, for example, UI components, etc.
09:00
We have one common library used called kujasa module tools, which has, like, blocking set up, some UI components, some of the helpful code that we often use in all of our plugins. And also, this should work with most other built-in libraries
09:24
that don't have any complex imports or binaries. For example, we took a dependency on a library called beauty-utils, which had some helpful code for managing our training issues. It's often a problem if you use the UI thread from another thread in kujas,
09:46
so it often crashes. So, with this Qt tools library, we found an integrated program that we can just use to execute some code always in the major. In that, there was actually quite complex logic
10:00
to handle the possibility of two different Qt implementations, like the PySide or the PyQT. So we also could replace that. So how does the dependency management work? When you add this tool or use this tool,
10:24
you declare your current built-in package name of your plugin, and then you declare the runtime packages that you have in the environment or in your code. And in your code, you can just use the import from the package,
10:42
as is, for example, here. If you use some dependency, import something from that, and just use those as is. And with this configuration and this code, after the bootstep, the code becomes like, shown in the picture, where actually the dependency is moved to a namespace
11:05
below your plugin, and all the imports in the code are actually replaced to match with the new namespace. This import-replacement logic, this is the reason why it works like for
11:22
multiple mobile-form Python packages, where there is no complex import logic or any dynamic imports. That we can do the replacing at the bootstep. So, for our improvements,
11:40
also in our sharing our code, with this style of using dependency libraries, we can also often share both the library as a normal built-in package, and we can share the plugin as like a pay-per-view implementation for the library.
12:01
For example, our quality checker, it's mostly used as a library. It's basically a GUI component, and it's basically a JSON geometry metadata, you name it, profit. So we use it as a library, but for development, there's a very simple wrapper
12:23
where you can basically create a wrapper and a JSON file, and it just calls the same API as we do in our code. With this, you can use the library API, instead of accessing, for example, a package instance from the utilities package in QGIS.
12:48
Also, with this structure, there's technically two versions, essentially, of the same code. For example, our segment reset plugin, also known as GitHub,
13:00
we use it as a library for another software integration, where we basically provide the same feature in other, like a remote editing application, and then we have it also available as a plugin to the user. So in our own code, we use this library, we can have a specific version, and the user can install another version of the plugin.
13:23
So those will not conflict. So we don't have to manage the, like for example, if you have a break and change, the user can install that, and we don't have yet changed our own code. So, how do you start using this tool?
13:40
You have to set up your repository as an installable Python package. It should work also if you have a root package in your repository, and use the Python N command, which magically adds that.
14:05
You have to have the package name, as shown previously, and if you have any dependencies, declare those as well. And then for development, you just set up your .env file, which has to add to your QGIS file.
14:22
So with that configuration, it's just basically long, just to be able to do that, and you can see the pointer path you provided with a bootstrapping code that will actually do all the debugger setup, the environment setup, and install it for you as well. And if you have dependencies as submodules correctly,
14:42
you could follow use of those as Python packages. You can do that even without changing the actual software code in that state. We actually did the change in the QGIS. The plugin tools repository, which was a good module before this implementation, and then you install your runtime dependencies
15:04
in the same environment, and you switch your submodule imports to the plain module name imports from the environment.
15:20
That's all for the tool. You can find it in our GitHub. There's also the other plugins mentioned in my talk, and you can find us at this, or you can find us by email.