We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

PgMetadata - A QGIS plugin to store the metadata of PostgreSQL layers inside the database, and use them inside QGIS

00:00

Formal Metadata

Title
PgMetadata - A QGIS plugin to store the metadata of PostgreSQL layers inside the database, and use them inside QGIS
Title of Series
Number of Parts
351
Author
License
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language
Production Year2022

Content Metadata

Subject Area
Genre
Abstract
PgMetadata is made for people using QGIS as their main GIS application, and PostgreSQL as their main vector data storage. The layers metadata are stored inside your PostgreSQL database, in a dedicated schema. Classical fields are supported, such as the title, description, categories, themes, links, and the spatial properties of your data. PgMetadata is not designed as a catalog application which lets you search among datasets and then download the data. It is designed to ease the use of the metadata inside QGIS, allowing to search for a data and open the corresponding layer, or to view the metadata of the already loaded PostgreSQL layers. By storing the metadata of the vector and raster tables inside the database: - QGIS can read the metadata easily by using the layer PostgreSQL connection: a dock panel shows the metadata for the active layer when the plugin detects metadata exists for this QGIS layer. - QGIS can run SQL queries: you can use the QGIS locator search bar to search for a layer, and load it easily in your project. The administrator in charge of editing the metadata will also benefit from the PostgreSQL storage: - PostgreSQL/PostGIS functions are used to automatically update some fields based on the table data (the layer extent, geometry type, feature count, projection, etc.). - The metadata is saved with your data anytime you backup the database - You do not need to share XML files across the network or install a new catalog application to manage your metadata and allow the users to get it. The plugin contains some processing algorithms to help the administrator. For example: - a script helps to create or update the needed "pgmetadata" PostgreSQL schema and tables in your database - a algorithm creates a QGIS project suitable for the metadata editing. This project uses the power of QGIS to create a rich user interface allowing to edit your metadata easily (forms, relations). Why use another interface when QGIS rocks ? More PgMetadata features will be shown during the presentation: - Modification of the template to tune the displayed metadata - Export a metadata dataset to PDF, HTML or DCAT - Publish the metadata as a DCAT catalog with Lizmap Web Client module for PgMetadata. It can then be harvested by external applications (Geonetwork, CKAN) - The data model is very close to the QGIS metadata storage and the DCAT vocabulary for compatibility. We will also show the last features such as the new support of the PostgreSQL rasters
Keywords
MetadataDatabaseComputer animation
MetadataLink (knot theory)FrequencyGeometryGroup actionData typeBuildingControl flowForm (programming)View (database)Constraint (mathematics)Raster graphicsArray data structureFunction (mathematics)MathematicsMetadataEndliche ModelltheorieCategory of beingSystem identificationLink (knot theory)LogicView (database)Game controllerRepresentation (politics)PlastikkarteConnected spaceDatabaseClient (computing)Multiplication signRelational databaseRouter (computing)Set (mathematics)Right angleAbelian categoryMessage passingSource code
MetadataSystem administratorComputer animation
DatabaseArray data structureInstallation artOverlay-NetzGrass (card game)GeometryComputer networkMathematical analysisRaster graphicsMetadataPlug-in (computing)View (database)Data structureDemo (music)Parameter (computer programming)StapeldateiProcess (computing)Plot (narrative)Table (information)System administratorInformationTemplate (C++)AlgorithmConfiguration spaceIntegrated development environmentCustomer relationship managementSelf-organizationExecution unitField (computer science)Content (media)TimestampElectronic mailing listCountingCuboidMetadataAlgorithmNetwork topologyTable (information)View (database)LogicExpressionProcess (computing)Integrated development environmentClassical physicsDatabaseProjektive GeometrieForm (programming)Centralizer and normalizerRelational databaseFunctional (mathematics)Number theoryGeometryDemo (music)Set (mathematics)Representation (politics)Client (computing)FrequencyPlug-in (computing)Attribute grammarContent (media)Translation (relic)Game controllerAbstractionExtension (kinesiology)Type theorySystem administratorField (computer science)Source codeComputer animation
Personal digital assistantLevel (video gaming)Process (computing)Demo (music)Maxima and minimaScale (map)Uniform resource locatorMetadataIntegrated development environmentBoundary value problemAbelian categoryCache (computing)Source codeType theoryDeterminantOpen setDatabaseMetadataSystem administratorNetwork topologyLine (geometry)DatabaseLink (knot theory)Descriptive statisticsDifferent (Kate Ryan album)File formatGame controllerProbability density functionMeta elementComputer animation
Content (media)PlastikkarteRepresentation (politics)Table (information)Configuration spaceVariable (mathematics)Decision tree learningMethodenbankMetadataWeb portalDatabaseVideoconferencingInstallation artPrice indexMereologyMetadataPresentation of a groupLocal ringProjektive GeometrieLevel (video gaming)System administratorGame controllerDatabaseRepresentation (politics)PlastikkarteConfiguration spaceFunctional (mathematics)Selectivity (electronic)Web 2.0Network topologyConnected spaceDefault (computer science)Table (information)Endliche ModelltheorieLatent heatWeb browserDifferent (Kate Ryan album)Formal languageComputer animationSource code
Source codeWeb portalMeta elementWeb pageMereologyMaxima and minimaLevel (video gaming)Data exchangeDifferent (Kate Ryan album)Web portalTranslation (relic)Cartesian coordinate systemLink (knot theory)SoftwareMetadataServer (computing)Presentation of a groupDatabaseClient (computing)File formatSlide ruleRevision controlWeb applicationOpen setClosed setMethodenbankWeb browserField (computer science)Centralizer and normalizerRaster graphicsOpen sourceShared memoryWeb 2.0Plug-in (computing)Computer animation
VolumenvisualisierungAbstractionMaxima and minimaScale (map)Abelian categoryFeedbackFeedbackComputer animation
Transcript: English(auto-generated)
Thanks a lot. I will speak about PgMetadata, which is a QGIS plug-in to manage metadata inside your PostgreSQL database. So what is metadata? Quick reminder, it's to help people to understand your data.
Like, if you're a creator, you need to tell them what is your data. So you have a title, an abstract. You can have categories, keywords. You can put in the metadata the spatial properties of the layer of the data. You can help to know when the data has been created,
modified, what is the license, how many features you have in your data. So it's a full identification card of your data. And it's very important if you need to share your data. And even more important, if your data changes a lot
throughout time, to know if it's still valid or not. So on the right, you have a small representation in the data set, contacts, and links for your data. It's a very simple model.
PgMetadata was designed for people using PostgreSQL to store their vector and restore data. Why? Because we think PostgreSQL is a great tool. And you have your data centralized in the same place. So if you have the metadata with your data, that's great.
It's very accessible. You only need a PostgreSQL connection to access your metadata, not a third-party tool. So you can benefit PostgreSQL-rich features like relations, constraints, triggers, views.
So you can use the logic of PostgreSQL to help create or update your metadata. You can also use write and access controls. Who can edit? Who can read the metadata? And obviously, you have many clients. You can use QGIS.
You can use PgAdmin. You can use the command line psql tool to see the data in PostgreSQL, so your metadata, too. And it's very easy to backup and restore your metadata. You just backup and restore your PostgreSQL database. So I will present three ways of using PgMetadata
as the GIS administrator, as the GIS user, and another one. So first, you need to, as the admin, you need to create in your database the schema called PgMetadata, which will store the needed tables and views and relations
and functions we created in this tool. So there is a, we always, in 3D, we always, we like to use QGIS processing algorithm because you can use that from the processing toolbox or the plugin can do that for you.
But it's a centralized way of having, of creating data or doing some processing. So you can use this algorithm, and it will install this schema inside your PostgreSQL database or databases. Then you can create with another tool, another processing
tool, a QGIS project aiming to edit your metadata because QGIS is a PostgreSQL client. It has very rich forms, very rich way of managing relations between data. You can use a QGIS project with no special layer,
only to edit your metadata. So for example, here, in this project, you can edit the themes. If you need one more theme, like environment and climate or other, you can add contacts. It's just a layer with attributes.
And you can have forms to edit that with controls, checks, expressions. So you prepare editing. You can also add translations if needed. And then you can use just a classical QGIS form to edit your data set. For example, the tree layer in this demo schema.
And you have the main field, like title, abstract, keywords. And you can use QGIS checkboxes. You can use the combo boxes to choose, for example, the publication frequency. It's very handy to use QGIS as an editing
tool for PostgreSQL layers. So you edit your metadata, for example, for the tree table. And then there are some logic under. And some data are calculated from the table content. For example, there is a valid unique ID for the data set.
We can compute the layer extent, the number of features, the geometry type, the projection ID and name. So PostgreSQL gives us, which triggers the capability to do that. And we have also created useful views to know the orphan PostgreSQL tables, which tables
in your data does do not have metadata. The opposite, orphan metadata, which sometimes you had edited metadata for a table, but you have then dropped the table. So you can see that. And we also managed some flat representations
of the data sets, for example, to export as CSV, to view that in LibreOffice or Excel. So that was for the admin guy, the guy editing the metadata. Now, for this user, so that was supposed to be an animated GIF.
You will have the link here. And basically, you can search with the Control-K in QGIS. You can search inside the title or the description or comments. And you can find your lines of metadata. For example, here, I typed metadata tree.
And I find the trees. And when I click, when I enter, I will have the layer popping up in QGIS and a small dock panel on the right side with all the metadata you wrote down before. So as a user, I can easily find
some layers in the Postgres database I can connect to and then have the metadata just visible when you click on the layer in the QGIS layer tree. The user can also export the metadata in different format, HTML, PDF,
and also DCAT, which is a standard. It's a vocabulary. It's a way of writing the metadata in XML. And so you can use that to import all the tools if needed. The third part of the presentation was the PgMetadata Advanced Features.
Because it's SQL-like and data-driven developed, you can easily change how metadata will show the metadata,
text representation, the layout. You can change this layout we've seen in the right dock. Inside the HTML templates table is just a table with values you can change and write down how you want to organize your metadata. We provided one default one, but you can change it easily.
And we have some SQL functions to, for example, generate an HTML card with SQL. So select PgMetadata, get dataset item HTML. You give the schema and layer a table and the localization.
In FR, it means French. You can have it in English or other compatible language. And we have also created some functions to export as DCAT, for example. And as an administrator in the big companies,
you can also deploy GIS with some already chosen configuration. For example, if the name of the connections to the PostgreSQL database, where the Control-K, where the QGIS search tool can use,
and if you want to open automatically the dock or not, there are some configuration available here. We have also, in 3D, we have also a tool created, which is LizMap Web Client. You can publish QGIS projects online in a web browser.
And so we have also created a model for that. So you can click on the layer in the web layer tree. And you have the same card because it's just select, give me the card. And we use that in LizMap, too,
to display the metadata of a specific layer. And we have also, yeah, we have tried to have a full or rich documentation for different usage for the administrator, for the end user, for the system administrators.
So you can go to docs.3liz.org and see the full documentations with the different parts. As a conclusion, I will try to answer the questions.
Why another metadata tool? There are plenty of metadata tool, server side, applications, client side. And why PG metadata? First answer, see the previous slide about PostgreSQL. It's centralized. It's very easy to access.
You don't need to install another application. You have your database. And data is in the same place. That's, I think, the main thing with that. You keep the metadata as close as possible to the data. And you don't have to install a new application and to maintain a new application.
So you just can focus on maintaining your PostgreSQL database, and that's OK. And the GIS users, they won't need to find your data inside the web application. They just brought that in QGIS. So they are in their tool, QGIS, and they'll just search the data and open it.
It's GIS user-oriented. It's inside QGIS that you use PG metadata, not the way around. And it was not designed to replace the existing metadata web portals, which provide a lot of rich features for sharing
and for importing, exporting. We have some capability to be harvested. If you use LISMAP and the modules, there is a link. You can harvest in DCAT the metadata. So other tools like National Open Data Portal
can harvest the data inside PG metadata, but you need LISMAP web client and the module, which are open source. The roadmap, we will release a new version. I hoped to release one just before today, but we did not manage it.
But in September, we will have a license. It's JPL 2.0, but we just forgot to add one. We will raise the QGIS minimum version to 3.16. It will have raster support, better handling of backslashes
in the links, new fields for the contact, for the license, and other things. And we would like to improve the import-export tools to facilitate the format, the data exchange between different softwares.
Some resources you will have in the presentation. And I would like to thank the French Gare Provence for funding and my colleague here, Etienne Trimay, which is maintaining and has developed a lot of things. I've done the PostgreSQL part,
is the responsible of the Python QGIS plug-in part. And also, many thanks to our external active contributors. We have one very, very active, Florian Yen, for ideas. He provided fixes, improvements, and now we have unknown but kind translators for Finnish,
German, Spanish, and Europe. If you want us to help you, you can contribute for all the translation. Thanks a lot for your attention, and if you have questions, and I would love to hear feedback from users,
because I know there are some, but I never have feedback, so feel free. Thank you.