PgMetadata - A QGIS plugin to store the metadata of PostgreSQL layers inside the database, and use them inside QGIS
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 351 | |
| Author | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/69127 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language | ||
| Production Year | 2022 |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
| |
| Keywords |
FOSS4G Firenze 2022117 / 351
1
7
13
22
25
31
33
36
39
41
43
44
46
52
53
55
58
59
60
76
80
93
98
104
108
127
128
133
135
141
142
143
150
151
168
173
176
178
190
196
200
201
202
204
211
219
225
226
236
242
251
258
263
270
284
285
292
00:00
MetadataDatabaseComputer animation
00:13
MetadataLink (knot theory)FrequencyGeometryGroup actionData typeBuildingControl flowForm (programming)View (database)Constraint (mathematics)Raster graphicsArray data structureFunction (mathematics)MathematicsMetadataEndliche ModelltheorieCategory of beingSystem identificationLink (knot theory)LogicView (database)Game controllerRepresentation (politics)PlastikkarteConnected spaceDatabaseClient (computing)Multiplication signRelational databaseRouter (computing)Set (mathematics)Right angleAbelian categoryMessage passingSource code
02:38
MetadataSystem administratorComputer animation
02:49
DatabaseArray data structureInstallation artOverlay-NetzGrass (card game)GeometryComputer networkMathematical analysisRaster graphicsMetadataPlug-in (computing)View (database)Data structureDemo (music)Parameter (computer programming)StapeldateiProcess (computing)Plot (narrative)Table (information)System administratorInformationTemplate (C++)AlgorithmConfiguration spaceIntegrated development environmentCustomer relationship managementSelf-organizationExecution unitField (computer science)Content (media)TimestampElectronic mailing listCountingCuboidMetadataAlgorithmNetwork topologyTable (information)View (database)LogicExpressionProcess (computing)Integrated development environmentClassical physicsDatabaseProjektive GeometrieForm (programming)Centralizer and normalizerRelational databaseFunctional (mathematics)Number theoryGeometryDemo (music)Set (mathematics)Representation (politics)Client (computing)FrequencyPlug-in (computing)Attribute grammarContent (media)Translation (relic)Game controllerAbstractionExtension (kinesiology)Type theorySystem administratorField (computer science)Source codeComputer animation
06:07
Personal digital assistantLevel (video gaming)Process (computing)Demo (music)Maxima and minimaScale (map)Uniform resource locatorMetadataIntegrated development environmentBoundary value problemAbelian categoryCache (computing)Source codeType theoryDeterminantOpen setDatabaseMetadataSystem administratorNetwork topologyLine (geometry)DatabaseLink (knot theory)Descriptive statisticsDifferent (Kate Ryan album)File formatGame controllerProbability density functionMeta elementComputer animation
07:34
Content (media)PlastikkarteRepresentation (politics)Table (information)Configuration spaceVariable (mathematics)Decision tree learningMethodenbankMetadataWeb portalDatabaseVideoconferencingInstallation artPrice indexMereologyMetadataPresentation of a groupLocal ringProjektive GeometrieLevel (video gaming)System administratorGame controllerDatabaseRepresentation (politics)PlastikkarteConfiguration spaceFunctional (mathematics)Selectivity (electronic)Web 2.0Network topologyConnected spaceDefault (computer science)Table (information)Endliche ModelltheorieLatent heatWeb browserDifferent (Kate Ryan album)Formal languageComputer animationSource code
10:30
Source codeWeb portalMeta elementWeb pageMereologyMaxima and minimaLevel (video gaming)Data exchangeDifferent (Kate Ryan album)Web portalTranslation (relic)Cartesian coordinate systemLink (knot theory)SoftwareMetadataServer (computing)Presentation of a groupDatabaseClient (computing)File formatSlide ruleRevision controlWeb applicationOpen setClosed setMethodenbankWeb browserField (computer science)Centralizer and normalizerRaster graphicsOpen sourceShared memoryWeb 2.0Plug-in (computing)Computer animation
14:13
VolumenvisualisierungAbstractionMaxima and minimaScale (map)Abelian categoryFeedbackFeedbackComputer animation
Transcript: English(auto-generated)
00:00
Thanks a lot. I will speak about PgMetadata, which is a QGIS plug-in to manage metadata inside your PostgreSQL database. So what is metadata? Quick reminder, it's to help people to understand your data.
00:21
Like, if you're a creator, you need to tell them what is your data. So you have a title, an abstract. You can have categories, keywords. You can put in the metadata the spatial properties of the layer of the data. You can help to know when the data has been created,
00:44
modified, what is the license, how many features you have in your data. So it's a full identification card of your data. And it's very important if you need to share your data. And even more important, if your data changes a lot
01:01
throughout time, to know if it's still valid or not. So on the right, you have a small representation in the data set, contacts, and links for your data. It's a very simple model.
01:22
PgMetadata was designed for people using PostgreSQL to store their vector and restore data. Why? Because we think PostgreSQL is a great tool. And you have your data centralized in the same place. So if you have the metadata with your data, that's great.
01:46
It's very accessible. You only need a PostgreSQL connection to access your metadata, not a third-party tool. So you can benefit PostgreSQL-rich features like relations, constraints, triggers, views.
02:02
So you can use the logic of PostgreSQL to help create or update your metadata. You can also use write and access controls. Who can edit? Who can read the metadata? And obviously, you have many clients. You can use QGIS.
02:21
You can use PgAdmin. You can use the command line psql tool to see the data in PostgreSQL, so your metadata, too. And it's very easy to backup and restore your metadata. You just backup and restore your PostgreSQL database. So I will present three ways of using PgMetadata
02:43
as the GIS administrator, as the GIS user, and another one. So first, you need to, as the admin, you need to create in your database the schema called PgMetadata, which will store the needed tables and views and relations
03:04
and functions we created in this tool. So there is a, we always, in 3D, we always, we like to use QGIS processing algorithm because you can use that from the processing toolbox or the plugin can do that for you.
03:21
But it's a centralized way of having, of creating data or doing some processing. So you can use this algorithm, and it will install this schema inside your PostgreSQL database or databases. Then you can create with another tool, another processing
03:42
tool, a QGIS project aiming to edit your metadata because QGIS is a PostgreSQL client. It has very rich forms, very rich way of managing relations between data. You can use a QGIS project with no special layer,
04:02
only to edit your metadata. So for example, here, in this project, you can edit the themes. If you need one more theme, like environment and climate or other, you can add contacts. It's just a layer with attributes.
04:21
And you can have forms to edit that with controls, checks, expressions. So you prepare editing. You can also add translations if needed. And then you can use just a classical QGIS form to edit your data set. For example, the tree layer in this demo schema.
04:45
And you have the main field, like title, abstract, keywords. And you can use QGIS checkboxes. You can use the combo boxes to choose, for example, the publication frequency. It's very handy to use QGIS as an editing
05:01
tool for PostgreSQL layers. So you edit your metadata, for example, for the tree table. And then there are some logic under. And some data are calculated from the table content. For example, there is a valid unique ID for the data set.
05:22
We can compute the layer extent, the number of features, the geometry type, the projection ID and name. So PostgreSQL gives us, which triggers the capability to do that. And we have also created useful views to know the orphan PostgreSQL tables, which tables
05:41
in your data does do not have metadata. The opposite, orphan metadata, which sometimes you had edited metadata for a table, but you have then dropped the table. So you can see that. And we also managed some flat representations
06:00
of the data sets, for example, to export as CSV, to view that in LibreOffice or Excel. So that was for the admin guy, the guy editing the metadata. Now, for this user, so that was supposed to be an animated GIF.
06:21
You will have the link here. And basically, you can search with the Control-K in QGIS. You can search inside the title or the description or comments. And you can find your lines of metadata. For example, here, I typed metadata tree.
06:43
And I find the trees. And when I click, when I enter, I will have the layer popping up in QGIS and a small dock panel on the right side with all the metadata you wrote down before. So as a user, I can easily find
07:02
some layers in the Postgres database I can connect to and then have the metadata just visible when you click on the layer in the QGIS layer tree. The user can also export the metadata in different format, HTML, PDF,
07:22
and also DCAT, which is a standard. It's a vocabulary. It's a way of writing the metadata in XML. And so you can use that to import all the tools if needed. The third part of the presentation was the PgMetadata Advanced Features.
07:44
Because it's SQL-like and data-driven developed, you can easily change how metadata will show the metadata,
08:03
text representation, the layout. You can change this layout we've seen in the right dock. Inside the HTML templates table is just a table with values you can change and write down how you want to organize your metadata. We provided one default one, but you can change it easily.
08:23
And we have some SQL functions to, for example, generate an HTML card with SQL. So select PgMetadata, get dataset item HTML. You give the schema and layer a table and the localization.
08:41
In FR, it means French. You can have it in English or other compatible language. And we have also created some functions to export as DCAT, for example. And as an administrator in the big companies,
09:03
you can also deploy GIS with some already chosen configuration. For example, if the name of the connections to the PostgreSQL database, where the Control-K, where the QGIS search tool can use,
09:21
and if you want to open automatically the dock or not, there are some configuration available here. We have also, in 3D, we have also a tool created, which is LizMap Web Client. You can publish QGIS projects online in a web browser.
09:42
And so we have also created a model for that. So you can click on the layer in the web layer tree. And you have the same card because it's just select, give me the card. And we use that in LizMap, too,
10:01
to display the metadata of a specific layer. And we have also, yeah, we have tried to have a full or rich documentation for different usage for the administrator, for the end user, for the system administrators.
10:21
So you can go to docs.3liz.org and see the full documentations with the different parts. As a conclusion, I will try to answer the questions.
10:42
Why another metadata tool? There are plenty of metadata tool, server side, applications, client side. And why PG metadata? First answer, see the previous slide about PostgreSQL. It's centralized. It's very easy to access.
11:01
You don't need to install another application. You have your database. And data is in the same place. That's, I think, the main thing with that. You keep the metadata as close as possible to the data. And you don't have to install a new application and to maintain a new application.
11:21
So you just can focus on maintaining your PostgreSQL database, and that's OK. And the GIS users, they won't need to find your data inside the web application. They just brought that in QGIS. So they are in their tool, QGIS, and they'll just search the data and open it.
11:44
It's GIS user-oriented. It's inside QGIS that you use PG metadata, not the way around. And it was not designed to replace the existing metadata web portals, which provide a lot of rich features for sharing
12:03
and for importing, exporting. We have some capability to be harvested. If you use LISMAP and the modules, there is a link. You can harvest in DCAT the metadata. So other tools like National Open Data Portal
12:22
can harvest the data inside PG metadata, but you need LISMAP web client and the module, which are open source. The roadmap, we will release a new version. I hoped to release one just before today, but we did not manage it.
12:41
But in September, we will have a license. It's JPL 2.0, but we just forgot to add one. We will raise the QGIS minimum version to 3.16. It will have raster support, better handling of backslashes
13:00
in the links, new fields for the contact, for the license, and other things. And we would like to improve the import-export tools to facilitate the format, the data exchange between different softwares.
13:23
Some resources you will have in the presentation. And I would like to thank the French Gare Provence for funding and my colleague here, Etienne Trimay, which is maintaining and has developed a lot of things. I've done the PostgreSQL part,
13:42
is the responsible of the Python QGIS plug-in part. And also, many thanks to our external active contributors. We have one very, very active, Florian Yen, for ideas. He provided fixes, improvements, and now we have unknown but kind translators for Finnish,
14:04
German, Spanish, and Europe. If you want us to help you, you can contribute for all the translation. Thanks a lot for your attention, and if you have questions, and I would love to hear feedback from users,
14:20
because I know there are some, but I never have feedback, so feel free. Thank you.