We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

From QGIS to Python: comparison of free and open tools for statistical analysis of cultural heritage and data representation

00:00

Formal Metadata

Title
From QGIS to Python: comparison of free and open tools for statistical analysis of cultural heritage and data representation
Title of Series
Number of Parts
351
Author
License
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language
Production Year2022

Content Metadata

Subject Area
Genre
Abstract
From QGIS to Python: comparison of free and open tools for statistical analysis of cultural heritage and data representation Thankfully to the European Commission initiatives such as INSPIRE (2007) and other governmental policies, spatial data are available publicly on different national, regional and municipality geoportals for further use. When it comes to the cultural heritage and Italian context, based on the decree of the Ministry of Culture (MiBACT, 2008), different activities concerning heritage has been assigned to the ICCD (i.e., Central Institute for Catalogue and Documentation) such as research and technical-scientific collection of the documentation and coordination of cataloguing of cultural heritage and its digitalization. These regulations allowed the public entities to share substantial information about geographical and spatial data with a wider audience. Specifically in the region of Lombardy, data about cultural heritage are catalogued in SIRBeC (i.e., Regional information System for Cultural Heritage) that has been promoted since 1992 and continues collecting, managing, and publishing a vast amount of information. Vector shapefiles are freely available for download on the Geoportale Lombardia. The scope of the research was collecting information about cultural heritage in Lombardy that is freely accessible online. Data downloaded are point and polygon features files of the position of the cultural heritage. Furtherly, the methodology developed deals with the use of QGIS, as the open and free software together with the Python console integrated into the software and finally using the online software of the integrated development environment (IDE) named Replit that is free, open, collaborative and in-browser Python coding application. The methodology is based exclusively on free and open sources, starting from the collection of data to their processing. Each vector file is enriched with the metadata in the attribute table but the methodology is providing a combination of software to obtain other data (e.g., coordination, area, etc.) and statistical analysis (e.g., ratio, percentage, position, distribution, etc.), which are the initial part of each elaborated cultural heritage project. Additionally, the methodology is discussing different approaches to reach the desired result and compares their differences. Firstly, the Python console in QGIS was examined, and metadata were extracted from the vector file to the .csv file to be used in Replit. The online codding application gave a higher degree of flexibility while coding, and it was possible to implement data extracted in a .csv file into a coding panel, using them to produce different statistical analyses. Furtherly, the methodology discusses the use of the plugin of QGIS called DataPlotly and data differences, from the representation to the utility level. Results through the Python Console in QGIS allowed the extraction of necessary data for further analysis, deleting the ones which are not needed. The good side of this approach is that metadata of the shapefile stay untacked, and the Python is simply extracting selected data in a new external file. There have been selected four categories of interest: Name, Category, Typology and Municipality of the cultural heritage. The area of interest was a northern part of Milan, in the province of Monza e Brianza which has a dense and diverse category of cultural heritage. Using the python code, these four categories are temporarily printed and saved in the console panel. Since there is no information about coordinates inside the metadata, there are two approaches that are tested to obtain them. The first one used was the QGIS integrated option "Add geometry attributes", which created the new shapefile enriched with the information about longitudinal and latitudinal coordinates. The second approach was extracting the coordinates through the Python console with the f.geometry() function. Information about the four categories selected and coordinates are printed temporarily in the console, and the user can control the order of the columns and delimited type, following the saving and extracting the .txt file. The second part of the analysis also discusses two methods that were tested for the creation of statistical analysis of extracted data and their representation, firstly in the QGIS plugin DataPlotly and then using Replit. Presenting statistical analysis in the form of different charts is available directly through the plugin. Nevertheless, when it comes to the great amount of data the plugin resulted not be very efficient for the representation nor easy to manage the view. Another constrain is that there is no option for exporting graphs in a .pdf file. On the other side, creating the charts through the Python packages such as matplotlib or pandas shows a better degree of control over a graph. The advantage is that there is a possibility of exporting it in many different files, such as a .pdf or .svg file. Additionally, through the Python in-browser application, there is a higher degree of control and change of the visual representation of charts.
Keywords
202
Thumbnail
1:16:05
226
242
Mathematical analysisStatisticsPairwise comparisonFreewareOpen setArchitectureIntegrated development environmentRepresentation (politics)Vertex (graph theory)PolygonCategory of beingType theoryForestStandard errorTurbo-CodeLine (geometry)Bridging (networking)Attribute grammarGeometryNumerical analysisMereologyDisintegrationField (computer science)Coma BerenicesPlot (narrative)HistogramImage resolutionGraph (mathematics)CodeVector spaceFile formatInstallation artService (economics)Process (computing)CASE <Informatik>Observational studyTraffic reportingResultantStatisticsCodeField (computer science)Plug-in (computing)PlotterCalculationVariety (linguistics)Range (statistics)InformationSystem administratorDifferent (Kate Ryan album)Universe (mathematics)Level (video gaming)SoftwarePopulation densityCore dumpStudent's t-testOpen setState of matterFreewareGeometryOrder (biology)Slide ruleContext awarenessEndliche ModelltheorieOpen sourceTable (information)Spline (mathematics)Set (mathematics)BuildingNumberScripting languageType theoryComputer architectureMultiplication signPhase transitionProjektive GeometrieGroup actionCategory of beingSinc functionComputational geometryReal-time operating systemGame controllerPresentation of a groupComputer programmingGraph theoryShared memorySoftware testing2 (number)Turbo-CodeOnline helpMereologyLine (geometry)MetadataMobile appDirected graphXMLComputer animation
Transcript: English(auto-generated)
Thank you very much. So I'm just going to leave the first slide to explain the slight context of this. So as all of you, also, we in a university that has researchers, PhD students, had a problem in COVID pandemics. So when it comes to I'm an architect, when it comes to heritage and general using an open street
map, you know the open street map is not made to hold these data. But we have relied on these open data and open source softwares. So we had a big problem since our main, let's say, goal and the beginning of any research of an architect, especially working with a heritage, is a basic statistical analysis.
So understanding the density of your urban core and your history. So we had a very simple. Is it working? OK. We had a very simple test. We had the students online. And we tested different open data from different OSM
and also from a GeoPort of Lombardy. And as you can see, the students realized that downloading them in a different way through the QGIS or downloading it with Overpass Turbo, we got different types of data. So sometimes we got lines. Sometimes we got with the different metadata, different even type of the data itself.
And as you can see, something official and administrative data as a GeoPort of Lombardy was the most accurate. But we have a problem there. Those data are officially, sometimes they're really old. And they are not very going into a time with this fragile architecture that we have
and actually that is changing really fast. So we had the first phase. And second phase, first phase is how we can enrich those data. Sometimes we miss these ground control points to reference the 3D models. So those are really simple things. But sometimes the geometries were missing. So how we can extract using the QGIS integrated tools
or using the Python console and how we can represent given data. So can we actually obtain everything in a free open source software or we need some help? So we use different integrated tools. And they behave in different ways. They provided some certain geometrical information
we needed. And sometimes they just didn't have any information, so they weren't from help. Or from a second part, so it's a very simple for a student that doesn't know how to use a code, doesn't know how to obtain data. However, the second one, we run a very simple Python code.
And we shared it in Python QGIS console. And to obtain, let's say, a table of information. As a conclusion, we have a different sets of, let's say, differences between those two things. Is while that building geometry tool, it's very easy to use.
You don't have to know something. You just click one button. But it's not flexible. We cannot filter so easily. There is no such a control as much as we have with the Python script. Then we have a presentation of the data.
So we do can have a structured table. And we can use it in another program. We can use it for a simple statistical analysis. If it comes to the plots tool or data plotly, which is basically used in a QGIS, we have graphs which are interactive, but through the HTML link.
And they are not providing, again, very big flexibility. So what we did, we wrote a code. We shared among students in Replit, which is a free and online share the information app, coding app. So we could be able to share the, let's say, we could share the project. So we can see what they are doing, the way,
how they are changing the code, and so on. So we could collaborate. It's still a free source online. So here we could have many more statistical information, how many percentage of this specific category of this specific town there is. And since the groups are distributed in a different case studies, that was actually very useful that all the groups
of all the students can see what somebody else is doing, which is not the case with the QGIS, since it's just in your desktop computer. So it provided, again, the results were different. Some of them were, with the Replit, of course, more flexible as the code is something that you're using,
real-time sharing, medium difficult use, especially if you're already provided a code. There is adjustable order. So there is a lots of numerical values can be calculated in many different ways. While in Plug-in Data Plotly, there is a field calculator and in a plots tool,
there is no such a flexibility of such a big variety in range of, so in just to mention for the last thing, like I know the OSM is not provided for the heritage, but let's say these little informations that we had actually were helpful and the people, the data's are much newer than from the administration
because the process is really, really long. So the people actually had some, the case study was Monza for this particular paper, had these information, let's say fallen wall, which is very important and immediately gives the report on the state of this very fragile and misused or wrongly used architecture.
Thank you.