Mapping Marine Diversity: using QGIS to visualise and extract biodiversity data
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 52 | |
| Author | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/44680 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSS4G SotM Oceania 201950 / 52
9
11
12
14
15
17
20
23
26
28
30
32
34
39
44
00:00
Software developerComputer animation
00:34
Distribution (mathematics)Scale (map)SpacetimeVariety (linguistics)Water vaporDatabaseOpen sourceVirtual realitySpeciesState observerSample (statistics)System identificationMacro (computer science)SoftwareNumberLine (geometry)Point (geometry)PolygonSuite (music)Type theoryFunction (mathematics)Square numberTexture mappingDrag (physics)Set (mathematics)Visualization (computer graphics)GoogolExecution unitDirected setAbstractionLibrary (computing)Link (knot theory)ThetafunktionCASE <Informatik>Online helpPasswordWorkstation <Musikinstrument>Conditional-access moduleDatabaseRange (statistics)CASE <Informatik>Different (Kate Ryan album)AreaComputer fileMultiplication signBitOpen sourceDrop (liquid)Office suitePoint (geometry)Query languageLevel (video gaming)CuboidComputer programmingWater vaporData typeValidity (statistics)Real-time operating systemLine (geometry)INTEGRALQuicksortPolygonType theoryWorkstation <Musikinstrument>Single-precision floating-point formatArc (geometry)Square numberMereologyData managementFunction (mathematics)Near-ringLibrary (computing)SpeciesDistribution (mathematics)GeometryRow (database)Physical systemSet (mathematics)MultiplicationVirtualizationNumerical taxonomySoftwarePlug-in (computing)Suite (music)MappingGroup actionLibrary catalogAbstractionNatural numberElement (mathematics)Cartesian coordinate systemSampling (statistics)State observerVideoconferencingDirection (geometry)Computer animation
08:05
Lecture/ConferenceComputer animation
08:20
Lecture/Conference
08:33
Point (geometry)WebsiteComputer fontVector spacePlotterTouchscreenComputer fileLevel (video gaming)VideoconferencingComputer animation
09:17
Shape (magazine)PolygonVideoconferencingPoint (geometry)AreaDot productDatabaseComputer filePlotterComputer animation
10:05
DatabaseProjective planeMappingBitRevision controlPoint (geometry)Different (Kate Ryan album)Distribution (mathematics)AreaMathematical analysisValidity (statistics)View (database)Vector spaceComputer filePolygonSet (mathematics)Menu (computing)Game controllerOpen sourceCuboidGraph (mathematics)Square numberSpeciesType theoryRight angleGraph coloringMemory management
13:15
DatabaseSoftware maintenanceAdditionRevision controlComputer filePresentation of a groupArchaeological field surveyInformationDot productSlide ruleVideoconferencingRange (statistics)Quicksort
13:50
Network topologyPoint (geometry)QuicksortDifferent (Kate Ryan album)AreaLevel (video gaming)Table (information)Ocean currentProduct (business)Natural numberType theoryField (computer science)CuboidBitSpacetimeData managementDatabaseComputer wormArc (geometry)Numerical taxonomyData storage deviceMereologySlide ruleDot productArchaeological field surveyComputer animationMeeting/Interview
Transcript: English(auto-generated)
00:02
So hello everybody, I'm Sadie Mills, this is just introduced from NIWA, and my co-authors on this talk and this piece of work are Brent Wood, who probably some of you may know and is currently presenting in the other talk in the Thornton Room. So he was going to be here to answer any particularly technical questions, so if
00:23
you have any of those, please go and hit him up afterwards. And Jane Robbins works in our software development team at NIWA and is a SQL whiz who helped out to develop the tool that I'm going to talk to you about today. So for those of you that may be from out of town or out of the country and don't
00:41
know what NIWA is, it's the National Institute for Water and Atmospheric Research. We are a crown research institute and we maintain nationally important databases there and science assets and where I work, in particular, a biological collection. And the area of the science that NIWA does that I work on in particular is enhancing
01:05
the stewardship of New Zealand's diversity, freshwater and marine, through the freshwater and marine ecosystems. And the program that I work on particularly is also the marine biological resources. So we deliver fundamental knowledge about the diversity and distribution of marine
01:22
biota in New Zealand and in the waters that we look after in the Antarctic and the Ross Sea and also in the wider South West Pacific. And so, yes, we're all in Wellington, we've got a little office at Greta Point, and I'm actually the collection manager of the NIWA invertebrate collection, so that's what
01:43
this little picture is up here of our shelving. So I'm going to go over the collections that we look after, the database we use, specify the GIS solution we wanted and what we found and how that works and some outputs to show you. So the NIWA invertebrate collection, I'm the collection manager of that facility.
02:04
We're funded by the Ministry of Business Innovation and Employment as a nationally significant collection. And we look after about 300,000 jars of preserved marine invertebrates. So it's kind of like a library of dead things just to really simplify it.
02:20
So we have specimens, I said, from New Zealand, from the Ross Sea and from the wider South West Pacific as well. We cover 8,500 species from 21 phyla, so that's the big groups, and we have about 142,000 registered in our database. So we have quite a long way to go to catalogue everything.
02:42
The collection began in the 1950s with the New Zealand Oceanographic Institute, and we're growing. We're still collecting out there today on biodiversity and fisheries research, but that's probably more minimal than it was in the old days. But we also receive really great samples from scientific observers on
03:00
commercial fishing vessels. We also house the Marine Invasive Taxonomic Collection, which has 74,000 samples and also some data for the Niwa algae collection, which is housed at PAPA. So we manage all of this in Specify, which is open source software developed
03:22
by the University of Kansas. It's for natural history collections. It's an amazing tool for managing natural history data. The Specify software recently lost their NSF funding, so we're now in a consortium, and we can decide if we want to pay a membership to get some support.
03:42
Otherwise, you can download and use it for free. It's used in 38 countries and supports over 450 collections. And believe me, the people that are not using this in their collections are jealous that they don't have this, so it's really useful. It's a suite of applications that access the data managed in the underlying
04:01
MySQL database. So MySQL does support spatial data types, like lines and polygons and points, but Specify doesn't actually use those. We store our lat and long as an x and y coordinate in the bottom box there, so we can store our station data. And we also have data for taxonomies, so the names of the different animals
04:26
in the database as well. So they've got an inbuilt plug-in for Google Earth, so we can just quickly visualize our data collection points on the map, which is really useful, but we can only drag and drop record sets from one query of data as a standard user.
04:45
So we can just query one taxon level, file them to species, we can plot the data from one cruise or the data inside one square box, bounding box of lat and longs. So it's quite useful, but it's not completely useful for everything we want
05:02
to do, so we get multiple requests for our data from a whole range of people wanting to use it for their research, and often from multiple different polygons and areas that they want the data from. So we would usually export the data into an Excel file, for example, to send off.
05:20
That has problems because those exports are made at different times, the data's out of date, we need to replace and update them, and if we've done some grooming afterwards, that's not captured in the original database. And also, we can only export 20,000 records at a time, which is a bit of a problem, and we have a lot more than that.
05:40
And not everything lives inside a single square box. This map is showing the polygons where there's vent activity, so that's where there's an undersea volcano or a seamount up the Kermadec arc north of New Zealand. So an ideal solution for us is something that accesses our database directly.
06:00
Something we can go right into the live database, we can have real-time validation of our points that we're entering in, make sure that they're not plotting in the middle of land, for example, and we want to be able to produce publication-quality maps and compare quickly the distributions and integrate with all sorts of other map layers to make the maps pretty.
06:21
So we did find that through using QGIS. It's a really powerful tool for us, and we also use the GDAL, the Geospatial Data Abstraction Library Spatial Data Access software to make a virtual data source to present our non-spatial data sources as spatial data.
06:41
So that uses a short file to describe the source of the data and present it to a spatial software data set to another program, which in our case is QGIS. So describing the data source, it looks at what kind of data we have, and in our case it's a MySQL database. It talks about where to get the data from, what the SQL is to get the data
07:04
out of the database, the type of geometry data that we want, and in our case it's a point, the columns that have the X and Y, which is the latitude and longitude start, and how to create the geometry and a coordinate reference system for our part of the world.
07:21
So when it uses the file, it points at the external data rather than saving all of our data in a file so it's really light and it makes it live, and it can be opened as a layer, and you can use that in QGIS like any other layer. So this piece of this file, just go through quickly and point out the
07:42
various parts. So this is what the layer name will be called in QGIS. There's our database details. You can also use it with a Postgres database or an Excel file. The columns that we want out of our database are listed there. And then how we're going to turn that into a spatial point and the spatial
08:03
reference system. So I've got a little run-through video which I'm hoping is going to work here. Is it going to run by itself? Not going to show.
08:25
Oh, okay, excuse me.
08:40
So basically what I was going to show you here is how we bring the file in. We just basically open it as a vector layer. So we can get the various layers of data that we want to plot on our map. We've got the bathymetry layers. They're coming from the NIWA website. You can download those. And just a simple outline of the New Zealand coast from LINZ,
09:05
from the coordinates, is downloaded in there. And if we were watching the video, it would show you where I'm loading in the file as a vector layer, and it plots all of our points up on the screen. And that's what the points look like.
09:22
And they're coming in live directly from our database. So it's really powerful for us. We can already see where there might be something wrong. What I also showed in the video was the points where some of them are on land. We do have a few freshwater points, but sometimes it's really helpful for us to be able to check that out live.
09:43
You can also bring in other layers, other shape files that you can lay on the top. You won't be able to see it because it's in the little video. So we've got benthic protected areas that are around New Zealand. And so we can use that to plot in where the dots are falling inside those layers. And I also went in and showed the vent polygons again
10:02
to show you those data points on there. So this is just a zoomed-in version onto a couple of seamounts. Basically showing the top right-hand corner there, if you can just make it out. I've started to do some analysis inside QGIS, where I've highlighted different taxon names, different animal species names
10:24
with different colors, and then I can already start to analyze the data inside QGIS. So it's a really powerful tool. And then looking again with these, I was going to show you the tool that I use a lot, which is under the vector menu.
10:40
There's geoprocessing tools, and there's a tool called Intersection. And what that does is it will just pull out all of the points that fall inside the polygons, and all of those points will then be merged into a new layer, which is called Intersection. And that, maybe I have a shot of that.
11:00
No, I don't have a shot of that. But just to give you the idea, this one here, I've used the benthic protected area polygons to put the points in, and only give me the points that fall inside a polygon. And then I can use that to export the data into a separate file, which has only got those points and those polygons. So we're going from what previously I could only get inside a square box
11:24
to something inside lots of different square boxes or squiggly round boxes. And then I can pull the data out directly in there. We can also use QGIS to do different projections. So Brent wrote me a polar projection so we can have New Zealand at the top.
11:44
It's a little bit different way of looking at things. And then down to the Ross Sea, which is the area that we look after. And then you can just see this is all of the database points in our database plotted on that projection. And we can produce these really great publication quality maps directly in QGIS.
12:04
And you can add in lots of layers and labels and things like that as well to illustrate what you want to show and scale bars, and that's just come straight out of QGIS. So this has really given us the ideal solution.
12:20
We have direct access into our database. We get real-time validation when we enter data, which is great if you're not sure where your points are plotting. We can quickly compare and view different taxon distributions. There's a lot more control over the style. And if you wanted to, you could put in like an institutional style
12:43
that you would set up for different maps. And we can easily bring in other types of layers and data sets that give new points and lines and polygons to make the data more meaningful. And we can change projections. We can produce publication quality maps, which is great.
13:00
And we can ideally do all of this using free and open source tools. And so this can work for your data too. As I said, you can bring it in from any database you're using or from even an Excel file. And yeah, I hope you give it a go. And so, yeah, if you have any technical questions, you can talk to Brent about those.
13:21
But otherwise, if there's anything else I can answer, I can do that. And I'm sorry, I might have gone quite short because my videos didn't go. Thank you very much. Do we have any questions?
13:41
Hi there. I was just wondering, I'm a little bit curious about the data. There was a whole range of dots that went out from the Kaikaurau Coast in a couple of your slides. I was wondering whether you could just talk to maybe why that is or what's interesting about those. So the big clump there in the middle, that's the Chatham Rise.
14:02
So the Chatham Rise is a really big fishing area. So as I said, a lot of our data comes from fisheries trawl survey or from commercial fishing vessels. So somewhere like that, the Chatham Rise is a really productive area. So there's lots of currents that run over there. It's very productive. So there's lots of food.
14:21
And so there's lots of fish that can be caught there. So a lot of those points will be from bycatch from fisheries. And also because it is such a big fishery area, that we want to do research there to make sure that we're informed about what what lives there in the different habitat types. Yeah.
14:41
And that's kind of what you can see from all of the points there, the sort of the interesting bits of the sea floor. So going running up north from New Zealand, we've got the Kermadec Trench and the Kermadec Arc. Actually, the trench has comparatively very few dots. It goes down to 10 kilometres deep at the deepest part. And so there's a lot of researchers interested in that.
15:01
But it's really hard to get there. So, yeah, the easier to get to places have more dots. And also, if we made the point size smaller and more realistic to what actually the point size was, it would be a lot of white space on that map as well. Was there some more questions?
15:24
Oh, I was just curious about Specify here. So it's something I knew. You said natural history database. Do you mean that it's primarily based around taxonomic classification? Yeah, yeah. So you can have a taxonomic tree in the background in one of the tables
15:41
so you can import that in from somewhere like Worms if you wanted to. Or you could build your own taxonomy of whatever you wanted to. We also have a storage taxonomic tree and a geography tree as well that sort of runs that feed into different tables in the database.
16:01
But it's been built. It was built by Kansas University for their natural history collection. So a lot of the tables and fields that are sort of come out of the box tailored towards natural history collection management. But you could you could adapt it to other things as well. Yeah. Yeah, it's really customizable fields in there.