And can I just remind you, this session is being recorded, and you switch off your mobile phones or send them to vibrate. And I suppose this session will be delayed for about 10 minutes.

Our first talk will be given by Mr Brent Wood, who is from NIWA, which stands for the National Institute for Water and Atmospheric Research in New Zealand.

He will show us the role that open source software and standards play in NIWA's strategies and the environmental information management framework.

Hopefully you can all hear me. I can just speak fast, so if you want to hear what I'm saying, I'm going too fast, slow me down. I'm assuming everyone here doesn't know what NIWA is. New Zealand is a long way away, and it's not a big multinational.

So I'll be starting with a quick overview of NIWA, what it is, what it does. An overview of what we do with open source, how we use open source, a brief digression into an open source case study within the case study, and then look at some of the things that NIWA is doing itself in the open source arena.

So NIWA is the National Institute for Water and Atmospheric Research. I don't know what happened to the R, but it's never been there. It's a government agency, and it's called the Crown Research Institute. It is owned by the government but operates as a commercial business

having to find its own funding and do its own work. The quality is not wonderful, but it carries out environmental research in atmosphere, climate, fisheries, oceans, aquatic areas in New Zealand. It's the lead agency for doing that work.

We do that research for government agencies, utilities, industry. We also have some international roles and responsibilities in the areas like climate science for New Zealand.

And all of that means that open standard interoperability are absolutely critical to NIWA performing and doing what it needs to do. This is about FOSS4G. One of the comments I'm making here is that the 4G side can only really flourish

in a place where FOSS itself flourishes. If you don't have an environment that supports open source, open source PIS tools themselves will be harder to sort of implement, keep running, get supported in general. So there won't be a quiz. You don't need to read all that.

But it gives you an idea of the range of open source tools used throughout NIWA, both infrastructurally, software development, GIS, data management, a wide range of tools. So it's a fairly FOSS friendly place to work. I like that.

We also have some of our open source tools. We have supported existing open source tools. We funded development in QGIS, SILC, CASL, JIRIS, SPM, myCAMS, CLIDESK are all NIWA initiatives.

They are open source tools in their own specific domains in climate, fisheries, wave action, stock monitoring, photo analysis, and climate visualization tools that we provide, we write, and they're all open source tools from NIWA. We funded some developments in QGIS-GMT, Postgres Studio,

our GIR network. It wasn't altruistic. It was to help those tools provide functionality that we had a need for. So it was quite pragmatic. Our environment is deliberately very OGC centric.

We're using OGC standards to support our interoperability capabilities. We're developing an information services stack, which is based around the OGC web services. We're currently working on some integration with the Kivri services to help interoperability, particularly in the central observation service arena.

Does anyone know what these acronyms all mean? Or do I need to explain some of them? Dead silence doesn't tell me much. We're looking at sort of, because we're using that platform,

that framework, we're also trying to improve the way that metadata can be harvested from web services to make them self-documenting, and that the catalogs that we build from harvesting those can be used by clients to drill back to the original data sources, if that makes sense.

So I did say we have a digression of a case study within this case study. I think it's a nice little story, so you're stuck with it. We went to Oman for a fishery survey off the Arabian coast. That's where we went. It was divided into depth-based strata, as well as latitudinal strata.

That's our boss collecting souvenirs, and a very happy Omani. All the gear we took on that survey was planned months, years in advance.

It was very rigorous. We put it all together, shipped it to Oman in a container, and Murphy took over. We were using a couple of small shuttles as our Linux boxes. We had two redundant Linux servers that had all the database tools, all the satellite comms tools for email from the boat, et cetera,

and they were both dead on arrival when we opened up the container. Flying on route, I boarded a Linux magazine in Dubai airport. It had a Fedora DVD on the cover. The entire survey was run, from then on, from those servers,

rebuilt from that DVD. You try doing that with commercial software. It was very open-source based. The entire survey we carried out there was using open-source tools. I like mentioning libraries that sit behind the tools,

because they tend to be forgotten, and I think they deserve an awful lot of credit for the casualties they provide throughout the cost GIS environment. We had GMT scripts, so you could simply, for all the species we caught, you could run the script, access the database, and generate length frequency plots, length weight plots, distribution, catch plots, et cetera.

We catch a thousand species, so we could generate a thousand maps in two minutes, running one script. What's GMT? It's an open-source tool called Generic Mapping Tools. Thanks. I did suggest when generic, called Greenwich Mean Time, became UTC,

that they renamed Generic Mapping Tools to Universal Tools for Cartography, but they didn't do it. As you can see in the bottom graph, on the bottom line there, you can see outliers, so the ability of tools like this not only let you visualize the data, but that visualization also allows you to error-check the data.

QGIS had advanced to 0.8 at that stage, and that was our default desktop GIS tool, all the data going into PostGIS. We carried out a wide range of sampling techniques, methods,

gear being deployed, et cetera. All of that went into the PostGIS database. All of it was sort of managed quite successfully there. And because it was all open-source, we could leave the Omanis in their recent center with a complete operating system of the working database.

All the scripts, all the tools were theirs at no cost, no license issues, worked perfectly. So again, from that particular one, all the open-source developers here have contributed. Thank you for your contribution. And back to the main presentation.

In 2011, the New Zealand government issued a directive on open and transparent government that required or directed all core government agencies to make publicly funded data available to the public. It encouraged non-core agencies such as councils and CRIs to do the same.

And one of the decisions NIWA made, or I made on behalf of NIWA, was that we can release all our spatial data, set up web services, allow people to download shapefiles, and if they can't afford an ESRI license or a MapInfo license, releasing the data was not particularly useful

to an awful lot of the people we wanted to benefit from it. The solution we came up with there was to take QGIS and have it customized to facilitate access to the data we were providing.

And we contracted Sourcepol to do that, and we have, I think, a good working relationship with Sourcepol ongoing. Doing more customization. So rather than building lots of portals to sort of allow people to look at the data that's particular to domain in that portal, this tool now allows people to look at all NIWA data,

plus their own data, plus other agencies who are using OGC services in the same way. So it's a much more generic tool that allows people to make much better use of the open data that's being provided. So it's hopefully a very empowering approach to information delivery.

We're working on a SaaS client, so as well as the spatial environmental data, we will be using SaaS, it's a sensor observation service from OGC, to deliver time series data. We have SaaS services built on our hydrometric and climate databases.

The SaaS client in QGIS will allow people to see the various stations, click on a station, see what readings are available, select the interval for which they want those readings, view a graph of them, and download them. So we're moving away from the straight GIS arena with spatial data,

into time series data as well. We've had a lot of success with structured conservation planning, for managing marine protected areas in Antarctica, with the toothfish fishery, and we're working again with Sourcepol on a generic tool, for that capability being built into QGIS.

So apologies for the image quality, but essentially the NIWA plug-in here, parvests a list of OGC web services from the NIWA website. We can add NIWA and other agencies' URLs to that website.

This tool, the clicker button, will harvest those, provide you a directory tree-like structure, that allows you to click on a service, see what the layers are, add those layers to a favourites list, and you then don't have to reconnect, reload that information until you want to.

It also allows you to, for both WFS and WMS layers, once you've opened them, you can cache them locally, and use them offline. When you go back online, you can continue using the cached version, or you can refresh from the online source.

That is pretty hard to read, but that's the NIWA web page that currently has the services that are available, from a range of agencies, district councils, city councils, CRIs, universities. A wide range of agencies are now providing data using that approach.

One of the other tools we've built is a survey tool, that's being endorsed by the Ministry for Fisheries, as a survey design tool. One of the interesting things with coastal surveys, is trying to propagate statistically robust transacts, in a sensible direction, off a convoluted coastline.

We think we've got a good solution for that. We're doing some testing of the robustness of that algorithm. The idea in particular, in New Zealand, along the coast, is that Maori, the indigenous people, now have rights, customary rights over that. They also have management obligations as part of that.

This is a free tool. We've received funding from Maori groups to help develop this, so they can now run statistically robust surveys, for the coastal areas they are responsible for, and contribute to the management of those resources, regions.

And, as I've told, source pollens have been here. We do have other things in the pipeline coming as well. From a web mapping perspective, we undertook a survey, Ocean Survey 2020, as in the top website. They funded a two year program, looking off the northeast coast of New Zealand.

One of the contractual obligations from that project, was to make all the results available to the public. And, we went for a totally open source solution. We co-funded the development of an open layers administration module,

for the Silver Stripe CMS. The CMS now has a tool in it, that allows you to set up a connection to OGC web services. And, because it's within the CMS, we can have method pages that describe,

I can't show you live here, but we can have method pages that describe each of the sets of gear that are deployed. We can click on the method name in the map legend, and retrieve that information. So, we can, we set up a map layer, into some keywords.

From the map, we can instigate a search on the metadata catalog, that uses those keywords to find all the relevant entries in the catalog, that pertain to the layer on the map. So, we're able to access reference information in the CMS,

we're able to access metadata searches, all from a map centric approach. The map in this case has data coming from PostGIS, and from Esri. It's all OGC service compliant, the data source is abstracted, so it's quite portable. And, we also have some input,

where people want to be able to change the symbology for the layers, vector layers of points on the map. You can see in the middle little black box there, there's a simple pick list that allows people to use, choose the color and shape to use for each symbol on the map.

And, that's fairly unusual. Most web mapping tools do not allow users any control of symbology. That's fairly rudimentary, but it met the sort of user requirements. We collected over 20,000 CVED photos. Those photos are managed in a commercial product called Atlas,

AtlasMD, it's a digital asset management tool. We have paid the developers of that tool, to migrate it to PostGIS, as the underlying database. That means we can associate a location as a point with every image, and have a map interface so that all the images in the database,

can be retrieved via a map. We've also funded a web service, which is used by this tool, so by clicking on the point, that represents that station on the map, you can browse through the carousel here, and go through the photos from each site. The developers at Silverstripe thoroughly enjoyed this.

20,000 photos, and after about a week of them working on this tool, I was told, did I know there are three photos with bottles in them? So between them, they'd gone through all 20,000 photos. As I mentioned, there's a metadata catalog integrated with this.

Every report, every presentation, every data set, captured, derived, part of the survey is in the catalog. It's a geonetwork catalog, and the website has a simple CSW client as a front end,

so users aren't faced with a complex catalog. A very simple interface that allows them to browse it, or search it. Results will come back, and this is one of the main reasons it's used. Councils want data. Environmental conservation organizations want data. This is where most of them are getting it from.

So the whole tool itself is sitting on top of Postgres, maps, OpenLayers, Silverstripe, Geonetwork, totally open source. And a few months after we put it together,

we were asked by another government agency who were quoted what I thought was an extravagant sum, to spec out a website for New Zealand marine biosecurity data. For less than the price they were quoted for specing it, we could take our open source solution from the Bay of Islands survey

and wrap this into it. It is now the New Zealand marine biosecurity portal. Similar tools available. Note it works across 180. If you work with GIS tools, 180 is the biggest pain you will ever face.

Because this involves a lot of work with individual invasive species, we now have a species picker tool. And that tool allows you to select a species from a pick list. You can see the occurrences of that species. There's also a WMS service that shows you the modelled distribution,

or predicted distribution of that species. Racing through it. EI browser, because we're using web services, we can set up a single facility that allows people to go to that one point and access and search multiple databases across NIWA.

So we have bio data, we have climate data, we have hydrometric data, we have water quality data. The idea is this one tool allows them to enter a set of keywords and search across all those databases for relevant information.

Linked to the system, but not strictly GIS related, we have taxonomic data. We have a taxonomic reference system, which is your core taxonomic hierarchy. Linked to that, we have attribute data.

And that's used by a publishing and reporting system. Here we have some species data, which inherently has nothing spatial about it whatsoever. It's simply a species in a taxonomic hierarchy. There's the hierarchy in a tabular form.

There's a tree form. We're in the process of linking this information to our bio data distribution, the GIS, the biodiversity information, so that all the taxonomic and the species information is actually linked inherently with the spatial data.

Here's an example of species data. As I said, PRS takes that and generates output. This output has some photos and some text. And it's for those who know what oatlets are. They're the ear bones of fish. They're used for both aging fish

because they have rings like a tree. The shape is also diagnostic and in middens for archaeological research and in stomach contents of birds, fish, marine mammals, they're used to identify prey species. So we can now have a fat-sheep generator and potentially a diagnostic tool to enable it to happen. Similarly, we have freshwater pest species.

The data is in the database. We can do this and access that information. In-house tools, Niiwa's got the spatial population model that we developed in-house and has been used globally now for people working in the fisheries arena. MyCAMS is used for analysing seabed photos,

what species are we seeing, what substrates are we seeing. It uses ImageJ, an open source tool from the US National Institute of Health and it uses PostGIS as the in-line data repository. That means that you can open up in QGIS, the in-line database and view where the species are being observed.

ClidESK is for climate data visualisation. That is a tool sitting on top of a Clid database developed by the Bureau of Meteorology in Australia. This is being deployed across the Pacific at the moment. So, that's where I'm at.

I'd like to acknowledge the open source developers, the community that gives us these tools to do all these fun stuff and the people who provided the funding to keep us going.

I'm afraid we are a little bit late for the time. May I ask you to leave your question to the end of this whole session and we will ask the speakers to answer your questions altogether, please. So, we will move to our next talk.