Thank you very much for the introduction, Tommy, and slight promotion as well. I haven't actually done a PhD yet anyway, but I have a master's in Earth observation

and geo-information management. As mentioned, I work at an organization called Satellite Applications Catapult. We're based in the UK, and I'll go into a little bit more detail about the organization in just a minute.

I've got a geography background with a master's in Earth observation or as it's in elsewhere remote sensing. I'm here to talk to you today about an implementation of the open data cube. It's a core technology that's enabled the creation of multiple Earth observation data cubes around the world, and several others of which are seen to be presented in this

conference, which is fantastic. This talk will discuss the ongoing implementation of the IPP Common Sensing Data Cube, which is located over the South Pacific region, focusing on small island nations, such as Fiji, Vanuatu, and Solomon Islands.

This project has been funded by the UK Space Agency's International Partnership Program, and the key technologies used as well as realities of developing a data cube from the perspective of an EO data scientist will be presented. Who we are as an organization, Satellite Applications Catapult, we're quite a unique

business. We're an innovation and technology company, and we're trying to transform the way the world uses satellite technology and data. We're partially backed by UK government and the UK Space Agency.

We're an independent organization, not related to government, but we're there to bring together industry researchers and end users and help organizations, particularly within the UK, grow their businesses within the space industry. Our focus is across several areas.

We have those connectivity type satellites, but the area in which I work within the organization is within geospatial intelligence, and we're there to energize the market and power technology and enable businesses throughout this Earth observation and geospatial sector.

Application-wise, we have multiple vertical value streams from agriculture, transport, health, sustainable development, and they're continually growing as well with recently branching out into more sustainable finance type objectives. The project itself, of which the data keep is part of a larger solution.

The overall vision or aim of common sensing is to improve national resilience towards climate change, including disaster risk reduction and food production, and contribute to sustainable development in Fiji, solar lines, and Vanuatu with support of information derived from satellites.

For Pacific island nations to respond to, prepare for, and anticipate hazardous events and disturbances related to the climate known as climate resilience is increasingly vital. We've been taking a user-centered design approach where our designers have been talking

to many of those in countries and gathering requirements. The project has developed a platform to assist government-level ministers in making decisions around climate resilience, of which the data keep is a part of the solution. With building these four areas of focus, one is to build climate resilience,

where you need detailed understanding of future climate conditions as well as past, in both a timely and scalable manner. Next is to build climate resilient food systems, which needs detailed understanding of future climate and their impact on food sources, as well as building disaster-resilient communities.

Changing climate requires a detailed understanding of both current and future risk and vulnerabilities. To build climate resilience requires a level of resource and investment, which is often beyond the capacity of individual countries.

Therefore, having access to international climate finance funds is crucial and hopefully what the platform being developed here can help support with. So it's important to note at this point that the end users of this project are government ministries and climate finance advisors within these countries,

of which we have the data over. We have fantastic in-country representatives within those governments who are providing feedback and support as this project has been developed right from the start. So the overall goals are to develop satellite-based information services,

which directly match the challenges and needs for each of these countries, strengthen each country's capacity to improve their climate resilience and disaster risk management. So this includes us doing two things. The project focuses on delivering geospatial and climate information,

decision-making tools and capacity building for technical experts, sector specialists and decision-makers. We also place specialists within government structures to ensure local knowledge and data systems benefit from the added capabilities afforded by the Common Sensing Project.

So we're working as one sort of organization within a much larger consortium, and these are kind of our prime partners. So some of them are providing kind of data services.

Others are providing sort of training type training and capacity building and sort of understanding and knowledge, whilst others such as DevEx is providing more of that kind of marketing support. So the main kind of data providers here are ourselves with the DataCube solution

based off of the Open DataCube technologies. The University of Portsmouth are providing expertise on products such as bathymetry and landslide risk, the Met Office. This is the UK's Met Office providing data on climate,

sort of going back 30 years, as well as predictions. Spatial days are providing geospatial sort of technical support and Centinomic are providing data on food systems, as well as Unisap providing kind of a disaster resilience app.

So, yeah, all of this is provided by the UK Space Agency International Partnership Program funding. OK, so now I'll sort of move on to what many of you on this call are here for. The technology which we have both produced,

as well as the technologies that we've used to sort of help produce this solution. As those of you who are familiar with these logos, most of these are open source, which has been absolutely essential to the development of this project and kind of the communities around this,

the communities which have also developed around many of these softwares are also a really important kind of resource, particularly kind of the Open DataCube community. This list is by no means exhaustive, but some of the really important tools which have enabled us to develop our solutions are here

and I hope I'll put some of those into the books coming up. So I think it's important to kind of highlight this point that I'm an Earth observation scientist. I like working with satellite imagery. I've been working on this project in conjunction with many others within the wider consortium,

but also within the catapult for several years. We've had a team of developers, GIS specialists and EOD scientists, as well as the designers and all the others who are required to kind of ensure a technical project runs smoothly. Although many partners have also contributed to this project,

the catapult's been responsible for kind of bringing together this wider picture and kind of portal. So as you saw on the slide before, we kind of have an Esri sort of client side.

The reason for this is the government ministries within one of our countries, Fiji, already use sort of an Esri enterprise system and wanted that to be integrated with their existing systems. Whereas Solomon Islands and Vanuatu don't have that system already set up, so they're using an open source system based on Ontario GIS.

So sort of here in the top right of this kind of architecture diagram, we've got these other data services which are coming in, particularly this UNITAR disaster support app.

And it's important to note now these data centres. So currently this system is based out of the UK, this data system. Within the catapult, we have a data store called CEMS, which is currently hosting the data,

but we're in the process now of moving that out to the South Pacific region. The University of the South Pacific, which is based within Fiji, have sort of taken on responsibility to host this, which is fantastic from a sustainability point of view of the project, because it means that it's not being sort of held within the UK and will sort of be dropped when this funding is up.

It's been taken on by University of South Pacific and they're sort of taking on responsibility for maintaining it, as well as sort of their students being able to use this data cube system. Also, they obviously have close relationships with the government ministries who are the end users.

And then sort of this ITC data centre here, which is Fiji's existing ESRI structure. So here is kind of that. This sort of box down here is what we've sort of really been responsible for bringing together, which are these applications.

I assume that you can see my pointer. I'll keep going. So this box in the bottom right, I've sort of put together a slightly messy diagram of how this looks with kind of the technologies that we're talking about. So this is what I would call our data cube processing chain, where we're going from sort of having this wealth of satellite imagery.

Most of these sort of, well, originally the sort of programme was to have Sentinel-1, Sentinel-2 from ESA.

Well, sort of the imagery, the data from ESA downloaded from various portals for various reasons, as well as Landsat. But we've actually sort of expanded beyond this, bringing in kind of MODIS data. There's also some spot imagery in there as well.

And more recently, NovaSAR, which is a new S-band SAR satellite, for those of you who are sort of more satellitey experts, which is a kind of a UK-operated satellite, but we've tasked some imagery over Fiji. So what we've sort of developed is a system where it's quite easy to sort of bring in any sort of satellite data,

as well as any other kind of geospatial data into this data cube. So the first kind of step of the data cube is to create this ARD, which as an Earth observation specialist,

I can say is the majority of the legwork of using satellite data. So we sort of, our ARD processing chains are in line with kind of industry standards. There's a lot of work being done, particularly out of Australia on kind of these standardisations.

And we've sort of integrated these with technologies such as Stack, which enables kind of the indexing of the data in a commonly known way. It's all kind of dockerised and containerised within GitHub, all of these processing chains,

and the outputs are cloud-optimised geotiffs. So all that's been really interesting work, and as I said, kind of takes out a lot of the legwork along the way. And these kind of open source technologies have been really integral to that. I'm sure that most of the technologies within these slides have been mentioned sort of yesterday in talks,

as well as today, and I imagine they will be highlighted fairly heavily as well. These sort of three across the top here are ones which have been used throughout the whole process.

So once we've got our ARD, which is analysis ready data, we've put that into an S3-like storage. Currently this is hosted on CEMS, our kind of internal computing system, cloud computing system, but to be moved to the University of South Pacific. Many of the other data cubes operate on kind of AWS currently,

including kind of the African data cube and, sorry, Digital Earth Africa and Digital Earth Australia. We began with this system being hosted on AWS, but made because of this kind of sustainability aspect and sort of conversations,

which took us in the direction of hosting it elsewhere. We moved on to our own systems and then sort of made this connection with the University of South Pacific. So it's an S3-like storage that we're using, but it's not AWS S3. And then we've got this product generation, which I'll go into a bit more in a minute.

So this is sort of bringing in this really cool technology of the open data cube, alongside sort of Dask for scaling the processing, enabling us to do processing across scales

sort of up to kind of countrywide analysis, which as a kind of scientist is sort of a bit of a learning curve for me. But this sort of product generation is happening within Jupyter Notebooks, which I again will go into in a bit more detail quite soon.

And then we've got this kind of user access. So there's two ways that the users can kind of access this data. One is through Jupyter Notebooks directly, sort of talking back to our S3 storage of ARD.

And that's kind of taking advantage of this stack indexing in the way that you access and the data will have a little bit of code kind of later on. But we, but yeah, if you sort of looked into any of the other data cubes, we operate in a very similar way.

So the other way apart from accessing it directly through Notebooks is through in Fiji, kind of using this every system where we've developed kind of a portal along with the app provided by the climate app and the disaster resilience app.

And then for Solomon Islands, it's a Terrier GIS-based system or portal rather. So as kind of several of the talks I've heard recently, which have built systems off the open data cube, from a technical standpoint,

our data cube is an implementation of all of the amazing work which has gone on with the open data cube, particularly by the likes of Geoscience Australia. It's enabled us to sort of hit the ground running with implementation.

And for those of you who kind of don't know the principles of an open data cube, I'd certainly go and check out the open data cube website. The documentation is pretty readable. And yeah, I'd recommend going and have a look around.

There's some really nice examples. But in principle, it's kind of a way of taking satellite imagery and stacking it up. And then you can access it in a way of being able to take small chunks of data through space and time and provide analysis on it straight away,

rather than having to download whole images is really beneficial. So here's just kind of an overview of what kind of data is within our data cube. As I've said, this is sort of what the project, what we initially said we'd do on this project,

which was to bring in Sentinel and Sentinel 2 and the Landsat archive. We have added some more data sets as well, which is really exciting. And they will be accessible by kind of the users, especially once it's handed over.

When sort of the data cube was, well, sorry, when this project was set up, we're sort of saying, oh, look, we can go back to 30 years worth of Landsat data as you kind of do. But the realities are that Landsat 4 and Landsat 5 doesn't have great availability over the South Pacific.

And this is, the Landsat series wasn't sort of always imaging. It was being turned off over particularly over cloudy regions, which the South Pacific is for many months of the year. So we have got some quite large data gaps in this kind of time series going back, which is a shame for some of these cases that we have.

And also the sort of faults on board Landsat 7, which have led to strike mapping. But when you sort of mosaic a lot of data sets together, that does sort of go away. But from kind of Landsat 8 onwards, there's a really fantastic kind of data sets in there. I should have actually put in the statistics of how many images that we have loaded in

and that kind of thing across the South Pacific. But it is a very impressive amount. One thing I realize I haven't particularly covered yet is what sort of stage we are at in this project. We're coming very much to the end of it. All of the data is loaded in and all of the product are there and available

and we're currently kind of in a testing phase with some of those government sort of bodies who are going to be using this data cube as well as sort of at the point of doing trainings within University of the South Pacific as well to sort of ensure that there's a smooth handover of the system.

So all of the data is there and done. There's a couple of things with sort of integrating the data cube into Terrier GIS, which our developers are still working on. But again, it's sort of very near the end of that process. The Fiji solution is very much there.

That was sort of our starting point. So as well as this kind of these ALD products, which we've sort of preprocessed, we've also gone for some derived routine products into our data store.

So this is when we've developed things at a certain cadence and then pushed them back into the SQLite store. Currently kind of the more on demand products, which is the ones that the users are making, don't get pushed back into our S3 storage, but our routine products do.

So routine ones are two main products. We've got water masks, which have been developed for every sort of single scene across all of the sensors. Many of the other data cubes have used kind of something called WAFs, which is water observations from space. But we sort of found quite a few flaws with that,

especially looking at our sensors, and also we wanted this kind of Sentinel-1 to be a large part of this solution. So we went for producing water masks by machine learning techniques

to give a two-band raster, one of which is a binary mask and the other a kind of predictive confidence, which has meant that we can kind of use water masks with more certainty throughout our kind of analysis. So why did we make water masks routine

and sort of add to that ARD by adding a water mask to every single product? It's because they are commonly used across many of our products and having these pre-processed reduces the processing time and kind of prevents or makes it less common that we run into kind of processing issues.

So that's something which really helps us along and keeps everything running quickly. So as well as these kind of for every individual image, we're also doing it for annual aggregations.

So it's quite common that we found the users want to run one of our kind of products for a year. So if they can take that annual water mask, then that kind of again reduces that processing time and similar concept for the geomedians.

We've produced these, sorry, I've realized that the text on the bottom left is actually not correct. That's the water mask text again. I'll make sure that's changed for kind of the recording. But the geomedian products we've done on an annual basis and what this gives us is kind of a best cloud-free composite of sort of the optical datasets.

We've done this for Sentinel-2 and for Landsat-8. They're kind of our fullest datasets. What it is is kind of like a median function, but it's done on all bands kind of at the same time.

So to remove the likelihood of cloud in the best way possible and get the best kind of spectral representation across each of those bands. So that's something which took quite a lot of processing power to do those for each of the three countries for every year since 2013 and sort of certainly kept our DevOps team entertained for a while.

So just here kind of touching on access again. I've explained most of this already actually. So on the left here we have kind of this JupyterLab instance. It's kind of a sandbox environment similarly to how many of the other data cubes are kind of accessed.

We've got a set number of scripts which can be ran and they can also be changed, etc. Currently this sort of data cube system isn't completely open source. It's currently designed to be for users within the government ministries of Fiji, Vanuatu and Solomon Islands

as well as users within the University of South Pacific. But this could be subject to change in the future. But for those users they can go in and access this JupyterLab instance and run many products.

Or this shows our kind of Esri solution for Fiji where you can see how you can kind of order these products via a form. So the parameters in both sides are the same. You put in the parameters you want such as the area of interest, you want the time range, you want the resolution

and kind of the projection which are common across all of our products. As well as sort of ones which are more bespoke to each of the products. Which I'm going to show you the products in a minute. I realise I haven't really mentioned those so much yet.

One thing which has been quite the bugbear for us in this project is the anti-meridian which crosses Fiji as you can kind of see represented within this geo-median here. I think this was an attempt at making a geo-median for 2019 where you see we get data gaps along this anti-meridian.

We had quite a few issues with missing data, issues when displaying in certain productions and especially global ones such as EPSG 436 which any kind of geospatial people on the call will not be surprised about this but trying to get the open data cube technology to kind of work with this in a way which doesn't have too much missing data

has been a challenge particularly as the idea of this, the way the system works enables a user to sort of select any area of interest. Well if the area of interest sort of only partially crosses this anti-meridian or is too large over this area

then it quite often results in kind of missing data such as we had a large square up here missing for a while and quite a lot of these islands in the southeast were missing for a while as well. If you're interested in this then one of our DevOps resources, Luigi, he's no longer at the catapult anymore

and he did a lot of work on this about a year and a half ago and he's written most of it down in quite interesting blogs so if that sort of thing interests you then go and take a look there. So these are some of the data cube products which we are kind of supplied as kind of, they're almost demo apps in a way

in that we're hoping that the users will go and kind of create more afterwards and these were based on this user-centred design approach of talking to those within various ministries, government ministries

to see which products would be of most use to them. Many of them are similar to those produced by Digital Earth Africa and Digital Earth Australia. We've got ones around vegetation, ones around water quality, ones around permanency of water

all of these things which are important when we're looking at environmental factors such as sea level rise which are pertinent in these small island nations. So I think for time I'll not dwell on these too much

but if any of you have any kind of questions about specific products then we can cover those in the question session. This kind of shows one nice example of a change between the 2000 and 2010

going from kind of the greens being 2000 to the oranges being 2019 and we sort of looked at this one area and saw there was a lot of change going on in this sort of estuary and then we sort of spoke to our kind of in-country representatives about this. So you can see this box on the right shows where there's a lot of change going on

and the box on the bottom left, the sort of blue box shows there's not much change going on there at all and what had happened was we'd been dredging within this river here and a lot of the sand was being sort of deposited into this area here

which should sort of change the local morphology of this estuary. There's a lot of corals and things like that along this coastline and so it just kind of is one extremely quick and brief example of how we anticipate this sort of data being used towards these kind of more environmental type applications

of sort of evidencing how a decision has impacted or will impact kind of an ecosystem. So kind of some very brief reflections for me as an earth observation scientist.

So I came straight onto this project after I left university and the point at which I joined, we sort of had a working sandbox environment with sample data and not a huge amount, but enough that I could kind of start working on these products straight away.

And just the kind of the power which is in there and the ease which is in there of accessing these data sets compared to when you need to go and do work on other projects elsewhere and sort of download a whole room of data.

So yeah, for me, the kind of open data cube and all of the other sort of software which I've sort of briefly mentioned as well really comes together in a powerful way and you can see that by how many data cubes are being built all over the world based off these open source kind of solutions.

So I think that was kind of a brief reflections from me on sort of the ease of access which data cubes provided. So I suppose now we've got a couple of minutes for any questions. I don't know how you want to do this, Tommy, whether we're taking questions over chat or verbally.

I'm unable to hear you. I'm not sure if you're talking. Can you hear me? I can hear you now.

Yes, we are ready to start with questions. Please. Okay, I'm looking at the audience and I'm looking at the chat. If there are questions for Sara, let me kick off.

First, I want to apologize for promoting you to PhD. I do wish you to engage in science and related to that, I would like to ask you, in your talk you spoke a lot about technology and implementation and are you facing some specific scientific challenges also in terms of statistical methodology, spatial analysis methodology?

How do you deal with the uncertainties? So what are the top, let's say, two scientific challenges? So I think that it's a very good point, I've sort of focused on technological implementation

but there are all of these questions around if you were doing any one of the products which we've provided the baseline that you can run,

from a scientific standpoint, you would certainly want to go and validate those and implement them and see to what level of accuracy they are. Currently, there's been very limited ground data going into these and a way of validating.

Most of the products are based upon quite standardized earth observation techniques and things such as NDVI and quite often the outputs aren't definitive, they're arranged with a likelihood or a confidence attached to them of how those outputs look.

What the outputs are is a countrywide or regional wide initial assessment with the idea that it would highlight possible problem areas or areas of concern

to these ministries who could then go and take a closer look. So it's that first step in that analysis chain as well as providing evidence looking backwards. As I said, towards the end there, what's been provided is these earth observation products

and they're sort of there in the same way as with the other data cubes, almost as demonstrators of how you can get these systems working. So then more complex machine learning with training data, etc.

There's capacity for those to be built into the final product more through that kind of ability to access due to notebooks. We're especially in conversations with the University of South Pacific and they're sort of looking to take on that kind of scientific ownership of the products going forward within their kind of geospatial program.

We have one question from the online participant. If satellite applications catapults is also processing data for offshore marine part of the islands, so more like marine applications.

Yes, a lot of the marine areas are covered within the analysis ready data. We did take a bit of a look into what we could do with that. One of the products is water quality, which is looking at the turbidity

of the water coming out of estuaries in particular. There's a lot of GIS data, which is loaded within the same platform, provided by the government ministry. So it's kind of a one stop shop for data. So all of the marine protected areas and the known areas of coral

and that kind of thing are kind of marked within. As I've said, the ability to produce new products is there within this hub system. Sorry, fire alarm is going off. I'm not sure if you can hear that. So the ability to produce new products is there. The analysis ready data which has been produced is for land surface reflection.

Whereas you can do slightly different processing on data sets to adjust it more to marine applications. So that's kind of why we haven't gone into maybe some of the other marine applications. But certainly the scope is to kind of add chlorophyll-a type indices in there as well.

I have a couple of questions for you. Just keep them short for the sake of time. We have to continue. One question is about QGIS. We didn't see QGIS in your workflow or especially from the user perspective. And then the second question is people at Fiji.

They have a different education system than UK obviously. How do you get these people to use the tools and develop things independently? So two questions, QGIS and capacity. So with QGIS, it's something which we've used kind of in the background

as we've been developing to kind of check and validate products as we've been going in that kind of thing. But the two kind of GIS systems being used in country are Eteria GIS and ESRI GIS as the platforms. Of course, the users can download and use the sort of outputs

in whichever GIS system they would like. As I said, the list of kind of softwares wasn't exhaustive. And secondly, about the education system. So many of the government ministries that we've been sort of talking with and that we've been training up have kind of data scientists

or government, sorry, technical users, as we call them, as well as those kind of policy users. So we're doing separate trainings for both of those groups. So there are people within the ministries who have sort of that level of technical GIS knowledge, as well as we're hoping this connection with the University of South Pacific will enable them to bring

the DataCube into their teaching within their master's kind of GIS and satellite remote sensing programs. And many of those people already go into these government positions. So we're expecting kind of that location within the university to give us the DataCube to filter through into the ministries.