Deployment of AI-enhanced services in climate resilience information systems
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
Title |
| |
Title of Series | ||
Number of Parts | 351 | |
Author | ||
License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
Identifiers | 10.5446/68910 (DOI) | |
Publisher | ||
Release Date | ||
Language | ||
Production Year | 2022 |
Content Metadata
Subject Area | ||
Genre | ||
Abstract |
| |
Keywords |
00:00
Web serviceInformationSystem programmingModulo (jargon)TrailOpen setComputer programming1 (number)Web serviceBitInformation systemsProjective planeInformationCloud computingMechanism designTrailStaff (military)Artificial neural networkAlgorithmDifferent (Kate Ryan album)WebsiteProcess (computing)MereologyComputer animation
01:32
Thermal expansionStrategy gamePrototypeSystem programmingTranslation (relic)Software testingProduct (business)Integrated development environmentWeb serviceProcess (computing)DisintegrationFeasibility studyScale (map)Level (video gaming)Level (video gaming)Software testingDifferent (Kate Ryan album)Term (mathematics)TestbedProjective planeCodeFlow separationComputer programmingComputer animation
02:14
MathematicsWeb serviceLattice (order)Line (geometry)Domain nameDialectDifferent (Kate Ryan album)Computer animation
03:12
Time domainInternet forumInformationGroup actionFocus (optics)Pressure volume diagramDisintegrationVisualization (computer graphics)Variable (mathematics)Workstation <Musikinstrument>Reduction of orderData fusionStatisticsMathematicsProcess (computing)Function (mathematics)Artificial intelligenceProduct (business)Mathematical analysisDecision theoryWeb serviceInformationWeb serviceGroup actionLattice (order)Domain nameVariety (linguistics)System programmingLevel (video gaming)Information systemsIdentifiabilityComputer animation
04:58
InformationWeb serviceSoftware developerDecision theorySatelliteAsynchronous Transfer ModeNumerical analysisReduction of orderVolumeIntelInternetworkingStatisticsComputer networkArtificial neural networkSpacetimeContext awarenessContrast (vision)System programmingTerm (mathematics)Cubic graphComputer animationProgram flowchart
05:55
Client (computing)Web serviceOpen setOpen sourceTerm (mathematics)DampingMechanism designData storage deviceSelf-organizationPolygonCuboidSet (mathematics)CollaborationismWeb serviceSubsetClosed setComputer animation
07:04
Web serviceSkeleton (computer programming)Template (C++)HTTP cookieInstallation artProcess (computing)World Wide Web ConsortiumExact sequenceParameter (computer programming)Point cloudInternet service provideroutputMereologyComputer programmingProcess (computing)Scheduling (computing)PrototypeDomain nameAlgorithmTemplate (C++)HTTP cookieRepository (publishing)Line (geometry)Web serviceBitThermodynamischer ProzessGroup actionInternationalization and localizationProjective planeEvent horizonInformation systemsExtreme programmingWeb 2.0Skeleton (computer programming)CodeVapor barrierWordMultiplication signFrame problemHeat waveComputer animation
11:10
PrototypeProcess (computing)MathematicsDevice driverInformationHeat waveVirtual machineExtreme programmingEvent horizonComponent-based software engineeringCodeVulnerability (computing)Price indexWaveWeb servicePlastikkarteGraphical user interfaceSystem programmingSoftwareSelf-organizationDemo (music)Software frameworkSoftware testingDenial-of-service attackHeat waveSet (mathematics)PrototypeAlgorithmThermodynamischer ProzessWeb serviceEvent horizonPredictabilityVirtual machineLine (geometry)Flow separationProcess (computing)Data storage deviceResultantDialectUniform resource locatorWave packetDomain namePattern recognitionVulnerability (computing)Digital photographyDampingMultiplicationMultiplication signMoment (mathematics)Device driverType theoryWeb 2.0Raw image formatSource codeSubject indexingComputer animation
16:30
Time domainGroup actionProduct (business)Set (mathematics)Internet forumComputer animation
Transcript: English(auto-generated)
00:00
Yeah, welcome everybody and thanks a lot to everybody who made the way up until this hidden corner on the academic track. Yeah, the following talk is about the deployment or deployment mechanisms of artificial
00:21
intelligent enhanced services for climate resilience information systems. So basically the questions of how to bring scientific algorithms into cloud computing services and so that the end users can have an easy access to information on demand. That's and there for sure whether it's filled out with OGC
00:43
standards because yeah, I'm a staff in OGC. OGC has two big parts, big programs. So one is the standard standardization program and we had been hearing that in many different talks about the OGC standards and that then but then on the other side
01:02
there's also the innovation program where we are involved in quite a lot of different projects and the ones which is a bit related to climate are here, but feel free to check out the website. So there are plenty of other projects as well. And that's where we are fiddling around with
01:21
technical challenges and trying to find good recipes and if we are making a good job, then maybe they're ingesting into their standardization as well. And then also while the OGC is or in the OGC innovation program, we are working on quite different levels of
01:41
technical majority levels and so from very basic concepts up to then the testing and making the interoperability experiments and and then also well in terms of projects and we have small projects, little coding sprints for a couple of days or then also well the test beds
02:04
which are running over a couple of months or then long projects for several years. So that's just kind of showing the flexibility that we can do and and now if you are focusing on climate that's something what we are pitching up currently as a new road in OGC and with a new domain that we are
02:27
focusing on and last year and that we had kind of a lot of different ad hoc sessions, climate sessions during the OGC member meetings as well as then also dedicated or in line with the United Nations
02:44
COP26 or the the regional climate weeks in Africa and Asia Pacific, etc. And then colleagues from ECMWR, for example from ESA, from NASA, from from the Canadian colleagues, Australia, etc. They were
03:02
contributing to these sessions and then we had a very nice exchange and building up a community and about the questions on how to identify principles, so findable, accessible, interoperable, reusable
03:20
aspects in the climate domain. Not only for the data, but then also for the entire system itself. So how should climate information, climate resilience information systems be be designed so that we can have a better synergy and that we can have kind of a good exchange.
03:40
That's what we are talking about here in the Phosphogy very often and out of this all these discussions we have been setting up a climate resilience domain working group where you're all welcome to join and continue with the discussion and then the next working group meeting will be in the 5th of October during the OGC member meeting in Singapore
04:04
where we are electing the chairs and yeah, you're more than welcome to join. What we are talking about here is always the question that in climate or in climate related domain we have a huge amount of data, which you can see here on the left side and
04:23
the question is always how to concentrate that, how to merge that down, how to fusion the data, how to bring that from the raw data into a sensible information that is then useful for the scientists or policymakers, for non-technical persons, for a variety of very
04:43
homogeneous, a very heterogeneous user group. That's summarized in climate service or science service if you are including all different domains as well. On a technical level and here comes the the question of
05:03
interoperability and the fair principles. What we try to do especially then in the context of OGC is building up the blocks in the way so that they are fitting to each other quite easily. So that we then if you would like to set up your own system
05:23
wherever you are and however the system should look like and that all the blocks which are there are fitting together and you can build it up and in kind of in a cubic building block system. So that's the entire idea of all the standardization.
05:42
So why we are sticking to standards or why it makes sense to stick to standards in in contrast of building up your own individual ideas. So that's what we are here. And then in terms of the climate or climate related
06:01
building blocks, things that we're not starting from scratch, but there is quite a lot of things already out there. What I'm showing here is all open source. You can find it in the in a GitHub organization called Birdhouse, but then left and right we have other things also laying around.
06:21
There are two things already implemented in ECMWF C3S. It's the Rooks and then also the deployment mechanisms with Ansible Doggers and with Conda. Rooks is a service for polygon subsetting, bounding box subsetting. So that's already implemented there and we are in close collaboration with ECMWF and talking about how to bring the other blocks
06:46
into the climate data store as well. But yeah, feel free to to take that if you have a need on it and I'm showing you now a few other things what we have been
07:02
developed and which is upcoming where you can see that. And the idea is also well if it is working fine, so if we are kind of sticking to the standards, if you're sticking to OGC standards or what standard whatever, then it's possible to kind of exchange and and have the
07:22
synergies in between of the different services. And then during the or within the domain working group focusing on the United Nations policy frames, that is always the questions of how to enable the synergies in between of the different countries and the services which are around.
07:42
One of the things that we have, so now it starts to become a bit more academic and more technical, is to help out and building up these blocks yourself. So if you have a good algorithms that you would like to wrap up and then maybe provide that for C3S or for whatever service you are focusing on, then we have a template which is called the cookie cutter.
08:05
You find that also in the birdhouse repository. You just need to install that. So a couple of letters, a couple of words, a small command and then you have it in place and by running the cookie cutter
08:20
it's asking you a couple of questions, takes not so much time, some minutes, and you have the skeleton ready. And the skeleton is then in line with all the OGC standards. So it is respecting all the required yeah, all the all the required
08:40
standardizations. And then comes the the big work that you need to do is that fill it with your algorithms and kind of the code that you have needs to be set in place. And then also here you need to respect how to pass through the different arguments that it is then fully automatically running on on the cloud.
09:04
But once this has been done, once you have ingested that as well, then it's ready to implement and deploy that in C3S for example or wherever you you want to have that. So that's kind of here. We were working on that to lower the barrier so that the community can grow and then everybody who wants to
09:25
provide services in international or local climate resilience information systems. That's the way how you can do that or that's the way we are proposing. One of the things, this is a bigger project where OGC is involved together with a consortium, quite scientific
09:43
focused consortium. This is a European project CLINT. It's called CLINT climate intelligence. Here focusing on artificial intelligence to have a better understanding of extreme events and extreme event detection. So what we're doing here with the scientific colleagues is they are providing
10:06
the algorithms, the scientific algorithms and how to detect heat waves, tropical cyclones and etc. Things like that. And focusing on
10:22
seasonal forecasts so that's the dedicated these are the input data where we are focusing on. And then the part of OGC innovation program is that we are wrapping up these algorithms into OGC API processors or formally that was web processing service and
10:45
in the way I was showing before and having a building block and then with the idea that by the end of the project is going to be ingested or is going to be provided for the Copernicus
11:00
services. It's provided they have to do the deployment on their side then. But we are far ahead on the schedule. So the first processes, the first prototypes are already in place. So we have plenty of time to think about how to fiddle around and how to bring that into the climate data store and also where currently climate data store is being rebuilt and that we can think about how to
11:27
bring, yeah, how to deploy that in an easier way. So the first prototype that we have in place, which is up and running, there is a
11:41
the URL that Clint Dicker said, the German Climate Computing Center is hosting that for the moment. If you would like to play around on that, it's just type in this URL and what the process is doing is based on artificial intelligence.
12:00
So in the middle you have a data set with quite a lot of missing values. So basically everything is missing. And then on the left side you have a training data set. So the ground truth and where the artificial intelligence is being trained on and
12:21
kind of learning how the values should look like. And then on the right side you see the result. And the results are quite promising and quite astonishing. We took an example here in Africa because of, well, Clint is basically focusing on Europe,
12:42
but there were not so much data sets with too many missing values. So we had to take a data set from Africa. And currently we are fiddling around also with other sources, so radar data, and this is being possible as well. And the algorithms behind is coming from
13:03
your face recognition algorithms. So when you have a photo and there is something missing in the picture or there are scratches inside so that the algorithms could be trained and fill in the rest of the face.
13:23
So that's the the scientific algorithms behind. But if you have more questions on that then my colleagues from Deceresat can tell you all the details about that. Then another one is more on is we are facing here the the challenge of data pipelines and that is
13:44
heat waves and warm nights. And what we are doing is are several steps then. So there were processing services with multiple steps. And the first is a detection of the event, so the definition of the index, which has been detected by machine learning algorithms. And then once the
14:09
event is being detected and that we're piping through quite a lot of data and that making detection of the drivers to understand why this extreme event has been occurred. So if you have a heat wave, why is the heat wave occurring? So it is
14:27
depending on, I don't know, soil moisture or the westerlies or what is the reason for this event. And then that's been then detected by as much data as you can ingest there. And once this has been done
14:42
you can make the assessment of the drivers and do the prediction for the future. These are then the story lines which are coming out. We have several test regions because machine learning you need to reduce.
15:00
You can do that on a larger domain. So we have some test regions in Europe as well as in Africa and try to understand better how good that works. Okay, and to understand, well, so what we are promising for for the future that is missing values is already
15:23
very is already in place. You can take it and improve it on your side as well. Heat waves, we are on the way of having that as a prototype up and running quite soon. And then we are also focusing on hydrological floods. We'll
15:42
using artificial intelligence and then drought vulnerability and tropical cyclones, which is coming then from ECMWF itself. And I think with that, oh no, yeah, and for sure we have documentation in place. So if you would like to do it yourself, if you want to have a known
16:05
climate building block on your side, if you would like to contribute to that, then here is the documentation. It is the very draft and very raw beginning. So if there is something missing, don't hesitate to reach out and then we are
16:23
completing that. It will start to grow in the future. But you are more than welcome to contribute. And with that, I would like to thank you and I'm looking forward for the discussion.