We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

MOOC Cubes and Clouds - Cloud Native Open Data Sciences for Earth Observation

00:00

Formal Metadata

Title
MOOC Cubes and Clouds - Cloud Native Open Data Sciences for Earth Observation
Title of Series
Number of Parts
156
Author
Contributors
License
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language

Content Metadata

Subject Area
Genre
Abstract
*Motivation:* The Massive Open Online Course (MOOC) "Cubes and Clouds" teaches the concepts of data cubes, cloud platforms, and open science in the context of Earth Observation (EO). The course is designed to bridge the gap between relevant technological advancements and best practices and existing educational material. Successful participants will have acquired the necessary skills to work and engage themselves in a community adhering to the latest developments in the geospatial and EO world. *Target group:* The target group are earth science students, researchers, and data scientists who want to dive into the newest standards in EO cloud computing and open science. The course is designed as a MOOC that explains the concepts of cloud native EO and open science by applying them to a typical EO workflow from data discovery, data processing up to sharing the results in an open and FAIR way. *Content:* This MOOC is an open learning experience relying on a mixture of animated lecture content and hands-on coding exercises created together with community renowned experts. The course is structured into three main chapters Concepts, Discovery and Process and Share. The degree of interaction (e.g. hands-on coding exercises) is gradually increasing throughout the course. The theoretical basics are taught in the first chapter Concepts, comprising cloud platforms, data cubes and open science practices. In the second chapter the focus is on discovery of data and processes and the role of metadata in EO. In the final chapter the participants carry out complete processing workflows on cloud infrastructure and apply open science practices to the produced results. Every lesson is concluded with a quiz, ensuring that the content has been understood. The course contains 13 written lectures that convey the basic knowledge and theoretical concepts, 13 videos which have been created with a professional communication team and in collaboration with a leading expert on the topic and shines a light on a real world example (e.g. The role of GDAL in the geospatial and EO), 16 pieces of animated interactive content which engage the participants to actively interact with the content (e.g. Sentinel 2 Data Volume Calculator) and 11 hands-on coding exercises in the form of curated jupyter notebooks that access European EO cloud platforms (e.g. CDSE) and carry out analysis there using standardized API's like openEO (e.g. full EO workflow for snow cover mapping). *Infrastructure:* The EOCollege platform hosts the lectures and the animated content (e.g. videos, animations, interactive elements) of the course. The hands-on exercises are directly accessible from EOCollege via a dedicated JupyterHub environment, which accesses European EO cloud platforms, such as the Copernicus Data Space Ecosystem, using its open science tools like the Open Science Data Catalogue, openEO and STAC. Guaranteeing that the learned concepts are applied to real-world applications. In the final exercise the participants will map the snow cover of an area of interest they choose and make their results openly available according to the FAIR principles on an web viewer (STAC browser). This community mapping project actively lives the idea of open science, collaboration and community building. *Learning achievements:* After finishing the course, the participants will understand the concepts of cloud native EO, be capable of independently using cloud platforms to approach EO related research questions and be confident in how to share research by adhering to the concepts of open science. After the successful completion of the course the participants receive a certificate and diploma supplement and their personal map is persistently available in the web viewer as a proof of work. *Benefits for the open geospatial community:* The MOOC is valuable for the geospatial and EO community and open science as there is currently no learning resource available where the concepts of cloud native computing and open science in EO are taught jointly to bridge the gap towards the recent cloud native advancements. The course is open to everybody, thus serving as teaching material for a wide range of purposes including universities and industry, maximizing the outreach to potential participants. In this sense also the raw material of the course is created following open science practices (e.g. GitHub repository, Zenodo, STAC Browser for results) and can be reused and built upon. The "Cubes and Clouds" MOOC equips participants with essential skills in cloud native EO and open science, enhancing their ability to contribute meaningfully to the open geospatial community. By promoting transparency, reproducibility, and collaboration in research, graduates of the course strengthen the foundations of open science within the community. Access to cloud computing resources and European EO platforms empowers participants to undertake innovative research projects and share their findings openly, enriching the collective knowledge base. Ultimately, the MOOC fosters a culture of openness and collaboration, driving positive change and advancing the field of geospatial science for the benefit of all.
Keywords
127
CubeComputing platformThermodynamischer ProzessElectric currentTheoryLibrary catalogFile formatGoodness of fitCubeElectric generatorCircleSelf-organizationGroup actionState observerMassCloud computingE-learningCASE <Informatik>Open setTheoryFile formatMereologyFormal languagePoint (geometry)Library catalogWeb pageDimensional analysisBitService (economics)AreaLevel (video gaming)Thermodynamischer ProzessCategory of beingProcess (computing)Internet service providerVapor barrierData structureTerm (mathematics)Direction (geometry)LaptopMathematical analysisReal number2 (number)VirtualizationComputer fileClient (computing)Operator (mathematics)Server (computing)Thermal conductivityMusical ensembleMessage passingProjective planeLecture/ConferenceComputer animation
Thermodynamischer ProzessTheoryComputing platformCubeLibrary catalogFile formatContent (media)VideoconferencingStandard deviationInternet service providerYouTubeCoding theoryComputing platformWeb browserKey (cryptography)Game theoryCartesian coordinate systemPoint (geometry)Axiom of choiceLaptopCubePixelComputer programmingDifferent (Kate Ryan album)Cloud computingBinary codeSoftware developerSoftwareMereologyForm (programming)Stack (abstract data type)Field (computer science)Projective planeIntegrated development environmentResultantOpen setProgramming languageTelecommunicationVideoconferencingOpen sourceContent (media)Web browserLibrary catalogDirection (geometry)Computer architectureFormal languageLevel (video gaming)SpacetimeMultiplication signGUI widgetCentralizer and normalizerView (database)Computing platformExpert systemAreaSeries (mathematics)Electronic program guideE-learningMessage passingYouTubeReal numberSlide ruleRight angleLecture/ConferenceComputer animation
Content (media)VideoconferencingLibrary catalogComputing platformLanding pageOpen setPublic key certificateWeb browserStudent's t-testLanding pageCASE <Informatik>Multiplication signDirection (geometry)Link (knot theory)Point (geometry)Descriptive statisticsInformationUniverse (mathematics)Field (computer science)CubeComputer animationLecture/Conference
Integrated development environmentInteractive televisionVideoconferencingContent (media)Data storage deviceComputing platformDiagramBlock (periodic table)BuildingDrag (physics)Component-based software engineeringInformationLink (knot theory)View (database)AreaSeries (mathematics)Plot (narrative)Computing platformTheoryDigital filterKernel (computing)Computer fileLibrary catalogTemporal logicThermodynamischer ProzessType theoryWeb browserTexture mappingCubeCovering spaceOpen setCubeLibrary catalogWeb browserCovering spacePatch (Unix)AreaMathematical analysisValidity (statistics)Level (video gaming)Multiplication signComputing platformVolume (thermodynamics)PixelIntegrated development environmentBinary codeResultantComputer programmingDifferent (Kate Ryan album)Stack (abstract data type)Interactive televisionE-learningComputer animation
CubeStatisticsIndependence (probability theory)Open setWeb browserComa BerenicesThermodynamischer ProzessPublic key certificateProof theoryComputing platformLaptopProjective planeState observerE-learningWeb browserMoment (mathematics)NumberBitUniverse (mathematics)AdditionPoint (geometry)Numbering schemeDirection (geometry)Integrated development environmentField (computer science)Group actionStudent's t-testCloud computingLanding pageMereologyService (economics)Similarity (geometry)Musical ensembleSign (mathematics)Electronic data processing2 (number)Repository (publishing)Software maintenanceComputing platformRight angleAreaStatisticsNormal (geometry)Arithmetic progressionBit rateConfidence intervalIdeal (ethics)FeldrechnerVector spaceWritingComputer animationLecture/ConferenceEngineering drawingDiagramMeeting/Interview
Computer-assisted translationComputer animation
Transcript: English(auto-generated)
Good morning everybody Yeah, cubes and clouds What is this going to be today? Yesterday we've learned an exciting way of how to organize people in circles in an Estonian dance that we did together So that was a really interesting Exciting and I think it was a traditional way of doing things
So today we are going to take a different approach and we're not organizing people in circles now today we're going to organize spatial data in cubes and We're going to learn the new approaches that are available. So we're trying to leave the old approaches and Do a step in the new direction
and This talk is going to be about cubes and clouds So it's an online course to educate the next generation of Earth observation researchers or generally geospatial researchers In data cubes, so that is a big topic that we've heard about in this conference Cloud platforms cloud technologies, that's also very big topic these days and also in open science
So not only how do we operate but also how do we work together and below you see? Is a joint effort effort of a huge team and I'm here to talk in their name today so
What is our aim with this online course? Maybe I should say a MOOC is a massive open online course It's really available and open to everybody Who is interested in learning new things about the topics? We just talked about and why do we need? This kind of course. So just the last talk already said
making tutorials and this stuff is still underappreciated in the Scientific community and also the newer technologies are the less chances you have to find a curated course on the topics especially since we're talking about How to organize spatial data how to operate spatial data in clouds and how to share it is three different topics that you actually need
to conduct Research in the Nowadays in the modern ways and to collaborate so having these three things tied together Is an effort that we did and I think there is no other resources currently available that are
Doing this. So this is a good starting point for everybody who wants to dive into these topics And So I said we are trying to make the transition from the traditional ways of working with in Earth observation or geospatial Easier, so I guess everybody knows you've been working on laptops probably and you are well aware of how to do this
but It's good that somebody helps you to go in the direction of how to use a cloud for some Or if you've never heard the term cloud before it can be a little bit intimidating what's happening in the cloud What are the concepts behind this and why do I have to leave my old structure behind? What do I do this for?
So I think this little barrier to jump over we're trying to get people into Jumping this first hurdle and then I think they're free again to work in whichever way they want to so that's some our idea and the target group is that our young researchers are researchers that Want to advance their knowledge in cloud technologies
So, how do we do this? How do we convey the message? There we're using in three pillars on the one hand side. We're starting to teach the concepts and This is you before you start you really have to learn
What the concepts on behind data cubes in cloud computing actually are and also behind open science So this is a theoretical approach that we speak the same language that we speak on the same definitions, which is already Difficult when you think about cloud platforms and what is a service provider? What is a data provider these things? What is a server? What is a client?
So this can be a little bit intimidating So we're just trying to get everybody on the same page there The same is true for data cubes. And why do we need this? Why can't I just work with files? What is the virtual data cube? What is the dimension of a data cube? What are labels and so on? And then the same is true for open science. And why is this important? How can I actually do open science?
What are the paths that I can take to make my data open? So what are do eyes? How do I create them and so on? So this is the first step once we speak on the same language or all on the same page we're going to the second chapter that is discovery and There we're starting to understand and interact with the technologies that we have talked about. So we're going to
Discover these clouds and the cubes that are saved in clouds via various data catalogs So where can I actually find all this data that everybody's talking about in big data? But you need an access point where to find and you also need to know how to use these tools to find the data So that's one part and also we're going to search data properties because when you're talking about big data
You don't directly want to download the data first. Maybe you need to know how can I actually filter data? What are properties that I can use for that? What are file formats that are suitable for working in the cloud and for sharing data and What are the processes that are actually available on a cloud format?
So somehow you need to know how you can actually interact with the cloud in which kind of analyzes you can do there Once We're done with this. We're actually going to apply everything we learned in a real-world scenario So you're going to start processing a whole workflow from end to end
In the cloud, you're going to validate these results and in the end you're also going to share The map that you have created in an open community project. So in this case Here you're going to make a snow map You're going to select a small area of the Alps that hasn't been mapped before apply a process by loading the data
Subsetting the data to the area that is relevant for you Aggregating the data doing some cloud masking doing some thresholding and in the end You will arrive at a binary. It's no classification But that's not done. Then we're showing how you can validate this snow map and once you've validated it, we're also going to upload it onto a public available stack browser and
The participants are learning. How do I fill stack fields and how do I publish my data in the end? This data is constantly available on a Stack browser. So when you're done, you can go to your employer maybe and show them look This is a piece of a map that I've created on this cloud that I've been using and you can see that
I can actually adhere to all of the things that I told you that I'm able to do and How do we convey all of these messages? So that was a very let's say theoretical concept that we talked about so far, but how does this manifest itself?
On the one hand side we're using lectures, of course they are on the eLearning platform EO College So we have text forms We have the three chapters that we have already talked about and every chapter has sub chapters and a quiz at the end
But that's not everything so reading Tires is tiring I guess after a while. So we've also added a series of videos I think we have like 16 or 20 videos in this whole course that are picking out let's say they're shining a light on a very special part of The lecture that we have been talking about for example the one we see here
This is the chapter about open source software and here we are highlighting The role of GDAL in the open source view geospatial world. So why is GDAL important? Why is open source software important? And what is GDAL's role in cloud computing and
We've created these videos and with experts in the field and this one was the main developer was helping us to create the video So they're very short but they're very precise on one point and just show you the application of what you're learning in the real world and Besides the videos. We also have a lot of animated content. So little games that you can play here
For example, this is some what does fair stand for and we have a couple of keywords that you can drag on to the Different letters and you will get a little score in the end. So there are a lot of keys and And That's not all so so far we are learning but then the hands-on experience is very important as well
So you're also learning to code in this course There predefined jupyter notebooks that you can execute and Yeah that guide you through different steps of programming and you have different questions after these after you've completed these and notebooks
And if you have got the right answers, for example, how many pixels are in this data cube in the end of the exercise So, how does this actually look like because there's quite a complex infrastructure behind that and trying to go
Quick through that. That's quite a technical slide, but I think it's still important to understand the effort that is behind Making such a course also Trying to make it a one-stop shop that you start at your college and you don't have to worry about Registering to a jupyter notebook registering to a cloud. So this is all explained and you go through it in one step and
Of course, you have to make choices at a certain point which technologies are you going to use you cannot Depict all different ways of programming. For example, we're using Python now would be nice to also use R But we had just to make some choices to stay within the budget But the course is open everything is available on github and we're really happy if somebody wants to contribute and we're already starting now with a
Add-on project to also incorporate other programming languages So, but how does it work is that? There is a e-learning Platform, which is the central access point. That is EO College once you register there you also register directly at jupyterhub So that is where the EO access jupyterhub. That is where the notebooks are hosted and where you have your environment your programming environment
The notebooks that are hosted on jupyterhub come from github So you can also access everything that is available in this course on github. The whole course is on github It's also on cinodo so you can download versioned
Versioned editions of the whole course and you can also reuse it. You don't have to use it in your college You can just take the markdown files render them wherever you want You can pick whatever you want. You can open issues if something is not working or you have ideas to collaborate and We also did it to show that working openly is not just taught in the course, but it's also a real practice
Then the animated content we've created on the creation hub using h5p widgets the videos are hosted on YouTube and the results that you generate on the
Cloud so we choose This time the Copernicus data space ecosystem is the cloud that you're going to log into and you're doing it with the programming language Open EO or with the cloud communication language Open EO with this API and This said we're also working on integrating the Pangeo community that you have different ways of doing things
The results you create are uploaded from jupyterhub to a public stack catalog and Are made available Disseminated by a stack browser So that's the background architecture that is completely open and can be reused also for other courses if you want to continue in this direction
so I Think it's time to go on a user journey. So now we've talked about this course a lot But now let's look together how it actually looks in fields And I hope that this is going to motivate some of you to maybe subscribe to the course or in case your university teachers or whatever
Maybe have some students take the course so the first thing we're going to do is we're going to EO College and searching the Course cubes and clouds and you're arriving on the landing page with all the informations. You have all the links here to the GitHub community to the Zenodo community to the stack browser and so on you have the description of the course and
You see how far you are in the course and how many lessons are there? How many topics how many? quizzes and So this is the main entry point Then once you start you click on a lesson and you're going to start the course. This is how the look and feel of the
e-learning platform is so you have your navigation Bar on the side now. This is yeah, I've done the course So this screenshot indicates that I'm already done with everything but actually you're unlocking every step and by completing the last one and You also see how does and how do these
Interactive animations work. So this is where you will be learning most of the time This is how you Go to the hands-on exercises so you can stay within Jupiter within your college and you just click on the exercise button and it directly forwards you to your
personal Jupiter hub environment And Where you find? the different exercises There's also a catalog that you see up here that shows all the different exercises you just click on execute and you can start programming and Here you see this is the result of the
Of the final exercise or of the validation exercise. So here you see The snow covered area that is a binary pixel map that you have created aggregated for the catchment You see you have a high Volume of snow cover in the high winter time then it's decreasing and you have low snow cover up in July
and at the same time as a validation or a plausibility analysis you can look at a Discharge That is of course growing in the river once the snow is melting then these are the quizzes how they look like. So
Once you finish the quizzes and you finished them the exercises, this is your last Exercise let's say this is a snow. This is an example from the stack browser that is online So here's the participant Carolina Guna and you see she's calculated a snow map here It's available on the cubes and clouds
Stack browser and it's publicly available and There's also this is not a the stack item, but there's also the catalog So there's slowly every participant that is doing the course the next person can maybe pick the next patch here and start classifying the snow cover in a different region and once enough people have
classified as no cover We will at some day have a full snow cover map of the Alps or of the world So that's the idea behind I'm showing what a community actually what community is and how you can share your data once you're done with this there's a Certificate that you can also put on LinkedIn and they said you have
The proof that you've done the work on the publicly available stack browser. Here's some user numbers Yeah, we have so far we have 300 users at the moment or 400 around What we have seen so far a lot of people have subscribed and stopped at lesson one
so not so many people there are a lot of people still in in progress at the moment and 50 have completed the course so far. I think that's normal at the beginning a lot of people just want to look Does this actually work? What is it? But we hope that the number of people that complete the course is going to Increase in future and as I said, this is a community
Project we also happy to see that we have participants from all continents Mostly from Europe so far that's I think logically since this is a funded project, but it's also great to see that we have from participants from other continents and talking about the target group that we've identified before so young researchers in the field of
EO or geospatial I think that is we hit it quite well there. We see that most of the people are juniors PhD candidates or students With around a little bit less than five years experience. So that's actually the ideal target group
Then we also wanted to evaluate does this course actually help Or do people feel that they have a feeling of confidence after they finish the course? So here we had some questions that we asked before the course and some after We don't have so many people that answer the questions after the course so far because it's it's not mandatory
But you see it we have a writing from one to five. How confident am I using EO platforms? how independent am I using EO platforms and How well can I adhere to open science? I Said numbers are not really matching. So a statistical analysis would be said too much But at least I think you can see a tendency that after the course, there is no
participant who is completely afraid of Using cloud platforms, especially not doing open science. So we see people can adhere to open science afterwards They feel Independent on cloud platforms and they they want to use cloud platforms in the future
So I think that is a good sign in the right direction. We're looking forward to having more participants maybe some of you and If you're not participating there are also other ways to engage in the course on your college on github on Zenodo and By the stack browser are there questions from the audience?
Hi, I'm Henry from all the University of Finland first of all brilliant work
I really like what you have been doing and actually we have been doing something similar in Finland for vector Data processing related things. So I have a couple of questions that relate to our experiences with these So first thing is that do you provide some kind of support for the students when they are doing for example the exercises?
And the other thing is the certificate So would that certificate be also applicable so that actually the students taking the course would be able to use like get credits for the university Yeah
Yeah, thanks for the questions and great to hear that. There's a similar movement also for vector processing Maybe we can combine them or at least make them available somewhere in the same region that people can Yeah work together and somehow get their learning path and through modern these technologies and The first question was do we support?
students with the exercises that's a good question, especially in the let's say in this environment cloud technologies are changing so rapidly that It's a almost a miracle that the notebooks will work for a year without maintenance So I said we use the github community or our github repository for that
We encourage also and a landing page we encourage the students to raise the issues themselves. They find them and And then of course we once we get an issue log we we discuss it with the students there So that's the kind of support that we give at the moment And I think that's also a good idea to kind of push or the people in the direction of using github if they're not already
Used to it and the second question was about the certificates So We have Calculated the hours that you usually need for the course and we have translated this into ECTS points But there's no official University, I think that will that have has accepted this course yet
universities are open to do so, but I think you can you can take the course you can Show the certificate to your supervisor to whoever and if the university says, okay, I want to accept this course They can so we have let's say we've laid it out as far as we can This is the number of ECTS
on that certificate or is it that the Universities can actually directly see because with you know, as recourses they actually write also how many ECTS you could get. Mm-hmm I think we I have to check I think it's around two ECTS and I'm not 100% sure where this is written out
I think I'd have to look to the certificate to figure it out. But yeah, I think there's a there's a diploma supplement that's the name diploma supplement we give in addition to the Certificate that shows what have you learned and how many hours have you spent on different things? And what does that translate to in ECTS? So this is something you can give to your university to it makes it maybe a little bit easier to get the credits for it
question IHE Delft Institute for water education About the LMS that you're using is that custom-made or is it also an existing LMS like moodle?
That is the e-learning platform right? Yeah. Yeah your college that is a spinner from the University of Jena and they're specializing in spatial This is a spatial e-learning platform So they have different topics especially on earth observation so far a lot of courses that have been funded by ESA end up there
So it's already a big pool How to learn earth observation from Very basic things to a hyperspectral earth observation to SAR stuff to how these translate into SDGs So there's really already a well from your college So once you go there and click through it, you will see that there is really a whole lot of
Courses already available Question The hosting of the Jupiter notebooks you use the EOX services. So is that expensive? You X was a partner in the project we also have we had Tina here at least who is a part of the company
they also have their booth over in the Exhibition area and in this course, I think they're currently still doing it in kind So that is great for this course, but for the pricing generally I don't know their their pricing scheme so far. They're doing it in kind for education
So that's great and we're happy that they're on the team