Earth Observation DataCubes Multi-visualization Toolbox
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
Title |
| |
Title of Series | ||
Number of Parts | 351 | |
Author | ||
Contributors | ||
License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
Identifiers | 10.5446/68921 (DOI) | |
Publisher | ||
Release Date | ||
Language | ||
Production Year | 2022 |
Content Metadata
Subject Area | ||
Genre | ||
Abstract |
| |
Keywords |
00:00
MultiplicationVisualization (computer graphics)FingerprintFreewareOpen sourceContext awarenessTime domainDenial-of-service attackPersonal digital assistantRaster graphicsLaptopMathematical analysisSurfaceType theoryInformationTask (computing)VolumenvisualisierungCellular automatonMatrix (mathematics)Term (mathematics)Source codeCubeComputer simulationSoftware testingLocal GroupComputing platformFile formatGraphics processing unitServer (computing)PreprocessorTexture mappingField (computer science)Vector graphicsError messageCurveNichtlineares GleichungssystemParametrische ErregungSimulationCodeMathematicsDisplacement MappingElement (mathematics)Electronic mailing listMountain passCurveReal numberDemo (music)Message passingVolumenvisualisierungLevel (video gaming)MappingState observerSurfaceDomain nameSimulationCubeCurveWeb applicationProjective planeGraph coloringSet (mathematics)Physical systemOpen sourceObservational studyComputer simulationField (computer science)Latent heatPreprocessorImage resolutionTerm (mathematics)Electronic mailing listCellular automatonSampling (statistics)X-ray computed tomographyMessage passingVolumeComputer scienceReal numberLaptopTexture mappingDirection (geometry)DatabaseStructural loadThresholding (image processing)Volumen-RenderingDot productPoint (geometry)Key (cryptography)Different (Kate Ryan album)Series (mathematics)outputGroup actionError messageMultiplication signReal-time operating systemCollaborationismSlide ruleMobile WebCartesian coordinate systemStudent's t-testPresentation of a groupSoftware testingWeb 2.0FeedbackComputer programmingVirtualizationRaster graphicsSpacetimeCASE <Informatik>Arc (geometry)Mathematical analysisUniverse (mathematics)MathematicsGeometryTriangleMereologyComputer architectureDerivation (linguistics)Link (knot theory)Shared memoryInformation2 (number)ResultantDatabase normalizationObject-oriented programmingWordBitFinite-state machineMusical ensembleGraphics processing unitGeometric modelingGoodness of fitPort scannerStandard deviationComputer graphics (computer science)Visualization (computer graphics)Virtual realityType theoryCurveDampingNegative numberInstance (computer science)Demo (music)Computer animation
Transcript: English(auto-generated)
00:01
OK, thank you so much. Good morning, everyone. So this presentation is about, I mean, the title, of course, of the papers, this one. But it's a kind of summary of the project, of the whole project. So I made this presentation as a kind of storytelling
00:22
from the genesis to the end and the conclusion we had. And the sponsor were ESA and the Swiss Space Center. Well, my name is Stephan Gabron, and I am from HECO, which is the University of Applied Science and Art of Western Switzerland.
00:41
It's not very famous, so on the Swiss map. Basically, this university is covering 27 centers and institutes. And we are from the Acheu Arc in Neuchâtel. My specialty is definitely not about what we've seen just before, or for solution.
01:02
And I will tell you why I'm here today. I'm more about real-time 3D visualization, and computer graphics, and geometry, and virtual reality, and gamification. So it's quite far from what we've seen here. And especially in this presentation, you will see that there is no about standards.
01:22
And everything that you've seen just before, I'm quite happy to add that, because we didn't use all that because everything was done on the very low level. We had to add some very fast rendering and very fast computations. So it's very low-level computation and programming.
01:42
So as I was saying, my specialty is much more about health domains, such as this kind of rendering in real time with very high resolutions. And we actually use also very large data sets, also, for like pets and or CT scans or MMR.
02:05
And one day, like a couple of years ago, I met Jan Voumar, who is from the company called Solenix, also in Switzerland. And he is working quite a lot with Eza.
02:20
And he told me, well, what you are doing here, this is not the kind of things we see often in our domain. That would be kind of nice if you can transfer your technology that you are doing in health for Earth observation. And well, so I start digging.
02:41
And it is very true that Earth observation data cubes are definitely very large data sets, also. And the community is much more about data scientists. And here are some example of data cubes that you have, which is not the same that we had.
03:01
So we had to rethink everything in terms of computer science architectures. And there was also some plenty of new applied fields. So it's very interesting for us. So when we started this project,
03:22
we decided to find out, from the very beginning, a group of fields advisory people. And we found out seven institutes in the field. That will follow up from the beginning to the very end
03:41
and advise us what they would need in the field of Earth observation data. And this group we met quite often. We did a lot of brainstorming all together and separately. And we studied, developed, and tested two user case, which
04:03
were actually very close related to climate change issues. And also, what we realized is that they would have liked some tools efficiently.
04:23
To make it efficient for them, they would have need some application as a web app or using Python or Jupyter notebooks. So this was also a challenge for us because usually we were using some pure low-level
04:43
programming. And to integrate it using Python was quite a challenge. So here are the two use case we decided to study and to develop. The first one, elevation and time,
05:04
especially like floats issues. And we had to do some raster analysis using Jupyter notebooks. And also, in terms of geometric model,
05:22
raycast, implicit surface, and the raycast derived. That was the main models we developed. Please notice that all the graphics that you will see from now on, there is no triangle. It's only raycast rendering and implicit surface rendering.
05:47
So there's absolutely no triangle. It's too costly and it's too long if you want to visualize very large data sets. The second use case we studied was about agriculture.
06:03
And then that would be applied on a web application and especially with implicit curves in 2D and 3D and also a simulation of implicit surface. So of course, there's quite a lot of things
06:23
that we did and I will not be able to show and present every model we did. There's like 16 model we developed. But I will show you basically the specification and directly jump to the achievement.
06:43
So the specification, our university was supposed to work on the models and on Jupyter Notebooks application, on the web application, also on the true volumetric renderings, and on all the 16 novel models
07:03
in terms of application for Earth observation data cubes. And the Solenix company was making the link with all the people that was testing in the field and making sure that we had some feedback all along
07:20
so that we can change the programmings and also the sharing for the open source of the whole project. The achievements that we had. Basically, there was the pre-processing of the data. So as you've seen on the previous presentation,
07:41
we had a lake of data. In our case, it would be a kind of very specific fish. So we were aggregating some kind of fish to make a very strange fish that would fit directly on the GPDPU so that it would go very fast on the rendering,
08:04
basically. And this was about an issue with WebGL1 specification. Recently, because of actually one of the engineer that is just there, we found out that with WebGL2,
08:20
we can actually skip this part and go even faster and without this pre-processing system. In terms of volumetric rendering, we had all that. I will go on next slides about it. And on the implicit surface, we had all these models too for the respective types of data cubes.
08:45
And then here are some sampling of the achievement we have. And later, I will show and dig on two specific models. So we succeeded to integrate real 3D rendering
09:01
on our Jupyter notebooks with Python and behind WebGL. We also succeeded to do some simulation of real implicit surface volumetric systems and also on the fly color palette systems for users.
09:23
Multi-layer data so that you can add as many texture and data that you want on your maps. 3D step-based rendering, that was fun. And another fun part was also to start doing some simulation of load on the map.
09:45
So here are some highlights for two specific models. The first one is about derivative model for data cubes with x, y, and t time. So the issue here is that we had not
10:04
a lot of real 3D volumetric data to work on. But we had quite a lot of data with the Copernicus data sets about x, y, and time. So we thought, what can we do in 3D with that?
10:23
And we tried something quite fun. That was, let's see time. Let's see the change. And let's try to make a specific 3D rendering volumetric where you can see the change depending of any specific time that you select.
10:41
And any other region, you can make the derivative and make it appear like positive value would be in red and negative value, let's say, would be in blue. And so we made the access on the database. And there here are some sampling.
11:01
And from this sampling, we can have this kind of rendering. And you can see here, there's some reddish part and some blue part showing what was added on this specific region depending on this map and on bluish what disappeared.
11:20
And you can also add multi-layer data, like here some info on the map and accessing directly Google Maps and stuff like that. The second highlight is about implicit curves. The idea was to make a raycast.
11:43
And when you make a raycast, actually you can find out some threshold. And for instance, you can have these kind of results. All the dots are actually on a level, each level.
12:01
And the rendering that you can have from that is not that good. There's some gap, and there's some redundant data. And it can be an issue. So we try to find out something more like math-based. Even if the rendering can be not that bad and quite interesting,
12:23
this is in 2D, but you can actually map it on 3D. So we start to make a list of lists for each levels. And from this list of points, we can find out some key points and other reconstruction
12:46
with direct segmentation. Or you can use Fourier series and other models, such as Katman-Rohn models. And well, the actual data that you add is this one.
13:00
And sometimes there's quite a difference between the parametric reconstruction and the original data sets. So how can we validate that this model is better than the other one? Which parametric model would be the best? So what we did is that we went through all the data
13:24
in a very empiric way of computing the errors. And we can find out that the Katman-Rohn was quite the best and efficient way to make a very fast computation and rendering. If I show you these input data sets,
13:43
this is the original data that you can have. And this would be the reconstruction with implicit curves, real implicit curves. It's not just that the graphics is better.
14:00
In terms of data, there's only 3% of the data here, left. 97% just is useless. And even better, the rendering is about three times improved. And the access to the data set is multiplied around by 50 times.
14:27
And even better, the gap and the redundant data can be partially solved. We can talk about that later, if you want, why I said partially.
14:40
Anyway, if you want to have more detail about that, we were very happy and lucky because we just been published and accepted to journal for both of these models I've shown you. If you want a copy, I have a copy here. For the implicit surface and for the derivative 3D volumetric
15:08
systems using a Jupyter notebook. Anyway, oops, I don't have the mouse.
15:21
No, it's not working. Oh yeah, here it is. So I will not show you directly a demo, but we have the computers here. So if you want to have some live demo, we can do that later on. Here are some ending words, especially
15:43
feedbacks from the field people group. Take a message and some thanks. I will not show you all the things that the advisory group point out and would like to be improved, but they were quite very happy.
16:04
And we determined a list of stuff with actually some work to do, which a total of at least six months of work. But what I would like to point out on this slide is this last stuff here, is that we are still
16:23
lacking of data. So if you are interested by any collaboration or sharing some data, especially volumetric data, please do so because we have no volumetric data in this field. It's only on the medical field.
16:44
Here are some take home messages. For the open source, it's Apache license. For the real time data visualization, we can render up to one billion cells in real time, please, on a high end PC, not on a low level graphics card.
17:09
It's very convenient for the use because you can have application for a web app that is working basically everywhere, even on your mobile.
17:20
And you can also integrate it on Jupyter notebooks with Python commands. Furthermore, for the improvement relative to data, we have some rendering, compressing, and accessing data very fast and in a quite efficient way,
17:42
I think. And there are some potential structural issue resolving model that we can go on this one because I'm pretty sure we can actually improve also the data sets that you have with this model, with implicit surfaces.
18:02
I would like to, this is quite odd because you don't thanks yourself, but I would like actually to thank the team member and all the people that are just there. They are young and bright students and young engineers, especially Artin, Christophe, Matthias,
18:22
and Antoine, and also the Solenix team. And also some people from ESA, maybe you will recognize Olivier and Giuseppe, and all the advisory group people from the seven institutes. And they're a bit sad, but I must point out
18:40
that this presentation was also dedicated to Ron, who died last year, just before the end of the project. And he gave and provided a lot of advice, and that was a great guy. Anyway, thank you so much for your attention
19:02
and for listening. And if you have any questions, please do so.