We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Earth Observation DataCubes Multi-visualization Toolbox

00:00

Formal Metadata

Title
Earth Observation DataCubes Multi-visualization Toolbox
Title of Series
Number of Parts
351
Author
Contributors
License
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language
Production Year2022

Content Metadata

Subject Area
Genre
Abstract
Context It is said that data visualization is as important as the data itself. As the amount of data generated from Earth observation (EO) satellites – i.e. Copernicus program (Jutz and Milagro-Pérez, 2020) – is getting bigger and bigger, we need more efficient tools to deal with this onslaught of data. To help data scientists better extract relevant information from datacubes, we noticed that an under-exploited computer graphics tools could bring new perspectives to specialists. Datacubes are known to be the reference format to handle EO data; several techniques such as Web WorldWind developed by NASA exist to process and interact with them. Recent works have shown focus on the preparation of largescale geospatial data (Mazroob Semnani et al., 2020), a highly technical subject, could benefit from optimizations. QGIS is another tool frequently used in the field, that can be enhanced by plugins and can retrieve data from Web platforms. A modern approach to process efficiency is the use of GPUs. Still, when reviewing the use of GPUs to process geospatial data, the emphasis is often put on the parallel processing of geospatial datasets rather than focusing on their visualization (Saupi Teri et al., 2022). Objectives One of the main contributions of this paper is to consider geospatial data using GPU resources for intermediate computation and visualization. Considering the increasing interest to interact with this data directly using Web pages or Notebooks, this article presents tools allowing a program to run on the GPU and display the desired datacubes using the WebGL API. This can result in high performances thanks to its low-level control and possibility to use GPGPU algorithms. WebGL running natively on most web browsers, another benefit will be the end-user ease of use. The end goal is to display even large (i.e. 1024^3) datacubes rendered on the fly in real time on a PC, still well-equipped. Methodology To keep our applied research efforts focused, we have set up an independent international expert advisory group. Indeed, we wanted above all to provide something useful and concrete for the actors in the field. The represented institutes are ESA (EC), EURAC (Italy), GISAT (Czech Republic), Terrasigna (Romany), TU Wien (Austria), VITO (Belgium), and even a former NASA (USA) analyst. They have been regularly interviewed to get constant feedback on the suitability of our developed application, the final goal of our project being to build a toolbox of models to efficiently visualize different EO datacubes formats. Models This paper presents three main models applicable to datacubes from an EO context, some relatively standard and others innovative, still all revisited via the GPGPU architecture. Implicit curves model – This model has two main approaches: discrete and math-based sub-models. Especially adequate to process (x, y) or (x, y, t) datacubes in a 2D or 3D visualization we developed and compared both sub-models with their dependencies. Sets of given iso and δ values are extracted from the data and stored as sets of curves. They can be displayed in a 2D environment or in 3D with additional information such as: (1) the simulation of the data as a 3D surface; (2) different colormaps for the surface representing yet other external data; (3) surface render in steps to emphasize the given iso and δ values; (4) user customizable colormaps; and (5), a water level simulation rendering. Derivative 3D rendering model – This model is specialized in analyzing (x, y, t) datacubes as a volume where the time t is part of the visualization. Indeed, the aim is to visualize the evolution in time of a geographical area by highlighting the temporal differences within a volume. After selecting the (x, y) region of interest the user selects a reference layer representing the state of an area at a defined time t and a time interval Δt. The cumulated differences between the two are visible in a colored sub-volume defined by the time interval. In order to add more contextual information in the visualized geographical area, we have added the possibility to display an additional map (such as topographic data) at the reference layer level within the volume. Jupyter Notebook massive rendering model – To make the toolset even easier to use, we have developed a visualization model deployable in Jupyter. This model allows rendering of (x,y,z) and (x,y,z,t) data volumes. Two rendering algorithms are already available: (1) the implicit surface simulation for any iso intensity -- but only via the discrete approach -- and (2), an XRay-cast simulation. Discussion Results show our models can process large amounts of data and render them in real-time. Where large 3D datasets would normally become problematic to handle for any GPU, we developed specialized tools to overcome software and hardware limitations. For instance, a 3D datacube can be sorted into a 2D texture to be directly loaded into GPU memory, thus improving performance. When the textures become too big to work with WebGL, their information can be split in the RGBA channels of standard 2D textures for a four-fold decrease in memory use. Furthermore, when displaying our rendering models, and in the case of machines without sufficiently powerful graphics cards, we propose to display only the fraction of the data that interests the user. All of these highly efficient rendering models are assembled together in a toolbox dedicated to datacube visualization.
Keywords
MultiplicationVisualization (computer graphics)FingerprintFreewareOpen sourceContext awarenessTime domainDenial-of-service attackPersonal digital assistantRaster graphicsLaptopMathematical analysisSurfaceType theoryInformationTask (computing)VolumenvisualisierungCellular automatonMatrix (mathematics)Term (mathematics)Source codeCubeComputer simulationSoftware testingLocal GroupComputing platformFile formatGraphics processing unitServer (computing)PreprocessorTexture mappingField (computer science)Vector graphicsError messageCurveNichtlineares GleichungssystemParametrische ErregungSimulationCodeMathematicsDisplacement MappingElement (mathematics)Electronic mailing listMountain passCurveReal numberDemo (music)Message passingVolumenvisualisierungLevel (video gaming)MappingState observerSurfaceDomain nameSimulationCubeCurveWeb applicationProjective planeGraph coloringSet (mathematics)Physical systemOpen sourceObservational studyComputer simulationField (computer science)Latent heatPreprocessorImage resolutionTerm (mathematics)Electronic mailing listCellular automatonSampling (statistics)X-ray computed tomographyMessage passingVolumeComputer scienceReal numberLaptopTexture mappingDirection (geometry)DatabaseStructural loadThresholding (image processing)Volumen-RenderingDot productPoint (geometry)Key (cryptography)Different (Kate Ryan album)Series (mathematics)outputGroup actionError messageMultiplication signReal-time operating systemCollaborationismSlide ruleMobile WebCartesian coordinate systemStudent's t-testPresentation of a groupSoftware testingWeb 2.0FeedbackComputer programmingVirtualizationRaster graphicsSpacetimeCASE <Informatik>Arc (geometry)Mathematical analysisUniverse (mathematics)MathematicsGeometryTriangleMereologyComputer architectureDerivation (linguistics)Link (knot theory)Shared memoryInformation2 (number)ResultantDatabase normalizationObject-oriented programmingWordBitFinite-state machineMusical ensembleGraphics processing unitGeometric modelingGoodness of fitPort scannerStandard deviationComputer graphics (computer science)Visualization (computer graphics)Virtual realityType theoryCurveDampingNegative numberInstance (computer science)Demo (music)Computer animation
Transcript: English(auto-generated)
OK, thank you so much. Good morning, everyone. So this presentation is about, I mean, the title, of course, of the papers, this one. But it's a kind of summary of the project, of the whole project. So I made this presentation as a kind of storytelling
from the genesis to the end and the conclusion we had. And the sponsor were ESA and the Swiss Space Center. Well, my name is Stephan Gabron, and I am from HECO, which is the University of Applied Science and Art of Western Switzerland.
It's not very famous, so on the Swiss map. Basically, this university is covering 27 centers and institutes. And we are from the Acheu Arc in Neuchâtel. My specialty is definitely not about what we've seen just before, or for solution.
And I will tell you why I'm here today. I'm more about real-time 3D visualization, and computer graphics, and geometry, and virtual reality, and gamification. So it's quite far from what we've seen here. And especially in this presentation, you will see that there is no about standards.
And everything that you've seen just before, I'm quite happy to add that, because we didn't use all that because everything was done on the very low level. We had to add some very fast rendering and very fast computations. So it's very low-level computation and programming.
So as I was saying, my specialty is much more about health domains, such as this kind of rendering in real time with very high resolutions. And we actually use also very large data sets, also, for like pets and or CT scans or MMR.
And one day, like a couple of years ago, I met Jan Voumar, who is from the company called Solenix, also in Switzerland. And he is working quite a lot with Eza.
And he told me, well, what you are doing here, this is not the kind of things we see often in our domain. That would be kind of nice if you can transfer your technology that you are doing in health for Earth observation. And well, so I start digging.
And it is very true that Earth observation data cubes are definitely very large data sets, also. And the community is much more about data scientists. And here are some example of data cubes that you have, which is not the same that we had.
So we had to rethink everything in terms of computer science architectures. And there was also some plenty of new applied fields. So it's very interesting for us. So when we started this project,
we decided to find out, from the very beginning, a group of fields advisory people. And we found out seven institutes in the field. That will follow up from the beginning to the very end
and advise us what they would need in the field of Earth observation data. And this group we met quite often. We did a lot of brainstorming all together and separately. And we studied, developed, and tested two user case, which
were actually very close related to climate change issues. And also, what we realized is that they would have liked some tools efficiently.
To make it efficient for them, they would have need some application as a web app or using Python or Jupyter notebooks. So this was also a challenge for us because usually we were using some pure low-level
programming. And to integrate it using Python was quite a challenge. So here are the two use case we decided to study and to develop. The first one, elevation and time,
especially like floats issues. And we had to do some raster analysis using Jupyter notebooks. And also, in terms of geometric model,
raycast, implicit surface, and the raycast derived. That was the main models we developed. Please notice that all the graphics that you will see from now on, there is no triangle. It's only raycast rendering and implicit surface rendering.
So there's absolutely no triangle. It's too costly and it's too long if you want to visualize very large data sets. The second use case we studied was about agriculture.
And then that would be applied on a web application and especially with implicit curves in 2D and 3D and also a simulation of implicit surface. So of course, there's quite a lot of things
that we did and I will not be able to show and present every model we did. There's like 16 model we developed. But I will show you basically the specification and directly jump to the achievement.
So the specification, our university was supposed to work on the models and on Jupyter Notebooks application, on the web application, also on the true volumetric renderings, and on all the 16 novel models
in terms of application for Earth observation data cubes. And the Solenix company was making the link with all the people that was testing in the field and making sure that we had some feedback all along
so that we can change the programmings and also the sharing for the open source of the whole project. The achievements that we had. Basically, there was the pre-processing of the data. So as you've seen on the previous presentation,
we had a lake of data. In our case, it would be a kind of very specific fish. So we were aggregating some kind of fish to make a very strange fish that would fit directly on the GPDPU so that it would go very fast on the rendering,
basically. And this was about an issue with WebGL1 specification. Recently, because of actually one of the engineer that is just there, we found out that with WebGL2,
we can actually skip this part and go even faster and without this pre-processing system. In terms of volumetric rendering, we had all that. I will go on next slides about it. And on the implicit surface, we had all these models too for the respective types of data cubes.
And then here are some sampling of the achievement we have. And later, I will show and dig on two specific models. So we succeeded to integrate real 3D rendering
on our Jupyter notebooks with Python and behind WebGL. We also succeeded to do some simulation of real implicit surface volumetric systems and also on the fly color palette systems for users.
Multi-layer data so that you can add as many texture and data that you want on your maps. 3D step-based rendering, that was fun. And another fun part was also to start doing some simulation of load on the map.
So here are some highlights for two specific models. The first one is about derivative model for data cubes with x, y, and t time. So the issue here is that we had not
a lot of real 3D volumetric data to work on. But we had quite a lot of data with the Copernicus data sets about x, y, and time. So we thought, what can we do in 3D with that?
And we tried something quite fun. That was, let's see time. Let's see the change. And let's try to make a specific 3D rendering volumetric where you can see the change depending of any specific time that you select.
And any other region, you can make the derivative and make it appear like positive value would be in red and negative value, let's say, would be in blue. And so we made the access on the database. And there here are some sampling.
And from this sampling, we can have this kind of rendering. And you can see here, there's some reddish part and some blue part showing what was added on this specific region depending on this map and on bluish what disappeared.
And you can also add multi-layer data, like here some info on the map and accessing directly Google Maps and stuff like that. The second highlight is about implicit curves. The idea was to make a raycast.
And when you make a raycast, actually you can find out some threshold. And for instance, you can have these kind of results. All the dots are actually on a level, each level.
And the rendering that you can have from that is not that good. There's some gap, and there's some redundant data. And it can be an issue. So we try to find out something more like math-based. Even if the rendering can be not that bad and quite interesting,
this is in 2D, but you can actually map it on 3D. So we start to make a list of lists for each levels. And from this list of points, we can find out some key points and other reconstruction
with direct segmentation. Or you can use Fourier series and other models, such as Katman-Rohn models. And well, the actual data that you add is this one.
And sometimes there's quite a difference between the parametric reconstruction and the original data sets. So how can we validate that this model is better than the other one? Which parametric model would be the best? So what we did is that we went through all the data
in a very empiric way of computing the errors. And we can find out that the Katman-Rohn was quite the best and efficient way to make a very fast computation and rendering. If I show you these input data sets,
this is the original data that you can have. And this would be the reconstruction with implicit curves, real implicit curves. It's not just that the graphics is better.
In terms of data, there's only 3% of the data here, left. 97% just is useless. And even better, the rendering is about three times improved. And the access to the data set is multiplied around by 50 times.
And even better, the gap and the redundant data can be partially solved. We can talk about that later, if you want, why I said partially.
Anyway, if you want to have more detail about that, we were very happy and lucky because we just been published and accepted to journal for both of these models I've shown you. If you want a copy, I have a copy here. For the implicit surface and for the derivative 3D volumetric
systems using a Jupyter notebook. Anyway, oops, I don't have the mouse.
No, it's not working. Oh yeah, here it is. So I will not show you directly a demo, but we have the computers here. So if you want to have some live demo, we can do that later on. Here are some ending words, especially
feedbacks from the field people group. Take a message and some thanks. I will not show you all the things that the advisory group point out and would like to be improved, but they were quite very happy.
And we determined a list of stuff with actually some work to do, which a total of at least six months of work. But what I would like to point out on this slide is this last stuff here, is that we are still
lacking of data. So if you are interested by any collaboration or sharing some data, especially volumetric data, please do so because we have no volumetric data in this field. It's only on the medical field.
Here are some take home messages. For the open source, it's Apache license. For the real time data visualization, we can render up to one billion cells in real time, please, on a high end PC, not on a low level graphics card.
It's very convenient for the use because you can have application for a web app that is working basically everywhere, even on your mobile.
And you can also integrate it on Jupyter notebooks with Python commands. Furthermore, for the improvement relative to data, we have some rendering, compressing, and accessing data very fast and in a quite efficient way,
I think. And there are some potential structural issue resolving model that we can go on this one because I'm pretty sure we can actually improve also the data sets that you have with this model, with implicit surfaces.
I would like to, this is quite odd because you don't thanks yourself, but I would like actually to thank the team member and all the people that are just there. They are young and bright students and young engineers, especially Artin, Christophe, Matthias,
and Antoine, and also the Solenix team. And also some people from ESA, maybe you will recognize Olivier and Giuseppe, and all the advisory group people from the seven institutes. And they're a bit sad, but I must point out
that this presentation was also dedicated to Ron, who died last year, just before the end of the project. And he gave and provided a lot of advice, and that was a great guy. Anyway, thank you so much for your attention
and for listening. And if you have any questions, please do so.