Assessing the distribution of disease vectors and fruit crop pests from satellite in GRASS GIS 7
This is a modal window.
Das Video konnte nicht geladen werden, da entweder ein Server- oder Netzwerkfehler auftrat oder das Format nicht unterstützt wird.
Formale Metadaten
Titel |
| |
Serientitel | ||
Anzahl der Teile | 188 | |
Autor | ||
Lizenz | CC-Namensnennung 3.0 Deutschland: Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen. | |
Identifikatoren | 10.5446/31623 (DOI) | |
Herausgeber | ||
Erscheinungsjahr | ||
Sprache | ||
Produzent | ||
Produktionsjahr | 2014 | |
Produktionsort | Portland, Oregon, United States of America |
Inhaltliche Metadaten
Fachgebiet | ||
Genre | ||
Abstract |
| |
Schlagwörter |
FOSS4G 2014 Portland146 / 188
7
10
14
15
16
23
24
25
28
29
33
37
39
40
43
45
46
48
50
56
64
65
69
72
74
82
89
91
98
102
107
111
114
118
128
131
132
135
138
141
143
147
149
150
157
158
161
164
165
166
173
174
175
179
185
00:00
StabVektorraumElementargeometrieTotal <Mathematik>GruppenoperationMereologieSatellitensystemCASE <Informatik>DistributionenraumWort <Informatik>Open SourceGRASS <Programm>Digitale PhotographieSoftwareentwickler
01:20
DatenverwaltungImplementierungInformationMathematikMedizinische InformatikNatürliche ZahlOrdnung <Mathematik>SoftwareTopologieZeitreihenanalyseCluster-AnalyseKomplexes SystemKomplexitätstheorieProdukt <Mathematik>MAPVektorraumDialektProgrammierumgebungElementargeometrieTemporale LogikInterpolationPhysikalisches SystemProjektive EbeneSatellitensystemWärmeübergangFlächeninhaltGüte der AnpassungTeilbarkeitSpannweite <Stochastik>ComputervirusRuhmasseCASE <Informatik>Prozess <Informatik>ProgrammfehlerStrategisches SpielLuenberger-BeobachterAdditionPunktSchnittmengeOffene MengeEinfügungsdämpfungSoundverarbeitungWorkstation <Musikinstrument>MeterPunktwolkeGRASS <Programm>Endliche ModelltheorieAuflösung <Mathematik>MinimalgradMosaicing <Bildverarbeitung>Message-PassingKollaboration <Informatik>Mapping <Computergraphik>CodecMinkowski-MetrikGamecontrollerSoftwareentwickler
09:42
AlgorithmusBildgebendes VerfahrenOrdnung <Mathematik>SoftwareSprachsyntheseZeitreihenanalyseCluster-AnalyseComputerDeskriptive StatistikProgrammierungDateiverwaltungDatenverarbeitungssystemMAPStörungstheorieGenerator <Informatik>EreignisdatenanalyseAusdruck <Logik>Automatische IndexierungElementargeometrieBefehlsprozessorBildschirmmaskeBitKalkülMaßerweiterungMereologieMultiplikationPhysikalisches SystemRechenwerkSatellitensystemTabelleFlächeninhaltWasserdampftafelCASE <Informatik>Prozess <Informatik>RandomisierungParallele SchnittstelleAdditionNotebook-ComputerPunktSchnittmengeEin-AusgabePixelWort <Informatik>BimodulSkriptspracheGraphfärbungSoundverarbeitungWorkstation <Musikinstrument>Elektronische PublikationMeterOpen SourceGRASS <Programm>Endliche ModelltheorieDifferenteMini-DiscSchwellwertverfahrenAuflösung <Mathematik>MinimalgradKurvenanpassungMultiplikationsoperatorStandardabweichungTVD-VerfahrenWurm <Informatik>Mapping <Computergraphik>Wald <Graphentheorie>Festplattenrekorder
18:03
AnalysisDatenbankMathematikNatürliche ZahlRelativitätstheorieSoftwareStatistikZeitreihenanalyseComputerWürfelDateiverwaltungExpertensystemFrequenzDatenanalyseMittelwertVektorraumFaserbündelElementargeometrieArithmetisches MittelGruppenoperationMaterialisation <Physik>MereologieMetrologiePhysikalisches SystemProjektive EbeneSatellitensystemTermVerschlingungQuick-SortFlächeninhaltÄhnlichkeitsgeometrieSpannweite <Stochastik>ComputervirusGewicht <Ausgleichsrechnung>CASE <Informatik>Prozess <Informatik>ProgrammfehlerVerzeichnisdienstZusammenhängender GraphCoxeter-GruppeDatenfeldLuenberger-BeobachterGraphische BenutzeroberflächePunktGradientSchnittmengeWort <Informatik>Zirkulation <Strömungsmechanik>Bitmap-GraphikMetadatenStrömungsgleichrichterDateiformatFramework <Informatik>Elektronische PublikationPunktwolkeOpen SourceSichtenkonzeptGRASS <Programm>Endliche ModelltheorieMultiplikationsoperatorDatenreplikationFreewareStandardabweichungMapping <Computergraphik>Minkowski-MetrikRechter WinkelVirtualisierungSpezifisches Volumen
26:25
BitProgrammablaufplanComputeranimation
Transkript: Englisch(automatisch erzeugt)
00:01
Okay, welcome everybody. My name is Marcus Nitter. I'm from Italy origin from Germany. I would like to Speak about assessing the distribution of disease vectors and fruit crop pest From satellite in a grass gi7. So I try to combine Some one of our research topics along with of course open source GIS
00:25
development and usage in this case just a few words about our Of about my group and where we are working So I'm located in the northern part of Italy in the Italian Alps in Trentino working in a research foundation
00:42
Which is now 140 years old. We have about 400 scientific staff there and Overall 800 stuff in total. My group is on the photo very nice group about 10 people and we are fully open source based and also doing a lot of open source development actually myself
01:00
I'm since 1997 active in grass GIS development and Obviously we use also related tools and I was also involved in the OS geo Creation in 2006. So what is the problem nowadays? one of the emerging problems is the
01:20
Increase of emerging infectious disease in the world. This is of course related to globalization Not only to that but we have not only the exchange of pathogens We have also the exchange of vectors the exchange of diseases and so forth it is a complex system which we try to understand here and
01:42
I would like to focus here on some zoonotic diseases usually vector-borne diseases transmitted by ticks rodents or mosquitoes in this case here is here a health map zoonotic disease are Major health concern nowadays in many countries and It's affects wildlife as well as domestic animals in this case
02:05
Well, this is a snapshot from an online system, which is gathering health information and turning them into a Map actually, so what's relevant for that? Of course environmental factors cultural and political changes play a role and we are basically focused on
02:23
Environmental factors here. I cannot give a full lecture. Of course This is not the scope of my talk But just to beside the new essence which we have for example from mosquitoes We have also some kinds of hidden effects. This is a publication which got recently published in nature climate change about the effects of
02:43
Mosquitoes and infections on the blood supply so you may imagine that you don't want to receive a blood donation from From an infected sample, but you'll try to keep things clean and Obviously health public health has to take care That nothing like that happen to give you an example if you are
03:04
If you plan to donate blood for example in Trento in the city where I live your They ask you if you have been to the Veneto area Which is let's say south of Verona close to Venice and so on where a lot of tiger mosquitoes are
03:20
Flying around and also carrying sometimes West Nile virus For example West Nile West Nile virus is a big issue here in us But it's increasingly an issue also in Europe and this is of course the perfect topic for a GIS because Creating risk maps is naturally done in a GIS Completely different but yet related topic is the effect of the spotted wing drosophila
03:46
Also called drosophila zuzuki. This is an insect of insect pest of economical impact on Valuable small fruits and tree fruits crops and here especially in Oregon
04:00
You are Having this burden. It is also spreading in Europe. Meanwhile since 90 excuse me since 2009 and It appears to be established in many fruit growing regions around the country so this is a big issue here you see the damage and
04:21
Fem our which is my foundation at my foundation collaboration with Oregon State University tries to implement Effective pest control strategies and here we come naturally again to some spatial question Just a quick look onto the map of Europe the spread within Europe is as you can see from the map itself
04:41
pretty quick and We have of course the hidden transfer of insects By good transport and this is something which is really difficult to model The economic loss in this case is already in the range of millions of dollars The big problem is that the attack of the fruit happens. It's a ripening phase. So
05:03
at the very end and Usually no chemical treatment for example isn't possible because if you want to sell them You have obviously to also consider thresholds there. So coming back to spatial Data here a hot topic for myself for many years is meanwhile
05:22
the creation of temperature maps from space data and especially MODIS sensor data So we've been working on the reconstruction that is gap filling of sensor data in the past years What's why is that relevant to mosquitoes to?
05:41
Sping what a spotted wing drosophila and so forth because they are related on or yeah Dependent on temperature and if we are able to get temperature data in a spatial way from satellite this is pretty interesting with respect to data which have been interpolated from meteorological stations in case you have
06:01
Network a meteorological network, which is pretty dense. Then you're obviously fine also doing interpolation there but if you're in a complex terrain remember the first Picture from the southern Alps for example complex terrain Usually are not too many stations out there because it's simply too complicated to maintain them
06:20
So here satellite data are helpful What you can do is you can derive a lot of data once you have time series and those are relevant for The spatial modeling in the end so seasonal temperature values who can compare different seasons find anomalies Growing degree days is very important and so forth
06:42
Just to give you an idea about the message. It is published there. You can hardly read it It is called euro LST our product which you can partially download partially because it is about 20 terabyte and so we have been releasing let's say Some data from that which are useful for modeling
07:02
Cloud coverage is a big topic a big issue because in case of cloud coverage also that satellite measures the cloud temperature But not the ground temperature. So we have spent several years on inventing a method which is able to reconstruct The ground temperature under let's say in case of cloud coverage or contamination
07:21
So from one to three or four in this case Sorry this way around We Have not only outlier detection, but also the effect of using the time series already We're looking backwards and forwards to understand if probably previous observations were better
07:41
Got a better coverage and so on and this is relevant information, which we can use In this case, so we get a kind of spatial temporal interpolation here and this enables us along with additional variables like Something which is let's say depicting the climatic zones and other information
08:01
dependency of temperature from elevation and many more you can see this in this open access article, which is Sited down there From that we are able to reconstruct the each map and we are speaking about a lot of maps here. I will show later The software involved is now in the current version, which we have implemented
08:23
The Google Google 111 is perfectly fine. Also for doing massive data mosaic in for example in our case We are speaking about 20 more modi styles Which are put together for a single coverage and we have four coverages per day And then we have all together about 14 years of data to process
08:44
The common or geo tool set has been used here project for a grass seven We have been done a lot of grass development I will show that tomorrow in my talk at 10 a.m. What has happened in grass seven in the past years towards the new release Lots of things are also based on this project here in order to manage big geo data
09:06
great engine that is the software for Running jobs on a cluster and scientifically in access a base system So here you have a fly through through the various years These are already aggregated data which are monthly and you can see the various seasons, right?
09:24
But this is based on 17,000 maps all together and each of the map comes with 250 meter resolution Of course, this is a tiny screenshot here But if you zoom into you see here the gada lake close to Venice Verona Milan This is something which we see quite in detail at this point
09:41
The advantage of remote sensing is that with the new data On the left two sides, we have data interpolated from metrological stations, which are publicly available This CRO climate research unit data set as well as the EC ID European climate and assessment data set available online compared to what we get from the satellite and we can see the
10:05
Description of this area is completely different and here we have something which relates again to the habitats of various Animals and there we do not deal with eight or four pixels But we have something like 1 million in this small map here and given that we have time serious
10:22
We are able to also derive Variables which you may know like word Klim. So we have using been using this Form these formulas to derive the same data set again This is also available as an our grass extension r.bio Klim called you can simply install it
10:40
And if you have own data, you can recalculate this thing to give you some idea about The usage of these data and why time series are especially interesting here in the upper image You see a map of from 2003 this was a particularly hot year in
11:01
In outstandingly a hot year in Europe in some parts of Europe at least and you can see in the lower Map the effects on the subsequent winter This is a map from January and you can see the gada lake and some other lakes Southern alpine lakes which are way warmer than the surrounding. This is of course
11:21
Due to the heat capacity of water itself, but we got some extra heating Let's say in the hot summer of 2003 and this Can be considered as a local heating for that area This is every year liked it by here It was really outstanding and this is very helpful for various insects and ticks for overwintering
11:42
so in case of a harsh winter elsewhere still we get this heating effect from the lakes and overwintering leads to the potential Let's say survival of of parasites which then in turn appear with the subsequent Generation in the springtime in case they have some payload like infectious diseases. This is of course a multiplying effect
12:07
Another topic are growing degree days, which are well known for ecological modeling The difference is here that we get growing degree days, of course pixel per pixel So if I click into one random pixel here, I would obtain one of those curves which we can see there
12:25
This is an annual curve you accumulate the let's say the temperature contribution over a given threshold and this threshold is usually selected based on the biological threshold relevant to that insect or whatever it is or could also be wine just to
12:41
Tell something else wine dry painting is also controlled in a sense by This kind of index and so we get this curve for each of the pixels at this point You can consider this as virtual meteorological stations But here we have a few a million for example, and this is pretty useful
13:00
especially in complex terrain as already mentioned and then we can use the common GIS technology like R.map calc in in grass or we have some dedicated module to that to filter out the maps Telling you then At which day of the year a certain threshold was reached and so we get immediately the map and we can see okay for the
13:22
live stage Let's say adult of an insect we need for example 1400 degree days. We just filter these the stack of maps There's a time series tool in grass GIS which can easily do that And like this you get for each pixel the day of year when this threshold was reached
13:42
So since we couldn't resist we have also processed all US data. This is something new just finished two weeks ago So Still unvalidated, but it doesn't look so bad. I think so. We are of course working on that These are the annual average temperatures from 2012
14:01
We can of course also continue and do something else. This is the seasonal Variability of that particular year as it's more or less the standard deviation which is being considered In this case the higher the values in this case blue The more variation there is and if we zoom into some parts in California, for example
14:23
you see already the effect and Around this mountain. There are green areas with some forests as far as I know At least from the satellite maps and DVR you can see things like that or you go there You see cooling effects or let's say kind of stabilizing effects of water the big lakes over there
14:42
You can also see easily but we could still zoom in and reach something like 250 meter resolution So where are we doing this kind of stuff? We have a small cluster there in our foundation Which is which we used for this kind of computation. The parallelization is not yet
15:01
Usually at algorithm level which is fairly complicated in GIS But what we do is a kind of tiling approach that we split the maps into chunks or in this case We were calculating entire Days on one CPU and then sending out the 17,000 jobs to the clusters in order to subsequently
15:21
process them So just to give you an idea amount of data This is something which really comes nicely loud now in free and open source software that you can Deal with huge amounts of data in this case For the reconstruction of a single map, I mentioned that we have additional variables here. We use six additional input grids
15:43
each of the grid comes with approximately 400 million pixels or Rasta maps, let's say so all together. We also write out a map So that's another one and we probably have some interim maps in between so for a single job
16:00
We have how much is that? 2.4 billion pixels And this multiplied 17,000 is something like four trillion pixels which is are being processed in this approach and this is something which you can even do on your Let's say advanced laptop if you have enough time sequence in it as a sequence or if you can parallelize on multi CPU
16:23
Then you are done a little bit big quicker. So in our system, which is not that new It took us approximately one month to process these 20 terabytes, but this is something which is run Eventually completely automated in a sense that we wrote scripts to process the thing
16:42
To give you some ideas about challenges in the past years This is just a random collection of problems and their solution which we came through So net internal data traffic, that's a pretty big topic if you're if it comes to big geo data So what does it mean you have somewhere your data stored on a disk and somewhere?
17:03
It's your CPU or your CPUs and you want to make Your program Compute something on the CPU using those data But as I mentioned if we have per job something like two billion pixels Then it's quite heavy And if you do some of them in parallel
17:21
You can easily saturate a multi gigabit network as we had here in this case For those here who are let's say familiar with technology We we tuned the internal TCP IP network a bit to use long packets longer packets than usual We also modified the NFS network system in order to be faster than before
17:45
we also exceeded the various file system specifications because if you Save a map you have sometimes also additional maps like color table some statistical files and so on and with 17,000 maps you can quickly exceed also like I'm speaking here about 2009
18:04
The specifications of how many sub directories you can create in one directory But this has all been superseded with advanced more advanced file systems The latest one which we have adopted is the Gluster file system Yet another open source file system, which you can use to distribute your data over various systems
18:24
It also does geo replication and so on so pretty nice to use and well After saturating also that one we tried then to bundle Various cables into virtual cables and this for the time being seems to work So now if you say we can do cloud computing, I think this is a pretty nice idea
18:44
Still you need to get your data somewhere So maybe in future we are able to do the computation where the data are this would be much nicer Much less transfer, but we still have to go there I think the software is ready for that now since we are speaking about time series just to advertise
19:03
latest and greatest in grass 7 there's a new completely a completely new temporal Framework available which enables you to handle space time cubes yesterday. We had to workshop Yesterday morning the materials online So if you're interested also to exercises on top of that you can find them online
19:23
And as you can see here from the existing manual pages, there's a lot available for gap filling aggregation Statistics in time series. It does raster time series vector time series if you have some and also volume time series, which is raster 3d
19:41
we have the Related graphical user interface tools also available This helps you to navigate in your wealth of data. For example, you could have point in time data like metrological observations or field data, for example or other data which are continuous or say monthly averages which are
20:03
related to a certain period time and all this you can put together and Easily manage and you have the possibility to navigate in your stack of data. So in this case, we have the stack of 17,000 modus maps and if you want to cut out things to aggregate them to whatever you want seasonal data or summer temperature or
20:24
Deviation from something this can now easily be done and the technology is available for that Okay to sum up The vector borne diseases and also agro-crop pests
20:40
Do have dramatic impact. Meanwhile, and this is an emerging emerging problem, which needs to be addressed the fact that not only Let's say pathogens are traveling But also the vectors are traveling is introducing the problem in completely different areas of the world Where it was probably completely unexpected and there are many such cases in Europe, for example
21:03
Coming with a dramatic impact on welfare for the society, so In my view the spatial component is essential here. It is giving us a means on addressing these problems Free and open source software is for us the natural place to go in case of
21:23
Data analysis we are able to do high-resolution Processing nowadays and The data are luckily available We are also particularly grateful to NASA at this point to make those data available. This is fundamental many research groups out there and
21:42
Yeah, this is something which is Cutting edge. I would say in a sense that it enables us to combine from field data to satellite everything into spatial models so At this point, I would like to conclude and thank you for your attention I'm available for questions, of course, and please enjoy the microphone for them. Thank you. I have two completely unrelated questions
22:24
One is I do a lot of similar work in HDF 5 and net CDF and what sort of Facilities as grass 7 have to talk those file formats and leverage some of those database formats Okay. Yeah, we can import and export to a net CDF
22:44
Probably this is the full answer to the question you have the possibility to let's say get your data in let's like a Cube or whatever you want to call it do their processing and then write it out in the same format Okay, so at this point you are able to let's say a full import export in exactly this
23:16
With net CDF, I'm not sure I I believe it is possible
23:20
Because with we are using a Google for that right and Google is able to read net CDF at this point We could do those links there The system is a metadata system and at this point. It doesn't really matter where the data are But you have the possibility to link them into and as long as grass sees them as own grass data
23:41
Even if they're just a link who cares no, the system is happy and we can process the data And my other question is are you seeing any impacts of climate change since you've been doing this work on the past Any change in the range of these paths? Yeah, I mean the we are probably slowly driving from the word
24:03
Climate change to something which we currently call climate variability Climate change is something fairly complicated at least in my view, and I'm definitely not the expert here But climate variability is something which we do observe I'm I'm more focused on Europe at this point, but that doesn't really matter
24:21
We observe for example more anomalies and these anomalies appear to have impact This is ongoing research not only in our group, but in while we are connected. So various European projects on this matter and this is what we find at this point and In terms of satellite data we have the problem that let's see Modi started in 2000
24:47
This is nothing you want to look use directly for climate change analysis because that's just too short We try to let's say Do geo rectification of a VHR our data which would allow us to go back to the 70s
25:03
but it's fairly tricky as the experts here will know and we'll see if we manage or not, but You need definitely at least 30 years of data the other data sets which I've mentioned before They are way longer some start even on the first January of 1950 and here we can observe
25:21
in some parts of Italy for example a gradient like warming and this may lead to something or may not lead center something this is Depends really on the problem For sure, it is let's say off the over overwintering topic is something which is directly related To warmer winters which then drive more
25:42
Circulation of viruses for example and so forth. This is I would say confirmed To predict the future. It remains complicated
26:04
I was wondering if you could say you mentioned a mythology for the problem with clouds and I was wondering if is that documented some places how you Bypass the clouds or yes, sure
26:22
It's just on this bar, but let me quickly hijack my own presentation here so that you can read it We've just published a paper on that Let's see if this is big enough, sorry, I never mind I just do it like this
26:55
So is it readable big enough so it's published in remote sensing
27:13
If you just search for euro LST this year you are able to
27:20
Find everything there So the methodology is a little bit more complicated. There's this really simplified flowchart here There's also the more complicated flowchart in the paper itself Yeah, you can find everything there as well as some data is of course
27:46
Okay. Thank you very much for your attention