AV-Portal 3.23.3 (4dfb8a34932102951b25870966c61d06d6b97156)

A tool for localised cloud cover assessment

Video in TIB AV-Portal: A tool for localised cloud cover assessment

Formal Metadata

A tool for localised cloud cover assessment
Alternative Title
Extracting cloud free images in an area of interest from a remote sensing archive
Title of Series
CC Attribution 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Release Date

Content Metadata

Subject Area
Open source satellite imagery becomes increasingly important in many areas of research and business. In the case of agriculture, it is being used to assess land use, support subsidies, and to help farmers to manage their land more efficiently. Many of these applications require regular revisits to best catch temporal the variety that defines many land uses. Cloud cover can be a serious issue, often obscuring the majority of images acquired from passive sensors. Filtering an area of interest for cloud free images, can be labour intensive or lead to loss of valuable input data. Here we introduce a Python tool developed using open-source packages that will assess user defined areas of interest against supplied cloud masks for three satellite platforms: ESA’s Sentinel-2 (Level 1C products), NASA’s Landsat-8 (Level 1C and SR products) and the joint CNES and ISA platform Venµs. We present a case study from County Meath, Ireland to assess the number of cloud free Sentinel-2 images identified for fields surveyed in the EU Land Use/Cover Area Frame Survey (LUCAS) and compare it to the number of images that would have been obtained by threshold filtering images.
Keywords General

Related Material

Video is cited by the following resource
Presentation of a group Mass Bit Metropolitan area network 2 (number)
Data management Event horizon Extreme programming Spherical cap Authorization MiniDisc Online help Mereology System call
Medical imaging Event horizon Algorithm Extreme programming Temporal logic Order of magnitude Multiplication
Medical imaging Mathematics Event horizon Integrated development environment Extreme programming Spherical cap
Group action Image resolution Multiplication sign Computer-generated imagery Mass Total S.A. Demoscene Event horizon Thresholding (image processing) Field (computer science) Dressing (medical) Product (business) Neuroinformatik Medical imaging Mathematics Latent heat Computer cluster Term (mathematics) Spacetime Exception handling Rotation Multiplication Maxima and minima Extreme programming Thresholding (image processing) Grass (card game) Data management Uniform resource locator File archiver Video game Quicksort Local ring Spacetime
Overlay-Netz Point (geometry) Medical imaging Distribution (mathematics) Frequency Algorithm View (database) Multiplication sign Computer-generated imagery Computer file File archiver Spectrum (functional analysis)
Point (geometry) Satellite Polygon Group action Open source Computer file Computer-generated imagery Set (mathematics) Shape (magazine) Product (business) Medical imaging Single-precision floating-point format Intrusion detection system Computing platform Computing platform Overlay-Netz Satellite File format Software developer Polygon Moment (mathematics) Projective plane Open source Electronic mailing list Sound effect Frame problem Product (business) Process (computing) File archiver Computing platform Right angle Energy level Reading (process) Spacetime
Point (geometry) Hecke operator Observational study Image resolution Computer-generated imagery Archaeological field survey Price index Directory service Online help Generic programming Mereology Field (computer science) Medical imaging Uniqueness quantification Source code Software framework Default (computer science) Satellite Observational study Mapping Archaeological field survey Projective plane System administrator Sound effect Parameter (computer programming) Field (computer science) Line (geometry) Personal digital assistant Computing platform
Point (geometry) Computer font Observational study Satellite Observational study Tape drive Covering space Archaeological field survey Point (geometry) Field (computer science) Energy level System identification Physical system
Point (geometry) Computer font Algorithm Observational study Satellite Tape drive Covering space Archaeological field survey Point (geometry) Polygon Virtual machine Field (computer science) Bit Computer font Medical imaging Uniform resource locator Personal digital assistant Energy level System identification Freeware Physical system Position operator Physical system
Point (geometry) Satellite Medical imaging Exclusive or
Point (geometry) Medical imaging Exclusive or Observational study Tesselation 40 (number) Computer-generated imagery Query language Electronic mailing list Energy level Resultant Number
Covering space Medical imaging Latent heat Observational study Computer-generated imagery Maxima and minima Function (mathematics) Product (business)
Medical imaging Identifiability Function (mathematics) Modal logic File archiver Mass output Data compression Arithmetic progression Product (business) Product (business)
Pixel Open source Moment (mathematics) Source code Polygon Control flow Bit Thresholding (image processing) Product (business) Subset Online chat Medical imaging Goodness of fit Auditory masking File archiver Energy level Acoustic shadow Freeware Reading (process)
ok so in will start with the i'm second presentation in the session and which will be given day i'm just because i'm a man from a town research center.
so i'm going to scale back a bit this is a lot small a problem and a lot smaller tool has started it and give a bit of background tell the came to be even going to just briefly talk of it what i'm doing i'm working for the.
i wish i would call show research authority child disc and say i'm in what's called the data technologist i help of research supports stuff i help the researchers with analyzing acquiring a managing data and of course is one important part in agriculture research becoming more and more.
more prevalent is the use of remote sensing data is so remote sensing data. his. in. one. these. the. what's happening there.
the. it will move on.
should the images on there.
but lots of nice images on their budget by then of popping up to. i might be ok that's unfortunate talk over that sorry for that they were supposed to be nice images on their just showcasing the important so favourable sensing in agriculture so remote sensing is important for land use and and use change detection especially an environment like ireland where.
the average agriculture a field is incredibly slow and products like i corinne fail miserably in. presenting the diversity of the landscape and also it is important for and measuring productivity and again in ireland which is a very grassland based agricultural economy with high diversity of crosland and ross the management been crucial in terms of fertilisation rotation. remote sensing can help from us massively in managing the life stock in knowing where to fertilise the paddock where to move the cows and the next day where the grass going too fast and may need to be topped but it's also important in assessing impacts of extreme events for example last year was an extreme weather. the end thing all over europe and ireland suffered especially you to the drought causing the grass grows to fail and farm us having to import massive amounts of feet because the court and rely on the local glasgow anymore is also the more more useful disease detections for example. crop diseases can now be detected its from remote sensing imagery helping from us to respond very quickly and very targeted it to crop to to diseases that a problem with that with remote sensing especially in a country like ireland. his. the popping up. the goal. nick school and there we go is and clouds takes and so this is an amazing image of ireland for multiple reason reasons as and of one specific reason is that there is no cloud on it this doesn't happen usually get an image like that of ireland. even in twenty eighteen that was incredibly hot an incredibly dry is the exception most images will look like this so this poses a problem in remote sensing everyone walking remote sensing knows how important was the temporal imagery is land use change detection you. meat will temper imagery but also land used to take action as i said. in art and especially is very crosland focused but it's very different from what you may expect from other countries crosland an ardent is diverse and goals from barely managed where you just throw a few cows in february march leave them there until november and that's it to. credibly heavy managed well you have few fields divert to cut down into many paddocks and every day the cows a move to different fields to make sure that they get the maximum dress out of the land so to make any sort of land use assessment on these fields you need a on the. and city of the grass and you need very a high tempo resolution to take events like cutting like changes in the paddock. so. and also in terms of. extreme event to take action the impact might be very short lived so if you have happened to only have one image in a month or so you might not catch whatever an extreme event may have cost in terms of impact so the couple of ways to get a relevant working with clouds especially if you. like myself in the recession vironment with no access to any talent solutions.
and you have to process things locally on say local archive or download the imagery straight from company because as you need it so the first one is you just had a threshold for cloud cover and only down note imagery that has that meets this threshold advantages use a lot of space and you save time when you compete you say. a space a new computer you save time require lots of images but it's also useful when you say you need full coverage of an area where you can say ok i really want only twenty percent cloud cover the problem is that if you're only interested in specific location small location spread in a large area in imich. with say.
eighty percent cloud cover might still be useful because it just happened to be that your area of interest doesn't have a cloud so be it also has the disadvantage that you may still even though you download the archive few my stuff to go through it and check manually but the cloud mosque provided if the images really useful for your own small area so these. the dust. the distribution off different outcome of us in a sense of to tie acquisition this is a child over the the and east of ireland. most of the images as you see here as i said most of the images are ninety two hundred percent felt completely useless bought a few say take a special the fifty percent you may lose half of the remaining images that may still be useful same four and twenty seventeen and i just like to show up because was such. an exceptional year twenty eighteen you do end up with a lot of images that actually out free however the as i said it still wasn't great because a lot of pace was on them so they from a spectral point of view they may still not be useful. the other as a way to deal with that is just to manually go through the cloud must overlay them with your points of interests and six and extracts today to manually that is of course very accurate but it also takes a lot of time and work which you might not have.
so to turn the problem into a solution i try to.
develop a little tool that automatically takes the clout mosque in n.h. and an image archive and extracts at the moment as a file that tells you which of these images are useful for set of points. the tool is complete even in open source python package is so abuse was stereo for action out must processing uses deal pandas tape the infield enough for a safe and effective processing and final reading. so far i have included its to actually do as that light platforms one i had split i've recently changed the two over. the to reassess how i put in the cloud mosque so it works for sentinel. to imagery people with a one c. and to a products and both in the copernicus format and the fayad of formats musk eight. it also works again this too is going to build from this as a tea from our owns archives and we have access to the venus satellites which was launched by contests and these are eighty space agency so it works for that as well and what it does he give it a set of point. it's a polygon says it will scan through your whole image archive and then return the list of images for each point that our cloud free according to the product out mosque.
this is just the overviews sell you provide to emit satellite images as an archive for single images and your areas of interest. you select your platform it well ok if it's compressed if it's compressible to unpack the image. it was an overlay your and feature us or in shape five or whatever as a deal pandas that a frame with your satellite images will make sure that they're in the same projection and then have a right at the moment and it's under development and moment of have just returned to see his be filed way half your feature id. and for each of the future ids it gets in the name of the image that is allowed free at that stage.
so it's hardly a command line to have this is just to help file and so you just run it like any come on line to replace an. and it works fast unless you want to on pack imagery audience a images it then it can take a while because i saw and pack each of the images so i'm just going to present a case study so we have one research project that looks at assessing.
lucas injuries on lucas points lukas is a land use and then cover a survey done by the e.u. that covers all of your open in a fairly high resolution great where they send out survey as that got went to the fields recorded them according to the you land use and lend to have a framework. so in pictures and the problem is as i said grassley in ireland is very diverse but they just recorded everything is present that was cross and so we wanted to break it down to a more high resolution most suitable for i was far less or i wish research wrath of research. so one part of it was assessed visual assessment of remote sensing imagery and because they're so many lukas points all of ireland. there was no way and manually assessing each of them folks out the effects out for get the points so i ran the tool over that i'm just taking one county here county meath to be a map yes so county meath is in the east of ireland.
that's the county have got all the points i select that that specific me because county meath is looking he covered by a single sentence of two tiled making it easy as a case study.
i'm using the lukas points to identify the location said i'm interested in and also using the land positive occasion system which is in the european union at minister to tool where farmers can report the lands or do report the land use and audrey's receive subsidies and i just use that to delineates the actual feel so the lucas data. just giving us points but i wanted to show here as well that you can use it on polygons as well so in this case the two will exclude once you have a bit of clout in a polygon have excluded because so you take you mean for the polygon it screws up your flecked and stay to and my school your machine learning algorithms you want the whole body on cloud free. so these other points just some images some examples of how sense no two images might look and you immediately see the problem this is.
it's a fifty percent over fifty percent but seance fifty five percent cloud cover. a lot of the points would be useless but you actually do find areas we get a useful satellite image out of there. so say he would have run your just to extract exclusion by special value would have lost some useful data there and that's great image again that doesn't happen very often an island.
and if you other once again this image might still need a lot of manual checking to see if it works.
so just for the tile iran and i looked at the number of images if you just do an exclusion so you say up to ninety percent scout cover the end up with ninety nine image up to sixty percent out of a fifty idea which images forty percent cloud cover forty one image in. it is now this is the result given the tools you get a list so each point so you can see link that straight again to use a pile of and the image that is cloud fee for that point.
and what you get is a nice old of you.
so this shows for twenty sixty in this area obviously has a relatively high cloud cover like you get relatively low cloud cover it's still not greats thereabouts max and twenty sixteen maximum of twenty nine cloud for the images twenty seventeen very similar. there's no upland area that's why they're very cloudy. and twenty eighteen you get a lot more up to fifty nine clout the images so i think it's quite a useful tool there's some caviars of course just to say this only use cloud mahsud use the product specific out must i haven't developed new cloud last year so the output of the product is only as school.
but as the cloud mass that goes in and if the cloud not is not great if it's either over identifying on identifying. you might lose information or have to steal from was defined images that will actually be cloudy even though it was identified on cloudy especially at the center to copernicus products are. tend to be an identifying clouds and it's also work and is that it's a welcome on compressed archives a slow that's just because decompressing is slow. and it's also work in progress its kind of out of necessity. and the way it works is that when we need saying new images quite new archive of at to it but it is available and get up so if you're interested you can download it and thank you very much for listening.
thank you when. i. i am curious about the insides of the two i suppose you do pixel justifications nine. you pay as some the it depends on the on the product so if i use and be overlaid with the clout mosque so when i use sense no one at easter imagery they have the clout must provide is a g m l file. so i produce a cloud most from that and then depicts local a and if they're for example the most capable of products the cloud must call names as a i think it's been called it file as of us does so that's a bit easier at the moment for i use the highest threshold. in the coat so safe it's been told that five of the biggest more than zero then i say its clout and exclude it but that can actually because it's the source code can be accessed so you can change a violent how sense if you want to shout must to be or how sensitive you want the and clout must to be i interpret it. so if you want to say if you want to still include a serious and or cloud shadow you can do that. so the peaches come with it that mask yes so they have eight hundred fifty four products that half child mosques in them so sentinel long to one doesn't have twelve months it was after all it's a sentinel to add land said a products and been this product they all come with the clout must provide it. thank you. other questions. asian and western so few years ago i worked on a huge archive of imagery dot to in which we have to look for a cloud free much as for a certain area but we did was to extract. most of all gone off the cloud free areas so back to rise it simplified it's been quite a bit because otherwise it would have been huge and then stick it in a book bajo yes and then we were using that as a primary shelter and that was distracting that was giving us a small subset of the images that we would then check against their clout. mosque precisely and was a lot faster ok that i find that apart from the un and zipping the package is actually a very fast and reading and what i want to do in the future is actually for those area have pixel extraction straight away so that you get for its of average everything. i can see value in the polygon so i'll keep that it might actually it's an interesting solution thank you. ok so i have one question so cloud is a major problem but those look at chat that she cloud shadows is also a problem of familiar with a need good method to remove cloud shadows. no is a thing is i'm. when for example i do a at the moment when i do level to processing i used my other rhythm which does provide a couch at all mosque as wild and say i am and using the filter to exclude else as well straight away you can with is that when the excess the source the source code you can tweak that. said the bits that the detection to include or exclude and the couch and i clouds at all or none. sarah's and so on but it's no i don't see that i am i ok and the other questions. ok so thank you and will take a break and you can switch rooms thank you.