EarthExplorer - On-line Search Tool for USGS Remote Sensing Data
This is a modal window.
Das Video konnte nicht geladen werden, da entweder ein Server- oder Netzwerkfehler auftrat oder das Format nicht unterstützt wird.
Formale Metadaten
| Titel |
| |
| Serientitel | ||
| Anzahl der Teile | 188 | |
| Autor | ||
| Lizenz | CC-Namensnennung 3.0 Deutschland: Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen. | |
| Identifikatoren | 10.5446/31749 (DOI) | |
| Herausgeber | ||
| Erscheinungsjahr | ||
| Sprache | ||
| Produzent | ||
| Produktionsjahr | 2014 | |
| Produktionsort | Portland, Oregon, United States of America |
Inhaltliche Metadaten
| Fachgebiet | ||
| Genre | ||
| Abstract |
| |
| Schlagwörter |
00:00
Bildgebendes VerfahrenDatentypSoftwareSoftware EngineeringBitInverser LimesPhysikalisches SystemProjektive EbeneSatellitensystemZahlenbereichFestplatteQuick-SortSoftwarewartungZeitrichtungCoxeter-GruppeDistributionenraumPunktVarietät <Mathematik>SchnittmengeWeb-SeiteUmwandlungsenthalpiePortal <Internet>Domain <Netzwerk>SondierungMultiplikationsoperatorPackprogrammArithmetisches MittelLeistung <Physik>QuarkconfinementFehlermeldungLesen <Datenverarbeitung>sinc-Funktion
05:22
Bildgebendes VerfahrenDatentypInformationZeitreihenanalyseMAPOrthogonalitätBitGeradeGruppenoperationPhysikalisches SystemProjektive EbeneTeilmengeFestplatteQuick-SortGüte der AnpassungServerSchnittmengeOffene MengeFramework <Informatik>Diskrete-Elemente-MethodeAuflösung <Mathematik>Einfache GenauigkeitMultiplikationsoperatorOverlay-NetzKollaboration <Informatik>Demo <Programm>Interface <Schaltung>PackprogrammDemoszene <Programmierung>Natürliche ZahlInjektivitätMultiplikationFehlermeldungDichte <Physik>Klasse <Mathematik>Wort <Informatik>Data MiningRahmenproblemBitrateOpen SourceVierzigObjekt <Kategorie>MinimalgradWald <Graphentheorie>
10:44
Bildgebendes VerfahrenDatensatzDatentypInformationNatürliche ZahlTopologieProdukt <Mathematik>MAPTaskKonfiguration <Informatik>AggregatzustandBildschirmmaskeBitDivergente ReiheGruppenoperationPaarvergleichPolygonProjektive EbeneRechenschieberResultanteTUNIS <Programm>WärmestrahlungZahlenbereichVerschlingungQuick-SortFlächeninhaltServerÜberlagerung <Mathematik>RuhmasseCASE <Informatik>Prozess <Informatik>Formation <Mathematik>Gemeinsamer SpeicherAdditionPunktStrömungsrichtungQuaderBildschirmsymbolPixelKreisflächeWeb-SeiteUmwandlungsenthalpieMetadatenMailing-ListeRadiusGraphfärbungElektronische PublikationMeterTouchscreenAuflösung <Mathematik>SymboltabelleMultiplikationsoperatorOverlay-NetzDemoszene <Programmierung>GamecontrollerKoordinatenSchnittmengeShape <Informatik>BrowserAdressraumContent ManagementEinfache GenauigkeitRechter Winkel
19:24
DatentypInformationOrdnung <Mathematik>RückkopplungSelbst organisierendes SystemPhysikalisches SystemRechenschieberResultanteSatellitensystemWarteschlangeZentrische StreckungE-MailVerschlingungFlächeninhaltCASE <Informatik>DatenfeldStrömungsrichtungBildschirmsymbolSchnittmengeHilfesystemUmwandlungsenthalpieAppletMailing-ListeMeterWeb SiteTouchscreenMultiplikationsoperatorDigitale PhotographieRechter WinkelDienst <Informatik>Interface <Schaltung>Demoszene <Programmierung>SoftwareentwicklerAnalysisMathematikKonfiguration <Informatik>BetafunktionFunktionalInverser LimesMaterialisation <Physik>MultiplikationProjektive EbeneProzess <Informatik>Klasse <Mathematik>PixelSchätzfunktionMinimumPlastikkarteEndliche ModelltheorieAutorisierungStandardabweichung
Transkript: Englisch(automatisch erzeugt)
00:00
All right, I'll get started here. My name's Tim Mantell. I work for SGT Incorporated, which is a contractor to the US Geological Survey. I'll get into a little bit more of an introduction here in a little bit. But I'm here to talk about Earth Explorer, which is an online tool for searching USGS remote sensing data.
00:22
Which, what does that mean? But we'll talk about that a little bit more in the future here. But an overview of my presentation. I'll do a little introduction, talk about the history of Earth Explorer, talk about some of the challenges we had in developing it. I will mention the technology.
00:41
I'll talk a little bit about some of the data sets we have available, which we have plenty. And talk about the features. I'll go through a number of the features of Earth Explorer. I'll kind of give you a little screenshot walkthrough of how Earth Explorer works and how you can search for data, and then we'll go into questions. So a little bit of an introduction.
01:02
Obviously, Earth Explorer, earthexplorer.usgs.gov is the domain. You can go there. You can check it out any time you want. We are one of the main data distribution portals for the US Geological Survey, specifically remotely sensed data. So pictures of aerial photography, satellite data,
01:23
that sort of thing. And we do have some radar type data sets as well. So again, who am I? Like I said, Tim Mantell, SGT, Inc. I am the software engineering lead for the Earth Explorer project. I do lead a couple other projects, but since this
01:44
presentation is only about Earth Explorer, I'll just mention that here. And I represent the US Geological Survey's long-term archive project, which is a project out of the ARROWS data center, if you've heard of that, in Sioux Falls, South Dakota.
02:00
The long-term archive project is responsible for Earth Explorer, GloViz, if you've heard of that one. HDDS Explorer, EO1. I could probably fill the rest of this page with projects that the long-term archive takes care of. But the point is, as the name suggests, we do a lot of the
02:23
archiving of any of that remotely sensed data. So we have one of the largest civilian archives of remotely sensed data in the basement of ARROWS. I tried to find a picture of it, but I couldn't find one. But amazing that you can't find a picture of pictures. But we've got literally hundreds of thousands of
02:46
canisters of film down in our basement that we are working to digitize and put on Earth Explorer specifically. So a little bit of a history of Earth Explorer. In the 90s, somewhere in the mid 90s, it started as a
03:02
project called GLIS, the Global Land Imaging System. As you can imagine, mid 90s, it was pretty simple. It was text-based. Very limited number of data sets. I think there was maybe two or three data sets in there. So it was pretty limiting.
03:21
In the early 2000s, they started the Earth Explorer project. It's really where we came from. We passionately call it Old Earth Explorer. Very imaginative. But it was based on some clunky third-party software that over the years, as you can imagine, something that
03:41
starts in the early 2000s and lasts until 2010 almost, got pretty bloated and ugly and terrible and slow. So somewhere around 2009, I think actually 2008, they started talking, well, what do we do for a redesign? How do we make this better? How can we improve it?
04:01
So then came along New Earth Explorer. So again, an imaginative name. But one of the main goals was breaking that third-party dependency. We didn't want that software in there. It didn't really, it was serving its purpose, but it didn't really work that well.
04:21
We wanted to create something that was maintainable and extensible and look towards the future, because we knew that from the 90s when GLIS started, when we had two data sets, until 2010 when Old Earth Explorer really retired and went away, we went from two data sets to,
04:41
I think, 80 data sets. So it really got quite a bit of bloat in there and became a maintenance problem. So the New Earth Explorer was supposed to fix that. And obviously, we wanted it simple to use. We wanted anybody from the guy up the street, your neighbor
05:00
guy, to go in here and be able to find a picture of his house all the way up to a GIS user, a power user. We wanted them to be able to find things, too. So we wanted it simple. So the challenges in developing that New Earth Explorer, like I mentioned, the variety of data types at
05:22
the initial design. I think there was like 90 unique data sets. We're up to not including the HDDS Explorer data. We're up to 380 data sets now. So in four years, we've tripled or quadrupled what we've done.
05:41
We wanted to avoid that vendor lock-in if possible. It wasn't always possible, but we want to avoid it if we can. We do have a requirement that goes all the way up to the Senate to collect demographics information. So if you do go to Earth Explorer, you'll realize there's a log-in.
06:01
And we ask you a bunch of questions that we call demographics. That requirement goes way above me. It goes all the way up to some really important people. And they want to know who's using the data, what are they using it for, what type of research are they doing,
06:20
that sort of thing. Like I mentioned, we wanted extensible, so we wanted to create a system with future capabilities in mind. And the biggest goal of the New Earth Explorer project was to support Landsat 8. We knew that the data rates, as far as how much data we were going to receive, was larger than anything we've seen
06:41
in the past, as well as we wanted to handle that launch traffic. They were worried about millions and millions of users. Of course, we didn't quite get a million users, but they wanted us to be able to handle that. So some of the technologies we implemented with New Earth Explorer, we've actually had Map Server in there and in
07:04
different places for quite a while. We use it for the overlays, the browse overlays, the image overlays that we do on the map, which I'll show you in a bit here. We use PHP, which isn't specifically a FOS4G tool, but it's open source, so I feel good about that.
07:21
Obviously Apache, jQuery and jQuery UI are some JavaScript frameworks. We do use Oracle Spatial. That is the kind of came from the old system and lived on. It was kind of the de facto standard, and we are doing
07:41
some things to move slowly away from it. As you can see in the next line is PostGIS. We're starting to inject some PostGIS in there and use that a little bit. And then we do use the Google Maps API, again something that came from the old system and lived to the new system. Maybe that'll go away someday.
08:02
Not sure. OK, so data sets. What data sets do we have in Earth Explorer? This is actually a very small subset of data sets. If you ever go out there, like I said, there's 380 some unique data sets out there. But we've got everything from aerial data, some of these
08:20
Antarctic single frames, and high resolution ortho imagery. Those are from 1940s. We have data down in our basement that's even earlier than that that we're working to digitize. So we've got very old data, all the way up to on the right side, the Landsat archive. Landsat 8 is obviously very operational and very busy up
08:45
there for us. Going back to the left side, we've got declassified data. We have what we call D-Class 1 and D-Class 2, which are two different groups of declassified data. It's all aerial photography. We've got digital elevation data.
09:00
So we've got the SRTM, the Shell Radar Topography Mission, GTOPO 30, ask for global DEM. Global Land Survey, probably better known as GLS, at least S, is a collection of Landsat data. And there's different collections. There's a 2000, a 2010, a 2005.
09:24
It's supposed to be a global coverage map of the best scenes of Landsat data in this time series. So if you're looking for the best Landsat data, GLS is where it's probably at. LIDAR is, well, a lot of you know what LIDAR is.
09:43
I probably know less about LIDAR than you guys do. Landsat archive, we do have everything from Landsat 1 all the way up to the latest Landsat 8. We have almost all the data that there is for Landsat. And then we do have, in collaboration with NASA, we
10:00
have some of the land processes, DAC, data that comes out of AROS as well. So there's some ASTER and some MODIS, quite a bit of MODIS data out there. They added quite a few collections to us in the last year, so that's a good collaboration effort that we're doing with them. So on to some of the features.
10:21
I kind of wanted to give you guys an idea. I'm going to show you what the main interface looks like. So the interface is this. I'm probably not going to do a demo unless we get to the end and we have a whole bunch of time left. But if you're curious about a demo, catch me later and I
10:41
can walk you through it. But I'm hoping that the screenshot tour will give you an idea of what you can do here. So I wanted to focus on how you can find data, where that data exists, and how to search for it. We've got a number of search options. As you can see, this is kind of that left side of the
11:01
screen, take the map out. So we do coordinate polygon searches. So you can either enter them in there or you can use the map to enter the polygon or enter your points in there. You can do up to 30 points. It is limited to a single polygon right now, but you
11:22
can go up to 30 points, like I said. We have an address place name search that uses Google's geocoder. So if you type in Portland, Oregon, obviously you can find that. I'll give you a center point of that. WRS path row, if you're familiar with the Landsat series of data, you know what a WRS path row is.
11:44
That will either give you the center point or polygon of that path row. So we do have a feature search. So if you're familiar with the hundreds of thousands of features that are out there, you can find those in there.
12:04
We do allow circles. So if you want to say, in this case, 20 mile radius around Sioux Falls, South Dakota, you can do that. So if you just want to know what data lives within there. And then we have predefined areas. So this is something we've added recently, probably in the
12:24
last year or so. But it basically gives you a polygon of a specific area. So in this case, I chose the county around Portland. And then you can upload shape files and KMLs as well.
12:40
So if you've got a shape file of your area, just upload it and we can do a search based on that. Again, we are limited to 30 points and single polygon. We're hoping to fix that in the future. It's just a little difficult at this point. And then obviously, you can do temporal searches. You can change, you can do like a seasonal search.
13:02
If you click that month option down there, it lets you check off months. So if you only wanted to do growing season or winter season, whatever, you could do that as well. And then you can change the number of records. So onto what we call tab two, the data sets tab.
13:21
Here's a list of the groups of data sets we have here. Obviously, I'm not down to single data sets. So you can see we've got quite a few groups. Under each of those is a number of data sets. So up at the top there, if you look right up here, we
13:40
have what we call the data set pre-filter. And so what the data set pre-filter is, it kind of gives you an idea of what data might be in the area that you entered in the first tab. So if you've got a specific area of interest and you click this, it's going to say, OK, this data is definitely either, this data is definitely not in the area
14:03
that you are searching for. So you can probably ignore that. Now the one caveat I will say to this is we're just doing a bounding box. So if there's data in Alaska and data in Florida and you pick a spot in the center of the US, we grabbed a bounding box. We're going to say there's data there, even
14:21
though there's not. So it's not exactly accurate, but it should give you an idea of what data sets do exist in that area. Now down to data set options. The first option there, and you'll see this in most of the data sets, we've got a product page that tells you more about the data set.
14:41
It'll give you information as far as where it came from, what processing levels exist, how it's ingested, that sort of thing. And then we do have the coverage map, which was a recent addition as well. And the coverage map, again, is similar to that data set
15:00
pre-filter where it will tell you exactly where data is, doesn't tell you how much data, doesn't tell you if there's 300 scenes right in this one little spot, but it'll tell you if I'm searching in that little spot down there, I'm definitely guaranteed to at least get one scene. So moving on to the third tab, once you choose a data
15:21
set, so in this case I chose LIDAR, and I move on to the additional criteria page, this tab really fine tunes the search for that data set. So this is data set specific. So each data set's going to have different criteria here. You can see with LIDAR I've got state, state, province,
15:45
and country information, so something like Landsat doesn't have this, but LIDAR does. Project again is a specific thing to LIDAR, but a Landsat data set might have a WRS path row in here too, so differing information.
16:02
And then we move on to the fourth tab, which is where you actually find data. And you can see here I've chosen the Landsat 8 data set, it tells you how many results are within my area, which is right in here. Gives you some options for exporting, there's some different controls here, and then the bulk of what
16:23
you're doing is in this little tab row here, which I'm going to show you here. So each of these different icons means something different, some data sets have specific icons, some will have different icons, but this is the majority of them. So the first one is a footprint, you can get a
16:41
footprint of that scene, it tells you where the scene is, what it covers, in this case I was looking in Portland, I found some data that was over Portland. The second thing we call a browser overlay, which gives you essentially a full resolution WMS link is what it is.
17:02
This is where map server comes in. You can zoom into that, you can see all the way down into that, this is Landsat 8, so 30 meter pixel. And you can see if the data is going to work for what you need to do, if there's clouds over it or that sort of thing.
17:20
We have what we call the browse comparison tool, which is something again we've added in the last year. Browse comparison tool basically allows you to take a series of scenes, one or two or more than a couple, and once you hit this compare button, after you choose a couple, you hit that compare button
17:42
and it takes you to another page that allows you to compare those scenes, allows you to overlay them on top of a map, I don't know if I have a screenshot, no, I should have screenshotted that one, but it basically allows you to compare those scenes
18:01
and look at them on top of each other. So you can see, if you're looking at two scenes and you're trying to decide which one to grab, you can look at them and see which one has cloud cover over your specific point of interest or which one maybe has better seasonal data. You've got a scene from July and a scene from August,
18:22
which one looks a little bit better. Again, this is getting down to pixel type stuff where you're looking to get a specific piece of information at a specific point. The third one is metadata, obviously we're big on metadata so you can look at the metadata,
18:43
there's obviously more here, you can see the browse images, in the case of Landsat 8 they have two browse images, three browse images, and so you can take a look at those. Obviously the reason you're here is to download things, so you can download the data with that fifth icon there.
19:05
Again, each data set's gonna have specific products that are available, so in the case of Landsat 8, there's actually five products that are available, so there's a natural color, thermal quality band, geographic reference deal, and the level one geotiff.
19:26
Bulk download is something I'll talk about in a couple slides, but the bulk download, basically you can queue those up and create a queue of data that you are able to download later, so just clicking that icon will add it to the queue,
19:42
and then some data sets have an ordering option, so in this case, if I wanted the L1G, I can order that on demand by clicking that shopping basket icon. Another thing I wanted to talk about
20:00
was the standard request, which in the results screen at the bottom right of that results stuff is where you can submit a standard request. This is something that basically allows you to run the same search over and over, so if I had a specific area of interest and I wanted to know tomorrow if there's new data there
20:21
or next week if there's new data there, I would set up all my criteria, all my information that I wanted, and then I'd submit the standard request, and what you're doing here is you're telling it, run it once a day, run it once a week, once a month. You give it a start date and an end date. You can add the auto bulk download functionality,
20:44
so if you always just wanted to download any scenes that match this specific area of interest, you could do that, and again, I'll talk about bulk download, I think, in the next slide. And then you give it acquisition start and end dates, and you submit this, and this sits in our system,
21:01
and basically every day, in some cases, multiple times a day, we'll run that and we'll look for new data for you, so you don't actually have to come back to Earth Explorer until there's new data, then you come back and you can download it, so it's kind of a nice feature, something that I kinda wanted to show you guys because I think it's pretty useful.
21:22
So like I said, I'm gonna talk about bulk download. Bulk download is, basically allows you to generate a list of downloads, so if I know I want 100 scenes or I want 1,000 scenes, I can add those all to my bulk download list, and it's really meant to be kind of an unintended
21:41
system for doing downloads, so, I submitted an order here, I submitted two scenes, so this is actually a Java interface that we wrote that sits on your local system. So I come in, I select my order,
22:01
kinda see that, but you can see here, I've got two scenes pending, what I would do is I can come in here, I can change some things if I want, but if I just wanted these downloads to start, I'd click the begin download, and they would just start, so I'm downloading two scenes,
22:21
two scenes obviously is not probably a realistic use case for it, but imagine you had 1,000 scenes, you wouldn't wanna click 1,000 times, okay, this, this, this, and this, this gives you a way of downloading a large bulk of data. We released this about a year ago,
22:41
and the usage keeps going up, we're up to 100 terabytes of data a month now that we're pushing through this system, and I think we just passed the petabyte mark since we launched this thing a year ago, and the initial launch for the first about four months
23:01
was kind of a closed beta, so a petabyte of data going through this in a year is pretty amazing to us, we're happy about it, so we wanna push that more though, we're gonna hit a petabyte in a month, so help us get there. Okay, so I've got some links up here,
23:21
obviously the website is earthexplorer.usgs.gov as I mentioned earlier, if you have email questions, you can go to that lta.usgs.gov, if you ask them for a developer's help, they'll send it right to me, so I'll get them eventually, or if you confuse the customer service people,
23:42
they'll send it to me too. For feedback, this link is actually right on the website, if you go to the top right corner of the website, definitely if there's any feedback, if you have questions, I don't know what data to use, or I know I want this data, but having problems downloading it, anything,
24:03
again, that feedback button basically just goes back to the lta.usgs.gov, and then they send it to me, and we solve your problems, and then we do have a help document out there, again, this link is available on the website, in the top right corner, that does have some of the more standard scenarios,
24:21
you know, and clicking this and it doesn't work, or how do I do this, so, so with that, what time we got? Okay, let's go questions first, and then we've got kind of an aside that we can go to if we need to, any questions?
24:52
Do you guys have any like future feature stuff that you wanted to comment on, that you're working on? I do, I had actually hoped to come here
25:02
and kind of unveil a large feature that we've been working on for a couple months now, unfortunately my customer shut us down, so we do have some really cool stuff coming up that I can't comment on right now, but hopefully next year we can come back
25:22
and talk all about it, but other than that, we've been really focusing on a couple other websites, so one of the new websites that we're working on right now is something called Adopt Pixel,
25:40
and it's more based on, it's kind of away from the Earth Explorer project, it's based on ground truthing, so one of the things you'll be able to do is come into the website and submit photographs of specific areas, and then somebody could come through Earth Explorer and actually search for that data
26:01
and look for ground truthing type things, so if the satellite says, okay, this is grasslands, and you know, you pull up the pictures and it's actually something different, it's wheat fields or something, you'd be able to ground truth it, but that should be coming out, I don't have a real good timeline for it,
26:21
but sometime in the future, but that data will be available on Earth Explorer, but unfortunately, no, to answer your question, I don't have any future things, but there is something big we're working on. Any other questions up here?
26:46
Hello? I work for a sanitary district just south of Portland, and I've been wanting to do a septic tank failure analysis, and I've just never known where to get some current
27:02
data, aerials, and satellite imagery. Would the satellite imagery that I could get off here be at scale that I could do something that small? There's potentially some of our commercial data that we have available out there.
27:22
I didn't mention the commercial because it's kind of a limited, it's got restrictions as far as who can access it, but potentially depending on how you, which organizations you're associated with, you could have access to that commercial data, and that commercial data is down to,
27:41
there's some of it down to two meter pixels, so that could work for you. Can I talk to you about that? Yeah, yep. Any other questions? Okay, we got, any time or no? Okay, that's fine.