Add to Watchlist

OnEarth: NASA's Boundless Solution to Rapidly Serving Geographic Imagery

1 views

Citation of segment
Embed Code
Purchasing a DVD Cite video

Automated Media Analysis

Beta
Recognized Entities
Speech transcript
good afternoon but things are coming Monday Mr. Roberts I am from the NASA Jet Propulsion Laboratory just like the presenters of for me i or GP on managed by the California Institute technology I'm aware guide team called the global imagery browse services or Gibbs on is a collaboration between JPL and also the analogous NASA Goddard Flight Center and so today I'm and talk about what's gone on Earth the so this
brief outline and then the 1 on it actually is a more details about Gibbs get into some more de technical survive on their on the look at performance of metrics and there's time maybe from below preview of some the client applications that could built using the on the server an idea I'm going to just briefly go over on are open source project the
so what is on on is basically to an open source image processing and server software package from in other words is what tells server is intended to provide another box solution for generating geographic imagery on it's 11 more and then just to tell ever we also include components to Tau pyramids from our global mosaic and then store those on pyramids in this image archive but and so the intent of the of the archives field served tireless to some client application using standard what recalls for our case we use random you variant just tell the US for came out ontology mass is actually extension of 30 mass it was developed to your own well that quantum and overcome it is a way of creating a predefined set of Castile so when you do out there you must request you already retrieval set catch trials and now I guess the key selling point of honor monitors is as designed to be lightweight and very performance driven we
use this special far from format called another Rasta format for them or by is essentially it's nite in image format for stays in image container format and it's we use is because it has very fast access and we can were not limited by image size a resolution then we select the law to generate the MRS and we have this and what driver custom-built driver data on that we use to you know run or translate from 1 image format tape categorically and rubber and the other generates around the operates in into the former for the actual image server on it's essentially the same pattern much of the column on the n on top there we have some configuration tools to just kind of a whole sub server Ch that is all the background about on their it's 1 it's been around for a while this development began the originated PL or India early 2 thousands and the lead developer at the time of the chain he now necessary on all publicly being an open source project we we started some pretty contributions from from all around on those formerly known as a tower WS server this is back when we at the time were just using ontology must now we've expand to the more commonly used on them the UMTS specifications some other things
on if is in use for common queries from planned planetarium shows apps public museums on knows the 1st image server to serve out 15 you global Landsat data for imagery and for it for those of you who remember NASA world when there's kind of predecessor to grew worth on was used as the actual image server for for that applications well and as I plentiful there's we we have an server for Mars and the Moon and respectively we call that on Mars among them upload act which is the physical oceanography distributed active or archive visited the on they had this stay the oceans told to 1 % inventory of all the world's oceans not also built on this on the server on just a couple years ago we started this project Gibbs come back in 2011 and others was selected as the primary answer for for that project as well and on top of gives 2 client applications most commonly as NASA world view and finally just last year summary releases sovereign open-source and get out so it's really available and to develop an axis and on the software good OK
so I have a bit more but did so NASA as you all know has several river observation satellites constantly taking measurements of the year and we have this really rich and vast image archive so several useful methods available to the general public scientist and so forth so the purpose of gave is to provide a full resolution image archive and access services to get to you all these different inner products I'm currently there's about for over 75 global xi products most among model within 4 hours on its is furthest and Her conservation system Data Forum Earth Observation System Data and Information System as this and because it's ability of public we need something as really scalable faster responses and so forth from so from the Farm opt for more information there's a link there there 2 we also have another presentation gives related at 4 PM that's going to provide some more tea context and historical background the this so I
just wanna key selling points of on the figures on speed 0 the key to this being a story this morass format I mentioned earlier so most of the overheads in in actually tainted imagery where not most but much of the overhead is with is having to do with the file system if he had images sitting on on a so the honor machine some words disappointed PNG user cytologist them folders under introduces some bosses latency so to reduce that we put everything into 1 big data files so so when you're accessing they did this have to go to 1 1 point without any heavy file system operations on this but this is there under Florida's and I mean nowadays there's different image file format certain container format for top Herman such as and accountant and you back itself they're on before that wouldn't really consider anything else and at time see what from
so it MRF we can support groups the
sorry about that with MRF we can supports multiple arm projections we use geographical long Arctic and Antarctic polar stereographic and also when McKay to amass being maps and so forth some the compression types used on this is for when you plot images of a lotta tag induce audiences binary data on my out of any of any what class actually use contest by primarily it's going be Jupiter PNG I mentioned earlier there is the driver fatigue or that we used to generate mrs entered into to the to to to do that we basically this round cannot translate Whittaker from an image composite just that in mrs and on top that we residue optimist to build up the difference you levels of of that made you know where to get the final product from and what is not set itself by itself as single file is actually composed of 3 different files the 1st medium or headers is an XML based part containing the motive for the the MRF itself there's also this Annex 5 which is basically just a lookup 2 was somewhere in there binary data file which contains all the images cheap
the so in the metadata for in in the metadata it's it's pretty basic actually on the
it contains as the 4 base resolutions on the compression and information the size of the towers we normally use 512 to about 5 12 there's also additional element besides this and you can then you can have information the color maps and other projection information as well
so the data files is pretty simple actually it's just appointed J. Paganini 4 pgs 6 together side side sign for now for base resolution the bottom of the pyramid and moving upwards on each block itself is the actual J. tag image so once you pole on for the data that you actually about that that for on there are some drawbacks to this day updates only through a pen it so it's in there there are ways to go back and up this and the the but LaMarca complicated to do that so the units
size is a binary file containing 1 also in size so we do a look of fatalities what's really fast years on the for the role so on and so the top here imaginary of 0 0 0 . 1 0 1 0 2 and so forth this 40 offset and then read for death particular size of the town you get back images really really fast so moving on to the
actual what I call the on a server I should surpass margin so basically you you can run this software with and the Apache Server justice applied and that means that you can figure on it contains the cash metadata and the on this basically the information we saw no the market and the that's stored on a server and we were able to read information about the layer of really quick there's 1 if for cases where there's no data since so much the year 50 only have a layer that's on just the oceans are just when you might have something to do so we have a special index 0 0 that just returns a blanket tile I on measure the earlier about the common for the cost of the to see that the MTS tied that's in came out on From this you from a framed as doses stick to be below these data kind application read and figure out what there is available and so forth and this is isn't actually done through the Apache model we have a separate CGI scripts that that will return u t detectabilities information 1 we know exactly followed the specification for dimensions we this this added time extension of much of our data this summer as historical images we have to go back in time so we had a slow down time stability well to to build create at 1st and for on days in the past was Kano there are just too slow pointer is you always get back at lower images so when you him follows just for the entire of whereas with their interests were tied them as you could pull back the ready to get into individual tiles the whole purpose of the of the model is really just to translate HTTP requests and and you know to read or and return at a client application and so low bit
of information about the performance found is assembled from Otis data and I guess I guess the key thing again here's this is possible the gets these that thousand dollars per 2nd on this is without work right latency the so does in these numbers are a little bit dated user from 2011 back when we're doing evaluation for Gibbs I'm not sure how this compares with others and tall servers nowadays so be nice if someone could they check it out in evaluation and there's biotechnology out there a lot to to know about is well from so that encourage some just some comparison students who would yes thank you and just to see how on earth is
working on the wall of home deserves some sample matters and Gibbs there's about 40 million quest non 2 months ago so there's there's a pretty decent size of is visitors hitting the servants I really putting any dent into it so that's a good thing 1 currently there is over 90 image products and with which gives we wrapped around 50 meter data when the past we've we work with Landsat from 30 meters continuing to do this for so
to help configured as server we have a couple other tools just to make life a little easier if found using on Earth so there's a sperm are generated you can use G . to basically generate year and or a fast but this this MRF generated abstract so that you dog commands stuff away from you if you're if is the cash either by using that stuff on the also allows for on the processes we use x amount configuration files so when the system created on kick bounce a maximal falls and build it generates images for so there's also found this on flock integrator we use that to basically generate the server configurations on the Net idea for addictive abilities and so forth the thank aboard doses is on the flight in generator so you can take your own color map we have our own gives format for color maps and just below to generate a nice little graphic using that tool and there's also this matters to which basically it just converts Apache large into our format that's useful for for it if you're econometrics this the graphic here rises how the system works you prepared preferd imagery on this assumes you have the images available or globalization you're able to turn into an image pyramid and once accuracy in there something of figuration evolve with time deprived layer and the low down to the server and from that point and you you can serve out the energy imagery using these common protocols so forgives
we use on underfitting projection with the sufferer part time is an image management workflow system on it helps automate and manage our MRF generation and there really great thing about it is that it keeps track of on the metadata and this is if Data-Prompt prominence which is important for scientists to be able to go back and track word that did this come from on the great thing about ties that we can use it to you for searching and and do the search on the metadata so for and this is also in development it does not open source by something to the known the 1 the so singular
preview and client applications such like at some time to maybe check these I wasn't sure about the the network over trouble was on this is the so and
this and this is where do we can we use this as the gives reference client of In what
on earth is really doing is serving did you talented if I had refresh you you can actually see the imagery or details being loaded but I hadn't looked like Commissioner for this that works
the issues the so I'm I can do for them demo
there's another presentation coming up at 4 PM on exposing NASA's number of observations grows columns and on is anymore and and that type a world view
so you can actually see that call showing up here that space would understand is a couple more other
applications this somewhat this one's using the worth on Carosi on-screen here just to the screen resolution but in this this is showing in the surface temperature that and there's also a 2 D version this I believe is using the flat
mashine ascendancy service amateur there's a lot so I layers that did you can look up and so forth and 1 1 thing I forgot to measure earlier this on a survey only serves up imagery harm that doesn't stop you from using as separate server-to-server vector layers have leaders this the session vector information somewhere the the the we
we also have this a linear mapping moment portal it's the taking on earth to the moon I guess so we have then the disease frequency this image you back to the Apollo era and stop or Academy images them and so just particle so in
those are just a few couples sample can't can client applications on here some links here I wanted it on your own free time series to older so here's a list of other nouns for plants
and shut off worldview which uses open OpenLayers and firstly flight to maps the maps and of course and this this is also available for jazz clients and necessary the as we are just online actually has that gives available come to the end to the portal so you can use that for for for a variety of common justifications we have courses mobile clients and speckle axis which is is useful for a cure to answer down a bunch of tiles and humble claw-free map or something like that believe map Marx did that for for their own version coffee maps the so the
build up another open-source project and you have all all this too is available online on there's a couple other projects as well there's world you and some examples of how you can build your own clients as well on the primary source for D. on the software is on this NASA gives on the site and we're also were working to separating MRF Costa Rica bill that How would into its own standard there is some specification this now well documented at this point but there some specifications up there to provide you with the liberty of noble more intimate mission was code is in C or C + + some the non-performance centric software we use Python just because it's easier to develop and use and on the nice for you know the open source community to check to the software provides some feedback good you like to performance there's some other software that may be on to the benefit and also if you participated in and there's some there's also added the sense of helping out sense communion and feeding provides a made to NASA pushing technology for some thank this treaty on the future developments were working on the granular imagery the challenge is how to the search granules key on a server server without sacrificing performance on scholar items such as motor imagery and and several improvements in the MRF specifications and working some help from as rated reciprocal stuff of MRF I'm making more Paul friendly I guess and and also on the these groups assist can the know what is happening the so in this
this this lady inside and I the
1 advertisers there's an open position number too much about it because they had a
pretty good but on the it's
this of basically it around I know it it is so I
leave off couplings here the on their system target of page how these kids basically there's an API a new like cotton content in gives a is the e-mail address in my personal and notice assault there so I'm the because at this point likened to questions
if the I assume that the serving is all from those uh pre-generate tells like that and with the color map does not apply to a fly that apply during generation the tiles ah yeah that's correct volatiles the pre-generated and and doing every generation process were passing the color map the sticks with images and use the home so princes of you just generate J. pigs and that's the thing that server can serve as an interesting like being dynamic conversion as in and defeats the performance of idea there is interest but that's that that's a promise performance of might be slightly on unrelated is there the that the different data currencies of the different products within the viewer in another words that have the best monitor the freshness of the different data that's available on folder question I Overview miles the reasons we to have the best value the child so for attributes and we we served to of the imagery on so this is basically just timestamped into that and there's not really a weight here if you if you're looking at sub-daily invented to l there's not times therefore our recent evidence the i is just showing medial armed bands this and any other of the derived products like ciates constriction cloud cover analysis etc. that can go into the to get this the things that I thought the other topic yet there's so forgive serious if go world view there there's there's several layers of on science products on there so in this 1 you can know groups national disaster common intuitive so yeah there's there's different columns size parameters that this you can get the the list of compatible clients is quite large is that because of the w must support work you helps used also support specific other protocols hi it's this mainly because of the data Masson WMT which is commonly used for camel so if if in theory any client that can the false was proposed in deal that can access the data you match you mentioned that the W. MTS had extra thing for time as that above and beyond sort of the WTO standard and then there's the 2nd question is does the W. mouse support those time or a quarter of the time parameters and so Danube NTS is something we added to its assigned day aspects something we added and I think and I denotes expert I think there is a time extension how this so that as follows along you know so that right right right you know 0 yeah the 1 what's wondering with the Earth for if entries updated know 4 hours from life and the and the entire foreigners be appended for updates how does that happen how you dealt with that so is easy and fire is the entire founding be regenerated come when D is updated hi yet from pretty much at this point need about how long does it take to generate the and are a file hi Anatoly depends on the size of the imagery of its on favors kilometers sizes is pretty quick how much he deals with state 3 and Landsat data that could take hours so that there is some limitation there if you if you're dealing with really from high resolution images when took a long time unless you have some really the the computer power 10 them I think the was the time
Collaborationism
Context awareness
Web service
Goodness of fit
Computer animation
Analogy
Electronic program guide
Point (geometry)
Server (computing)
Open source
Key (cryptography)
Multiplication sign
Projective plane
Image processing
Client (computing)
Mass
Set (mathematics)
Computer font
Field (computer science)
Metric tensor
Word
Component-based software engineering
Computer animation
Personal digital assistant
Information retrieval
File archiver
Cuboid
Quantum
Extension (kinesiology)
Server (computing)
Open source
File format
Multiplication sign
Software developer
View (database)
Tape drive
Projective plane
Physical law
Client (computing)
Cartesian coordinate system
Image resolution
Chaining
Latent heat
Computer animation
Web-Designer
Tower
Query language
Ontology
Device driver
File archiver
Configuration space
Pattern language
Point (geometry)
Context awareness
Overhead (computing)
Presentation of a group
Computer file
Scientific modelling
Multiplication sign
Virtual machine
Image resolution
Web service
Linker (computing)
Operator (mathematics)
File system
Conservation law
Subtraction
Skalarproduktraum
Boss Corporation
Product (category theory)
Information
Key (cryptography)
File format
Bit
Measurement
Flow separation
Word
Computer animation
File archiver
Dependent and independent variables
Figurate number
Multiplication
Email
Texture mapping
Computer file
Information overload
Binary code
Projective plane
Mereology
Local Group
Residual (numerical analysis)
Roundness (object)
Polarization (waves)
Computer animation
Data compression
Device driver
Data type
Subtraction
Social class
Image resolution
Texture mapping
Computer animation
Information
Graph coloring
Tower
Data compression
Projective plane
Element (mathematics)
Metadata
Image resolution
Sign (mathematics)
Greatest element
Computer animation
Computer file
Block (periodic table)
Binary file
Units of measurement
Server (computing)
Multiplication sign
Scientific modelling
Translation (relic)
Client (computing)
Student's t-test
Metadata
Number
Latent heat
Scripting language
Extension (kinesiology)
Stability theory
Pairwise comparison
Computer-generated imagery
Information
Tesselation
Bit
Cartesian coordinate system
Measurement
Performance appraisal
Subject indexing
Pointer (computer programming)
Computer animation
Personal digital assistant
Hausdorff dimension
Right angle
Marginal distribution
Reading (process)
Separation axiom
Metre
Point (geometry)
Hidden surface determination
Server (computing)
Flock (web browser)
Product (category theory)
Process (computing)
Texture mapping
File format
INTEGRAL
Multiplication sign
Ökonometrie
Sampling (statistics)
Weight
Food energy
Video game
Computer animation
Graph coloring
Configuration space
Figurate number
Communications protocol
Physical system
Electric generator
Open source
Software developer
Multiplication sign
Projective plane
Client (computing)
Mereology
Metadata
Word
Image organizer
Computer animation
Software
Physical system
Computer animation
Client (computing)
State observer
Spacetime
Computer animation
Demo (music)
Presentation of a group
View (database)
Data type
System call
Number
Revision control
Image resolution
Web service
Touchscreen
Computer animation
Vector space
Information
Archaeological field survey
Cartesian coordinate system
Separation axiom
Frequency
Particle system
Computer animation
Moment (mathematics)
Linear map
Mobile Web
Texture mapping
Tesselation
Variety (linguistics)
Electronic mailing list
Sampling (statistics)
Time series
Client (computing)
Cartesian coordinate system
Revision control
Computer animation
Linker (computing)
Freeware
Point (geometry)
Standard deviation
Server (computing)
Open source
Code
Software developer
Projective plane
Feedback
Client (computing)
Local Group
Flow separation
Latent heat
Computer animation
Software
Website
Sign (mathematics)
Computer animation
Open set
Point (geometry)
Web page
Server (computing)
Musical ensemble
Computer file
State of matter
View (database)
Multiplication sign
Client (computing)
Parameter (computer programming)
Weight
Theory
Attribute grammar
Image resolution
Moore's law
Video game
Extension (kinesiology)
Subtraction
Address space
Physical system
Standard deviation
Process (computing)
Texture mapping
Product (category theory)
Electric generator
Tesselation
Content (media)
Mathematical analysis
Electronic mailing list
Expert system
Limit (category theory)
Local Group
Flow separation
Word
Computer animation
Graph coloring
File viewer
Right angle
Quicksort
Communications protocol

Metadata

Formal Metadata

Title OnEarth: NASA's Boundless Solution to Rapidly Serving Geographic Imagery
Title of Series FOSS4G 2014 Portland
Author Roberts, Joe
License CC Attribution 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
DOI 10.5446/31677
Publisher FOSS4G, Open Source Geospatial Foundation (OSGeo)
Release Date 2014
Language English
Producer Foss4G
Open Source Geospatial Foundation (OSGeo)
Production Year 2014
Production Place Portland, Oregon, United States of America

Content Metadata

Subject Area Information technology
Abstract OnEarth is an open source software package that efficiently serves georeferenced raster imagery with virtually zero latency, independent of image size or spatial resolution. The key to OnEarth's speed lies in the use of a unique, multi-resolution file format (Meta Raster Format, or MRF) combined with supporting open source software packages such as the Geospatial Data Abstraction Library (GDAL) and Apache to serve out images via web service protocols such as Web Map Tile Service (WMTS) and Tiled Web Map Service (TWMS), or visualization formats such as Keyhole Markup Language (KML). The emphasis on performance and scalability were strong drivers for developing this specialized package versus using existing software.While OnEarth is currently deployed operationally at several institutions, powering applications across the Earth Science and planetary spectrum, its active development is managed by NASA's Global Imagery Browse Services (GIBS) project. The purpose of GIBS is to provide a complementary historical and near real time (NRT) image archive to NASA's Earth Science data products for a multitude of uses: GIS ingestion, first responder and NRT applications, data search and discovery, decision support, education and outreach.Released as open source to GitHub in October 2013, NASA is encouraging members of the open source community to participate in the evolution of OnEarth—in the roles of developers, evaluators, and users—as a means to vet and enhance its capabilities. This leveraging of efforts not only benefits those who intend to use the software for their own endeavors, it effectively contributes back to NASA by strengthening GIBS and promoting the use and understanding of NASA's vast archive of science imagery and data. Several tools, including the GIBS reference client, Worldview, will be demonstrated as part of this presentation to illustrate the breadth of application and consistent image access speed across installations.https://github.com/nasa-gibs/onearth
Keywords NASA
OnEarth
GIBS
Development
Visualization
Earth Science
Collaboration

Recommendations

Loading...
Feedback
AV-Portal 3.5.0 (cb7a58240982536f976b3fae0db2d7d34ae7e46b)

Timings

  440 ms - page object