Using FOSS Tools, UAVs and Linear Referencing to Better Manage Federal Levee Data

Video in TIB AV-Portal: Using FOSS Tools, UAVs and Linear Referencing to Better Manage Federal Levee Data

Formal Metadata

Using FOSS Tools, UAVs and Linear Referencing to Better Manage Federal Levee Data
Title of Series
CC Attribution 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Release Date
Open Source Geospatial Foundation (OSGeo)
Production Year
Production Place
Portland, Oregon, United States of America

Content Metadata

Subject Area
The U.S. Army Corps of Engineers (USACE) is responsible for thousands of miles of levees in the United States. Over time the agency has developed a very specific way of evaluating levee systems and relationships between their constituent parts. This approach has served USACE well, at least until having to coordinate our internal data structures with additional data managed by other agencies or programs who potentially treat levees in very different ways. Solving this requirement led to a new approach being developed inside the USACE National Levee Database program referred to as Reaches. Based on linear referencing, the Reaches approach combines interactive creation of linear event themes, a graphical linear reference data editor, and user driven symbolization of the data within a single comprehensive web interface. Using a combination of OpenLayers, GeoExt on the client side and Oracle SDO, MapServer and GeoTools on the server, it is optimized both for speed and flexibility, putting the power of the linear referencing approach into the hands of a range of technical and non-technical users. This presentation presents an architectural overview of the system and how FOSS applications were leveraged during development of Reaches. The ability to manage data with a new level of flexibility and interactivity is supported in some cases by our new unmanned aerial vehicle (UAV) imagery pipeline. Flying imagery from altitudes only in the hundreds of feet give us images with resolution down to inches. By stitching these images into geotiff mosaic composites then processing into individual MBTiles databases using TileMill, we get a new level of perspective on construction projects and any other ground conditions, allowing for monitoring change over time at a convenience and price our agency has not previously had available. The full capability of this imagery is brought to light by our custom tile services that are generated down to zoom level 24. This presentation will show how by pairing the new imagery capabilities with the Reaches tools for collecting and storing data, our non-technical users now have access to tools and data normally reserved for the GIS/RS team.
Keywords USACE Federal OpenLayers MapServer GeoTools MBTiles
Computer programming Trail Data management Software developer Core dump Video game Database Oscillation Physical system
Web 2.0 Presentation of a group Data management Uniform resource locator Group action Data storage device Self-organization Right angle Position operator Spacetime
Web 2.0 Prototype Group action Momentum Demo (music) Funktor Hypermedia Linear programming Cartesian coordinate system Reference data Measurement Routing
Domain name Dynamical system Information Mapping Plotter Multiplication sign Tape drive Source code Bit Attribute grammar Tablet computer Data model Mathematics String (computer science) Right angle Endliche Modelltheorie Table (information) Position operator Social class
Domain name Multiplication Mapping Sound effect Line (geometry) Graph coloring Event horizon Field (computer science) Usability Proof theory User interface Selectivity (electronic) Reference data Social class Arc (geometry)
Area Greatest element Octahedron Mapping Linear programming Database Line (geometry) Table (information) Event horizon Reference data Social class
Axiom of choice Complex (psychology) Dynamical system Building Multiplication sign Client (computing) Mereology Food energy Medical imaging Core dump Physical system Social class Oracle Scripting language Touchscreen Mapping Software developer Maxima and minima Proof theory Data model Process (computing) Internet service provider Artistic rendering User interface Quicksort Row (database) Geometry Server (computing) Functional (mathematics) Mobile app GUI widget Real number Online help Distance Regular graph Event horizon Field (computer science) Product (business) Goodness of fit Causality Gastropod shell Software testing Data structure Computing platform Matching (graph theory) Projective plane Database Cartesian coordinate system Symbol table Subject indexing Spring (hydrology) Software Integrated development environment Personal digital assistant Video game Table (information) Service-oriented architecture Library (computing)
Regulator gene Product (business)
Point (geometry) Photographic mosaic Process (computing) Length State of matter Multiplication sign Image resolution Planning Mereology Measurement
Dot product Server (computing) Mapping Tesselation Software developer Virtual machine Database Cartesian coordinate system Computer Tessellation Medical imaging Digital photography Process (computing) Quicksort Position operator Physical system
Point (geometry) Collaborationism Presentation of a group Scaling (geometry) Computer file Observational study Weight Client (computing) Number Tessellation Product (business) Medical imaging Universe (mathematics) Quicksort Data compression Asynchronous Transfer Mode
island going get started them of high thanks for the hand of the last session uh 0 yeah we have the best life doing great I am I will records and with the yeah you say uh US Army Corps of Engineers uh this is rainy gasses but we work together program and that manages a lot of our love the data in the course the so we're talking about how we're using some fast oscillations in the tools were building to help us manage that data so what is a core Karabel levies will we've got a little over 14 thousand miles of levees out there that are in a are programs and we have to keep track of so that led to the development of what we call the national the database it's it's a pretty good system it's a little occured because the course managers are data based on stewardship so we divided all the levees into what we call segments based on stewardship that aren't necessarily based on the thing hydrologic anything a topologic and it works pretty well for the core but who directed
Americans I was true of the court but now were being challenged with helping other agencies to manage their data on Web as well so namely femur and the way we organize ladies doesn't work with the way female organizers Levi's and so historically they've kind of been hesitant to work with us Commerce is not telling us that they have to and so 1 of the ways that were trying to help that is to the I introduce linear
referencing into the in all the every year as a linear reference in its right again well if you don't this is kind of my definition for it to methods of group of methods use the store manage location relative position arrange positions long linear feature at all that means is you need to know what router on and how far along that you are and then you can find yourself in space a simple example that mile markers are right Superbase this is how it's is
working to functors were doing live demo of it India's lecture so hopefully if this works but this is a prototype application for managing that your linear reference data on top of levees up on the web so you're you're you're basically referencing year out you need measures right so got here I've done identify here and identify here I got the rout ID and measure offer this particularly and we go to in courses not momentum of the art FIL tool called from the cation cheers scanner group with check summer camp yet ready an the the the yeah so it would uh media from Asia I'm doing the I get
half that of Sarah landowning on been too which is not my allow us that that all you have been to running on a Mac so the the but with that little bit of information what we can do is come in here and dynamically plot that range on top that that so yea that's funny so blood
you yeah and that leads us into developing a little more intense requirements so I'm sure you guys are all familiar with the song I that the good idea for a right so we've been challenged we we always get tell about very highly specific yet infinitely flexible tools and so that really makes designing these tools hard some of things we don't to irksome requirements of a part of that that infinitely flexible ecosystem are the 1st is user-defined future costs we don't know what data folks at an ornament atop tablets we know some of it but there is a change of all of the time the yes actually it sets are evolved change over time and we don't wanna have to go in the database and add a new table or and columns 2 tables every time somebody wants to add new attributes so we come up with Oracle a dynamic data model that essentially transforms the data as from how deep you but this is we will dynamic in model allows users to go into our tool online and actually create their own feature class so they can increase you will recall reach class and then add the actress they want they can add domains to the attributes but the entry tell are tapes so I when the ashore and that is a swords the shortest string that how when the transform back into Jason we we typed them correctly of is he can increase your own the 2 and then edit that date on top of levees so and and only our positives is user-defined symbology needed to be able to uptake how they were the data displaying maps and then they wanna save that happen so so we we create this that we call reach layers there's really just data symbologies source restoring symbology definitions for prosumers rich layers it's going to work so you get
this little uh interface here it's not the most there user friendly interface yet but just keep in mind this is a proof of of concept and so what you can do is pick the yeah rich class they're interested in I pick the field that you want to distinguish this knowledge based on that you can pick a field to label this on we do have dynamic hospital we had offsets you can offset multiple reach classes against each other to see where they are the and then I will pick the take a value of that domain we can displayed as a solid acted out of line the line with pick a color the selections of me then show that on the map this effect this so it resumed Elfardy can can see where the the data is gettin plotted on a map on top of bloody and the again so we
also were required to allow people to edit those events online is so what we did and I'm pretty proud of this and right did all the work so I still take some credit which is nice and we we built a tool that I've always wished arc had for adding the reference data so to do that we've got our levees it hands open
the I so I decide to pick which the lira 1 edit Click on the delivery to create a social area accident pull up all the reaches of the on that living in this rich class finally you'd an Oct the attributable there at the bottom take on the road it's gonna highlight that event on the levee itself click on 1 of the vertices and drag up and down Levi's it's a start I in snap to the right line you can see the major value updates up there on the the on map and the updates the tables well once you're happy with that you well you could also commended as keon edits if you want to I what you're happy with that you hit Save that's gonna persisted to
the database that it is now saved in the database I think this can a powerful tool to let people begin to manage their IT turn on photography below this get really precise with that uh where you're linear reference data goes it so how do we do a
test ideas compatibility and the easy answers that we we widget to allow you to X to ship so we gotta reach class here's symbolized but if you wanna get that out into you just GIS uh we've go tool distances x for each us and pick a rich class here x 40 it will dump it into a shades of and dollars with them for a were eventually going to figure out a way to give people the rats themselves and an event table and let them play in arc litter of all they want but most people don't do it ourselves this is the help most helpful tool that we can before so fewer concerns issues we faced in continue faced throughout this project IUI complexity which is for building this sort of uh you to allow people again and manage the data it gets really hot so the tool you saw is great but it does the job and it's it's a pretty good proof of concept once we start implementing implementing this in production were and have to really take a hard look at you x UI design and make it more intuitive and then perform its been doing this can be competition in intensive released than a really good job of leveraging the open-source stack that we have let's make it really performance and it runs really well and to a certain time the offset that but that has been Madison really really good stuff at so that amateur and he's going to finish it thank go back to the slide real quick the performance really revolves around 2 things 1 is this dynamic data model which means when data is requested from our system we get a reconstructed on the fly and stored in such a simple way and the other thing is all the geometries that you see on the screen a derived you know we have the base geometry in the base map at everything has to be clipped and then delivered and so the derive geometries have some issues inherent the G databases when you can index and so that create a whole mother host of performance issues and so we had to consider that we started looking at the approach and the platform so that we pick so 1st off we are team uses what we call the core map platform core map platform is centered around were called Oracle is RG database of choice but we use map server as our spatial rendering engine and it's very fast and we've had a lot not successful matches 1 thing we did was we actually fork map server to add a 1 single simple functions and that was to deal with our performance issue and so we added keyword and that's index field so that instead of having to have an index some the geometry your displaying is you could use an index on another field such as the base map but still display the derived geometry and it really was quite simple to do that and and then we used OpenLayers and us of the client mapping side we design a
system we designed around other services we wanted to play well in a service-oriented environment and that's how it actually is deployed internal our system unfortunately because were part of the army when we tried to deploy this externally for clients to test we realize that that they don't have any interest externally in service-oriented architectures so they block everything so at the same time we had to make sure it became easy for us to make sure that we have the capability of actually providing every service we need internal to the application it's not optimum but it did allow us to get the system running and deployed application server wise we use a job a stack for the AP it's using spring itself hosted using jetties that means they don't need an application server run it we provide everything the IT team needs including the shell scripts and they just run the shell scripts assistance starts and it's completely self-contained summaries energy pulls library to do some of the so the unique rendering were doing more being able symbolize when you're referencing on the fly on and we decided but because of the environment we're working and go ahead and cache data so we just embedded GeoWebCache into our application so we again deploy it separately it's just wrapped up inside a vase it's just 1 out that is capable of providing the services a database approach but we had a provider really such a flexible approach Towards a Dynamic Data and but at same time we know that there's certain datasets that you might get created might become part of our established workflows and we all we have the ability to convert those back in the traditional database tables and optimally we want everything is a traditional database table cause we can index them a lot better than and perform better and so we do have the ability to just treat a regular table or dynamic table in the same manner as other software we do pretty much all of G processing an oracle were leveraging oracle any referencing functions and so Oracle pretty much handles all that makes my life a lot easier on development side and uh the 1 thing we did learn there was in this 1 we have to be very careful to only sort of a symbol and return the minimum amount of data we need because when you have dynamic data the way we did so you have to cross tabulate on the fly so we only return the few things we need and then work with that and I don't know there's probably only 1 case so we return all the records and that would be when you done to a shapefile otherwise it's only going to get the specific things you need to you to do and identify or to to render it and get an image the other part of this talk and was a UAS burden that were starting to use to look at hydrologic structures and more building capability and our apps so
our Community of Practice lead in Washington causes boys with toys she doesn't she's as she's kind of right I mean it is kind of fun and we just need to enjoy this stuff a lot this is actually the they're crapper flying right now and it's small it's handheld which gets us around some regulations but it's actively used in production noun it's pretty much flying around you know every day and the the team that runs it has to be licensed to fly it have to be licensed as and that about
and right now we're using all
over the place so here's example
we have another application internal and this is a screenshot from that what this shows is just a flight plan along lake and South Florida this is called Herbert Hoover dyke it's part of Lake Okeechobee which is a large lake in South Florida and there's a construction project ongoing and 1 thing we've identified this really useful is the flight construction projects were getting extremely high resolution imagery were flying about a thousand feet and so the tenant document state construction over time but down at a point where you catching a measure the length of a piece of rebar and so it is it's an excellent way to document ongoing processes as well so you see the flight plan in purple and the mosaic image underneath here
is the same image we turn off the base map here the dots that you see all the photos centers or application also contains everything designers and so you can
go on and on and on the imagery and even mouse over the original so see the original photo is the thing the 1 reason I wanted to talk about here and is because we talked the UAS guys into leveraging a tile mill and in the tiles and it's been it's worked out really well for us so after they're on a flight now and they create a mosaic then process at Mozaic with townhall and create a single in the tiles database we maintain it in the tiles and we're at a server that allows us to stack as many in the tells databases as we want of inconsiderate and here's an example in our ap I loaded for this system all imagery we have but this way we can just deploy new imagery as it comes in very easily and the user experiences what they like the most because it's just a cash so I don't the my cursor go from here to there was no and so this would to say this is some of the 1st imagery they had of the positioning is off along at Indian initial sort of weird artifacts but I don't have any of the good stuff on this computer is so that the development machine occasionally take on
the road and but the main point was just a have jury that is the instantaneous and you obviously you you pretty much needed cash for that because what they like to do is just move back and forth and hang around as fast as they can and they don't want any weight on it and so it's worked really well for us so that that particular stack of technologies as were good 1 other interesting thing we done with it is it that were actually build generating tiles down to the scale or timescale 24 I don't know that we can practically go any lower than that 24 the number of tiles even on a small flight is just enormous I 23 seems to be a sweet spot you get incredibly detailed injury i and it doesn't take too long to crunch it and so it it works out pretty good for us and it seems at at z until 23 decided immotile file is roughly equivalent to a tiff with sort of the a non lossy compression like NLC demure deflate so we've been real happy with what but
now concludes the presentation so if there's any questions be happy to take them FIL B yeah what camera to have on the yes I will direct you to the people that can answer that I I crunch the data that set up I'm not really involved with the flight team than working with them to get it in in database and make sure their products that their their clients can work with I the I don't know they they change they actually change their imaging frequently now is mean the technology is moving so fast that they just added infrared to it that they're doing different things and so they're experimenting with light are and it changes every 6 months there's new equipment they're running on it it it was a collaboration with the University of Florida so they finally got it out of sort of the the university study mode now it is in full production but they still have a team that is always evaluating new equipment they're modifying the aircraft and try to make it more stable and the only thing they want to continue to do those habit at that size if they can hand wanted it and it your clock enjoy the rest of the conference


   78 ms - page object


AV-Portal 3.21.3 (19e43a18c8aa08bcbdf3e35b975c18acb737c630)