Logo TIB AV-Portal Logo TIB AV-Portal

The Development and Evolution of an open source mapping application within the USG <- Now with More Google Glass

Video in TIB AV-Portal: The Development and Evolution of an open source mapping application within the USG <- Now with More Google Glass

Formal Metadata

The Development and Evolution of an open source mapping application within the USG <- Now with More Google Glass
Title of Series
CC Attribution 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Release Date
Open Source Geospatial Foundation (OSGeo)
Production Year
Production Place
Portland, Oregon, United States of America

Content Metadata

Subject Area
The United States Government has a history of developing applications using legacy systems and continuing to use brittle software. This approach has managed to minimize data collection, sharing and use of open standards. With this in mind NGA has several groups focused on a rapid, innovative, and open approaches to application development. One of the recent applications developed in this fashion is the Mobile Analytic GEOINT Environment (MAGE), which evolved from earlier applications that were used for Disaster Response as well as various special events. Each of these earlier applications had their own strengths and weaknesses that were factored in during the development of MAGE. MAGE is built on an open source stack with a mobile and html5 application designed for geospatial data collection, imagery sharing, tracking, and communication. It is designed to be a lightweight, fully portable software stack that can be placed in front or behind firewalls with ease. It is fully customizable to a wide variety of mission needs so administrators can easily change the data collection parameters. MAGE is fully service enabled allowing easy access to the data via REST requests and returns multiple formats including GeoJSON, KML, and Shapefile to ensure ease of access and sharing. The app has also been ported to Google Glass for field collection and enhanced visualization.
Keywords Disaster Response MongoDB NodeJS Leaflet Google Glass
Revision control Slide rule Mobile app Group action Computer animation Descriptive statistics Computer icon Algebraic variety Power (physics)
Web page State observer Greatest element Building Service (economics) Multiplication sign Student's t-test Regular graph Field (computer science) Thermodynamisches System Videoconferencing Descriptive statistics Addition Texture mapping Electric generator Spacetime Audio file format Denial-of-service attack Incidence algebra Connected space Type theory Uniform resource locator Digital photography Computer animation Right angle Window
Point (geometry) Web page Game controller Server (computing) Service (economics) Computer file State of matter Multiplication sign View (database) Mereology Event horizon Rule of inference Field (computer science) Roundness (object) Operator (mathematics) Touch typing Cuboid Address space God Standard deviation Fitness function Denial-of-service attack Cartesian coordinate system Symbol table Degree (graph theory) Subject indexing Word Uniform resource locator Computer animation Point cloud Procedural programming Figurate number Computer worm
Web page Point (geometry) Android (robot) Mobile app Game controller Group action Service (economics) Multiplication sign Client (computing) Coma Berenices Theory Front and back ends Goodness of fit Component-based software engineering Average Videoconferencing Ideal (ethics) Energy level Cuboid Form (programming) Area Texture mapping Mapping Tesselation Debugger Audio file format Connected space Digital photography Computer animation Buffer solution Quicksort
Android (robot) State observer Building Multiplication sign Archaeological field survey Water vapor Run-time system Disk read-and-write head Mereology Web 2.0 Medical imaging Array data structure Casting (performing arts) Bit rate Core dump Videoconferencing Soliton Office suite Overlay-Netz Texture mapping Spacetime File format Data storage device Bit Representational state transfer Connected space Type theory Data mining Benutzerhandbuch Digital photography Process (computing) Telecommunication output Right angle Summierbarkeit Quicksort Point (geometry) Web page Server (computing) Functional (mathematics) Game controller Computer file Open source Drop (liquid) Event horizon Field (computer science) Number Revision control Thermodynamisches System Traffic reporting Plug-in (computing) Form (programming) Algebraic variety Tesselation Interface (computing) Weight Debugger Database Set (mathematics) Uniform resource locator Computer animation Software Blog Table (information) Local ring Library (computing)
Point (geometry) Ocean current Building Open source State of matter View (database) Multiplication sign Open set Disk read-and-write head Event horizon Field (computer science) Expected value Mathematics Circle Social class Area Mobile Web Addition Boss Corporation Standard deviation Augmented reality Moment (mathematics) Instance (computer science) Shareware Connected space Digital photography Uniform resource locator Process (computing) Computer animation Software Website Video game Right angle Summierbarkeit Cycle (graph theory) Quicksort
Point (geometry) Laptop State observer Texture mapping Statistical dispersion Forcing (mathematics) Bit Water vapor Client (computing) P (complexity) Field (computer science) Revision control Digital photography Word Uniform resource locator Loop (music) Computer animation Videoconferencing Right angle Data structure
Slide rule Computer animation Quicksort
Computer animation Quicksort
Trail Group action Server (computing) Presentation of a group Building Open source Code State of matter Decision theory Mereology Perspective (visual) Product (business) Expected value Medical imaging Thermodynamisches System Profil (magazine) Energy level Addition Dialect Mapping Planning Sound effect Bit Set (mathematics) Cartesian coordinate system Type theory Digital rights management Computer animation Software Telecommunication Self-organization Figurate number Reading (process)
Perfect group Computer animation Bit rate Information Right angle Volume (thermodynamics) Quicksort Cycle (graph theory) Lattice (order) Cartesian coordinate system Window Algebraic variety
Laptop Variety (linguistics) Direction (geometry) Multiplication sign Web browser Disk read-and-write head Event horizon Product (business) Operator (mathematics) Algebraic variety Information High availability Volume (thermodynamics) Total S.A. Incidence algebra Lattice (order) Cartesian coordinate system Dirac delta function Digital rights management Process (computing) Computer animation Telecommunication Smartphone Right angle Object (grammar) Window
Trail Building Functional (mathematics) Multiplication Scheduling (computing) Touchscreen Perfect group Information Decision theory Volume (thermodynamics) Lattice (order) Price index Graph coloring Word Computer animation Bit rate Point cloud Right angle Game theory Routing
Point (geometry) Multiplication Group action Algorithm Multiplication sign Decision theory Direction (geometry) Device driver Line (geometry) Array data structure Computer animation Visualization (computer graphics) Term (mathematics) Arrow of time Right angle
Identifiability Computer animation Data storage device Right angle Bit Quicksort
Game controller Arithmetic mean Computer animation Plotter View (database) Multiplication sign Right angle Cartesian coordinate system
Content delivery network Open source Variety (linguistics) Code Chemical equation 1 (number) Content (media) Design by contract Branch (computer science) Product (business) Spring (hydrology) Computer animation Core dump Website Mathematical optimization
Content delivery network Dataflow Server (computing) Presentation of a group Group action Scheduling (computing) Service (economics) Computer file Multiplication sign View (database) Execution unit Letterpress printing Theory Product (business) Number Frequency Mixture model Term (mathematics) Energy level Software testing Address space Algebraic variety Tesselation Content (media) Counting Volume (thermodynamics) Database Bit Maxima and minima Line (geometry) Timestamp Digital rights management Process (computing) Computer animation Software Telecommunication Data center Volumenvisualisierung Right angle Cycle (graph theory) Table (information) Routing
Point (geometry) Noise (electronics) Smoothing Lecture/Conference Personal digital assistant 1 (number) Right angle Bit Quicksort Line (geometry) Number 2 (number)
Lecture/Conference Quicksort Measurement
Area Covering space Key (cryptography) Divisor Direction (geometry) Mathematical analysis Sheaf (mathematics) Degree (graph theory) Word Mechanism design Spherical cap Lecture/Conference Cuboid Quicksort Routing Form (programming)
so I'm with the future Experience group has therefore a fund name and that comes with the G of mobile apps team we have official descriptions of what our teams do both effectively you know we wanna try work and stuff is useful and innovative and we don't want in research projects that summaries personal last soapbox of their research for 3 years and then to people wanna use it so we wanna get fieldable technology that of can be used still immediately so of
the history this act the initial versions zapiske older the UFO after her really cool icon was spaceship but then we were told to pull back the spaceship icons people might read into that battle but but out so the history of this work came about from urban search and rescue people working on and disaster after Hurricane Irene and will go through with all the slides the power for the powerful usage so whatever better analysts went and
supported irony of and he saw 1 of the firefighters that was on research a rescue team from Virginia came back and he presented his zeal who notes to an analyst to make it into a map In this with the field notes there was 3 pieces and very a little like 3 by 5 no that's the last of a pretty In this amount of time sitting out the road intersections making notes and 0 the passing off for others to use and you know I'm sure that took a lot of time you know if the rains strong wind hits you might have some other issues going on but are analyst of looked at it and kind of just went with this stupid we can do better we do a lot better so cut to 2 days later the so we had
stood up the 1st generation of work became called the Robinson search arrest you feel observation the what happened here was a space we assume all 5 page that was wrapped of the wrapped into of the regular students ethnicity all 5 web page that you can access from and road I West Blackberry windows or your desktop it would pull the location from the device of it is filling up with an x y could reverse geocoding you put a date stamp on there and then down at the bottom of the thing that time there's about 30 different incident types collapsed building animal issue of you know burning vehicle or human remains that you can drop down fill in your team title and description if it's a minute and then a writes at 2 in the the rights attuale every endpoint and this is the this is great people like this and the nice thing is you all work with the mobile environment is you have for about installing anything right the it was just a web pages but the this technology of all the people wanted more of you know they want to be able submit photos videos and audio files to the endpoint service in addition to that they also want to have a disconnected ability because do not necessarily have Ltd connection in after tornadoes hit someplace or you have flooding are you have earthquakes so that when the old
detour here what the researcher rescue teams general currently do is they have a handheld GPS solution which is a garment and they have a custom symbol set the payload into the gardens and they then extract them at the end of the day and normally what happens is having done this in the field the and the search rescue teams will come back and all dropped 10 garments at a desk and tell the analyst extracted their older approach the extracted at the converted to KML file and then they e-mailed of tonic KML files around so if you have a 4 5 day event you could potentially have 50 KML files in another round 5 different times you might be dropped off to or through the military instabilities search indexes God spun out of control pretty fast but the so what we end up divorce evolving it will wind up writing there was basically just a a webpage importer where when extracted data you get a GPx file import the GPX file you prove you know it gives you some drop-down combo box many the says what's your team with that the search is that the future squad leader and then it will load this all into an service of the nice thing with this is the when it gets used it gets used and of the everyone has the same endpoint if it's we do not will march shall not foot stamp to the nth degree but dictate what what is the the word that the user wants to know we want right the service and we want to let them ingested however they see fit of the urban address teams while this it's now officially part of their standard operating procedure so it is written into their rules that the show this the approach in this application of the 1st beta view of it came about
from my home the from of the Colorado floods so these are actually points the rescue teams used for Polaroid flooding in there and it's so coming out party was the mudslides in Washington state so the mudslides washing state we had north of 6 thousand points in the field of so you have a I think it was 12 teams they're doing the points of that but you had 6 thousand points where people could see that there was a remains found at this location or there were other search-and-rescue point events that they needed to find out this approach of this approach is also we work with NGA that's part of the and that's part of the I see there's a lot of this data that you're not allowed to handle in touch so all of this from and server fun and web page as well as with as well as all the data is actually living on seamless service in the cloud so it's need to God is all amazon base so you know in a big event in Figure it is then a big disaster that can ramp it up and put more servers on it as needed my as all get back to your follower
who will get back in the day after the UFO up so the 1st episode just may seem 0 5 web page the 2nd half we developed in Iowa less and Android of the client to install wealth will talk will more in depth on the Android app the injury after that was written for the research arrested guys but it also had an offline component were you could catch you could zoom into centroid be on OpenStreetMap and catch the tiles with a you can define the buffered like I want 5 mile buffer around this area and go down as level 18 and I'll say you rotted grab 452 theory was and then you could if you have a tile on the map of so if you're area with no can connectivity you still have a mapping component and I wanna work disconnected you could also take photos of photos videos and audio files on the Sony when you're back in a y fire good connectivity it would upload them to the service of the so this was good this is great this is popular people liked it but the problem with the problem with this was we control stack so the front end of books that the the
front end of this form is really designed for the researcher STT so they're com drop-down boxes were disaster-related in the back end was an entry and points that of but we repurpose this app at least 6 or 7 times in each client wanted a new front end the just war stuff related to what they were dealing with and then they need to stand up their own as a service on the on the back the problem there was ah developer needed to rescan the front and then whoever we were doing this for needed to stand up an appropriate every service and my if you had as a tech support need to do pretty quickly that but some the analyst in some of the tech support from other teams and groups were less than ideal of the fast we ever managed to repurpose an average of 3 days on average it took about 2 weeks ago we had a couple that when north of 4 5 so it was popular at but the with the ability to read purpose it was less than ideal so that evolved into marriage and all passes that's the sort of Benson's team the so the the promise people
kept changing how those endpoints work we had no control over the right people change the format and so what worked and 1 version 1 work in the next nobody would tell us that until we get it that so right a long story short we said but right there's a bunch of people's office and you can imagine that these people even outside urban search and rescue were coming to us and saying OK I want the space capability but I want the interface a little bit of I wanna report on these types of events and things like that but and what initially started happening was we ended up with all these little fiefdoms of people going already customer for that can already cosmetic black and already has some of that of which very quickly people started to realize was again to be sustainable and so I came to the table and said look right there's a basic core functionality here that people want of I can build you something that does that and as a customizable you are right so we swap this all out of and started we really quickly put together a server back end with an API and a front end on that's built on Monday b node j s and then using a new and leaflet for the front end up and so it was really quick to send them up and get going and it's got a form where each team that's running this can go in and say this is what I want mine to look like and they can define what all that collection looks like and because we're using mongos our back there's no schemas it doesn't care if you build up what kind of data you want collect images happily populated into the database so we got rid of that problem that we ended up with our right we essentially have poured over but started again to make a clean up and build native Android and IOS sets but they pre-tax a bunch of data rates of tiles vectors all protection by so if you're disconnected from the network at any point you still have access to all of that data that you want but as well as the ability to take right we think that if somebody creates something in the field and the disconnected that should be the end all be all of they should have to worry about it again so somebody creates an observation and hit submit out in the field and they're not connected the network the out drops into a local database and things onto its weight so there's a network connection again and submits it without the user having to do anything right so once you hit submit you're done it's good to go it will get to the server as long as you have network again at some point the so let's people do all sorts thing text for this photos videos can all be attached you can submit all that send it up it appears on the map that all goes out uh to other people but were also connected to that system and surveys getting real-time updates them we use this a few exercises with the user guides and bring it incest and instead of talking directly to as research we just wrote a little plugin on the back and it takes all the data the system in a plug it into that RTs online but set up if that's what they wanna do up or they can use the web map uh we've created and you can also export files can also give some of that stuff so we're not trying to what the data into the system that we built at you can get it out of that's what you want to of the so I I no but you don't care about this the so but
right is essentially what looks like on Android you got a map you can see all the stuff these are all were employed and bring up see what's happening you can edit them up if you're part of a team of you're available overlays right so any cast maps the you have the apple go out and find mines device for you and show you and you can click goes on and off and if you don't want the map you we also put in sort of these news the use of you can just go see water the latest observations that are coming and what are the latest locations for people on my team the coming and depending on what your interests in the given time and a lot of people are more interested in where somebody on my team at a given time right we've heard lots of war stories about and use on other places of people sorted disappearing from the communications with their home base and people start to freak out about where they are and where did we last see him and so what are people really just like to see right where's the rest my team around it the sum of war where was the last place that we saw somebody for having a hard time in all the of so we got the soliton running out 1 of the cool things right going to the open source point here but some of our right so Paul Ramsey of anyway lesser known and up and pointed out right being a supporter of open source doesn't just mean I use open and support right it means giving back so and so we built on a lot of all some open source stuff that other people have contributed but our guys in the process of doing this were also encouraging them to contribute back and so some other libraries that were using and the 3rd 1 we've run into a blog users or places where we saw opportunities for enhancement our guys are doing pull requests and submitting that stuff back to those projects and going beyond that and even then looking to start open-sourcing some the stuff that we have developed a so this is 1 of the things that we're going to be looking at is a lengthy lengthy bureaucratic process to make that happen but we we have managed to get the things that I give have now and you can go see and page up there and see what's out there and were working to see if we can get through this process with this out to you because we've talked to a number of people that would like to get their hands on users from the things that we like to get out there where they can just take it and run with it on the the so going beyond that right a bunch people are coming to us and said this is cool but a big thing we've learned besides right the people that we deal with what disconnected right 1 of the top 5 questions if the network stand was again the right is a step in store but then the next thing is a lot of these guys right they're used to doing things the way they do so and they don't want something that's going to get in the way of or make you feel like it's interfering with the safety so to agree that the opportunities for us to take advantage of some other wearables and things that are coming out to make them getting these alerts and observations that to them without necessarily having the whole this this phone in their hand and head that distract from what they're doing were starting to explore that 1 of the places we started was with the last election time and
and deceptive earlier demo where every single kind a connection to the website for alive demos did not work well were not the thing to but when not could try and sink upper-class upload here but right but we area we were on a p k that we loaded into glass and this was the person co Walker they could click the photo and it's hit submit and they cycle through an pull there event and what's submit the point to glass pulling the location pulling the of embedding the GPS location from i in the excerpt that are on the photo I get submitted and in addition to what's right to the end point 9 and then in addition down here on the lower right if you see there was also an augmented reality view so if you had a glass on turn your head around you could see that there is a human remains issue 250 meters in in that direction and as you walk it would update and I and reflect accordingly now if anybody's ever use glass of it's only 15 hundred dollars and that kind of and the battery life on it is now while you're actually using stuff so it's a corpus of technology I I don't view as eminently field of at least current state so you know were were were were in the process of trying to write an instance of this with an Android where watch which has a 24 25 hour battery life of a lot more her under the radar class of the and it also cost 200 always up so if anything happens to that field it's going to be less of a problem so we that's our that's the the glass time here here and
will pull up a demo will pull of the website there's 2nd so we have 2 signatures and take a soapbox moment probably everybody already knows the spectra of dried this point of what I'm I'm talking inside government circles of some you appreciate this of the right like we really are trying to drive within the circle of right there's there's a lot of proprietary stovepipe stuff but the open standards are going to be t and getting access to the data sources right not having the situations where uh proprietary things are changing in or not hearing about until after the fact and and I I don't just mean official right now some in what I call the fact red so like I'm I'm all about GeoPackage which is a full on Open Geospatial consortia of mobile data container right but I'm also all about the adjacent so you know looking at both of those is this open standards that we need to be supporting and taking advantage of OWL and there's some culture change stuff for the government that 7 happen right but the current expectations that people have from the stuff that they're able to get home means we sort of past this point that I think the Government got numb fool with and saying I can give you a piece of crap was on the government in this music so that we have to right we have to keep up and provide things that are just as good if not better and if we want people to adopt them and and really take advantage of that and the open source software data and standards are all going to help their sum and then the most important 1 and I I'm I'm proud to say that uh Eric's boss's boss actually came up with this and has been shouting at around the bar agency uh that we need to embrace change is a constant and that's not something that anybody who's worked in the government knows that the data but so were were were were trying to drive it right these these things are happening way faster when when we're building uh this major thing it's it's turning around that releases right sometimes in a matter of days and we had 1 customer working with the used it during an event and they came back to us afterward and they said it was awesome but we really wish you would do x y and z and at the next event for weeks later it'd x y and z and they called us up and said where all that stuff in there and the city and they that we thought that would be in like 2 years and that's the life cycle the user living and so trying to change out trying to take advantage of of what mobile brings the and make that up in front of you so I actually
took over in well I I submitted this point before the start so hopefully will of uploaded with the variables
require we have heard of right so I
install the major client on me and operates on the injured from over there an the submitted appointed photo and it was a little bit to the west of
pork so I'm going on to field observations loop the should be 1 coming up here it is our stop at all to the world on the to so my up this might be a I I right so this is the point I submit it was from the farm go photo of the aforementioned water barley you can see it on the left and other people can download the phone those photos videos audio clips in the field on a laptop and then we also have a we also have a blue force tractability where people can see where the where where folks locations are so I and turn this off and then I can see that I was updated my last location on this updated 4 hours ago the maps not we drive but we have a lot has basically came to the this Convention Center or you will pay at the focal have this and any questions the it the and again just repeating it for the other folks with the updates for in the new version of post presently considered flipping that over there so the among the the following what I'm not going to the and I I I astonishing about advanced into on on and words avoid above all else and my pet peeve is the battery life but the GPS on the battery life the way polls that I think you had some the ran like 3 days were drawn from the current yeah so it's it's not an excessive drain on the battery which is a person that the of my a the mind mind the and I the to the but if at any additional questions what structure of the could this
my a the mind
room room room
room in my mind I see my but room room room to room my son my son my room room who might might see my room room room in my my slide my room room room my my sort my
so some room who might say my son my room room room in my mind to see my room room room my my
sort my room room room on them through my mind to see my
hopefully you're all here here my talk on track plans for york city of and with that the way the so just want to go over sort of
as a reprise of Mandarin shared level set expectations with the the audience right so this is not a heavy technical of presentation is really intended for managers decision makers and curiosity seekers those ridges plows of those generally interested in what happens in New York City of so if that's what you will feel upset if you going going to any other presentations if that hopefully you stick around and I'll entertain inform you that so with that let's plow ahead the each on sound effect but I couldn't figure it out so what the so why why should you listen to this presentation I am so a bright regions require different that study of about how we use source right from my perspective working in city government of those images story to show demonstrate at least how open source can be used in an otherwise of very conservative organization of I see in federal government there's a fair bit of open source work a lot of city and state governments as a real reluctance to it of because of obviously governments are often risk averse and there's a perception that open sources you know everybody's managing this code it's all willy-nilly you should be really careful of but is is also a prevailing belief that if you buy shrink-wrapped software commercial software with paid support year-round it the reading about a product well you obviously need to do the same due diligence in selecting whatever tools that you use whether it's closed or open source of so near T is an example of using the open-source to develop an application of fairly high profile targets a lotta use of that is part of the so very briefly I work for the city of New York Department of Information Technology and Telecommunications the mouth full I bring that up is where 1 of 50 some odd agencies within the city or mandate is IT services we don't plow the streets we provide services specifically I manage the mapping GIS group within the city and my role doing that is a manage group about 6 16 people split between drug developers additional analysts and systems admin type folks how we manage a lot this geospatial data data for the City of New York he also the building support applications so what we build is what we support so very careful in what we select and how we build things of so the the
situation is what you would have seen looking at your window on December 27 2010 right it's been referred to as snow getting it was sort of the perfect storm right on the mayor and his 1st deputies were on vacation the Bahamas right back of the snow rate was very heavy the accumulation was really high volume and the temperature was perfect quite often the streets a warm it's the sort human that accumulating on the streets very quickly this was every variable played into a bad storm really difficult to plow people lost their jobs because of it on eminences got stuck with the busses get stuck so on and so forth so the project alone if you
fast forward I my fast forward about a year January 2011 mayor bloomberg does a weekly radio show and the interviewer start asking questions about Snow mediating in in the 16 . planets city put in places like cycle by the way are you know providing information to the general public as they might wanna know the streets and platinum enough as I get will be doing that will be delivering something this here so I was in a meeting with them an hour and found out that yes we would be developing this application so we hadn't then but anyway we were given it was January we had to deliver something by the end of the winter so is approximately 6 the uh 6 weeks to get this out some of the so with that you know the 1st thing so to do was you know think about
things right so what exactly should we develop here because really all we had was a sound bite from any of the radio show and so you would track lousy with the with progress of plows reliable want you just look out your window because you have visual visual as evidence is looking outside window yes and no right you really need to look at a smartphone a web browser on your desktop or laptop to see that street and platinum so were scratching our heads trying to figure it out n initially here we
came up with some objectives and what we would try to achieve of so really knew that there was convey the snow operation the residents and visitors of the war organizational Aubrey us no operations because the useful so plows and spreaders right and we need the ability to handle a large volume of traffic right so this is a sort of and an incident where there's a snowstorm the apple application gets activated everybody's gonna come on in your the duration of the snowstorm is maybe a day a lead on for a bit longer just to show the street you continue to be plowed but it's a very short events a lot of traffic and any 1 given time so high availability of support a lot of applications excuse me in other users really keep it struck simple and straightforward right convey information understandable way to the lay public and it was really about conveying information not really about the technology behind the scenes direct so we wanted to build deliver them with other the minimal viable product right go out initial release in meeting most of the undefined requirements and then go out with future releases and build upon so was the total team effort in what you mean by a a cadre of very developers of so we did deltas entirely in-house by using the tools that we already been comfortable with so learning on the job new technologies when you have a very aggressive time aligned is not advisable i n on e existing infrastructure which was frail and aging but we had to make it work for and it's the 1st Winter why we hardened things the next so essentially the project became 2 separate al efforts right you had the mayor's mandate and then you have a variety of stakeholders partner sanitation the streets Office of Emergency Management managers snow emergencies they handle communication and then you had city hall and then had my team the development effort rights of 1 taking more of a waterfall let's me every other week pontificate talk you know for occasional requirements out there and the development team taking more of an agile approach we had the least grounds were actually building what they were trying to formulate in their heads right so the 1st thing we really did was let's take a
surveillance of the of the of the lay of the land is out there on with with some of the other cities are doing right see along the top Chicago there on the rate I'm not to be critical but OK so seeing acute plow I can't dance around on the screen doesn't really tell you what's happening out there doesn't tell you whether you street it's cute and all my make a better game but it certainly doesn't convey information on the top over here you know the plow and more buildings and the our street so but we looked at those and we realize there was a really anything out there on that really helped inform what it is that we should be doing
so we brought you some other challenges in place right so we need to realize the vision of from a person we really have access to the near was in this in any of our meeting just said we had to do something we had a whole that we hit the target or weren't too far off the target and just according to multiple stay coop holders the dreaded decision by committee of when decisions are reached and they focus on the new show like colors and things like that as opposed to like the functionality of the application of what we really want to do initially was we would let's let's track progress against schedule for our right so plows are given schedule streets that the news about their or out and use what you do for the day and month track track their progress against what they to cover the the well those plow routs were actually in narratives in word perfect documents to just give you some indication of how well they work right that and if you read them and try to map them there were huge gaps between where they started where they made left turns in its can I wonder how the streets actually a cloud of that's the narrative the folic so quickly we realize that was in the work right the so we had to come up with something else we had a very aggressive schedule and then we need to handle a large volume of traffic in in in a fairly short the period of so the fact that so we started doing some initial
visualizations of the GPS points right creating vectors and showing most of arrows there the bearing of the vehicle of any the 1 going down 2nd and you would pretty good the 1 on 1st Avenue you that the driver of the plow was drinking that day you the person drawing that fake line I had made much to drink that nite or there is a multiple of so so it was sick but anyway we looked at that and realize that's similar to what Seattle was done and this is not gonna really be very helpful right so but we realize that we the entirely different approach and while we decided to do a
natural transition to yet but anyway will we tried to do was or what we did do I was we see that those GPS points to the street segment they should have been plowing and we had some intelligence in there in terms of looking at the directionality of street the bearing of the plow looking at previous segments to ensure you were automatically making a quick return right so there's an intelligent and algorithms snap to the nearest street segment and then we created time buckets so we're showing here when the street was previously plowed whether it's about the last hour of the last 12 24 hours on and this was a decision in terms of the time bucket that took ever for the Committee make but anyway this is what we worked with so we get GPS theme of and this is that so that sucks
right so this is the 1st iteration rare and so what happens here is we you here the were standing as a so that street segment which has a unique identifier every 15 minutes were holding the GPS data store n looking at the last problem-set stamp on a street segment and then putting those into the different buckets and then rendering it and I'll give you sort of technology torn between how that's been done but this was probably release 1 . 5 right it was a there was an earlier release but this is this still sort of is somewhat emulating the sort of GIS desktop with the concept of layers on the right a bit more functionality problem wanted right so this is the sort of of more
recent release right it's a completely responsive mobile compliant right get away from having turning layers on and off right is 2 different things that you can say no view activity right took this screenshot recently so we worked snowplows cities but also shows the designation of each streets whether their primary street secondary tertiary meaning plot 1st 2nd and 3rd or all bets are off we don't cover your street so so this is the current application of only and has a controls as well as annotation to activate the right hand side it'll it'll indicate whether it's active or not
so here we review the top right and tell you it's current as of what date time and then the next time the ETL runs the data and ended
so is that looks on the mobile
of Android device so the the so the the the variety of technologies that we use in our and 1 really getting that really is a real plus for open source if we had worked with 1 of the proprietary are solutions that shall go nameless but we ran into a defect 1 of the core products were working and balance provide support to us tips for contract with them with an engineer on site within 24 hours we had those defects corrected and posted back to GeoServer within 48 hours and they gave us a branch of the code which we had deployed within 3 days so I had that it happened with 1 of that of 1 of those unnamed vendors that would have been probably months not days right so as a real log for open source right there so is the rise the is we use the spring bracht to write the ETL I you from most known with all the other ones these optimizer content delivery networks were caching all the contents of to that closest to from the end user so how does it work and the previously but I had a very simple non-technical graphic that I
did to explain this to managers within the City of New York right so here's the data flow years helping workers they're all like wow has the work there are GPS devices within the which initially started out as being essentially cell phones and now we're were embedding them with full AVL with on dead reckoning of it goes up to a is or database which is it's a partner of Verizon right which then goes to nice when which is our internal wireless network they have a data center where they get all the GPS feeds coming in at the red line is we're really and my he took over right we wrote that the quality of management services you here runs every 15 minutes it's polling the database looking at each segment and the time stamp on every 1 of those populates of the detail runs populates a table we have another view on that table that tells us all the segments where there where the time stamp had changed we then stands and the you know juice GWC requests to render reader new tiles everything style but the Men's me serializes tiles store them on this I and then all that content is in the test again on optimize the 1st time something comes in tights for you know whatever Street address all those files that are being rendered than cast so the next within the next 15 minutes all that content is then coming from the content delivery network and not coming back from a service so that multiple levels of caching involved just to ensure that we can handle the volume of traffic that we I and we can say a fair bit so but
the presentation would wouldn't would be nothing without a certain number of steps right you have to read the small print of a year-on-year answered by itself read that 1 but anyway so fear in theoretical maximum on a 24 hour period given the number of at times that the ETL runs in the number of levels that we have there is a up to 190 million tiles that get regenerated in 24 hour period so those servers along right we get during at a big activation we probably we get anywhere from 1 to 2 million visits so a lot of traffic coming in there's 10 thousand kilometers the roads from getting lane miles from each 4 times that for what they have to plow there's 20 100 snow vehicles mixture of files and spreaders so fair so large volume of GPS data GPS data from American 2nd where people database in a 15 minute cycle of variant effect so lessons learned about somebody's learner and from projects a lot of these were just a reinforced right you always have to listen listen listen of new dealing with a committee of it's it's certainly a challenge communications always a k bread and then in terms of more on the technical side of you know putting out the minimum viable product especially not in an aggressive timeframe is unit is really key get which can out there and then add to it it's easier to add than it is to subtract of and then there you go with what you know especially an aggressive schedule what we had you know don't don't pick technologies that you haven't used before your experimenting use what you comfortable with you know hopefully you have processes in place that when an emergency hits you're able to respond quickly and you have some qualified individuals behind you of and I certainly do so the the the euro's this but why fight and flaky down here so maybe attempted but this new really really nothing to say but you know if you if you want to in entropic this presentation on their next time we activate during a snow emergency you can check and hopefully things are moving along I'm and hopefully in the foreseeable future when we do actually have no routs will be able to track progress at you where they've been against where the expected debate on so somebody can come in and say OK my streets can be plowed at 3 PM right as opposed to our hasn't been done yet you know what's going happen so um with that that's you know my presentation have to answer any questions if yes the I and a yes of
and yeah for the I mean so there there is and there has been cases where we snap to the wrong street and it's it's been shown on public and you
actually some of the papers in the city of written about that right of the but yes there are usually multiple nodes being paying their GPS points coming every 10 seconds is usually there's multiple hits the streets at right there traveling quite slow and about the streets right and when they're going faster there on the highway so we get usually of at least 2 points for a street segment when said the stabbing to linear street we're looking where the plow was previously been right to be doing 50 miles an hour you can't you know banger right very quickly right as you'll see it straight was street and it'll go next so we we hold a certain number of points we then snap and then we move on so we're kind of to doing a bit of smoothing and sort of you know normalizing the data to ensure that we can remove some straight points some of the noise that the really problematic ones are like when I should you 1st Avenue when you have a whole line that's completely off that's when were mistaken right so in that case that was actually a writer bless you we
had we had falsely
from indicated that the adjacent street York Avenue at comply with that 1st and you have several complaining that telling us the streets and plan it had in fact of the matter is we should really need need to do snap the street because if we had real AVL in these vehicles that wouldn't be an issue and that's being implemented this year so was sort of it was 1 of those stopgap measures that need to be done and it lived a bit longer than expected yeah to and yeah we so you know when when something like that happens we have no microphone then if I start
but anyway the question was had he defends something like
that for us it was more with the defendant to internally to the unless he defended it to the press we had to defend what happened to the mayor in it and it really became a strong realization hey the current technology that were using the GPS technology is insufficient you know we could spend a lot of money try to prove the snap-to-grid never going to work the best solution is get a really the l so we took 1 on the chin but we ultimately as you can see the shared flying UAV up yes go ahead yeah within of people very equipped with
GPS or is that installed for the further that was it was being prototyped in a couple of areas and then it was quickly rolled out to all vehicles it wasn't really the L it's more a in a cell phone in a steel box on the dashboard you know so it was really low tech that now but the key is being collected it been used to optimize the plowing that really equipped so that that's the direction that the heading and so the department sanitation is sort of re-engineering how they do things in their caps there now going to digitize all the rout they're going to hopefully were tracking progress against those so they're using it not as just a mechanism for informing the public but also to the form themselves and to improve how they plow things the thing about you know the sanitation department is its union but it's a union shop so there's only so much flexibility have and how you change the way they do things on many all handle separate sections so so there's is a degree of autonomy in each 1 of the sections that they're covered analysis person out there with a walking talking helping orchestrate things so yes there will be improvements made but you know sometimes they're limiting factors unions being 1 of for a word I will go toward star and then we might go a word or words that a Google Docs through Microsoft of any last questions that so if I spoke to quickly but with the technical difficulties if it had been that of saying thank you