Bestand wählen
Merken

Scaling for NYC while Tracking Plows

Zitierlink des Filmsegments
Embed Code

Automatisierte Medienanalyse

Beta
Erkannte Entitäten
Sprachtranskript
the just want to go over sort of as a reprise of Mandarin should level set expectations with you decline the audience right so the and this is not a heavy technical of presentation is really intended for managers decision makers and curiosity seekers those Richerson plows those generally interested in what happens in New York City of so if that's what you will feel upset if you going to Italy representations if that hopefully I can stick around and I'll entertain inform you of so with that let's plow ahead it ch ch on sound effect but I couldn't figure it out homeless on that the language so why why should you listen to this presentation of them so a bright regions require different that study out about how we use open-source right from my perspective working in city government at those images story to show demonstrated least how open source can be used in an otherwise of very conservative organization of I see in the federal government there's a fair bit of open source work a lot of city and state governments as a real reluctance to it of because obviously governments are often risk averse and there's a perception that open sources you know everybody's managing this code it's all willing nearly and you should be really careful of but it is also a prevailing belief that if you buy shrink-wrapped software commercial software with paid support around it a better product well the obviously need to do the same due diligence in selecting whatever tools that you use whether it's closed or open source of continuity is an example of using the open source to develop an application of query fairly high profile targets a lotta use of and that is 1 is that so very briefly I work for the city of New York Department of Information Technology and Telecommunications at the mouthful I bring that up where 1 of 50 some odd agencies within the city or mandate is IT services we don't plow the streets we provide services specifically I manage the mapping GIS group within the city and my role in doing that is I manage a group of about 6 16 people split between drug developers official analysts and systems admin type folks how we manage a lot of this stuff geospatial data data for the City of New York we also support applications so what we build is what we support so very careful in what we select and how we build things on the so this
situation is what you would have seen looking at your window on December 27 2010 right it's been referred to as Sonoma getting it was sort of the perfect storm right so the mayor and his 1st deputies were on vacation the Bahamas right back on the snow was rate was very heavy the accumulation was really a high volume and a temperature was perfect squared off the streets of warm it's a sort of human accumulating on the streets very quickly this was every variable played into a bad storm really difficult to plow people lost their jobs because of it on ambulances got stuck with other busses get stuck so on and so forth so the project alone if
you that for fast forward about a year January 2011 mayor bloomberg does a weekly radio show and the interviewer start asking questions about snow mediating in in the 16 . planets city put in places like book by the way are you know providing any information to the general public as they might wanna another the streets and cloud enough as I get will be doing that will be delivering something this year so I was in a meeting within an hour and found out that yes we would be developing this application so we hadn't then but anyway we were given here was January we had to deliver something by the end of the winter so is approximately 6 the 6 weeks to get this out from other so with that you know the 1st thing started to do was
you know think about things right so what exactly should we develop here because really all we had was a sound bite from a from a radio show up and so you would attract lousy with the with progress of plows reliable why don't you just look out your window because you have visual visual as evidence is looking outside window yes and no right you really need to look at a smartphone a web browser on your desktop or laptop to see that street and platinum so were scratching our heads trying to figure it out
on n initially there we came up with some objectives and what we would try to achieve it so really knew that there was convey the snow operation progress to the residents and visitors the new optimization Aubrey us no operations because the useful so clouds and spreaders right and we need the ability to handle a large volume of traffic right so this is the sort of thing at incident where there's a snowstorm the apple application gets activated everybody's gonna come on in the duration of the snowstorm is maybe a day a on for a bit longer just to show the street you continue to be plowed but it's a very short events a lot of traffic at any 1 given time so high availability of support a lot of applications excuse me users really keep it struck a simple and straightforward right convey information understandable way to the lay public the and it was really about conveying information not really about the technology behind the scenes so we wanted to build deliver them with of the minimal viable product right go out initial release but meeting most of the undefined requirements and then go out with future releases and build upon that right so was the total team effort of on what to learn that right of a cadre of very developers and so we did deltas entirely in-house by using the tools that we already been comfortable with so learning on the job of new technologies when you have a very aggressive time aligned is not advisable and on the existing infrastructure which was frail and aging but we had to make it work for and it's the 1st Winter why we pardon things the next so essentially the project became 2 separate L efforts right you had the mayor's mandate and then you have a variety of stakeholders partner sanitation streets Office of Emergency Management manages snow emergencies they handle communication and then you had city hall and then had my team the development effort right of 1 taking more of a waterfall let's meet every other week pontificate talk you know for occasional requirements out there and the development team taking more of an agile approach we had least grounds were actually building what they were trying to formulate in their heads right so the 1st thing we really did was let's take a
surveillance of the of the of the lay of the land is out there on with with some of the other cities are doing right Seattle atop Chicago there on the right are not to be critical but OK so seeing acute plow I can't dance around on the screen doesn't really tell you what's happening out there it doesn't tell you whether you street flatten it's cute and all my make a better game but it certainly doesn't convey information on the top over here you know the plow and more buildings and the our street so column we looked at those whom we realized there was really anything out there on that really helped inform what it is that we should be doing so and we
brought you some other challenges in place right so we need to realize the vision of from a person we really have access to the mayor was in any of our meetings he just said we had to do something we had a whole that we hit the target or weren't too far off the target and adjust just according to multiple stay code holders the dreaded decision by committee all when decisions are reached and they focus on menu shell like colors and things like that as opposed to like the functionality of the application of what we really want to do initially was we would let's let's track progress against schedule plow right so plows are given of streets that they need about there are out and here's what you do for the day and month track track their progress against what they expect to cover well those plow routs were actually in narratives in word perfect documents to just give you some indication of how old they work right there and if you read them and try to map them there were huge gaps between where they started where they made left turns in its can wonder how the streets actually a cloud of dust the narrative the folic so quickly we realize that was the work right and so we had to come up with something else we had a very aggressive schedule and then we need to handle a large volume of traffic in in a in a fairly short of time so the fact that so of we started doing some initial
visualizations of the GPS points right creating vectors and showing of arrows there the bearing of the vehicle of any on the 1 going down 2nd Avenue was pretty good the 1 on 1st Avenue you that the driver of the plow was drinking that day is the person drawing that fake line I had made much to drink that nite or there is a multiple of it so I it was sick but anyway we looked at that realize that similar to what Seattle was done and this is not gonna really be very helpful right the so what we realize we we needed the entirely different approach and while we decided to
do a natural transition to
yet but any will we tried to do was or what we did do I was we see that those GPS points to the street segment they should have been plowing and we had some intelligence in there in terms of looking at the directionality of the street the bearing of the plow looking at previous segments to ensure you were automatically making a quick right turn right so there's an intelligent in algorithms snap to the nearest street segment yeah and then we created time buckets so we're showing here when the street was previously plowed whether it's about 1 last hour up to the last 12 24 hours on and this was a decision in terms of the time bucket that took forever for the committee may but anyway this is what we worked with so we have GPS thing of
this is that so that starts so this is the
1st iteration and so what happens here is that we get here the for starting as a so that street segment which has a unique identifier every 15 minutes were holding mentioned years data store and looking at the last time said stand on a street segment and then putting those into the different buckets and then rendering it and I'll give sort of technology torn between how that's been done but this was probably release 1 . 5 right it was a the there was an earlier release but this is this still sort of is somewhat emulating the sort of GIS desktop with the concept of layers on the right a bit more functionality problem wanted right so this is the sort
of of more recent release right it's a completely responsive mobile compliant right get away from having turning layers on and off right is 2 different things that you can say so you activity and I took this screenshot recently so we worked snowplows cities but also shows the designation of each streets whether their primary street secondary tertiary meaning plot 1st 2nd 3rd or all bets are off we don't cover your street so I the so this is the current application of only has the controls as well as annotation to activate the right hand side it'll it'll indicate whether it's active or not
so here we review the top right and tell you it's current as of what date time and then the next time the ETL runs for the data and render
on so this outlooks on a mobile Android device
so thank here and so the there's the variety of technologies that we use our and 1 really good and that really is a real plus for open source if we had worked with 1 of the proprietary solutions that shall go nameless but we ran into a defect in 1 of the core product working on and balance provide support to us tips for contract with them we an engineer on site within 24 hours we had those defects corrected and posted back to GeoServer within 48 hours and they gave us a branch of the code which we had deployed within 3 days so I had that it happened with 1 of that of 1 of those unnamed vendors that would have been probably months not days right so as a real log for open-source right there so is the variety that gods is we use the spring bracht to write the ETL on the problem most known with all the other ones we use optimizer content delivery networks were caching all the contents of to that closest to from the end user so how does it work and the previously but at a very simple
non-technical graphic that I did to explain this to managers within the City of New York right so here's the data flow years helping workers they're all like wow has worked there are GPS devices within the clouds which initially started out as being essentially cell phones and now we're were embedding them with full AVL with on dead reckoning so it goes up to a is or a database which is it's a partner of Verizon right which goes to nice when which is our on internal wireless network have a data center where they get all the GPS feeds coming in at the red line is really and my slide he took over right we wrote that what you know expenses are UCL runs every 15 minutes it's polling the database looking at each segment and a time stamp on every 1 of those populates of detail runs populates a table we have another view on that table that tells us all the segments were there where the time stamp changed wouldn't send and geo juice GWC request to render reader new tiles with everything style and I then SME serializes tiles whether this I and then all that content is then I can again on optimize for the 1st time someone comes in tights for you know whatever street address all those files that are being rendered cast so the next within the next 15 minutes all their content is then coming from the content delivery network and not going back from a service so that multiple levels of caching involved just to ensure that we can handle so the volume of traffic that we see so we do see a
fair bit so that the presentation would wouldn't would be nothing without a certain number of steps right you have to read the small print of a year-on-year search by itself that 1 but anyway so fear in theoretical maximum on a 24 hour period given the number of at times that the ETL runs in the number of the levels that we have there is a up to 190 million tiles that get regenerated in a 24 hour period so they serve as a humming along right we get during at a big activation we probably we get anywhere from 1 to 2 million visits so a lot of traffic coming in there's 10 thousand kilometers were the roads from to lane miles from E 4 times that for what they have to plow there's 20 100 snow vehicles mixture of clouds and spreaders so fair so large volume of GPS data GPS data from in every 10 seconds were poet database and a 15 minute cycle of president
the so lessons learned somebody's learner from uh projects a lot of these were just a reinforced right you always have to listen listen listen of new dealing with a committee of it's it's certainly a challenge communications always a k bread and then in terms of more on the technical side of you know putting out the minimum viable product especially in it in an aggressive frame is unit is really key get which you can out there and then add to its it's easier to add than it is to subtract rate of and then you go with what you know especially an aggressive schedule what we had you know don't don't pick technologies that you haven't used before your experimenting use what you comfortable with you know hopefully you have processes in place that when an emergency hits you're able to respond quickly and you have some qualified individuals behind you know and I certainly do so the the euro's this but why fights and flaky down here so maybe attempted but others new written really nothing to say but you know if you if you want to in entropic this presentation on their next time we activate during a snow emergency you can check and hopefully things are moving along I'm and hopefully in the foreseeable future when we do actually have snow routs will be able to track progress but you know where they've been against where the expected debate on so somebody can come in and say OK my streets can be plowed at 3 PM right as opposed to our hasn't been done yet you know what's going to happen the so with that that's you know my presentation have to answer any questions that the yes the yes
and and you the the I mean so there various there has been cases where we snap to the wrong street and it's it's been shown on public and you know actually some of the papers
in the city of written about that right of the but yes there are usually multiple it's being paying their GPS points coming every 10 seconds is usually there's multiple hits the streets at right there traveling quite slow and about the streets right and when the going faster there on the highway so we get usually of at least 2 points for a street segment when said just having to street we're looking where the plow was previously been right to be doing 50 miles an hour you can't you know banger right very quickly right because you'll see it straight was street and it'll go next so we we hold certain number of points we then snap and then we move on so we're kind of to doing a bit of smoothing and sort of you know normalizing the data to ensure that we can remove some straight points some of the noise that the
really problematic ones are like when I should you 1st Avenue when you have a whole line that's completely off that's when were mistaken right so in that case that was actually a right up bless you we had
we had falsely from indicated that the adjacent street York Avenue at comply with fat 1st and hadn't arose complaining that telling us the streets and plot it had in fact of the matter is we should really need me to do snap the street because we had real ADL in these vehicles that wouldn't be an issue and that's being implemented this year so was sort of it was 1 of those stopgap measures that needed to be done the and it lived a bit longer
than expected yeah to you yeah we so you know when when something like that happens we have no microphone then start but anyway the question
was had defends something like that for us it was more
with the defendant to internally to the mayor unless he defended it to the
press we had to defend what happened to the mayor in it and it really became a strong realization hey the current technology that were using the GPS technology is insufficient you know we could spend a lot of money try to prove the snap-to-grid it's never going to work the best solution is get a really the l so we took 1 on the chain but we ultimately as you can see a shared flying UAV up yes go ahead get rid of people very quick with the GPS
or is that installed for the further that was it was being prototyped in a couple of areas and then it was quickly rolled out to all vehicles it wasn't really L it's more a in a cell phone in a steel box on the dashboard you know so it was really low tech that now but the core is being collected been used to optimize the plowing let's really equipped so that that's the direction that they're heading and so department sanitation is sort of re-engineering how they do things in their caps there now going to digitize all the rout they're going to hopefully were tracking progress against those so they're using it not as just a mechanism for informing the public but also to the form themselves and to improve how they plow things the thing about you know the sanitation apartment is its union so it's a union shop so there's only so much flexibility have and how you change the way they do things on men they all handle separate sections so so there is a degree of autonomy in each 1 of the sections that they're covered analysis person out there with a walking talking helping orchestrate things so yes there will be improvements made but you know sometimes they're limiting factors unions being 1 of of a word I will go WordStar and then we might go word or words that Google Docs through Microsoft of any last questions so if I spoke to quickly but with the technical difficulties of
in the middle of saying that you
Bit
Selbst organisierendes System
Selbstrepräsentation
Formale Sprache
Gruppenkeim
Kartesische Koordinaten
Kombinatorische Gruppentheorie
Code
Computeranimation
Übergang
Erwartungswert
Datenmanagement
Software
Perspektive
Datentyp
Softwareentwickler
Figurierte Zahl
Analytische Fortsetzung
Bildgebendes Verfahren
Soundverarbeitung
Open Source
Abfrage
Profil <Aerodynamik>
Physikalisches System
Biprodukt
Dialekt
Entscheidungstheorie
Mapping <Computergraphik>
Dienst <Informatik>
Menge
Rechter Winkel
Aggregatzustand
Perfekte Gruppe
Verbandstheorie
Rechter Winkel
Bildschirmfenster
Kartesische Koordinaten
Projektive Ebene
Information
Spezifisches Volumen
Bitrate
Streuungsdiagramm
Quick-Sort
Computeranimation
Telekommunikation
Total <Mathematik>
Browser
Minimierung
Kartesische Koordinaten
Inzidenzalgebra
Computeranimation
Demoszene <Programmierung>
Arithmetische Folge
Prozess <Informatik>
Notebook-Computer
Bildschirmfenster
Spezifisches Volumen
Softwareentwickler
Schreib-Lese-Kopf
Trennungsaxiom
Nichtlinearer Operator
Deltafunktion
Gebäude <Mathematik>
Hochverfügbarkeit
Biprodukt
Quick-Sort
Ereignishorizont
Objekt <Kategorie>
Verbandstheorie
Rechter Winkel
Projektive Ebene
Information
Smartphone
Streuungsdiagramm
Varietät <Mathematik>
Lineares Funktional
Nabel <Mathematik>
Gebäude <Mathematik>
Routing
Kartesische Koordinaten
Code
Computeranimation
Entscheidungstheorie
Scheduling
Weg <Topologie>
Multiplikation
Perfekte Gruppe
Arithmetische Folge
Verbandstheorie
Rechter Winkel
Spieltheorie
Wort <Informatik>
Indexberechnung
Spezifisches Volumen
Information
Kantenfärbung
Streuungsdiagramm
Touchscreen
Multiplikation
Druckertreiber
Punkt
Rechter Winkel
Gruppenoperation
Visualisierung
Zeitrichtung
Vektorraum
Gerade
Computeranimation
Algorithmus
Punkt
Rechter Winkel
Trigonometrie
Term
Computeranimation
Entscheidungstheorie
Richtung
Arithmetisches Mittel
Mailing-Liste
Bit
Rechter Winkel
Gamecontroller
Iteration
Kartesische Koordinaten
Plot <Graphische Darstellung>
Identifizierbarkeit
Speicher <Informatik>
Quick-Sort
Quelle <Physik>
CDN-Netzwerk
Offene Menge
Web Site
Open Source
Minimierung
Stapelverarbeitung
Verzweigendes Programm
Orakel <Informatik>
Biprodukt
Code
Computeranimation
Eins
Design by Contract
Summengleichung
Quelle <Physik>
Rechter Winkel
Volumenvisualisierung
Speicherabzug
Inhalt <Mathematik>
Gravitationsgesetz
Varietät <Mathematik>
Mittelwert
CDN-Netzwerk
Extrempunkt
Minimierung
Hochdruck
Adressraum
Zahlenbereich
Kombinatorische Gruppentheorie
Zählen
Räumliche Anordnung
Physikalische Theorie
Computeranimation
Übergang
Rechenzentrum
Datenmanagement
Schätzung
Statistische Analyse
Volumenvisualisierung
Zeitstempel
Inhalt <Mathematik>
Spezifisches Volumen
Gerade
Sichtenkonzept
Datennetz
Datenhaltung
Zwei
Elektronische Publikation
Frequenz
Datenfluss
Zusammengesetzte Verteilung
Rechenschieber
Dienst <Informatik>
Rechter Winkel
Parkettierung
Dreiecksfreier Graph
Serielle Schnittstelle
Streuungsdiagramm
Tabelle <Informatik>
Telekommunikation
Prozess <Physik>
Rahmenproblem
Extrempunkt
Routing
Biprodukt
Kombinatorische Gruppentheorie
Term
Scheduling
Einheit <Mathematik>
Arithmetische Folge
Rechter Winkel
Projektive Ebene
Bit
Punkt
Rechter Winkel
Zwei
Geräusch
Zahlenbereich
Vorlesung/Konferenz
Glättung
Gerade
Quick-Sort
Eins
Vorlesung/Konferenz
Plot <Graphische Darstellung>
Einflussgröße
Quick-Sort
Vorlesung/Konferenz
Verkettung <Informatik>
Vorlesung/Konferenz
Kraftfahrzeugmechatroniker
Quader
Routing
Quick-Sort
Teilbarkeit
Kugelkappe
Überlagerung <Mathematik>
Richtung
Bildschirmmaske
Minimalgrad
Arithmetische Folge
Flächeninhalt
Vorlesung/Konferenz
Wort <Informatik>
Garbentheorie
Speicherabzug
Analysis

Metadaten

Formale Metadaten

Titel Scaling for NYC while Tracking Plows
Serientitel FOSS4G 2014 Portland
Autor Reilly, Colin
Lizenz CC-Namensnennung 3.0 Deutschland:
Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen.
DOI 10.5446/31634
Herausgeber FOSS4G, Open Source Geospatial Foundation (OSGeo)
Erscheinungsjahr 2014
Sprache Englisch
Produzent FOSS4G
Open Source Geospatial Foundation (OSGeo)
Produktionsjahr 2014
Produktionsort Portland, Oregon, United States of America

Inhaltliche Metadaten

Fachgebiet Informatik
Abstract In the winter of 2012, NYC's Department of Information Technology and Telecommunications (DoITT) was tasked with developing an application to track snow vehicle operations. The DoITT GIS team was given a mandate to have the application in production before the end of the winter. Due to the aggressive schedule, our approach was to get something up as quickly as possible while enhancing and improving over time. Beyond the schedule constraint, additional challenges were minimal requirements and decision-making by committee with no clear business owner.Three major tasks were required to complete the project: scale the existing infrastructure to better handle the expected demand, determine an approach for communicating the information to the public in a legible and understandable way, and develop and test the application. The team quickly undertook a multi-pronged approach to complete these tasks within a roughly two-month timeframe.Of all the impossible tasks, scaling the infrastructure was the most challenging and difficult. High-profile application launches in NYC that come with press announcements tend to garner traditional and social media coverage and with that national exposure and demand. And although the application would have been a perfect candidate to deploy in the cloud, that was not an option. Additional servers were added and the application was optimized and tuned for performance. To do so, multiple-layers of caching were employed including GeoWebCache and a Content Delivery Network. In terms of visualizing the data, we conducted a quick review of existing public-facing applications. There were not many examples at the time with most cities choosing to show 'breadcrumbs' of a plow's path. We felt this method was not an effective way of conveying plow coverage; our objective being, to show which streets had been plowed and not to show where a plow had been at specific time. As such, we decided on visualizing the data by the time a street was last plowed. Five time-buckets were established and the street segments were color-coded based on the last GPS ping received on the segment. Every 15 minutes an ETL pulls the GPS data and renders tiles using GeoServer and GeoWebCache.The application, PlowNYC, was developed using open source and commercial software and custom code. These include OpenLayers, Geoserver, GeoWebCache, GeoTools and Oracle. Since its release, the application has been enhanced to handle greater traffic, support mobile clients and to simplify the interface. The presentation will cover these aspects of the project.
Schlagwörter PlowNYC
NYC
GeoServer
GeoWebCache

Ähnliche Filme

Loading...
Feedback