An Open Source Approach to Communicating Weather Risks
This is a modal window.
Das Video konnte nicht geladen werden, da entweder ein Server- oder Netzwerkfehler auftrat oder das Format nicht unterstützt wird.
Formale Metadaten
| Titel |
| |
| Serientitel | ||
| Anzahl der Teile | 188 | |
| Autor | ||
| Lizenz | CC-Namensnennung 3.0 Deutschland: Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen. | |
| Identifikatoren | 10.5446/31617 (DOI) | |
| Herausgeber | ||
| Erscheinungsjahr | ||
| Sprache | ||
| Produzent | ||
| Produktionsjahr | 2014 | |
| Produktionsort | Portland, Oregon, United States of America |
Inhaltliche Metadaten
| Fachgebiet | ||
| Genre | ||
| Abstract |
| |
| Schlagwörter |
FOSS4G 2014 Portland152 / 188
7
10
14
15
16
23
24
25
28
29
33
37
39
40
43
45
46
48
50
56
64
65
69
72
74
82
89
91
98
102
107
111
114
118
128
131
132
135
138
141
143
147
149
150
157
158
161
164
165
166
173
174
175
179
185
00:00
Textur-MappingBitZusammenhängender GraphFreewareKartesische KoordinatenSkalarproduktOffice-PaketEinsHypermediaBildverstehenDienst <Informatik>DifferenteHasard <Digitaltechnik>PrognoseKategorie <Mathematik>
00:58
RechenschieberDatenbankPolstelleDesign by ContractDienst <Informatik>MinimalgradFlächeninhaltExogene VariableRoutingDichtefunktionalBildschirmsymbolPlug inAuflösung <Mathematik>DigitalisierungKartesische KoordinatenEinsBitRegulärer GraphOffice-PaketDarstellung <Mathematik>TypentheorieDomain <Netzwerk>PunktZusammenhängender GraphInformationDatenfeldGeradeCASE <Informatik>BeweistheorieMultiplikationsoperatorLokales MinimumPixelBildgebendes VerfahrenKonditionszahlLeistung <Physik>Interface <Schaltung>HypermediaZoomTesselationRechter WinkelAdditionSichtenkonzeptGraphOpen SourceDatenverwaltungQuaderMapping <Computergraphik>Textur-MappingAffine VarietätProdukt <Mathematik>KugelSchnittmengeGemeinsamer SpeicherAbstraktionsebeneTransformation <Mathematik>Prozess <Informatik>ElementargeometrieDateiformatFreewareZeitzoneSchwellwertverfahrenDifferenteInformationsüberlastungMathematikWeb-SeiteURLKonfiguration <Informatik>ZeichenketteSystemaufrufGraphfärbungSelbstrepräsentationHasard <Digitaltechnik>Message-PassingTelekommunikationMittelwertReelle ZahlPhysikalischer EffektShape <Informatik>Arithmetische FolgeTwitter <Softwareplattform>ZeitrichtungInterpretiererBitmap-GraphikFluidFront-End <Software>Graphische Benutzeroberfläche
09:50
Dienst <Informatik>RechenwerkDichte <Physik>StrömungsrichtungTextur-MappingBildschirmfensterHasard <Digitaltechnik>DatensatzZusammenhängender GraphNormalvektorInformationMultiplikationsoperatorURLProgrammierparadigmaGraphfärbungRechter WinkelZeitrichtungFlächeninhaltObjekt <Kategorie>FrequenzLaufzeitfehlerAusnahmebehandlungMinimumWeb-ApplikationHilfesystemCodeNichtlinearer OperatorStabilitätstheorie <Logik>StandardabweichungStereometrieEntscheidungstheoriePunktBildverstehenSoundverarbeitungSchreib-Lese-KopfOpen SourceZeitreihenanalyseDifferenteVersionsverwaltungPunktspektrumProdukt <Mathematik>DatenfeldRichtungVektorpotenzialPlotterInteraktives FernsehenBereichsschätzungGraphKonzentrizitätDatenbankHypermediaGeradeArithmetisches MittelDiagrammSkalarproduktQuick-SortQuaderPolygonGüte der AnpassungMapping <Computergraphik>Coxeter-GruppeWort <Informatik>Bildgebendes VerfahrenBitMetadatenStatistikGruppenoperationTabelleHeegaard-ZerlegungNatürliche ZahlSummierbarkeitSelbst organisierendes SystemKette <Mathematik>Kategorie <Mathematik>FreewareInterface <Schaltung>Total <Mathematik>MaschinenschreibenSchnittmengeExtreme programmingZellularer AutomatZählenDefault
18:42
Partielle DifferentiationRückkopplungWeb-SeiteDatenverwaltungKartesische KoordinatenGruppenoperationHilfesystemMereologieGarbentheorieMultiplikationsoperatorOpen SourceEntscheidungstheorieVideokonferenzEreignishorizontElektronischer ProgrammführerStreaming <Kommunikationstechnik>ZweiInformationServerPunktspektrumPerfekte GruppeDemo <Programm>VisualisierungSondierungTypentheorieRechter WinkelFormale GrammatikFormation <Mathematik>Komplex <Algebra>Varietät <Mathematik>Dienst <Informatik>Nichtlinearer OperatorBitDifferentePhysikalisches SystemMenütechnikSchnittmengeGeradeVorlesung/Konferenz
Transkript: Englisch(automatisch erzeugt)
00:00
A little bit of background on the U.S. National Weather Service. Our mission is to save lives and property, so this definitely melds well with that. And our vision is to build a weather-ready nation where we give basically everyone the ability to prepare and respond appropriately to weather-related hazards.
00:21
And the weather service is pretty wide stretch. So there's 122 different weather forecast offices around the country, and each of these red dots on this map here is one of those forecast offices. And then there's also six regional centers, which are probably the most publicized ones, such as the Storm Prediction Center in Norman, Oklahoma,
00:42
and the National Hurricane Center down in Miami, Florida. Those ones get a lot of media attention, but there's a lot of other components in here. But I guess the gist of it is that all these places are generating a lot of free data that is – or that can be used in GIS applications.
01:01
The U.S. is privy to a really hazardous slew of weather. All these different things combine to basically cause about 650 deaths every year in the average year of U.S. weather, and about $15 billion in damages. And the graphic in the lower left-hand corner here shows the weather-related cost
01:24
to society from 1980 to 2010. And regardless of the trend or what's causing it, weather is becoming more and more impactful to the country every year. So there's a few issues with weather data, the first one being communication.
01:40
Communicating risk is a very difficult process for forecasters to get their message to the public and have the public take the proper response to that potential risk. And there's a lot of reasons for this. A lot of people just interpret data differently. And then other people have different risk thresholds.
02:02
So those are two of the big ones. And then there's the false comfort zone, where basically it's something we call warning fatigue, or you get nine tornado warnings for your house, but nothing ever happens. But on the 10th time, it gets you, and that's what we refer to as warning fatigue. And weather is a pretty abstract concept.
02:24
You can think four-dimensional on this, where you have a transient fluid wrapped around a rotating sphere. A lot of people have a hard time comprehending how all that works, and it's pretty chaotic too. So I'll finish that up with data format. There's a lot of free and plentiful weather data out there, but until recently,
02:43
it hasn't been in a very usable format, especially not in a GIS format. So it's getting better, and the product that I've developed is called the Enhanced Data Display. It's taken all the different types of data sets that the Weather Service offers and put them all into kind of one place.
03:01
I hate saying one-stop shop, but that's kind of what it is, as far as a weather-related one-stop shop. And so this is the interface if you come to that page. That's the URL up there on the top. And here's a few of the highlights of the interface. On the left-hand side, or in the green, is something I call Quick Layers,
03:20
where it gives you access to the most common layers, which you can toggle on and off, change opacity, change the different fields in there. And underneath that is what I call a cornucopia of additional layers, but it's about 350 searchable, filterable, weather-related layers that you can add to the map. And if that's not enough, you can go to the top right-hand corner
03:41
and click on the little tool icon and bring your own data. And bring your own data, you can import KMLs, shape files, WMS, and tile services. So it's pretty flexible in the amount of things it can do. In the red box there, that's a layer manager, which you can control the order and opacity of all the layers.
04:02
And on the very bottom center, there's a little button that allows you to quickly share this view with anyone else, all the layers, the zoom, everything via URL. And I create a tiny URL so it's easily shareable via social media. So a lot of things that go into it. It is kind of a busy interface, but it's very powerful
04:21
at the same time. So the interface is built on open layers for the mapping API. And KUKS do, which is a JavaScript API, does a lot of the, I guess, GUI manipulation and buttons, where I use FLOT and just started recently
04:40
using D3 for the plotting of different meteorological fields. And the back end is basically a post GIS database and postgres. And GDAL does the geometric transformations of a lot of those kind of old data sets into kind of a more spherical locator projection.
05:02
So there's a lot of components that go into this application. One of the biggest ones is the National Digital Forecast Database, or the NDFD. This data is being generated by all the forecasters at all the weather service offices across the country that I showed you on a few slides back. And this is an example for last winter
05:20
when we had a big snow and ice storm down in the southeast of the US here. And just shows you some of the things that are potentially possible with that database. And it's a pretty gigantic database. Extends from Western Africa all the way to Eastern Asia and from Central South America all the way up to about the North Pole.
05:40
And so it's a gigantic database with a 2 and 1 half kilometer grid. So it's very high resolution. This image right here is of Portland. And Mount Hood is that little 50 degree pictural of maximum temperature. The database goes out seven to eight days with hourly grids
06:01
out to three days. And it has about 3 million grid points per domain, per grid, with about 40 different fields. And all those are updated hourly. So you can imagine how much data that is actually going into this. And if you want more information on this, I'm going to put a plug in for Tim Campisti's talk tomorrow at 10.30. He's going to go over how he rapidly pushes all this data
06:23
in and out of this database. So be sure to catch that if you're interested. So I leveraged that database to make different components in the edDisplay. And one of them is the travel hazard forecast, which shows you the hazardous weather color coded
06:43
along the line, along the travel route, at the time that you arrive at that specific location. And the way that works is you can quickly right click the map and set a start point or an end point. And if you want to, you can add a waypoint too. And that'll bring up this little GUI
07:01
and actually fill it out for you. Or alternatively, you can just bring this up by itself and search your own locations. And different options such as when you want to leave. And once you actually have data in there, there's a few other things that you can add, such as display options and stuff. But in the end, you're going to end up with that kind of forecast where you start in New Jersey,
07:22
go through El Paso, and hit Portland. And what I have displayed here is temperature along the route. And the icons denote different weather types that you can hit across the way. So right now, the orange is just thunderstorms. But I think I'm going to add a little more granularity to that in the future. So I can depict severe thunderstorms versus just
07:42
your regular ordinary thunderstorms. And then there's a little, you can see the red text kind of over in the West Virginia area. I threw a filter on there for just highlighting places where the temperature is above 85 degrees, just for display purposes. And the data I was making, I couldn't find a whole lot of really good weather.
08:00
So how does that whole thing work? Well, it uses geolocation to grab the start and end points. And then I feed routing service. Right now it's Google Maps, but that'll probably be going away because we're losing our contract with them. So I plan to convert that to more open source feeds. And I think MapQuest has one that I've just started to investigate. And then I use the response from that routing service
08:22
to calculate the arrival times along the route. Then I query the database at each of those points and basically color code the segments by the worst conditions on either side of the point. And then I display the values as a feature. And those are what the arrows are pointing to.
08:41
But as you can see, this kind of was a proof of concept thing. And I got it working, but then I found out that there's some flaws. So in this case right here, there's a high wind warning out on a segment of, or across a segment of your route. And it didn't pick it up because I was sampling on both sides of that line. And so what I thought is if I start sampling along the line
09:02
segments between those points, that I'll get a lot better representation to the public. So here's an example call of doing such a thing. And again, I'm still developing this, so it's still a work in progress. But this grid right here represents the 2 and 1
09:21
half kilometer raster, NDFD grid that the NDFD is based on. And then the line segment is one of those paths between the points. So the first thing I do is I clip the giant raster down to this smaller raster right around the line segment there. And that's using the ST clip from post GIS.
09:43
And the coordinates of that line string go into that geom from text call. And then from there, I intersected at this specific valid time. That's one big assumption is that your time along this line is all valid at the same time. That's where valid time is equal to such and such date.
10:04
That is a pretty big assumption, but at the same time, I think it's a really good assumption because those line segments are generally going to be less than an hour, and all of our grids are an hour long. So that's all right. So if I do intersect that with that clipped raster,
10:21
then I pull out all the data points that that line feature touches. And then I just union them up by basically merging all those together, and then I do a value count to pull out the hazards or whatever else I'm interested in along there. And the cool thing is the whole total execution time for that is about three milliseconds. So very, very quick.
10:42
Another component to the interface is the impact hazards where basically what you see now on if you ever see a weather service warning is a polygon. And so this one adds a little bit more flexibility to it where you get the name of what it is, what time it's valid to, and a whole hazard watch GUI there that you can sort by end time, population affected,
11:03
and so on and so forth. And it's all interactive too. So if you mouse over one of these features, you'll get all that metadata that's associated with that warning there, including impact data. And there's a little bit more information you can get if you click on the little red target in that table.
11:21
It'll bring up something that Brian Wallowender and his group have been working on. And this is something that's really powerful because it shows the radar data on the left with the warning, but also kind of a heat map of population density on the right hand side. And all the population impacted statistics are on the bottom. This is really useful for FEMA.
11:42
And since it is an image, you can tweet it out via social media and get the word out really quickly to a lot of people. So very powerful stuff there. And the way that he does that is he uses the Landscan database, which is a one kilometer grid of the ambient population, basically where people are during the day.
12:02
And then he also uses the 2012 HTSIP Gold data set for public venues and the National Transportation Atlas database for highways, rails, and national parks. And he rasterizes the warning polygon that we have and then sums up all of the data within there to get those impact statistics that I was showing you.
12:22
So really, really neat stuff there. And again, this is a pretty powerful interface. And it's kind of built for the decision makers. So one thing they would be concerned about is hazardous material releases or something of that nature. So NOAA runs a high split model, which is a model that predicts where these chemicals will go if there is such a release.
12:42
And so there's an example of a fictitious release from Chicago. And this shows the time of arrival to certain destinations along the way. And this one right here is just the concentration along there. And again, this is all within ed that you can interact with. Another thing decision makers told me
13:01
about when they were out there is they really want to know a confidence in a forecast that we have. And so I came up with this kind of plume diagram that shows at least the probability of exceeding a certain amount of rainfall. These are all percentiles in there. And the mean is the solid black line there.
13:21
And similarly, you can throw on the National Weather Service forecast and throw in a couple other different fields there. And it really gives them a good idea for what our forecast is and how that relates to what the models are saying. Another thing is called the model spectrum. This is all, again, within ed. And these show up as points on the map. So you click them. And you can get a lot of weather data out of them.
13:42
This one right here is really good for extreme heat and cold, where the background colors are the record highs, record lows, normal highs, normal low temperatures. The dark blue dots are the NWS forecast. And the box and whiskers are the model spread at a certain point for that period.
14:02
And it's colored by the standard deviation. So the warmer colors are, of course, a lot higher spread. And the greener or cooler colors are low spread. So when they're green shaded, you're thinking that's a pretty high confidence forecast, where it's red, you might not be sure. Like that very last one over there on the right-hand side
14:20
in purple, the model is saying you might get record heat that day. But the Weather Service is saying you're going to be above normal, but probably not record. So a lot of really interesting stuff in there. If you click on the map just anywhere, what you'll get is this little window here pop up. And what it'll tell you is a little Gantt chart at the top
14:42
shows the current active hazards and it watches morning's advisories in effect for your point, and what period they're in effect for. And then the first tab is the default one, which just gives you a forecast. But if you switch over to the hourly graph tab, that's where I tap into the NDFD database again. And I plot a mediogram, or basically a weather time
15:02
series, of data from all these different potential fields that you can plot and toggle on and off down here and interact with. So if you have the show wind rose button checked, it'll pop up a wind rose over there. If you don't know what that is, it's basically what direction is the wind coming from
15:20
and what frequency and what speed is it coming from. And I have to really think Nelson Minar, who is also in the room, for the code to do that. And you can actually click and drag a swipe on that graph there, and it will update all of the fields and kind of summarize that data for you. So again, really powerful stuff within there.
15:43
And one of the really cool things is that open source technology is allowing us to change the present warning paradigm of the weather service. Right now, we're in what I call a binary warning, where anything inside of this big yellow polygon, you're warned.
16:01
And anything outside of it, you're unwarned. And it doesn't really have a lot of good information, except for the fact that you know you're warned. The person in the top left-hand corner has the same exact chance of getting hit by the storm as the one in the bottom right-hand corner. But as a meteorologist, you know that's not true. The person in the path of the storm
16:20
has the highest chance. So what we're doing is we're using open layers on the right-hand side to draw a threat area, or we're calling them threat objects, around a certain storm. And then we're propagating that out in time using a Gaussian, which is the typical damage path of a storm.
16:40
And what that gives you is a whole lot more information about that cell. You can calculate the probability of that storm hitting you and the time of arrival of that storm cell in your specific location. So that's a huge improvement over what we're currently feeding out. So that's just us changing the warning paradigm
17:02
using open source tools. Here's a bunch of screenshots from the enhanced data display. I just kind of cherry picked these over the last year or so of different things that you can do with it. And there is a mobile version, but it needs a lot of work. So I'll just put that giant caveat on there.
17:22
So in summary, the open source community has really provided a really stable and solid foundation to build a professional web application. And I underlined support in that first bullet just because without it, this thing would still be a vision in my head. And with all of your guys' help, it's really made this come to fruition.
17:43
So I really thank the community for all the help that they've given me over the past few years. Open source code also enables us to generate new data display techniques and get those into our partners' hands as quickly as possible, speeding up research to operations, which is very important.
18:02
And finally, to end with, I think this is probably the most important point of all, is that it's changing the way that we're communicating weather risks. And ultimately, that leads to saving lives and property. So that's all I have for you. If you have any questions, be free to answer them now. If you want to, you can scan this QR code,
18:21
and it will give you a trip to Seaside, Oregon, or over on the coast this weekend, using ed. So that's all I have for you. Thanks.
18:42
Any questions?
19:04
Yeah, so the question is, can you pull the data from the EDD into an existing system? And that is actually in the works right now. A lot of the data sources in this application come from a variety of different sources. And the Weather Service, they're almost all experimental,
19:22
so they're not really stably supported. But that's changing, because they've set up this group called the IDP, which they actually took all the layers that I have in here, and are using that as a baseline. So when that does come online, you should have access to all of them. And if there's anything in particular that you're interested in, such as radar data,
19:41
I can show you how to do that, because in the help section in there, it has examples of how to pull that into your application partially.
20:03
Yes, so the question is, some of the visualizations are complicated and complex, and he wanted to know if I have gotten around to testing them with actual users. And the answer is yes. I've given this demo of Ed, like a live demo, probably three dozen times to emergency managers
20:22
and other decision makers. And when they first see it, they are overwhelmed. I'll give them that. But a lot of them that have come up to me afterward and started using it say they're priceless. And a lot of them, especially on the mild spectrum one, it's pretty complex, because there's so much data going into that, but they do, in the end of the day,
20:42
really like seeing that information. And it's not too complicated for them. At least that's what they've told me, and that's what a lot of the feedback that I've gotten from this display so far has shown. Yeah, it's really targeted for decision makers,
21:00
but there's a lot of really weather savvy people out there that really appreciate this display, too. And it really has a huge, broad user group, from hospitals to school districts to emergency managers to grandma. So it's a wide user group. Nelson.
21:27
Oh, well, what they do is there's a feedback survey that you can take in the top right-hand corner, and they give you 10 questions of the reliability of the data, how well does it work, do you like it, should it be offered by the Weather Service, stuff like that.
21:41
And they compile all those reviews, and then they make a decision saying, should we continue this, should we get rid of it, or should we keep on modifying it and making it better and then put it out for operation. So it just, oh, I forgot to repeat the question. His question was, how does the Weather Service
22:01
use this kind of application, or get feedback for this type of application? So sorry about that. Yes. The question is, on average, how many times is the application opened? I actually don't know, because the server people
22:20
that I have to go through to get this pushed out to the public haven't given me those stats. But I suspect it probably gets 100,000 hits or so a day, something like that. Any other questions? And to follow up on that, weather data
22:40
is very, it depends on the weather of the day. I mean, a normal day, you'll sit there and take out, but all of a sudden, some big event happens, and all of a sudden, you spike into the millions of hits per hour or second. So any other questions?
23:02
Yes. Yeah, so the question was, if you didn't hear the live stream or me demoing this,
23:21
is there a way to go back and get a feel for how to use it or use the features? Or is there a paper in it? There is a help section in the top right-hand corner of the page. I don't know if I have it up here. Give me one second. Perfect. But it has a whole user demo. There we go.
23:40
So the Help menu, far top right-hand corner, it has videos on how to use the application, has a paper rough draft guide kind of thing that's about 26 pages long of how to use each individual part of it. And it probably needs a little bit of updating. But yeah, so yes, there is.
24:02
Any other questions? Or are we running out of time? Running out of time. All right, thank you.