Simple is Better: An Intro to Event-Driven Serverless Architectures for Faster Disaster Response

Video in TIB AV-Portal: Simple is Better: An Intro to Event-Driven Serverless Architectures for Faster Disaster Response

Formal Metadata

Title
Simple is Better: An Intro to Event-Driven Serverless Architectures for Faster Disaster Response
Title of Series
Author
Contributors
License
CC Attribution 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
2019
Language
English

Content Metadata

Subject Area
Abstract
In a disaster or humanitarian relief context, time obviously matters. This talk will show how, without concern for the details of servers and storage, you can use small amounts of code to quickly build powerful solutions, using a prototype disaster response pipeline as an example.
Keywords General

Related Material

Video is cited by the following resource
Loading...
Architecture Mathematics Web service Dependent and independent variables Event horizon Server (computing) Bit Open set Open set
Software Open source Right angle
Web page Area Dependent and independent variables Theory of relativity Link (knot theory) Channel capacity File format Computer program Set (mathematics) Open set Mass Information privacy Mereology Computer programming Field (computer science) Workload Web service Process (computing) Point cloud Right angle Spacetime
Group action Building Dependent and independent variables Texture mapping Set (mathematics) Water vapor Online help Open set Information privacy Goodness of fit Information Task (computing) Area Dependent and independent variables Mapping Channel capacity Building Moment (mathematics) Computer program Bit Basis <Mathematik> Cloud computing Cartesian coordinate system Connected space Connectivity (graph theory) Quicksort
Satellite Windows Registry Group action Dependent and independent variables Variety (linguistics) Motion capture Set (mathematics) Water vapor Real-time operating system Open set Directed set Software testing Hill differential equation Service (economics) Workstation <Musikinstrument> Dependent and independent variables Distribution (mathematics) Information Windows Registry Open set Distance Software Point cloud Right angle Quicksort Scheduling (computing)
Point (geometry) Building Set (mathematics) Library catalog Price index Information privacy Focus (optics) Entire function Element (mathematics) Physical system Point cloud Satellite Temporal logic Interface (computing) Amsterdam Ordnance Datum Core dump Digital signal Cartesian coordinate system Element (mathematics) Plane (geometry) Subject indexing Process (computing) Point cloud Endliche Modelltheorie Summierbarkeit Quicksort Videoconferencing Extension (kinesiology) Elasticity (physics)
Server (computing) Mapping Interface (computing) Maxima and minima Demoscene
Workload Information management Wechselseitige Information Dependent and independent variables Server (computing) Context awareness Mapping Interface (computing) Interface (computing) Demoscene Newton's law of universal gravitation
Area Category of being Server (computing) Dependent and independent variables Interface (computing) Demoscene
Context awareness Server (computing) Group action Process (computing) Visualization (computer graphics) Interface (computing) Mathematical analysis Quicksort Demoscene Web browser
Server (computing) Building Scaling (geometry) Mapping Open source Server (computing) Fatou-Menge Data storage device Mathematical analysis Bit Dimensional analysis Mechanism design Interface (computing) Lambda calculus
Point (geometry) Satellite Predictability File format State of matter Software developer Projective plane Archaeological field survey Shape (magazine) Prediction Order (biology) Interface (computing) Point cloud Automation Endliche Modelltheorie Mathematical optimization
Area Dependent and independent variables Server (computing) Dot product Scaling (geometry) Open source Sequel State of matter Projective plane Source code Water vapor Price index Code Measurement Process (computing) Forest Point cloud Right angle Maize Game theory
Source code Group action Observational study Mapping Observational study Server (computing) Software developer Analogy Computer network Mathematical analysis Open set Group action Field (computer science) Element (mathematics) Query language Website Right angle Quicksort Electric current
Cloud computing Grass (card game) Data storage device Instance (computer science) Icosahedron Computer Subset Type theory Peripheral Whiteboard Green's function Data center Connectivity (graph theory) Point cloud Software testing Right angle Process (computing) Graphics processing unit Point cloud
Dependent and independent variables Event horizon User interface Bit rate Endliche Modelltheorie Water vapor Quicksort Event horizon Element (mathematics)
Service (economics) Server (computing) Mapping Open source State of matter Multiplication sign Interior (topology) Open set Content (media) Element (mathematics) Field (computer science) Number Frequency Geometry CAN bus Event horizon Bit rate Peripheral Website Process (computing)
Laptop Server (computing) Dependent and independent variables Time zone Data storage device Computer Field (computer science) Area Medical imaging Process (computing) Information Local ring Loop (music) Mobile Web Source code Service (economics) Axiom of choice Mapping Data storage device Field (computer science) Computer network Stack (abstract data type) Element (mathematics) Medical imaging Software Visualization (computer graphics) Local ring
Goodness of fit Prototype Mapping Texture mapping Interface (computing) Field (computer science) Mathematical analysis Open set Enumerated type
Source code Group action Texture mapping Image resolution Weight Artificial intelligence Event horizon Telephone number mapping Point cloud Boundary value problem Spacetime Right angle Quicksort Figurate number Spacetime
Metre Satellite Multiplication sign Addition Context awareness Dependent and independent variables Building Satellite Image resolution Multiplication sign Mereology Medical imaging Software Different (Kate Ryan album) Bridging (networking) Spacetime Quicksort Endliche Modelltheorie Spacetime Routing
Point (geometry) Dependent and independent variables Interface (computing) Mereology Element (mathematics) Field (computer science) Computer programming Prototype Peripheral Point cloud Right angle Quicksort Point cloud
thank you i know everyone.
i'm just a shame the open geospatial daily for amazon web services and i'm going to talk you little bit about the some of the self said we're doing been mostly about what our customers are doing related to disaster response so first i'd actually like to start and propose and name change because i realized as some of what i'm talking about is related to saw.
where but it's not just software it's also going to be open data great so i thought we should call it like free and open source software and data produced a show that sounds terrible so then i thought this is a really great ideas after like the other night so we will see i thought that we could just add another letter as it may be like. every in reuse will open source software and data for just a show. like this because then we get to have a cool mascot right apologies i had to find something that's labeled for commercial reasons as were commercial entity so that's why you got this and i would just like to propose that we do this in time for calgary because it seems like it would be appropriate so anyways i don't know how to actually make this petition but i'm all into a frosty four.
c n n so.
said our friends on oil web services as i work in the open data team part of my job is over see our public data set program in particular the jiu special datasets if you see me top before you've almost everything you talk about this we work with a lot of awesome customers to make data available in the cloud and we try to do this and. more ways that make it easy to analyze in the cloud so this is as opposed to a lot of traditional ways of making data available which requires you to download massive amounts of data to your own computer store yourself and work with some not necessarily friendly formats we try to change that and we work with a lot of our customers to do this space.
typically for geospatial for the last couple years now we've had something called earth and eight of us which is basically a home before geospatial related things on a to b. us so if you go to this page you'll see a link out to a whole bunch of data sets that we have available that you can use and access as well as stories from our customers talking about how their. do you do good performing their geospatial workloads and the cloud because it's my firm belief that it's not just enough to make data available you need to help people figure out how to use that data right or else it's just kind of pointless and so if you see the top before or hopefully you know that we make a lot of the state available if you don't please. so they're check it out but what you might not know is that eight of us also works with a lot of our customers around disaster response related activities so we do this and couple different ways we actually do deploy into disaster response in areas we will send people out into the field i myself have deployed in this capacity i was down in north carolina.
airline after hurricane michael in the us and we work with partners on the ground like i did year see to actually rebuild connectivity in communities we also work with groups like humanitarian open street map team to use are rather large employee base to do map a thons and help map areas. around sort of the hot tasks right just like hopefully you'll do and we also work with a lot of groups to try to incentivize and support the building of applications for good and so i'm going to talk a little bit about that more in a moment because that's really leveraging our customers and that capacity so i sort of wanted.
find these two things. open data public data set piece that i see on a regular basis as well as the disaster response peace that we spend up around two years ago and so i wanted a little sort of guiding sense for what i just sort of wanted to think about and so was to write meaningful action when from nation as quickly as possible to respond to disasters situations and and again eight of us. we're were cloud provider were in infrastructure company so i'm trying to think of how can we leverage our infrastructure knowledge and water customers are doing to sort of help in these sorts of scenarios so the first thing we can bring to bear is the publicly available data that we make available on eight of us guys tattling for this earlier but if you didn't see.
the best place you can find all this pope the available data is a registry of open data us and just registry data open data that eight of us and here i have the link that if you just want to see data sets that are tagged for disaster response you can do so so these are data sets a lot of its the satellite imagery that folks have been talking about like lance at central to data. but also there's like elevation data so if you're trying to figure out which way waters going to flow or like people often walking down the hill right so if you want to figure out which way somebody is going to walk if they're like lost out the wood somewhere but you can use is to figure out which way they're going to go and we also have other is a really cool group called grieco they have. the open earthquake early warning network i think but it's a seismic sensors real time seismic sensors that are putting data out through eight of us in chile and mexico i believe i'm so there's a whole bunch of a very variety of data sets there to look at. and just sort of a i just can't not include is like as i think it's awesome.
when you think about access to data for disaster response one of the mean things that you need to think about his latency right if you're getting access to imagery but it's hours and hours delayed that's less helpful to you so we heard from a lot of our customers who own satellites that they were unhappy with the latency that there is seeing and how long it was taking them to get information to their customers. summers and so we said ok we will put a ground station network for you so last year we launched a to us ground station to help customers who have satellites get their data into the cloud much more quickly so macs are as a large company for this a digital globe you might know them as previously so it took them previously it took them about an hour to get the data. of from their satellite capture into the cloud which is where they're doing all are processing and distribution of the data and they started using ground station and their test the intent to a minute. that's huge in a disaster response an aerial right like you can get dated to people on the ground much more quickly so i'd i just love including the side so.
so ok we've got a lot of data but that's not it right you need to be able to index it and find it and so i'm not going to believe this point too much because hopefully verdi seen a bunch of the stack talks but were trying to do a better job of making the data that we make available have associated stack meditative alongside it right so if you do that then.
you can then build index is on top of it and so this is work that element eighty four has done so many answers here for million eighty four but this is built on top of sad a.p.i. it's run by an eighty four and so it is an a.p.i. that you can query all the that geospatial data sets that are made publicly available in eight of us. have associated stacked made a data so this isn't everything because we haven't gotten sacramento data in for everything yet but as we add more stacked made a data to this public data sets you'll see them show up here so if you're building any applications are just want to try this out for yourself you can go here like i said this is all managed by element eighty four and it will be a good representation of all. all the geospatial data that's on a to be less so we've got data we have some sort of system that you you can use to see what data is there so what are people building with this or what can you do with data that's made available in the clouds marley and so there's always of interfaces that you can go on top of the story and sums going to go through some of these quickly this is work that was done by vincent years.
though he won't give me something new to put in here i did ask him the other day but he said is to use this so this is a slippery map like all of you have seen in this isn't new any more but what this is doing is actually using surveillance technologies to do map tiling right so the reason why this is important is twofold there's no underlying server.
running here right and so insincere in the audience right so like vincent doesn't have to pay to have a server running continuously so if nobody's making any requests to this mapping interface vincent doesn't have to pay to keep a server running if nobody's using it right the other piece that's important in the disaster response context is that disaster response workloads are very spooky.
right like you have nothing going on and thousand have fled comes in and you have a lot of people that are very interested in what you have this service technologies are designed to scale up so you have zero cost when nobody's using it and they immediately scale up as they get used right which is a very cool property that you would like to see and disaster response in areas of so this is something similar this is work.
by center job as powered by said no hub and so. just same sort of things using service in the back and but this is showing at an analytical context instead of a visualization context but the same idea there's a trigger being made. action is spinning up very quickly grabbed a dated needs every turn it in some context this is just showing it for analysis rather than visualisation.
this is work that was done by david vitter before foster g. and may this year he actually put map server inside of them to function so that's the surveillance technology i was talking about so again you don't need something running continuously if nobody's using it will spin up on demand for the ngo has been talked about a little bit i think.
julia my dimension to the other day and one of the keynotes penned using open source community building large scale distributed analysis tools but the ability of pangea go to work efficiently is predicated on the fact that has access to optimize data sitting in an efficient storage mechanism. and this is work at home who did i think connor is out here somewhere.
not met him before so you might actually be sitting here nobody looks like but this is work that was done on top of the u.s.g.s. lighter data i don't know if you can read the point the top there but that is eleven billion points that were analyzed and put together this is a whole bunch of light are point cloud data over the u.s. the cool thing about this is who did this for us yes. u.s.g.s. themselves as our u.s. geological survey and yes yes themselves said that they had never been able to see all their data in a cohesive manner in till this work was done right and so this is putting light are into a cloud optimise format which is pretty cool since our government pay for it also is nice for them to be able to see it so couple other projects just call out this development see day.
did hurricane intensity predictions this is on top of our u.s. meteorological agency data this goes data these are geostationary satellites state is available on eight of us but then what they did was to have a deep learning model that would basically automate hurricane intensity predictions because they can look at. and if we can look at clouds like the shape of clouds and you can figure out the hurricane intensity and this is work that was actually i think done by hand previously but it would take on the order of six hours and they were able to automate this get it down to about fifteen minutes which is you know that seems prius.
blue dot observatory on j i think maybe gave a workshop earlier on in the week but this is worked as a bystander jobs so they were using sentinel to data they were analysing water bodies around the globe this is actually in cape town when one of their major water bodies was a running out of water when they had a water crisis. is this a previously and so the cool thing here is again all the state is made available in the cloud this is essential to data but they are able to analyze this its scale but they're able to do this for a few dollars a month which is fantastic great they analyze water bodies from around the globe. and they can do so for a few dollars a month which is pretty cool and i believe the edges and i don't think but i believe this codes open source but it's so it's the blue dot water observatory and all of these can be triggers if you're trying to set up a disaster a pipe a disaster response by playwright like these could all be triggers so if you start to notice water bodies deteriorating like that's potentially.
the indication of drought conditions things like that she muses triggers this is another project the data is available any of us this is open a cute sought to open source community and open source project and so this is an aggregation of air quality measurements from around the world i think it's something like seventy five countries one hundred twenty different sources if i was more on top of my game i would have taken a picture. of like an actual forest fire happening somewhere but i just took this the other day and so you can see that one red dot that one red dot is hide p.m. ten particulate matter in the air so this is a potentially a trigger of like a fire going on in the area right or you could use its after the fact and see what the exposure is so when you do look at this over like a populated area. the like california with his wildfires you'll see a lot of dots quo low light up read those are all sensors on the ground and we also make those and data available so you can easily query that great if you're looking so you can write something like to see where this is worked on by to set that simmons you can make a simple sequel equerry and again you don't need around any service to to do any of this.
so i sort of showed you a bunch of these examples but we wanted to know what people who are deploying actually wanted so we work of the element eighty four to a user needs studies and we talked groups like american red cross community an open street map doctors without borders as well as to developers like open road map map larry to see like what we could sort of try to put to.
other offer people on the ground so i would recommend you go there to that site on the bottom right there and you can see what sort of came out of all that but for the rest of the stock is going to focus on the one piece which is people wanted access to data in the field and like being able to compute into the field with them would likely be very helpful and so on.
the main tool that we have to offer there is something called the snowball edge this is a physically hard and device that you can actually pick up and take with you so a lot of people think of the cloud is just something that exists like thing is you've on the other days as other people's computers which is true but those computers don't need to be sitting somewhere far away you can actually take one with you so this noble edge actually gives you access to. do the same fouts or visit some subset of the same cloud services and you can actually physically take that with you like you can see there's an airline shipping label on their right like you can you can check it out on an airline and gives you access to cloud services so this is actually made as an import export devise a cool little example this is a u.s.g.s. recently when they're volcanoes in hawaii.
they were worried that their data center was going to get overtaken by the volcano so they were going to lose all their data so we actually ship one of the well we ship this noble to them write this is a physical device so shifted to him they stuck other data on it and it's got a little kindle so you just flip the you flip the front of it and it changes the shipping label get sent back to us and all of the data is now out of their physical like. status and it was located in the lava flow path and i got moved into the cloud house wanted to include a phone gets so so again.
i've got five minutes last summer to go through this pretty quickly i apologise but disaster response to the pipeline what would this look like great so you can think of event triggers rate so we already sell some of these if you're getting earthquake notifications wildfires you see smoke you see sort of water disappearing from water bodies you can trigger on that so.
you can start gathering up tools right so you pull open street map data like we saw you gather sent on to land said a few of commercial data available you can gather that is well there are a number of open source packages to allow you to do this then you can start provisioning servers rates you can use things like surrealist or you can actually provision like always on servers for the period of time.
that you need it you can start putting together static content websites that you know people are going to start accessing you can start cashing them ahead of time since you know you're going to get a lot of demand and then you can start putting the state on to the physical device when you can ship that device into the field so you can walk into the field with something that you have locally that requires no external can.
activity and it will give you all your base map that will give you all your preachers astor imagery and then you can start doing things in the field you can start collecting drone imagery that can go right on the device you can start using things like feel papers any way that you can sort of collect data locally you can put that back on to the device and that device gives you computer on the edge storage.
and it will give you a little run a network you can also just stick a network in front of it and so any of your mobile devices or computers can access that so couple of examples that this is work we did with seth about two years ago said simmons so this is open aerial map running on this noble edge for local data visualization this is an example of using huge us on a laptop using the device as an image server.
you can also use something like open map good and positive running on the device is all things that we've already prototype running on the device a l m eighty four as a some interfaces that are using to try and bring all these pieces together.
and if you're collecting data locally and we also work with groups to sort of figure out how best to push the boundaries of a i m l on top of geospatial data so if you're doing something like grown collects or if you're getting high resolution imagery after the event what can you do sort of locally or in the cloud to like.
the best analyze the data right and so if you so ryan talk the other day or taken it and you are here as well this is a work that we're doing to space net and so we're working with space net to push the boundaries of ai and am now on top of juice we should data and sort of try to help the community get there as well and so there's.
i was high resolution imagery as well as labels for a bunch of different cities around the world so there's two challenges i just want to call out real quick space that for his face and five as i think they're both very important for the disaster response context.
we set for took off meter imagery so the question was how often meter can you get and have your model still perform the reason why that's important in a disaster response context if you want to quit this imagery you're not necessarily going to have a satellite directly overhead so you want to know how far off meter will your model still work and so how quickly can you get those sort of first look images and have your model still work. space that five is cool because in addition to doing road extraction in building actual roadable road networks it's looking at travel times this is extremely important because if you have like a bridge that no longer exists need to reapply your routing locally you want to know how fast you're going to be able to get any sort of response vehicles around to any given play. straight so being able to do this on the fly with updated imagery that's coming in post disasters is pretty cool so space that five is just starting here in the very near future so if this is interesting to you please check it out.
just remember this is all part of this sort of what would it look like to have a disaster response pipeline that you could spin up at any point rights even trigger you gather data you build the interfaces you to play to a physical device and the new ship it out into the field with someone and so with that i'm going to stop it.
i just want to say if any of this seems interesting to you any of the work that any of those folks are doing please let me know we have credits program if anyone looking to prototype any work clothes and top of geospatial data in the cloud please just let me know or go to the website we're always looking to sort of support those efforts and with that i will stop.
you are.
Loading...
Feedback

Timings

  395 ms - page object

Version

AV-Portal 3.19.2 (70adb5fbc8bbcafb435210ef7d62ffee973cf172)
hidden