Automating Generating a Web Map from Online Tabular Data: UC Davis Potential Worksite Exposure Interactive Web Map
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
Title |
| |
Title of Series | ||
Number of Parts | 351 | |
Author | ||
License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
Identifiers | 10.5446/69141 (DOI) | |
Publisher | ||
Release Date | ||
Language | ||
Production Year | 2022 |
Content Metadata
Subject Area | ||
Genre | ||
Abstract |
| |
Keywords |
00:00
Universe (mathematics)Interactive televisionWeb 2.0Level (video gaming)Computer animation
00:26
BuildingInteractive televisionScripting languageVisualization (computer graphics)Group actionAssembly languageTraffic reportingWebsitePolygonTable (information)File formatAutomationSource codeNetwork topologyCodeQuadrilateralTable (information)Web 2.0BuildingGroup actionMultiplication signResultantBitGreatest elementForm (programming)CodeExtension (kinesiology)Set (mathematics)Computer fileType theoryWebsiteMereologyInteractive televisionLevel (video gaming)Data dictionaryElectronic mailing listUltraviolet photoelectron spectroscopyProgramming languageStudent's t-testScripting languageMotion captureState of matterPhysical lawUniverse (mathematics)TouchscreenFluid staticsInformationMoving averagePerfect groupFile formatComputer animationProgram flowchartEngineering drawing
Transcript: English(auto-generated)
00:01
All right, so my name is Michelle Tobias and I'm going to tell you a little story about a web map I put together about a year ago. This is a web map that automates itself. It builds its own data every day twice a day from an online tabular data set. So very exciting.
00:22
How do we advance with that mouse button? So a little bit of background, the university that I work for, the University of California Davis, is required by our state, the state of California, to list potential COVID exposures that happen in work sites.
00:41
It's something that's mandated by our state. And so the way that they do this to meet the requirement and fulfill the law is to produce a table that they post on a website every day. The table lists the building name, where the exposure happened, and the dates. I personally, as a map person, think a map would be a lot more useful for this.
01:04
So the goal of my web map was to create an interactive web map that uses the campus potential work site exposure data that's on that website in tabular form, and then to automate it so that it updates twice a day because we don't know when they're going to update the data. So I figure twice a day is probably good enough.
01:22
So this is the workflow that I put together with some of my students. Essentially, it runs as an R script. So we wrote the code in R. It's a programming language. And it runs on GitHub as a GitHub action. So the GitHub action triggers twice a day.
01:41
Every time it triggers, it runs this R script. And what it's doing when it runs that script is it's starting out with web scraping. It's going to the COVID exposures website, grabbing the table from the site, and then extracting the table from the site. And then it assembles the data. So it takes that new data set it got.
02:01
It tacks it onto the old data set so that we have all the data from yesterday and today. And then we have to clean it. You can check out the code to see all the cleaning that has to happen, but this is a very not standardized data set. So we have to do things like clean building names and clean up dates and stuff like that. And then finally, once the data is cleaned and the building name dictionary has been
02:24
applied, then we have to make it spatial. So we join that data set to our campus building data set to make it spatial. We save it as TopoJSON because the file has gotten so big that TopoJSON is the only file type that will go on GitHub and not trigger problems.
02:41
So found that out a couple months ago and devised that solution. And then that TopoJSON file is piped into a leaflet website in HTML. And also an extension called Leaflet Timeline JS that makes the timeline slider that you're gonna see in a minute. And so that makes our web map.
03:01
So the result is we have an interactive web map. You can go play with it online right now. Just Google UC Davis COVID exposures and you'll find it. And it updates daily and it's automated so I don't have to run it every morning like I did last August. So this is what it looks like in static form. The time slider has a play button at the bottom where you can hit play and
03:23
it will run through all of the data that we've assembled. And it's really cool but it's also really frightening cuz you kind of see the spikes on campus and when people are off campus. The other nice thing is that because it's Leaflet we can configure pop ups. So I kind of simulated this one, thank you.
03:44
Because I couldn't do the screen capture but generally if you click on a building that's highlighted, it will tell you what the workplace name is. So the campus building and then what the exposure date is. So if you ever want to dig into the data a little bit more that's how it works.
04:02
And honestly I think this is a really nice way to visualize this particular data set. For me as a geographer it's a lot more intuitive than looking at building names. I should also mention that one of the reasons why we built this in the first place was I was looking at the list of tabular data going,
04:20
it looks like there is a cluster of exposures going on at the vet school. Is that really happening? What happened was it was actually just me recognizing those building names and not the other building names cuz I didn't know the other buildings. There was not actually a cluster of exposures there. So this helps us understand that data set better and really know like are we having a problem on a particular part of campus or not.
04:43
So thank you, I'm perfect on time. If there's my contact information, I'd love to chat more about this. Thank you Michelle.