The Mapossum: A System for Creating, Collecting and Displaying Spatially-Referenced Answers to User-Contributed Questions
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 188 | |
| Author | ||
| License | CC Attribution 3.0 Germany: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/31727 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language | ||
| Producer | ||
| Production Year | 2014 | |
| Production Place | Portland, Oregon, United States of America |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
| |
| Keywords |
FOSS4G 2014 Portland42 / 188
7
10
14
15
16
23
24
25
28
29
33
37
39
40
43
45
46
48
50
56
64
65
69
72
74
82
89
91
98
102
107
111
114
118
128
131
132
135
138
141
143
147
149
150
157
158
161
164
165
166
173
174
175
179
185
00:00
Calculus of variationsProcess (computing)Archaeological field surveyCountingWebsiteWordTerm (mathematics)MereologyOrder (biology)Goodness of fitLevel (video gaming)
01:02
Different (Kate Ryan album)Strategy gameLevel (video gaming)Inclusion mapFront and back endsScaling (geometry)CodeWater vaporGraph coloringCartesian coordinate systemHypermediaMultiplication signSoftware frameworkRight anglePoint (geometry)State of matter
01:49
Type theoryUniform resource locatorLevel (video gaming)Process (computing)Query languageCountingWeb browserStatisticsCartesian coordinate systemHyperlinkString (computer science)DebuggerPoint (geometry)MereologyPie chartLecture/Conference
03:38
Context awarenessGraph coloringSoftwareComputer fileOrder (biology)MereologyFacebookTemplate (C++)Link (knot theory)Configuration spacePoint (geometry)Mathematics1 (number)NumberWater vaporDifferent (Kate Ryan album)Server (computing)Type theoryDatabaseGreatest elementSet (mathematics)Artificial neural networkComputer fontState of matterTable (information)Web applicationTrailMultiplicationAxiom of choiceDependent and independent variablesDebuggerDisk read-and-write headTerm (mathematics)Electronic mailing listAudiovisualisierungFile formatUniform resource locatorCycle (graph theory)Level (video gaming)Natural numberSystem administratorAreaTesselationCross-correlationCartesian coordinate systemHexagonEndliche ModelltheorieBoundary value problemOpen sourceMultiplication signWorld Wide Web ConsortiumSystem programmingPower (physics)InformationLibrary (computing)Basis <Mathematik>Web serviceMatching (graph theory)Operator (mathematics)AdditionMonster groupCoroutineQuicksortMedical imagingComputer programmingRule of inferenceInternet service providerSoftware testingTessellationStaff (military)Extension (kinesiology)Mobile appSeries (mathematics)Latent heatLine (geometry)Revision controlVideo gameFlow separationSingle-precision floating-point formatFront and back endsCodeQuery languageComputer clusterInteractive television
13:33
MiniDiscTessellationComputer fileCache (computing)Point (geometry)Configuration spaceSymbol tableTesselationTouchscreenUniform resource locatorLevel (video gaming)Motion captureGraph coloringWebsiteCivil engineeringDependent and independent variablesDatabaseMereologyFigurate numberPanel painting
15:02
Process (computing)Query languageArchaeological field surveyLevel (video gaming)Visualization (computer graphics)Type theoryBlogDifferent (Kate Ryan album)Dependent and independent variablesWater vaporDrop (liquid)Uniform resource locatorException handlingTrailGraph coloringMultiplication signWeightWebsiteIncidence algebraPoint (geometry)Line (geometry)SpreadsheetComputer fileErgodentheorieDot productSymbol tableTexture mappingTable (information)Client (computing)TessellationQuicksortSound effectWeb serviceTask (computing)NumberMathematicsString (computer science)Set (mathematics)Semiconductor memoryElectronic mailing listInterface (computing)BitEndliche ModelltheorieOnline helpFile formatLatent heatAreaComputer configurationResultantSystem programmingSelf-organizationLocal ringInformationHypermediaWeb applicationDatabaseFront and back endsTesselationInclusion mapWorld Wide Web ConsortiumShape (magazine)DebuggerFacebookZoom lens
Transcript: English(auto-generated)
00:00
Thanks for coming to our talk today on the mount possum. My name is James Dickens. This is George Raber. We're going to be presenting this together. The mount possum was inspired by this mount, which is the pop versus soda map. It was made some years ago. And essentially what they
00:20
tried to do is map spatial variation in the term used to describe pop, soda or coke. In order to do this, they set up a web-based survey where users go and provide their location and give their answer and then they map it. This website or this map does a good job of kind of
00:41
highlighting a geographic concept that place is important. Where you're from or where you currently live has a large impact on words you use like coke or pop or things you say like you guys or y'all or even your cultural beliefs or knowledge that you're given. So what we wanted to do is kind of generalize this map, give people the chance to make their own questions
01:05
and get answers. So that's what we set out to do. We made an application where users can come create questions and then through social media or through any kind of marketing strategy they want to use, get answers that are spatially located for those questions. We also tried to
01:25
include many different scales and aggregation levels, which are going to continue to grow through time. Right now it's points and counties and states and countries and a new thing we developed called watercolor, which we'll talk about more later. Before we go into the
01:43
back end of the application and the framework, we're going to kind of give you a preview of the front end and kind of how it operates. So we developed the application specifically for
02:01
mobile devices at this point because we kind of feel like that's how it's going to most be used initially. We use jQuery mobile and the map possum kind of has two components. You can come and you can just visualize questions that have been created or you can answer questions that have been created or you can create your own questions. To answer questions or view, you
02:23
don't need to set up an account or anything, but if you'd like to create a question, you do need to set up an account. So it's important when you're looking at things like this that your location, maybe not right now, is not important, but where you're from or where you currently reside is more important. So we've kind of set up two ways where you can
02:44
report your location. You can search and use the map centroid. So if you're from someplace that you don't live now and you kind of feel like that has more of an influence over your thought process, you can use this method or you can just use the location provided through
03:00
your browser. As part of creating a question, you do have to log in, like we said, and once you log in, you're provided an account tab where kind of the statistics for your questions are tracked, and right now it's pretty bland. We have just pie charts that we did
03:23
using ChartJS, but you can also see there's hyperlinks that will go directly to your question that you can pass in using query strings, but you can also pass in map types as well. So if you'd prefer your question to open with a different boundary, you can do so.
03:42
Just using the map type. Kind of in the future, we want to include more visualizations, more ways to consume your data. So give people the ability to download their answers so they can put them in a GIS system or whatever format they want to do,
04:04
but also kind of make it a little more friendly to use in the future. That's kind of the gist of how it works right now. You can essentially come cycle through questions and answer them. You can send out those through Facebook or whatever using the link that's
04:23
provided, and then you can create your own questions to kind of explore spatial change. So now I'm going to let George here talk about the back end and kind of tie in how the front end and the back end work together and kind of what technologies we use
04:40
to make this happen. So I'm going to give you George now. Thanks. So when we set out to create this application, one of our ideas was to develop this software, develop this web application using all open source software. We're at the
05:09
effort was a learning effort on could we do this using GIS software and implement this idea that we developed here. And you can see the different components to the web app listed below
05:27
and the different pieces of open source software, both GIS and mainstream non-GIS software pieces were utilized. I'm going to step through each of these
05:43
different parts of the system and describe how they fit together. First, we have the database itself. The database has two different sets of data. I guess an artificial delineation between these two sets of data. But the data that
06:04
application creates as users interact with the application is the first set of data I like to talk about. So we've got a few, basically a very small number of tables to keep track of the users and then another one to keep track of the questions that are generated, the user submitted
06:22
questions. And we have both answers and responses and we could have given different terms to those, but for the purpose of what we're talking about here, the answers are the list of multiple choice, the valid choices when the user encounters a question. So with the pop versus soda pop
06:47
question, which we talked about at the beginning, those would be Coke, pop, soda, other. And then the responses is the table of responses by the users to the questions.
07:06
And that one is spatially, has a spatial component to it. So it records the point at which they submit as their location. And so those are those tables. And then the other tables that
07:23
are in the database are the support tables. And most of these are spatial in nature. Basically, we've got a number of tables that correspond with the different aggregation levels that we want to be able to visualize. And currently, we have supported in the app counties, what we call
07:45
level one administration areas, the state level. In the United States, it would be states, but we have these for worldwide in the system. So if somebody answers outside of the United States, their data is still aggregated up and you can see that response at that level. And
08:01
then countries is also a worldwide data set. We don't have these in the system yet. We're working on generating them, but we want to have multi-scale hexagons in the system so people can choose those values at different levels. The map that you saw earlier that we showed that watercolor layer, that's what we're calling it right now. And that is generated from the
08:25
raw points. So there's no, the raw points meaning the responses table. So no separate support table needed for that. We also have a table that keeps track of which map tiles need to be redrawn. The maps that are shown are tiled out. And at this point, we haven't
08:46
actually implemented that. We refresh the tiles manually, periodically. When a question gets a certain number of responses, the tiles move from being redrawn dynamically every
09:04
time there's a request to being drawn on a less regular basis. Because first of all, you don't need to draw the tiles as often because a single response doesn't change the map that much as more and more points are drawn. And then in addition to that,
09:22
the map tiles take longer to draw once there's more and more responses obviously in the database. So, to draw the tiles, we use, we've utilized a piece of software called TileStash. It's a tile, a tiling creation, map tile creation program
09:45
that was written in Python. It'll serve up your data in lots of different formats. There's lots of different providers for TileStash, but we're utilizing the the MapNIC provider. It basically will draw tiles based on a MapNIC XML file.
10:08
And the way that TileStash works is that you feed it a configuration file and then you start TileStash running on your server. And at that point, you get the ability to access tiles using a URL like this where you have your server, then your configuration layer.
10:25
So your TileStash configuration file can have multiple layers. And so you request for for one layer and then you request the z, the x, and the y for that layer, the specific tile for that layer. And we modified it only very slightly,
10:45
probably changed five or ten lines of code. What our version of TileStash does is that it allows the map user, I mean, sorry, the application to serve up tiles based on multiple configuration files. And each of these configuration files for us
11:05
represent a question ID in the database. So the new URL becomes like the bottom here where we have the server, the question ID, the map type, which are the different layers in our configuration file. And they can be the raw points, the counties, the states, and so on.
11:25
And so when you're, you know, when you're moving on the web app that James demonstrated, when you're moving from question to question or from map type to map type, it's just changing these values to request a different set of tiles.
11:41
All right. The application server itself, which is what allows the database and the tiles server and the front end to talk to each other, that was written in Python. It's a Flask app that's handling requests in from the server. And it has routines for all sorts of things
12:03
like creating users, requesting the legend. The legend is generated using the Python image library based on the colors. So when a user sets up a question, he specifies, he or she specifies what colors are going to be utilized for each of the answers that they provide.
12:22
And right now that's just, we don't actually allow the front end user to do that, but that's one of the changes that we'll make. Also, there's a few other just housekeeping type things it does. It allows you to get the extent of the response so that the app can handle doing that. And then the important ones, creating the questions and adding responses
12:43
that I'll go over here real quick. So when a user goes to create a question, this is the incoming request, what happens is that that information is passed to the app, to the web service, the Python Flask app. And the Python Flask app handles creating a new
13:01
Kiles-configuration file from a template. It also handles creating a series of Mapnik XML files based on the map types that are currently supported. And so there's a set of templates that are currently supported. Right now it's points, counties, those are the ones that we're talking about, including the watercolor map. And this includes generating
13:27
the SQL query that will need to go into that XML file in order to make it work. So here's a screen capture of the tile cache configuration file that's created. This is for question 25, which is the Coke versus soda pop. By the way, I don't think James
13:44
mentioned this. We imported the data from that question, just so that we have a demonstration question. The other questions on the site are pretty young, so they don't have as many responses. But anyway, this is the tile cache configuration file for that question.
14:05
And you can see how it creates a location on disk to store the tiles for that question. And then it provides the layers, the points, the counties, the subs, the countries, the
14:26
to generate these maps that are in this file. Those are in the Mapnik XML file that gets created. It gets cut off here, but here's the SQL that gets generated to create the points. And you can see over, actually I don't know, but over just to the left of
14:42
the logo there, you can see where it generates a symbol for each of the answers, each of that, and gives it a color. And that color, it's drawing from the database, what the user
15:01
had specified as that color. And so this is pretty straightforward for all of the map types, except for the one that we call watercolor. It's kind of like a heat map. Anyway, the way that I like to describe it is that if the map was a sheet of paper
15:20
and each map symbol will be drawn by kind of like dropping a droplet of colored water onto the map at that particular place. And so as you get more droplets around a certain location, it'll be more saturated with that color. And the way it's implemented on disk, I mean, sorry, the way it's implemented in the web app, in the Python
15:41
Flask app, is that these point symbols are created dynamically when the question is created. You specify what the colors are and it generates these small PNG files that will be used to symbolize the map and places them on disk so that they can be referenced
16:01
when the tiles go to be drawn. And so that's just the aggregation of all those, all the little dots that look like that on the map is what turns into that effect. All right. This is just what happens when a user answers a question. Of course, we know
16:23
that that information, whether they specified, you know, what their response was and where they were is what's passed into the database, into the responses table. And that information is then stored. But then on the front end, the client is responsible for redrawing the
16:42
tiles. And on the back end, the tiles aren't always redrawn. So depending on how many responses there are, we redraw the tiles so that you can immediately see, if there's only a few responses, you can immediately see your answer. Whereas if there's a lot,
17:01
then it might take a day to show up as your particular answer. So the future task is to automate this process so that automatically it figures out which tiles need to be taken out and then decide, based on the number of questions, when that tile will be redrawn,
17:21
when a new tile will be created. So what we want to, Jim's mentioned a few of these things and I talked about a few of these things during the course of the talk. We want to update or change in the future, or the ability to basically create a number
17:42
of questions and link them together as a survey so that when you get a certain URL with a certain query string and you send that to somebody, they will see only those questions that you've listed in the query string so that you can tell them to answer a set of questions
18:01
or one or two questions at the same time. I actually have it implemented for a single question but to tie it together to make it more like a survey. And the second thing here on the list is to be able to embed the maps like in a Facebook post or social media or the ability to embed the map in a news article or blog post. And right now you
18:27
can do a little bit of that. Basically you can automatically specify which, you know, we demonstrated the map type and the question that shows up. But we also want to make it so that they can specify a specific zoom level so they can look at a specific
18:41
area and everything that the site allows you to customize to be customized in the URL so that those can be shared and make it easier, provide some help for doing that. We want to continue to enhance and build the visualization tools, include more map
19:02
types, more aggregation levels. You know, looking back at that pop versus soda pop question at the very beginning, they symbolized it using kind of a tertiary soil type symbolization of using red, green, and blue in the corners. And so we could provide something like that
19:22
or, you know, the possibilities are endless for the different map types that we could include in our front end or our web app. Also the ability to include different charting options and the ability to view results, both the map results and the chart results
19:42
based on spatial and temporal queries. So the ability to look at your data for a particular area. So if like James does a lot of work for different local government organizations and somebody might want to look at their question just for a particular area or they
20:03
might have designed a question just for a particular area and they might want to see the responses through time if something in the media changed the value of a response. So we want to provide the ability to do that. And right now the database keeps track of all that information so that we could do that. And then one of the glaring
20:23
pieces that's missing right now is the ability to kind of hover over or click on the aggregated units and for it to describe what's below that point, particularly to show the aggregation. So you could hover over a particular state and find out not just what the majority of people have chosen but kind of a breakdown of that state's
20:46
responses or that unit's responses. And then, you know, he showed the ability to look at your data for each of the questions that you created. We also want to add the ability to export the data for all the registered users so they can export the data as a shape
21:04
file or as an Excel spreadsheet or a common delimited file so that they would be able to use that data offline or however else they wanted to do it. And the reason why we're kind of going over this, if anybody has any additional ideas, that would be great.
21:21
At this time, we'll open it up for questions. And we have our contact information and that's the URL to the website up at the top. Thank you.