ScribeUI: MapServer Mapfile management made easy
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 188 | |
| Author | ||
| License | CC Attribution 3.0 Germany: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/31588 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language | ||
| Producer | ||
| Production Year | 2014 | |
| Production Place | Portland, Oregon, United States of America |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
| |
| Keywords |
FOSS4G 2014 Portland181 / 188
7
10
14
15
16
23
24
25
28
29
33
37
39
40
43
45
46
48
50
56
64
65
69
72
74
82
89
91
98
102
107
111
114
118
128
131
132
135
138
141
143
147
149
150
157
158
161
164
165
166
173
174
175
179
185
00:00
Data managementScripting languageInclusion mapVariety (linguistics)Source codeScale (map)Variable (mathematics)Text editorVideo game consoleFunction (mathematics)Web browserAttribute grammarSoftware developerRegular graphFile formatData typeSocial classProcess (computing)Local GroupDefault (computer science)Block (periodic table)Template (C++)PasswordGroup actionMappingLine (geometry)Interior (topology)Point (geometry)NavigationDisintegrationPoint cloudCache (computing)CloningAreaServer (computing)Interface (computing)Physical systemDirectory serviceModule (mathematics)Plug-in (computing)Machine codeVisualization (computer graphics)Limit (category theory)ArchitectureGoogolCodeLevel (video gaming)BlogInflection pointConstraint (mathematics)Error messageService (economics)Software bugPower (physics)Water vaporTexture mappingWindowFilm editingMaxima and minimaMathematicsMereologyMultiplication signSource codeClient (computing)Interface (computing)Functional (mathematics)Point (geometry)HTTP cookieGraph coloringFile formatPlug-in (computing)Software developerLine (geometry)MassPhysicsMappingPoint cloudResultantProjective planeGroup actionSpacetimeLevel (video gaming)WordCoefficientComputer fileInformationGenderServer (computing)Endliche ModelltheorieFunction (mathematics)Direction (geometry)Artificial lifeService (economics)Data managementDataflowVirtual machineRight angleDirectory serviceCore dumpKey (cryptography)PredictabilityTemplate (C++)Computer architectureMachine codeOrder (biology)INTEGRALBlock (periodic table)Doubling the cubeCodeWebsiteSoftware frameworkPiFrequencyScaling (geometry)Metropolitan area networkInteractive televisionoutputAdditionCartesian coordinate systemPlanningTwitterScripting languageOrbitConfiguration spaceComputer programmingStudent's t-testParameter (computer programming)Process (computing)Presentation of a groupFeedbackWeb browserDecision theoryHypermediaRepresentation (politics)NumberState of matterRectangleCountingRepository (publishing)Web crawlerOpen sourceConnected spaceMehrplatzsystem40 (number)Social classVideo gameShape (magazine)Fluid staticsVariable (mathematics)Network topologyInclusion mapDifferent (Kate Ryan album)CuboidGraphics tabletInheritance (object-oriented programming)PasswordRefractionSoftware testingProduct (business)Video game consoleIntegrated development environmentSubject indexingLogic gateElectric generatorModule (mathematics)Cache (computing)Goodness of fitFront and back endsBitRaw image formatComputer clusterDatabaseMultiplicationZoom lensTesselationVisualization (computer graphics)Single-precision floating-point formatRegular graphSelectivity (electronic)Graphical user interfaceText editorSet (mathematics)Standard deviationExtension (kinesiology)Modul <Datentyp>Computer animation
Transcript: English(auto-generated)
00:03
Hello, everyone, and welcome to this presentation of ScribeUE. First of all, I would like to know if there is anyone in the room that is familiar with Map Server? Nearly everyone. Good, good. So I will pass more time on the stuff
00:23
that will be interested. So most of you or all of you know that the map files are great, and Map Server is great. And it's possible to get a really beautiful result out of Map Server.
00:41
Map files are human readable, and we can create our own custom script to generate them. It's also possible to create big, big, big map files with hundreds of layers in them. And for that, we usually use includes.
01:02
But there is some downside to the way we do things with map files. First of all, there is a lot of copy pasting. Then you can't have multiple data sources in a layer. There is also a lot of copy pasting. And the scales are a bit confusing in what
01:24
is the min scale and the max scale. And yeah, so it's a bit confused. There's a lot of copy pasting. There's no variable or comments. There's no copy pasting. Did I say that there is a lot of copy pasting? Anyway, so when we develop new maps,
01:44
we all have our own way to do things. So we will have a text editor open to edit the map, a browser open to browse the map, a console open to see the debug outputs, a file browser to see what are our data files, SQL window or QGIS
02:07
desktop to see the data. Maybe some geek among us will use OGR info or GDAL info to explore the data or the projections. So that's a lot of tools to be able to master
02:23
and to get in our workflow that we have to learn. So at MapGears, we developed a new tool that we open sourced and provided to the maps of our community two years ago, or is it, yeah, two years ago, called Scribe UI.
02:43
Scribe UI is made of two things. First, UI, of course, and the Scribe syntax. It's a new syntax to make it easier to develop map file. And you should use it. There's even some talk among the developers
03:02
that we are, well, we are talking about the developer to implement the Scribe syntax inside the core map server because it makes things so much easier. So Scribe UI is there to make things easier for map file developer, and it's developed by map file developers, by people actually doing map files.
03:25
So first of all, the interesting stuff. The Scribe syntax. Like I said, it's an alternative way to write map files, just like XML map files for machines, base maps for people doing map cache, or any in-house format
03:44
that you have or script that you've built that generate map files at the end. It's actually a Python script that just reads a kind of JSON input and outputs a map file. The Scribe syntax in itself is a way
04:04
to make the work of configuring maps over a lot easier, and I mean a lot easier. It makes the project more manageable and does save a lot of time.
04:20
So you can create multiple layers from one configuration and have exactly the result you want. You don't have to mess with the scales anymore. You play with the zoom levels. And it changes the way that you view things when you configure the map files.
04:41
It's still based on the map file syntax, though, so it's really familiar to everyone already familiar with Map Server. So it's a lot easy. It's really easy to get into it and start using it. And the command is as simple as the small command line
05:01
that we have at the end of this slide. So now there's a few key features. So the scales are now defined inside the Scribe syntax. You define at which scale you want each layer to render based on your configuration.
05:22
And from the inside, it will generate one layer per scale per data source. But for the map file developer, it will only configure one layer per layer, what it should be.
05:42
So that's how it works. You can define scales group. For example, here, this layer is available from the zoom level 1 through 16 and have three different data sources that are generalized depending on the scales. Normally, you would have written three layers,
06:02
each with its own data source and its own classes and its own connection information and everything. You can also use the scales at the style level or at the class level to redefine how you draw your lines or your points, the size of them.
06:22
So you can really quickly and without any copy-paste change a few key things in your layers so that your map file will adjust to what you really want it to be. There's also other cool things like variables.
06:43
You can define variables that are a group of map file instruction or that are a single value. For example, here, we have a layer that has, if all your layers use the same configuration to post this, for example, you can add them in a single layer config
07:04
variable. And if you have, for example, a color that you want all your water layer to use, you can define it in a variable and just use this value inside, well, beside map file key syntax.
07:24
There's also commands, commands where you can comment a block of text. And that's a really, really great thing. You can have commands that will reflect and be inside your generated map file.
07:42
That's double dash. Or if you comment with a slash star, you will comment things inside the scribe syntax, and you won't get the result in the resulting map file. So that makes things really, really easier when you are debugging things.
08:01
So scribe is great. Use it. It's there. It's open source. It's working. We have been using it for two years now. And we really want your feedback on it. And like I said, there have been discussions
08:21
to include the syntax in Map Server because it's so useful. Please give us your opinion on this. Now, scribe UI. That's scribe UI. That's it. No, OK. Scribe UI is, well, once we have developed a new syntax for Map Server, things were so much better.
08:44
But then we just integrated that syntax inside that GUI to have the syntax or the configuration in one panel and the map file in the other panel. And that made things easier because we didn't have
09:03
to switch between two windows. And then we added another key feature here and another key feature there. So in the end, we built this interface that has a lot of cool things in it. There is a map file editor, of course.
09:21
But it's also a productivity tool to help you manage your data. So you can upload data to the right directory. You can see the debug outputs and everything. So we'll go through the key feature of scribe UI now. But it's a really, really great tool
09:41
to make the creation of map files and the configuration of Map Server easier and faster. So we save time and money. So the basic things are there's workspaces, which are a kind of project that can contain multiple map
10:02
files, templates. There's the editor, of course. You can have groups of layers in your configuration, see your map definition. And the thing that is the most important, see your changes live.
10:20
So here we have the workspace. A workspace, it's a kind of project. When it's like a group of map file. So when you have a new client, for example, you can create a new workspace just for this client, password protect your workspace, and give access
10:41
to your client to his map file. Or you can be multiple people working on the map files together in a single installation. There are also templates. So who here started a map file from nothing?
11:05
Like a few people? Everyone or most of the time people copy an existing map file, remove all the layers, and start from something.
11:21
So that's why Scribe UI, because we always do that, have this key functionality built in. Every time you start a new project, you copy an existing one. So if you have an existing project for a client that always have the same database connection,
11:42
the same set of key layers, you can have a template for this client and just copy this template, and it will start from there. You can also use as template any map file that is configured on your installation.
12:00
Now the editor, it's color coded to make it more fancy. And you can group your layers together so it's more manageable. You also have line numbers, which is really useful. Your groups can be, you can change
12:22
the order of your groups. And thanks to Vincent, it's working better now, because he opened a ticket a few weeks ago. On that, you can create new groups, delete groups, edit the groups, or change the orders.
12:41
The map definition, you have access to it. For now, we have only seen the layer definition, but you also have access to the map definition. As you can see here, it's a Scribe syntax, but you can also have a map definition in a standard map
13:04
file syntax. In fact, existing map files can be used in Scribe UI, because we don't have an export tool to describe UI syntax yet. So we still use regular syntax map files in that too.
13:22
But the thing is that you can hide it, because the map definition, we define it at the beginning of the project and come back to it once in a while. But when you don't need it, you don't want to see it. So we hide this map file, the map block definition
13:44
when it's not necessary. Now, some other more advanced feature. There is the debug output, the resulting map file, some point of interest, and a few key things that will go over. So the debug output is directly in the Scribe UI syntax.
14:04
No more debug console beside your map that you refresh and check the debug and search. You have everything in there. Right now, it's only the raw debug output, but we plan or we would like to change this
14:23
to make it more human readable. So all the debug output would be grouped by layers or by function calls, depending on the level of debug you've set. And each time you plan the map, you get a new debug. So you can see what are the problems more easily.
14:42
You also have access to the resulting map file that Scribe UIs has generated. So you can copy it elsewhere. Another really cool thing, we have added point of interest. So you can register a point of interest somewhere that you always go when you test.
15:02
So you start your map zoomed out, and just select street level. And it will zoom to the Portland downtown, where you want to test the change you made to the styles. So you can configure multiple point of interests to be able to test your changes live and quickly
15:23
and efficiently. There is also a Browse tab, where you can manage your files, your shape files. You can upload new files with that tool. Multiple users can use the same tool to upload their files or download them.
15:43
So it's centralized all the work to manage the data. This is really useful. So it's not everyone that has to install the map file configuration or the data in the right directories, things
16:04
like that. So it's really useful to manage the data. In the last year, we've also added, that's new for people who saw the presentation last year. In the last year, we added a Git integration. So now you can copy or clone a scribe UI map configuration
16:25
from Git and to Git. So if you are multiple people working on the same project, or if you just want to have an historic of all your changes you've done to your map files or your mapping project,
16:40
it is now directly accessible inside the interface. So you configure the server. And every time you want to use it, you simply push commit, enter a comment, and your map file is in the repository. You can connect to GitHub if it's a public project
17:02
or to an internal Git server if it's a private project. Here we have the one push to commit or to clone. Like I said earlier, with the browse tab, scribe UI can be used in the cloud.
17:20
That means that you can have a group of map integrator working together on the same project at the same time with the same map configuration because they all have access to their password protected workspace and each work in the same or in different maps. So they can work together and don't
17:41
have to duplicate all the data sources and duplicate the time it takes to configure their own development environment. So that's really useful as well. We also added in the last year a new plugin for Map Cache.
18:00
So now we can start and manage caching jobs. So you want to generate, for example, downtown Portland from scale 12 to 16. You can start it there, and you will see the status of the tile generation and the size of the tiles that will be generated. And you will have access directly in the interface
18:23
to the generated tiles. So you can see the end result of your tile generation. Then you can directly make a copy of the tiles and push it to your production environment. Talking of plugins, Map Cache was a plugin.
18:42
We have a plugin infrastructure. So Scrub UI provides a few key functions to manipulate the interface. So you can add buttons or tabs. And it's a really simple Python module inclusion
19:00
to create new plugins. Plugins are made in Python. It's based on the pyramidal framework. But they are really easy to develop. There is a tutorial on the scribeui.org website that describes exactly what to do to create new plugins. And there is an example, a Nellow World example,
19:21
that creates a pop-up. But you could create plugins to add new key functionality to your own workflow when developing map files. And we know that with the years, we will get new tricks. Like when we come to conferences,
19:41
we learn new tricks and new trades. And so the way we work with change over time, then that's why with this modular configuration, we'll be able to adapt to the new way people work. A plugin example, we've developed a set extend plugin
20:03
that sets the map extent based on the current view or from a rectangle that you wrote in the map. So that's a really good example that you can base your plugins on. The plugin you would like to see in the future, maybe a color swatch to be able to easily select colors,
20:22
data visualization so you can explore a shape file or post a stable zerking the interface instead of always opening it in QGIS. Human readable output, like I said earlier, and everything else that people will suggest us, of course.
20:40
In 2014, this summer, we've had the chance to have a Google Summer of Code student work on the Scribe UI. Jessica LaPoint worked on the new pyramid backend before we were based on Flask. And now we are based on pyramid because Flask was a bit harder to manage with plugins.
21:03
And it was a nightmare to install on Windows. So now with pyramid, it's a lot easier and a lot more robust. We've also added a good integration map cache plugin. We all now have a nice MVC architecture for the server
21:21
side. So it's a lot easier to get in the code for outsiders. Some tricks now. You can really now think of map configuration with a designer eyes instead of a GIS scientific
21:46
or technicians. So now it's the levels at which you will see your map that are important and the style that you give your map that is important.
22:00
So you work more like a cartographer or a designer. Variables are the best thing in the world. Use them. Seriously, use them. And Scribe UI also gives you logs when it fails. So there's a log tab that I didn't show.
22:20
But there is a log tab that when it fails to compile your map file, it tells you why. If you forgot a column or put a space somewhere or forgot to close your quotes, it will try to tell you where at which line. So it's really useful as well. And POI are your friends.
22:41
Use them. Any questions? Please use the microphone. Great presentation. Very interesting. I was wondering if you intend to have an API for that.
23:00
So it's based on programming. So you could have a REST API so that we can, for example, create a workspace or, I don't know, create a new map server, upload some files to the API from another application. We didn't plan that yet, but it's a really good idea.
23:22
The way things are currently built would support that kind of interaction. The client side is really detached from the server side, so it would be possible to do it already. But it would not be very clean, I guess.
23:40
So it's a matter of getting things in order. Thank you for the suggestion. Yes, question? It's not clear to me if the Sorry, use the microphone. It's for the to be able to get it. It's not clear to me if the Scribe and Scribe UI work
24:01
directly on files on the server in a production or you work in a test environment and afterwards upload. How does that work? The way it works, you use in a designer or development box Scribe and Scribe UI to develop a map file
24:22
and upload your produced map file to the production environment. Or you can connect your production environment directly to the designer-generated map file. But it's always safer to have things in production static and not editable.
24:46
Any other questions? So thank you very much, everyone. I'm Julien Samuel-Racroix, working for MadGears. And it has been a pleasure.