Scribe: MapServer Mapfile Development Made Easy
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 95 | |
| Author | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/15576 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language | ||
| Production Place | Nottingham |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSS4G Nottingham 201375 / 95
17
25
29
31
32
34
48
50
56
58
68
69
70
82
89
91
00:00
Scripting languageInclusion mapWide area networkPersonal area networkValue-added networkArtificial neural networkComputer clusterQuadrilateralEmulationMetropolitan area networkOrdinary differential equationUniform resource nameChi-squared distributionSummierbarkeitHTTP cookieVariety (linguistics)Source codeScale (map)Variable (mathematics)Text editorVideo game consoleFunction (mathematics)Web browserWindowAttribute grammarSoftware developerVideo gameRegular graphFile formatPolygonData typeSocial classClique-widthLocal GroupProcess (computing)MIDIBlock (periodic table)Texture mappingServer (computing)Computer fileSoftware developerFile formatOnline helpMultiplicationRaster graphicsResultantPower (physics)WindowFlow separationInformationScaling (geometry)Source codeMultiplication signProduct (business)Video game consoleMathematicsWeb browserBookmark (World Wide Web)Level (video gaming)Zoom lensSlide ruleReal numberText editorStatement (computer science)Computer configurationWeb applicationElement (mathematics)Block (periodic table)Parameter (computer programming)MetadataProjective planeSocial classConfiguration spaceScripting languageGroup actionGraph coloringLine (geometry)Variable (mathematics)Clique-widthAttribute grammarVideo gameFigurate number1 (number)WordCondition numberMassDevice driverCASE <Informatik>Shape (magazine)QuicksortRight angleMoment (mathematics)Single-precision floating-point formatSet (mathematics)Control flowOperator (mathematics)Point (geometry)View (database)Subject indexingWhiteboardState of matterGradientBitStatisticsWorkstation <Musikinstrument>Open sourceComputer animation
09:16
3 (number)Line (geometry)Text editorMappingPasswordTemplate (C++)Local GroupNumberDesign of experimentsData typeSocial classGroup actionOrder (biology)Metropolitan area networkMenu (computing)WebsitePoint (geometry)Point cloudBlogLink (knot theory)AreaComputerLogarithmInformationPersonal area networkCurve fittingSupremumSoftware bugNewton's law of universal gravitationMIDIMaxima and minimaSet (mathematics)Computer fontSoftware testingSoftware developerPhysical systemWorld Wide Web ConsortiumInterface (computing)Server (computing)Computer fileTexture mappingText editorInterface (computing)Computer fileSoftware developerMultiplicationPasswordProjective planeScaling (geometry)Template (C++)WindowGroup actionLine (geometry)Directory serviceUser interfacePoint cloudWeb applicationServer (computing)Demo (music)Set (mathematics)Error messageMathematicsFunction (mathematics)Point (geometry)Installation artZoom lensOpen setBlock (periodic table)Right angleResultantDoubling the cubeBitRevision controlProduct (business)Multiplication signSoftware maintenanceDialectExtension (kinesiology)Cartesian coordinate systemLevel (video gaming)Web 2.0MappingVirtual machinePower (physics)Data storage deviceIntegrated development environmentPhysical systemDescriptive statisticsLocal ringNumberSpacetimeTouchscreenMedical imagingView (database)CodeRegular graphPlanningAreaMusical ensembleMassComputer configurationFrequencyArithmetic meanRenewal theoryNear-ringCondition numberGreen's functionDampingVideo gameRule of inferenceWorkstation <Musikinstrument>Goodness of fitOrder (biology)PredictabilityComputer animation
18:32
Server (computing)Computer fileDirectory servicePlug-in (computing)MappingSet (mathematics)OvalCodeMulti-agent systemCache (computing)Configuration spaceDisintegrationLimit (category theory)Visualization (computer graphics)Function (mathematics)Newton's law of universal gravitationLevel (video gaming)Variable (mathematics)BlogInflection pointConstraint (mathematics)Software developerError messageService (economics)Demo (music)EmulationProcess (computing)Social classData typeSpecial unitary groupMIDIValue-added networkLine (geometry)Mountain passArmPartial derivativeElectronic program guideExponential functionMetropolitan area networkClique-widthIRIS-TSupremumMaxima and minimaUniform resource nameNatural numberNormal (geometry)Port scannerSummierbarkeit3 (number)Execution unitStatisticsComa BerenicesSatelliteSoftware development kitMenu (computing)Interior (topology)Library (computing)Electronic mailing listPointer (computer programming)CodeSpacetimeDisk read-and-write headRight angleServer (computing)HypermediaConnectivity (graph theory)Functional (mathematics)AreaProduct (business)Direction (geometry)Social classFlow separationPopulation densitySource codeOperator (mathematics)Subject indexingDirectory serviceReal numberCartesian coordinate systemMultiplication signPlanningQuicksortMatching (graph theory)NumberInformationInsertion lossExtension (kinesiology)MathematicsTouchscreenMappingPlug-in (computing)Revision controlWeb 2.0Texture mappingTheory of relativityGame controllerNegative numberGame theoryWordIdentity managementService (economics)Level (video gaming)Observational studyScaling (geometry)Physical lawWater vaporBlock (periodic table)Endliche ModelltheorieString (computer science)Square numberVisualization (computer graphics)Attribute grammarRepository (publishing)Zoom lensINTEGRALProcess (computing)Software developerComputer fileFunction (mathematics)Limit (category theory)Classical physicsClique-widthDifferent (Kate Ryan album)BitVariable (mathematics)LoginConstraint (mathematics)Line (geometry)Regular graphPoint (geometry)Demo (music)User interfaceError messageBackupProjective planeConfiguration spaceWeb applicationMultiplicationWebdesignColor spaceGraph coloringCodeSet (mathematics)Key (cryptography)Scripting languageCache (computing)Electronic mailing listRule of inferenceWritingRectangleTesselationComputer animation
27:48
Execution unitSlide ruleUltraviolet photoelectron spectroscopyTable (information)Computer animation
Transcript: English(auto-generated)
00:00
Hello everyone, so let's begin for a question for you. Who in the room is already using, loving Map Server? Cool, a lot of people. Who is new to Map Server or some people? Okay, that's good. And who is here just to get a country picture and check emails?
00:22
Okay So Map Server is great We just saw the new features of Map Server. Map Server is great, Map Server is powerful, Map Server is the fastest map rendering engine there is, right now on the market. Map Server is easy to install thanks to Jeff on Windows and thanks to people doing the builds on for Linux.
00:46
Map Server is easy to configure with a nice human readable configuration file. Well It is human readable. It is human readable.
01:03
And Map file development is great. It's quite a work, but it's great. Map files are great, powerful. It's possible to get really really beautiful results
01:22
from your map files. Here is a nice Map Server map with OSM data with a style that replicates Google Maps. Here's the same map with a gray shade so you can put your colorful statistical data above it.
01:41
Here's nice rasters, absolutely full map, and in this case, there's also the ocean ocean boundaries, but also oceans. There's a raster data for the ocean that's metrics, but thank you.
02:03
Here's a nice glow. I don't know if you can see it at the back of the room, but there's a nice glow around the Montreal Island. So just to make your features appear more nicely in the map and get out, get your information out of it.
02:24
Yes, that will be just if you want to do all of that, it will help you to use Scribe and Scribe UI. It will help you a lot, and that's the goal of this presentation. And if you want more detail, I'll be
02:41
we can fix it around a time where we can meet or set up a buff or something. So map file, they are great. You can do powerful stuff. With big map file, you can even use include to get to be able to manage all the information because there will be a lot. But the thing is that
03:02
everyone that has developed big map file has gone through a variety of issues. There's a lot of copy-paste involved. When you have multiple data sources that are generalized for multiple levels, you have to copy-paste your layers, not anymore in 6.4, but on all the alterations you have to.
03:23
Scales are a bit confusing. There is no variable, so you have to copy-paste a lot of information all around, and so on. For example, when you develop map file, you have to have several tools and windows all around. So you need a text editor, your favorite text editor for map file development.
03:42
You will need a browser to see the result of your changes. You will also need a console open to see your debug output, a file browser, an SQL window so you can test and explore your data. Even QGIST to see your data and attributes or OGR if you're a geek like us.
04:05
And the goal of Scribe and Scribe UI is to make your life and our life easier, to make things easier to manage, easier to develop. So in this talk, I will talk about mainly about two things. The first thing is the Scribe syntax,
04:24
which is a way to trim down your work when developing map file. And then I'll talk about Scribe UI, which is a nice user interface, a map file editor, and a productivity tool above the map file editing work. And at the end, I will give you some tips and tricks with Scribe UI.
04:43
So Scribe UI is there to make things easier for map file developers by map file developers. So it's really, we are not trying to reinvent the wheel, we are not trying to
05:00
replace the map file development. It's the request or the work that map file developers do day to day that we are the requirement for the tool. The Scribe syntax first, in itself,
05:22
is an alternative syntax to the map file syntax, like XML map file, like base map, or like any in-house format that you have that you store in a database or something. It's still based on the map file syntax, so you have to know the map file syntax
05:40
to be efficient with the Scribe syntax. It's just really a Python script that runs and that gives you a real map file at the end. Scribe is a way to make map file development more productive by removing a lot of copy pasting.
06:01
It makes project more manageable and it does save a lot of time when developing map files. Some details. What Scribe does, we will see a few examples in the next few slides. What Scribe does is, it takes your Scribe configuration and creates a lot of layers. It will create one layer
06:24
for each zoom level that you have defined. The result will be perfectly indented, you will be able to get comments inside your resulting map file. It prevents a lot of layer class and style duplication, and it's still based on the map file syntax, so we do not seek to replace
06:46
map file just to help people that work with them. So the command to run Scribe is as simple as python scribe.py. There's also a bunch of options that you can pass but I won't go over,
07:01
just read the documentation. The first thing which is great in Scribe is the scales definition. So when you start a project, you define the scales that your map will be used at, and from there on, there will be no more min-scale-denim and max-scale-denim keywords
07:22
anymore. So that's really great. It will give you things like a data statement here, where you just pass the scales, the now zoom level, because you work with the zoom level from there on to match with your web application that you're working with. So you define the zoom level that you will work at and each data source that will be used
07:47
at each zoom level. Any keyword can be used with the syntax, so here we redefine the width of a line depending on the zoom level we are at. It's as simple as that. So no more copy paste,
08:03
you just define which zoom level you want to use with each parameter. You can use either a block or simply define it for a single element. The other great thing, the other marvelous thing, are variables. You can now define variables, variables that are a group
08:27
of instructions or that are simply values. And those variables, you can put them inside your scribe syntax, so no more duplication of post-disconnection information, no more duplication
08:43
of your most common WMS metadata. And if you want all your streets or river or oceans to be the same colors, then you define the color once and you just change it once. No more find
09:00
and replace of this single color statement. Another thing that is really useful is the common blocks. If you wanted, if you already developed with massover or if you're new to it, when you want to comment out a layer, you have to
09:23
put a little hashtag like that next to every line. So that's a little bit of a pain. So now in scribe you will support command block with slash star and that will put a block of code in comment. If you want to still have comments in your
09:45
resulting math file from scribe, then you use the double hashtag comment and those comments will be in the resulting math file. So you can still leave notes for application integrator, you can still leave notes to yourself when you will try to debug or change the
10:03
resulting math file when it's in production or in a staging environment. So scribe, use it. It's really cool. It does save a lot of time. We already have a lot of projects that we use it and it's at least twice or twice as fast as regular math file development.
10:30
Now, scribe UI. Like I said, developing a map file or for maps over is great, but it could be
10:47
easier. Scribe is a good step to make it easier, but now it will be even easier if we add all our tools in the same place, all our small tricks shared together.
11:08
So scribe UI is indeed a map file editor like there was before. There was, I think, the two most popular in the past has been map lab and map storage. There's been a few other as well,
11:22
but each map file editor eventually died down because when you have a tool that is aimed for newbies, those newbies usually don't really contribute to the tool because they are newbies. So scribe UI is indeed a map file editor as well, but
11:43
it's more than that. It's a tool for the power user as well. That's why we hope it will live long and prosper. So a basic tour of the scribe UI interface.
12:00
First, we have workspaces. Workspaces are projects. When you create a workspace, you can, or you're, but you're not, it's not mandatory. You can put a password to it, so you can be multiple map file developers working on the same project, maybe different
12:21
players, maybe different scales. A single project can obviously contain multiple maps. Then you can create new maps inside the project and in scribe UI there are small goodies hidden everywhere, well not hidden, but everywhere there are small goodies.
12:41
For example, who in the room when starting a new project start the map file from scratch? Nobody. That's what I thought. Nobody ever does that. So in scribe UI, we are well aware of that. So you can copy any map file that you already have as a template for your new project.
13:01
So you can either define template map file for all of your project or just copy a map file that you had in other projects two years ago. That still works. So when creating a new map file, you always copy an existing one, give it a description and it will now be available for
13:21
to copy for your next new project. Now the editor, which is the main block of scribe UI, in the editor, we edit layer by groups. So when you select that group, you will have all the layers in this group. It's color-coded, there's line numbering
13:47
and when you make a mistake in the scribe syntax, you will also have nice error message that tell you which line, which layer is problematic. Of course, there's a
14:01
map view of this for the sake of the window screen and the people at the end of the room. I didn't put yet an image of the whole application, but we have a demo at the end if times permit. The layer groups, a new layer group can be created, deleted, or you can modify
14:24
over of your layers inside the application. The map definition, one of the other great thing is that we trimmed down the user interface. So you don't have the map definition at the same place as your layer definition. So it's easy to access if you want to access to modify the
14:45
global map definition and you can hide it when you're done so it doesn't take screen size, screen space. Now that was the basic introduction to scribe UI interface.
15:02
Now more advanced tools for developers. One of the first and most important one is that inside the UI we added a small block where your debug output are displayed. So that's the first step towards getting everything together in the same interface so you don't have eight windows
15:26
open while you develop map files. As soon as you plan the map or zoom in, you will get the output there and at some point it would be cool to have more human readable or trimmed down debug logs. You can also get the resulting map file in another tab so you can immediately see
15:51
the result, copy paste it somewhere in another file or whatever. You can also export your whole map file with all the data in a tarball that you will be able to download and
16:04
install on a projection server. A really really cool tool is the point of interest that we added in scribe UI. So when developing map files, what we notice is that we are always coming
16:21
back to the same points. We have two, three, four, ten points of interest that we want to test our change against to see if it's the the result are good. So we added point of interest so you can just select them, the map will automatically zoom at the right zoom level and the right
16:41
extent to show you the result where you want to see them. It makes it easy to test several regions when editing styles. Now scribe UI in the cloud. Scribe UI is a web interface, a web application and it can be in the cloud or simply installed on a server that you share
17:05
with your co-workers. So it allows developers to work in groups on the same project without duplicating the data all over the place. So it removes a lot of headache for the data maintenance
17:22
so you can work on a centralized system all together on the same project at the same time and you can password protect your workspace like I said. So you can, if you want, give access only to a few key developers to a single project or to your production version of the application.
17:44
No more installation puzzle. You install scribe UI once and it's available for everyone because it's in the cloud. You can also install scribe UI only on your machine and work locally. There's no problem with that but it's a feature that is that we found was cool because when you're
18:05
working when three people are working in the same data set especially on an OSM planet data set you don't want all of them to install the data on their machine. And to work in the cloud we also have a browse tab where you can download or upload new data on the server so you don't have to
18:28
SSH or SCP your data files on the server, install them in the right directory and things like that. You now have a nice plug-in here where you can simply upload with a few clicks
18:43
your data file in the right directory on the server. I mentioned plugins, that's because scribe UI support plugins. The key for this project to work is of course people using it but also make it even better as a productivity tool. So for that we added plugins which are
19:07
small component that developer will be able to use or develop new one. So we added a few JavaScript functions like the add button or add tab that will be allow you to add components inside
19:24
the web application and with you will be able to develop small python script that will do the server side of your of your plugin. Scribe UI is a flask application written in python
19:44
and if you're familiar with that well each plugin gets their custom rules so you can access the code directly from from the plugin directory. We hope that in the future there will be a lot of new plugins implemented. We already have plans for some of them if time permits obviously
20:04
and we hope that other map file developer will identify an area where they need a new tool develop a new one and share it with us. So here's the example plugin that we have. The first example plugin that we have in the application is the set extend plugin where you
20:26
have a small button that you can see on the screen and it will allow you to drag as a nice orange square or rectangle on the map and it will automatically change the extent of your map file
20:41
instead of trying to write numbers or copy pasting the output of an OGR request or things like that like we always have done. So you can refer to this plugin to get a code example it's really simple and it will show you how to create new plugins if you're interested.
21:05
The plugins that that are to come will be the probably the first one will be the color swatch so you can have a list of predefined color that you just click or if you add a new color it will add a new square little square in your application that you will be able to click to
21:23
just identify which color is going where. Then I would like to have a map cache configuration plugin where you can define your map cache configuration what is on your server and launch and monitor pre-caching jobs. Again it's to have every tools that we use every day
21:48
in the same application integrated with each other so when you create a new map file you will be able to just click a few buttons and a pre-caching job will be started to generate
22:02
all the tiles for all the words at all zoom levels on your server. Some other plugins that we talked about a git integration so you can when you save your map it goes automatically in a git repository so you can have a source revision control. Data visualization so you
22:21
can upload a shapefile and see the attributes. Human readable output like I said earlier sky is the limit or the imagination and time of the map file developers. Now some tips and tricks basics with Scribe UI. The first thing is that you can now
22:46
think of your maps as levels instead of services. Each zoom level can become can now become a map in itself because you will be able to easily define the width color spacing
23:02
of all your features and this will allow you to consider your map with cartographer or web designer eyes instead of with from the application developer constraint my data are this and that and I have to make the line blue things like that so it makes things easier and
23:29
more beautiful for the end user as well. Of course variables are the best thing in the world so use them. Scribe UI logs will tell you where your syntax are so it will be easy for
23:48
you to define to know when you have an error and point of interest or your friend use them so a couple of minutes maybe a small demo so it's installed locally hopefully it will work
24:07
right so here I only have one workspace with the Scribe syntax of course Scribe UI still works
24:22
with the regular map file syntax here is a map of the world Scribe UI when you install it can come with a few two well one that I set with two example map files so you can start from something instead of nothing here we have the regular map file syntax
24:43
inside a map file let's open the Scribe map file quickly here's the ocean layer you can see that I use variables for my classical layer configuration and some zoom levels because my data is generalized depending on where it is in the roads layer there are a little bit
25:11
complex you can see that we have some blocks to define several classes at different scales
25:21
and you can also include components that comes from multiple data sources here I have defined a layer that comes from PostGIS that is all the OSM roads in Nottingham this little button here opens the debug output so if I pan the map I should get some
25:47
didn't try it I should get some debug information yeah yeah debug information here variables available from the click of a button the map configuration available from the click
26:04
of a button and if I make a change to a layer for example the I didn't zoom at the right place
26:22
let's go to Nottingham here so let's change the color hopefully I change the right layer yay
26:44
so that's how scribes work scribe uis work I would recommend to use scribe so source
27:13
control your scribe file yes and if you have a file going into production then
27:22
make sure to source control the projection version as well the resulting just to have so you have a backup of it no not yet sorry