The role of geospatial open source (FOSS4G) as a component of hybrid systems


Formal Metadata

The role of geospatial open source (FOSS4G) as a component of hybrid systems
Title of Series
Terner, Michael
CC Attribution 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
FOSS4G, Open Source Geospatial Foundation (OSGeo)
Release Date
Open Source Geospatial Foundation (OSGeo)
Production Year
Production Place
Portland, Oregon, United States of America

Content Metadata

Subject Area
Currently, it is rare that an organization faces a choice between "just commercial" or "just open source" geospatial software. Increasingly, the answer isn't about "or", it is now about solutions that may involve both. Indeed, Commercial systems - both installed software and cloud-based platforms - may have both functional and cost effectiveness gaps. Free and open source software for geospatial (FOSS4G) can be a critical component that helps to bridge these gaps. And, given that organizations may have large existing investments and sunk costs in commercial software it can make sense to leverage those investments by building hybrid systems that incorporate open source components. This presentation will describe several specific examples of hybrid systems that incorporate geospatial open source (e.g., OpenGeo Suite) alongside both Esri and Google components. The presentation will focus on both the business reasoning for choosing hybrid as well as the technical approaches taken. Ultimately, hybrid projects or systems can be very important as they provide a key entry-point for geospatial open source into organizations that might not otherwise consider it.
hybrid systems
hybrid architecture
Axiom of choice Point (geometry) Slide rule Hidden surface determination Server (computing) Open source Geometry Water vapor Mereology Goodness of fit Computer animation Integrated development environment Database Hybrid computer Internet forum Blog Software framework Quicksort Freeware Matching (graph theory) Physical system Task (computing)
Axiom of choice Polar coordinate system Complex (psychology) State observer Musical ensemble Greatest element Concurrency (computer science) State of matter Confidence interval Multiplication sign Orientation (vector space) Mereology Web 2.0 Maxima and minima Component-based software engineering Stress (mechanics) Core dump Software framework Physical system Area Theory of relativity Touchscreen Mapping Channel capacity Block (periodic table) Software developer Electronic mailing list Basis (linear algebra) Interface (computing) Staff (military) Bit Tablet computer Digital rights management Exterior algebra Hexagon Process (computing) Graph coloring Information systems Moving average Quicksort Cycle (graph theory) Row (database) Laptop Server (computing) Service (economics) Open source Computer file Variety (linguistics) Real number Geometry Virtual machine Online help Point cloud Rule of inference Open set Scalability Attribute grammar Goodness of fit Natural number Term (mathematics) Operator (mathematics) Boundary value problem Associative property Units of measurement Computing platform Condition number Task (computing) User interface Scaling (geometry) Forcing (mathematics) Cellular automaton Projective plane Neighbourhood (graph theory) Polygon Model theory Mathematical analysis Set (mathematics) Cartesian coordinate system Local Group Computer animation Integrated development environment Software Personal digital assistant Query language Hybrid computer Universe (mathematics) Vertex (graph theory) Lageparameter
Gateway (telecommunications) Database transaction Complex (psychology) Code Multiplication sign Mereology Software bug Component-based software engineering Mathematics Enterprise architecture Mapping File format Fitness function Shared memory Bit Tessellation Digital rights management Process (computing) Buffer solution Right angle Quicksort Freeware Point (geometry) Server (computing) Service (economics) Open source Geometry Online help Open set Revision control Goodness of fit Database Authorization Traffic reporting Computing platform Capability Maturity Model Condition number Addition Scaling (geometry) Projective plane Model theory Polygon Line (geometry) Limit (category theory) Human migration Word Summation Computer animation Grand Unified Theory Integrated development environment Google Maps Query language Personal digital assistant Hybrid computer Vertex (graph theory) Local ring
Gateway (telecommunications) Implementation Multiplication sign Geometry Mass Open set Revision control Goodness of fit Mathematics Natural number Database Subtraction Computing platform Overlay-Netz Enterprise architecture Standard deviation Projective plane Gradient Planning Demoscene Digital rights management Computer animation Visualization (computer graphics) Software Speech synthesis Game theory
and good morning land my name's Michael Turner I might want partners and apply Geographics and I guess in the spirit of free and open source it's an open moderation here so I'm going to just dive in talk about 20 minutes on a hybrid systems now I think on the place to start is with the definition of of hybrid least my definition of hybrid it's essentially on any solution that requires more than 1 piece of technology 1 flavor of technology 1 stack of technology in those technologies may be open for commercial or both and I'm I really like this quote by from a blog that as we did a few years back on those of you have seen a recent blog posting at it I am I quoted this is as well but that the real point is that the technology doesn't matter what really matters is what are you trying to do and you to pick the best tools for getting it done and it does all have to be 1 tool they can be intermixed I really appreciate that a synthetic on another way of thinking about it is at its best of breed and don't be afraid to mix and match things because some things are very good for 1 thing or the other thing you can have the best geocoder and the best match I have met publishing environment and the and best desktop environment if that's if that's part of your solution and you just as with dogs in where were best of breed comes from and some dogs are good in the water some does a good hunting some dogs a good herding are you can you get the 1 that that you need and but it's a big market place are a geospatial environment is a bigger market place and it's hard to know what can be difficult to make choices when there are so many choices of you know and so many pieces of technology whether it's desktop or JavaScript frameworks for Geos out for geospatial servers or even database servers and then what's going what's best commercial what's best open source so it's not a simple task to sort of implementing a best of breed you have to be a good consumer a smart shopper if you will the I'm at this slide this
morning armor-piercing Paul Ramsey's talk I'm on posted posters for managers and I think it's a it's a real important set of observations that it's not an all or nothing that sometimes starting with a small constrained open-source the component can get the ball rolling and what he was saying in the suppose just but substitute any geospatial open source is maybe have 2 things for a little while your staff get used to it they build confidence and then bit by bit you migrate or build a dual capacity and you know it's important also you respect out where commercial software came from just because open source is readily available readily downloadable the savings on immediately their most commercial software is paid for was paid for a long time ago and they're real business cost to switch it on the command economic terms switching costs i can be trivial if you have a big existing mature operations the I and again you know a few more details on on the commercial side and new we've we've been around 23 + years and you know I grew up in an environment in the in the nite you professionally grew up in an environment in the 19 eighties on where there weren't geospatial salt source alternatives so a lot of people have commercial stuff because they started a long time ago and and it's not just the costs and money organizations have a lot of expertise and staff knowledge about how the commercial systems work on and in my opinion there are you know some partial systems remain the best of breed you wanna do real high and cartography you wanna do sort of SO Tariq geospatial analysis etc. you might judge the bad on commercial systems of it are very very competitive and obviously 1 of things it's starting to emerge in the open source system our ecosystem is a support network on places to go for help places where you can buy support and things of that nature again commercial that's always been that's 1 of the things you you very much a buying land so that sort of a unit that you know of you some of things that as we really does does well on and off at the core of the business men and young can talk about it some as stuff in some Google and non Google you what people were looking for is near the scalability of their of their cloud ability to have applications that can support hundreds of thousands of concurrent users pretty pretty seamlessly there's a user interface that you know a billion people on the planet used every month and and are familiar with and you can leverage that so users have an experience the familiar with and I and if you want things it's that the Google on another big web exercise in all device orientation and making it work on tablets or laptops a max of PCs of firefox or safari just happens because that's part of how the how that company has has has been built and so that's kind might my arms opening and monitors was walk through for the for use cases and basically just say you know what is the solution about why was hybrid chosen and then give you peak and that how it works the 1st 1 is a pretty simple 1 on project we did an association with a company that uses a bunch of Google technology era a newspaper conglomerate in Ontario Canada and 1 of things they have is a big online real estate listings and 1 of the things that they do with their online real estate listings is defined neighborhood so people can search for listings in a particular neighborhood and our neighborhood mappings complex and you were a neighborhood boundaries because they choose to do it is groupings of census blocks and are constantly changing and refining them so the what I make this neighborhood bigger so we got out in the census block were when add a new neighborhood and cut it out of these 3 other neighborhoods or whatever the case may be so they need to do basically polygon aggregation out of Canadian census geometry now and 1 of the problems is I Google Maps Engine does not support which which is a publication platform and does not support polygon aggregation so how do you add a perfect opportunity for a hybrid solution and armed we built a capability that does that end users posters on to do that kind of thing analysis and I'm toss walk you through with the would and this quarter's long and time at about trying to but just lucky through the work so essentially they build in aggregation file said these are the polygons we want to be in everything that we put on a web server on the world we build this side of the machine on the notorious going to get the fusion of OK I use something this because of this the the relation between the 2 you because layer all the and rover is on the low and replaces the only in those with the and seamlessly the service to the public a google search scale and with and the Google based user interface that they've had for the quite a long time that's that's a hybrid system Scott posters that some node got some Google and they all work together yeah so the 2nd examples of a little bit deeper and I you for Western Governors Association of crucial habitat assessment tool and and eventually it's the this is the publication of a data set on which habitats in the West the most important I'm from variety of criteria they ran a model and it's darker blue it's it's more important and they're making these data available about the people who are making land use choices development choices on the prostate analysis there 17 states in the West a so they put out an
RFP to have the solution built and on 1 things it's interesting about the WGA is there don't have a GIS team they don't have any GIS software they don't but they needed a GIS solutions and so that was young so the landscape was wide open for the technology was like we were doing this make it work on our thing there were like give us a good solution so we saw had the freedom to propose what we thought was the best solution and it ended up being a hybrid of arms found was geothermal the forces that Google and as we know it can start from the bottom of the core of the the the set is a giant grid of the west of of the outcome by the end of the cell hexagonal grids and about 1 . 8 million grid so it's a pretty big our dataset on in each state takes care of their own peace and so the university Kansas was was part of the team and they manage putting all the puzzle pieces together to create 1 giant grade out of the component pieces and that's done and again there's rituals for youth member rule of and JavaScript framework again a good base map imagery across the West serve very fast on geocoding I can take band Yellowstone it takes me Yellowstone National Park those kinds of all kinds of things and on and it can play nice with with other services and then we had to serve the customer layer on top of that is 1 way millions million of hexagons on color coded and allow people to click on the hexagons and get attributes through the draw shapes and all the hexagons in the summary statistics and so but 1st we were made a bunch of trials for the west so it remains fast possesses Cobol-based now put them in Amazon and then GeoServer on and poster so used to manage the geometry in the query of the geometry and and this is what it looks like architecturally inorganic Monteverde with them here on the area I put the here the hexagons ocular helplessly the files along with your with that the title and the hexagons posters and then we have web server the unit is the interface calls do all the job of service to the job of it the and serves the JavaScript et cetera manages the user interface and when people are doing various clicks that involve data activates GeoServer propose just to get the at tributes to highlight the polygons etc. that they've selected and stand up for about a year and a half and in running really really flawlessly and with good performance manager our are lunch spike gun day 1 when 10 governors mentioned in the in the press releases yeah yeah and then that the last 1 I want to talk about is the cycle drive Texas that'll work on its public traveller information system now in Texas on Google technology was chosen for scalability their arms are nightmare use cases a hurricane in the Gulf of Mexico maybe Galveston maybe even Houston need to be evacuated on and potentially you have a whole lot users like hundreds of thousands or millions hitting it at the same time and in the that this application was stress tested with 100 thousand concurrent users matters 100 thousand connected 100 thousand hitting the button at the same time and that no no easy no easy task and again the global infrastructure perform very well of but the road conditions the DOT has been maintaining row conditions data per very long time in the Esri environment and so that trick was how do you get that road conditions data into the Google infrastructure on a reliable regular basis that and to me this is 1 of my finer parts so that you get a splash screen when you come in and and it tells you when the data was updated and you'll see in 1 minute before it happened to take take a screenshot will not be ever more than than 10 minutes and that's basically this routine running silently the background I'm following
this workflow and so the DOT has a mature environment as customers retools desktop tools for maintaining the road conditions they have an on-premise database of those conditions and they publish those out without yeah server from the we built a component of runs on Linux at the node j component on that accesses the Esri services tidy any changes in the conditions from the last time it was updated and then again pushes those changes up to Google maps engine so that the current data is available to be served at a global scale and you know so again there's no open Source per say it's a hybrid work of basically go 1 has really but the thing that we built on the snow GAS tool on has they made available on open-source I'm not the whole thing there's a very specific details that pertain just to that use case but the guts of it taking data in node and push to Google Maps Engine is openly available under the MIT license and he mn through our I get repository on this if this is clear you'd see that the authors the MWA Calvin Metcalfe was not sitting in the audience and know in fact we use this exact same component in the metro line project that I am refer to to earlier so you know you try and share and share alike we leverage a lot of good work from other people and building the node tool and we wanna make that no tool contribution we added and equally available and then here's the last the last 1 I'm Maggio migration that Acció is a hosted solution we offered a local governments about 175 cities towns counties come mostly in the Northeast on but as far as well as the US Virgin Islands I use map GO-term to publish their their data and to foster G North America's ago I did a whole talk on the architecture of matching up on it's a it's a great tool it's both entirely GeoServer GeoWebCache and and posters but 1 of the things we have to do is lots of tiling and we have to tile all the data for the 70 customers and it's it's a it's a complicated ballet and and a bit of a hassle and so as Google Maps Engine has evolved we're moving the sort of tiling piece to Google Maps Engine but basically we don't have to tell we move our data up there in google serves it very fast a child fashion but Google Maps Engine doesn't do things like polygon buffers intersect queries and things like that you have to do I wanna capabilities as it is in a batter's last so even as we migrate parts of this away from an open solution outpost yes what will very much remains In the next 4 again geometry serving geospatial queries etc. and it begins its kind of the best of both world we have easy tiling fast tile delivery but we have the ability to do more geo geek queries etc. by using on posters yeah so I'm you know that wraps up the the use cases and really you might take homes there several models for 4 hybrid you it's not 1 size fits all you can mix and match and you you as a correct might instead said yeah picking the best solution the best fitness for purpose to meet the customers needs on any as you'll see 1 of the things we've been doing particularly with Google's platform is to cost effective way of filling platform commercial gaps if you don't have what you need in the commercial solution you Kannadigas yourself and as Paul Ramsay observed on a can doing these kinds of things can be the gateway get your feet wet with open source see how it works see how it works alongside existing commercial solutions and again you know all of you were here you get when you're doing these things in the open environment don't be afraid to share with share what you're doing make available of your contributions your additions to the projects are working on and I really appreciated work I know would both said about me and and on and I might that is you know Calvin no contributes code but there are other parts of our company that can contribute documentation and bug reports etc. In staying involved with the projects that that you access and leverage free on business it and I think we have time for a few questions if anyone has any so yes you know you you know you and the price of the stock data editing like many local governments is use strife so but we also use post induce a review of cash and now we're doing GME as well but I 1 of the things we've run into is getting data from st you know some sort of steel format into GME and using the Gini API is no we haven't found a way to do that you are using transactional of updates 2 g in the words of a true ETL loading the entire layer every 10 minutes houses or the the other on the but I didn't get that you have to work and I would
do know you lies in the this only the whole thing to be in the the the the but as of yet there's some but on limits in Jimmy I hit repeatedly and then it's sort of like go complained to that person with Google and they another Collier you can do that and so I it ends up being ended up being then back to me as much as possible the add 4 leaves because you get updates of the so yes this is the other funny thing and so and of being I can talk to more accessible what exactly black magic I had to do to get that to work but hang in there he did you yeah and dealers find that the solutions where the becoming unnecessarily more complex big because had to do like a detail of because he had to do like EDL or just consuming services or what will you expedient assignments that it's a it's a it's a totally fair fair question non but it hinges on that on your definition of unnecessarily on this was necessary they had an existing solution that could not handle the scale was was failing with a few hundred users in snowstorms and in West Texas they they needed that piece and so we had to just work work through the problems but on geospatial people pretty good at handling complexity it's it's 1 of those things about as refute you've been there as real world for a long time I started then there's real world and in 19 86 on prime you know minicomputer pretty good at complexity and you know you're dealing with art map RTS server RTS online there they're all different components and that you have to to make work and they do a reasonable job making the components work together but it's it's hardly simple and so it's a judgment that you that you have to make I and my point is we don't do it this way just because I say so but don't be afraid to to look at alternative models and then assess hybrid help hybrid may or may not fit for your for your solution for what you the primary trying to solve the it was no more trying to work around the current issues and then they just don't really some of the the guys the yeah very good right yeah I'm sorry that you really know the we don't want to deal with the In the end of the at the end of the day and it's it's the it's the early days and there's no question Google's enterprise offerings are at version 1 in there there's a way to go but they they do work in the sum all other third-party things on we we might have well looked into using something like FME which handshakes with GME we had to build this project in 6 weeks before the start of hurricane season and they were license Stephanie uses that would have been etc. etc. so it's the it was quick and 1 so and that the is that it it may think too quickly is 1 I think if if you if if you accept for someone who's advocating to bring more of the things in your organization you need all the best tool for the job a best of breed thing that you don't wanna be the guy that brought in some really crappy open-source thinkers is open source and the managers see it and then say if intended for all projects but but to me the the the
Open gateway drug in as re enterprise
world is you need to replace a special
database use STE but this time these posters as that proposed the back and then you know you as a fire the you can still get geometries you can still do all those things on the back and ferric register PostGIS and you can do some cool things like Ed triggers from doing visual overlays audiences it and and you can also then hit with other back and pass it on to the that it's exactly the vendor I agree entirely yeah this is the game plan this is Paul Ramsey speaking and you know and as gradient in the in the talk some of you may have been there who is just a manners that how many of you managers in about year 10 per cent of the room raise a hand goes while the rest of you here and there like we have to convince our managers and we know that that's where we are at the early days at that time for 1 more year work at the New Zealand on the Canterbury earthquake rebuild and there are problems as trying to merge data from lots of different organizations trying to work together so it's essentially an STI project and what I'm kind of curious on and that approach here because what we face all the time it's the addressing standards and how that the various software vendors don't implement standards in the same way they say this work WFS but the versions differ in their implementations differ and that it and so of 1 of the standard 1 of those of trans we take is that we don't really care what platform is behind the scenes that adhering to stand to make it to make it it's interoperable as possible and that just kind of wondering any comments on yes I agree with that I agree entirely with that and I you google on publishers added GME with a good dimuon mass very fast standards compliant 1 . 3 dB a mass but obviously GeoServer and on has WMS as has has a decent of your mass although we however to date is changes with different releases but I think that's a very smart thing have to face in the public and what other people to consume it nature you put it out there in a consumable standards fashion and and the lecture the thank you very much


  635 ms - page object


AV-Portal 3.11.0 (be3ed8ed057d0e90118571ff94e9ca84ad5a2265)