An Open Source Approach to Multi-user Distributed Geospatial Data Management

and the other African talk open

source approach to multiuser distributed geospatial data and so that's the thing about taking away good afternoon everyone my name is Danny I work for the hell out of Africa layout Africa is a non-profit organization based in northern Nigeria with their quarters in Canada we

have in offices in on Guinea and Liberia I and Nigeria and we so-called public health interventions using data driven approach in in multiple African countries

and do so do different areas where about projects that focused on how delivery systems and deploying and call centers operating in emergency operations centers you know and and supporting them for their education and nutrition and severe acute non-traditional across different across different

parts of West Africa and now this defined on projects and interventions that we support required Rocky data retrieval and decision making which means we deploy multiple

that to support decision making and the different modes

of wood that what's and you to support near-real-time decision-making and require geographic information to power of D did decision making and we do a lot of we have a lot of people without bulleted if you want to do in data collection I'm in different parts of northern Nigeria especially in places with a bad natural kind how really 2 areas and to use and

require as those deploying multiple mobile devices for offline data collection and in the we

have GIS and especially is that take these different data as they are synchronized and near real-time all them off line you know editing using

satellite imagery and committing invention what did you

do that is to make different maps at the same time and feeding the information into then the dual database to support decision making now

and the main thing with this is when we started we started with its proprietary softly

rice whereby we deployed a multi-user data management system as you can see it in a sequel server believes that agrees with need to wear on top of it and and then and different versions created to to so ports more use editing you know you create your checkouts of and unto your editing comets we didn't know how this works for us and then we define the workflow but the main challenge wake and the the you know the different sort of workflows that we've be using uh with a proprietary software now the main challenge with disease it's really expensive and for projects that we work on their require us to transition and the data and the systems that we built to the government and sometimes it becomes not sustainable so we have to find a sustainable we're working

we talk about using an open source approach to to

the this and the 1st thing we did come along the line was to 1st use and so we wanted a situation whereby we can deploy proprietary software because proprietary software something is always easy to deploy and especially in emergency situations and we also wanted to have an open service and in the system and to to continue to work so the 1st thing we did was to run the hybrid system

whereby the federal the with several posterior samples grants and several not server unsynchronized did to databases by running from jobs and synchronizing them together using a unique identifier so with the proprietary software generates a GUI the ball with they're not adopted in the open source software was also use a UUID system and and credence to much each other and keep them synchronized and Bowie we then needed to take this for that so we looked into the solutions and in the month in the open source markets and 1 of the things we that with the and and solutions which we already talked about here and it's good to see and that these are really improved a lot of developments that have gone into it and so we looked at duty for example we agreed that's and the was was PG versioning and those of the president ball and all of them have their different limitations for our workflow this all we needed exactly was it

clients you know what Q GIS and being able to do regrows geoprocessing processing operations on on the text of periodicity in time synchronizing these changes back to the database so we could eventually found acute especially in so those already been deployed by all slander and we said and as you can see when we finished and said in all these both in a certain object versioning environment this is what the proprietary looked like and this is what our open-source environment looks like so whereby the user check out your data from it GIS that this into a special lights on and there these 2 do we look at editing and if you wanted to connected editing you check it out from GIS to post from the Judaizing to freeze and then you do your editing unsynchronized back into the hysteresis them so with version is there between the different pose GIS systems online on the server then you can QC of this synchronized true from the master database and then do your buckles all using the POS GIS QG areas and spatially so what if all

we can't got here and we have to do is a lot of development because of the

Auslan 2 was not in the from register I did so and 1 of the things we do we have to spend a lot of money and hired and

develop person to do this and so being in Nigeria for example in Africa you know do something you people get the perception that when you say opens it's free book books in the real sense open source is not free you have developers who are commuting yet only when it is made of a label-free and it's not actually free so we what we all slander to improve on the 2 and I'll talk question was how we got from where we will have in proprietary software to where we are in the open source software by

the 1st thing we needed to do was to look at and I walk through In the proprietary

and environments and we went through the list all the different the thing is that we normally do using the proprietary software and we did the prioritization of what we needed and for example you things like and checking out locally with very important for us we might

different priorities to be sure we developing and

so the the the other priorities only created

now I don't believe what I should of this in South Korea last year when we about start so we saw the offline there on the website we spoke to them and then you

know desired the basic characteristics of the 2 when we started you know it was created in 2014 for example and so called enterprise

and you data management you can do branch branch in and manage your branches when you check out checking and but changes accommodate different branches of then you contract the history of all the different changes I have made them to your database and right now it's not our integrated into repository and when we started it wasn't so you have to download a plug in and what we started and 1 of the challenges of which the tool when we started was the window and beats you could do a post postprocess and Bryant creation as there was some boxing right and also the dissipation light version was out of the there was no support for these and at the same time so that that means you couldn't do it you know you check out into spatial I tend to locality and also 1 of the main requirements I mean we've been talking about conflict resolution here but when you have like that then you know hundreds of people working on the view that people working on laptops creating edits it would if your software has cropped up conflict resolution strategy you need to manage your walk through to reduce the conflict so for example even if you have a conflict resolution and mechanism if you have 2 thousand conflicts you know when I'm going to finish resolving those conflicts I have to do this you know on a consistent basis so 1 of the is we normally at all to do this is the structure of the web by limited people to specific geographic areas and specific putrid so you do I attribute and geographic filters when you are creating your check out so people can only edit specific areas now these 2 also according to the right so we have to see them and down in gets out what we did develop developers to build this functionality into the into the 2 so looking at the

different problems we have these are the different fixed is that we did to add to the software so for example and we fixed the spatial light accounts and and now you can check out from more your posed serious database and check out into a spatial light and and do your editing merge branches back to depose GIS and database and when checking out you can do and geographic and filters on the on the GIS interface and create your check out into your spatial all into your post post yeah that these and then when you finish editing you can made all the different branches branches this so 1 of the other things that was also a you in which is also a very which is always a very very big challenge in the open source community is documentation right there was no proper documentation for this so and 1 of the things we did was to implement Sphinx and to do and the documentation I will shortly interface after these so on and then while we did all these works and did you I was also not very very intuitive and these are some

of the improvements we did need to and graphic user interface as you can see it and you can't see the difference differences when you create a check out and they're working on it and you can then merge these branches by and we also and improved the the GUI TQ GIS interface whereby when you want to create your own you check out to you you know you you point on the the gives you the right information of what you need to do and you know how and when doing the revision history we also did a lot of some improvement on the user interface to to make it more intuitive right and indeed

documentation so we implemented so we started from the it temporary documentation which is the the 1 we deployed on a local server and then move these over to most land to integrating the normal changes version of the

documentation so this is not a very very technical presentation but I cannot answer technical questions where I'm here to show was used to show that how there was indeed implementation community can walk with open source and developers to make improvements to open source software and you make community better because you need money right to creates tools for people to use and then you need the appropriate feedback on how people are using to to to create a you know an effective with so this a perfect example of how we work with the open source community to create products that are not very useful for the rest of the community based on use cases of how we envision open source software to to move from

an intense although enhancements and we are not there yet and destiny a lot of things and to be done right now with the software is the only thing you can do with offline editing and the spatial so if you create your and check out this week's post progressive you need to be connected to to the editing so you'll be good to be able to create a check out to my post Chris and just go line right and finished my work and manage the branches but so these need improvement also you have to create a technology spatial to a specific location when you move to location you know all of where you create general and specialized account it loses the connection and you can't answer merger branches but so these are the areas of for the improvements that we're looking into working on in the in the future that happened I mean we've seen the different presentations on on you know only in the previous president the conflict resolution user interface is very very intuitive so this is again something that needs to be improved so especially now that this is a tool that is readily integrated with you GIS that is going to be available to users to use it will be good to see it properly conflict management and you know

conflict resolution interface whereby you can't make intuitive decisions emerging and branches or doing you know choosing what geometry of edits to to speak and when the resulting your conflicts and indexing and these are the people that I

worked on the project and Eve is the development that we have from the open source community to to do this development and we were so what did the developments in and out all slander to objectives working yeah that's it thank you thank thank you so that was very rude the all 3 of

the 4 and the questions but the so it is that's going to overlap here with the Geoghegan what they're doing had and as right so things by developed to meet some of your kind of each development needs in the spirit of open source collaboration Jim is a case for merging the best that's about project sodium actually have different use cases as well and so while listening to the that's a good question while listening to and the dual presentation 1 of the block for us in adopting duties words because he was not integrated with the proper GIS clients you know like Q GIS and connecting directly to it database to manage your history so you have to use duty to do the is true which is a the which is the bottleneck because we're talking of where the network is slow for example you're going to have problems with this so and it will be good to see and and so this is again this is why I'm here right in this period of open source collaboration and it would be good to see these players work together that's what we just did is to support an existing work if we we're trying to create the product what we would have done was to for about product and then creating new product out of it but that's not what we after what we our users what we just 1 is tools that can help will solve our problem so it will be good I think there should be collaboration to answer questions and these will be be good to see this collaboration especially because you know did GIS community of adopted is to let the default within so should Judaized community facilitates slide there to work very closely with 2 and merge these together and then maybe they come up with business collaborations will support that's something we would like to see that does not say question it question other questions the and talking about the users I was wondering if now you changed for more close so fair to a more open source projects I'm wondering the users of what after any can you shows any experiences what how to do they like it's a healthy thing to think about this change into and you will probably for them but not the that would 2 questions so with every change in technology commons capacity building and and what we look at when doing this is the gains in the long run I mean so for example you pay developers to improve the tool you use money to train the people right to get then from with open source but in the long run the cost savings you get is different from paying expensive licensing costs onto the you consistently so the answer to your question is what we did during the development of the 2 was to start training all the users within the team on how to use q GIS on how to use Paul's GIS you know so that by the time you know the whole 2 was set they were comfortable to move over and if you notice what I said in the 1st place the 1st thing we did was to set up the 2 architectures together then move some people to open source while some people stay on you know the proprietary software the reason is because for every technology daddy any all right so the idea adult as the people that best is always best to choose your best people in proprietary are them all over to open source because they then crock everything that is needed within the open source so that's what we adopted and in that stands as the challenge you know for the rest of it and because they have to meet so eventually they moved over several more questions yes and no could you tell us a little bit about a timeline going and on and did it take you to do the transformation from the proprietary software to the open source solution of the good and so we started this project that are September last year right and we completed the development the dental became ready to use our own February in February March this year but very mind we had just to developers and to work this and 1 of them had the data that was working on it in at a later time while someone else was working on its own full-time and then there was a developer from was larger those working with the developers that we high so so roughly took about 6 months you know to have his work but in terms of the migration rates the migration and also to I would say about 6 months right because the 1st thing we did was to say it all and the architecture and then you know how people work on it incrementally ball right now in hindsight looking at it and I would say you can migrate and we have a fairly large database supporting units on the interventions in in different African countries so I would say you can move over i in like 3 months right both training your users having been used to solve them the the applications everything understand so the question yeah any other questions them and then give my best to connect you with the other speakers at that that think this was really a very good example of like the whole state of the art in good you can also little bit more about how often you have 2 thousand user whatever I think that you you're quite even if another use cases that they may not have to the a 3 and so on and this is brings me to the main reason why joined eHealth Africa and now because this problem even existed when we were using the proprietary software so you just create your checkouts right and the whole area and people are educated now the challenge that is someone does select all I'm just moves the features on which the book is change right and then when someone does the proper right so imagine that you now have people working in different states doing proper edits died then results in the in in conflicts right so how how do you manage that right so it would be proprietary software couldn't solve the problem when I joined the organization so what we did was to create walk-throughs right so sometimes well there may be a way to solve the technologically book I wouldn't so when I can solve something technologically then I decide you know it is not a technology problem then is a process of problems so if you can't change a technology thank you change of processes where again it would be good for you know for the technology developers to understand the processes of the users and structure that technology in such a way that it's you know makes life easier for the user or even you know what with the user closely and you know for example if you to this documentation now you can see that you should you can't geographic filters and use that to minimize conflicts right but would be good for things like this to be included even in the documentation activities and eventually did technology developed to solve the problems like this or what they can do very much group recently