Migration To Open Source Database Systems


Formal Metadata

Migration To Open Source Database Systems
Title of Series
Nielsen, Age (Danish Geodata Agency)
Lund Nielsen, Jonas (Danish Geodata Agency)
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license.
FOSS4G, Open Source Geospatial Foundation (OSGeo)
Release Date
Production Place

Content Metadata

Subject Area
Changing data distribution from one relational database system to another should be an easy task. SQL is a standardized database language and concepts concerning spatial data is much the same through OGC standards. Still, some tasks has to be done in a slightly different manner. The Danish Geodata Agency decided to explore changing a major part of its data distribution from a commercial Oracle Spatial database to an open source PostGres/PostGIS database. A pilot project was set up to evaluate PostGIS as a productive distribution database accessed by a lot of users through open source services. Experiences were positive and the pilot system was upsized to a full scale production system. The database setup is designed to facilitate sufficient performance and ensure constantly running service. Databases and services are replicated and a master-slave relation is established between the databases to ensure immediate copying when new data are transferred from the authoritative database. A special challenge was the change-over from the old system to the new one while services were still running. New data are copied on a daily basis. Old and new system were run in parallel for a short while to be sure that the new system was stable. The change-over has mainly been done by inhouse employees, which were non-specialists in open source products. Documentation and expert service companies are available if help is needed. Experiences are positive. The Danish Geodata Agency decided to explore changing a major part of its data distribution from a commercial Oracle Spatial database to an open source PostGres/PostGIS database. A pilot project was set up to evaluate PostGIS as a productive distribution database accessed by a lot of users through open source services. Experiences were positive and the pilot system was upsized to a full scale production system.
Human migration Digital photography Thermodynamic system Human migration Computer animation Mapping Database Database Range (statistics) Open source System programming Mass
Product (category theory) Thermodynamic system View (database) Distribution (mathematics) Multiplication sign View (database) Distribution (mathematics) Physicalism Limit (category theory) Symbol table Power (physics) Flow separation Web service Computer animation Database Personal digital assistant Web service Database Authorization Data storage device Oracle Physical system World Wide Web Consortium
Point (geometry) Thermodynamic system Mathematics Computer animation Presentation of a group Forest Database Mass Usability
Complex (psychology) Set (mathematics) Model theory Multiplication sign Distribution (mathematics) Auto mechanic Web 2.0 Chaining Web service Deadlock Database Data structure Oracle Scale (map) Product (category theory) Thermodynamic system Channel capacity Distribution (mathematics) Server (computing) Computer file Expression Symbol table Table (information) Computer animation Database Web service Data structure
Enterprise architecture Implementation Thermodynamic system Scaling (geometry) Mapping Channel capacity Distribution (mathematics) Set (mathematics) Multiplication sign Model theory Virtual machine Cartesian coordinate system Replication (computing) Web service Computer animation Database Web service Database Distribution (mathematics) Matching (graph theory) Oracle
Logical constant Point (geometry) Distribution (mathematics) Thermodynamic system Number 2 (number) Response time (technology) Web service Goodness of fit Mathematics Database Dependent and independent variables Data storage device Task (computing) Thermodynamic system Point (geometry) Ripping Scalability Single-precision floating-point format Process (computing) Computer animation Database Web service Mathematics Oracle
Point (geometry) Purchasing Thermodynamic system Product (category theory) Process (computing) Structural load Multiplication sign Projective plane Expert system Line (geometry) Thermodynamic system Similarity (geometry) Word Web service Roundness (object) Computer animation Database Mathematics Database Speech synthesis Self-organization Metropolitan area network Position operator
Purchasing Email Observational study Multiplication sign Insertion loss Shape (magazine) Thermodynamic system Scalability Web service Component-based software engineering Database Core dump Medizinische Informatik Software testing Data conversion Task (computing) Thermodynamic system Process (computing) Mapping Bit Local Group Arithmetic mean Process (computing) Computer animation Self-organization Data type
Computer animation Thermodynamic system Identity management
for 3 to 4 times that of the US and photo so is is used in the Nielsen a refund the system means that 0 0 1 4 0 or see the of his which led to the United States from the day he died at the age of the of misuse body into about migrate the sound of that indeed it's and well made the and we Ouija was not so that will be seen knows what mostly the stories about him are migration database and systems for from Oracle spatial to and and yet as the that dated insisted Danish national mapping could after latencies and we have a range web-services W W mass W W sisterly and this is mostly bid W Masback back and that this will be about
also about that in the beginning of the situation we came from and what it was no longer viable for what we did and it not to this is a turned up quite well was said somewhat focused use case and reasonably symbols well and and some experience as we go along the way the
situation Hughes back to us that we had a what would call a monolithic or spatial database doing pretty much everything because that will power the agency was structured everything went into this 1 database to and all databases and so full production Systems everything went through the database was also a new authority to date stall and it was to bring data for Web services well so pretty much everything was some interacting with each other some of interacting with nothing and in this great database the was and still we still have a residing on the same day though several databases in and you can see all the same System physical system is so that was a half physical limit and there was licensing licensing issues as well they were somewhat limiting to more and more time went by and the which we have all this production flanker and in the end they would databases views created for web-services distribution and they were well increasingly complex and not at all times before only
as we would like and and
what would happen this basic this island of comedy and you went to a massive presentation yesterday morning is my body he chose this 1 as well and this is are many requests ease among and in although we were a just shy of of a 120 million which this awful database system who is no longer able to and it's not supplying this think that we made the change in 2 thousand and 11 this somewhere on the and which is here and that point the Oracle database where it was no longer scaling forests at Zola the
big increasing data distribution through a services it kept increasing that a we were doing well paid much whether we could no during this database trying to push speed how about we could and at same time and new database structure for for the commercial data was implemented which was not as simple as that might have been on the increase the complexity of the database to use of the whip services as well and did not make it any quicker we had W request going databases while they were technically legal with very expensive to and to go for it and we had at the time not a no reasonable self defence mechanism so that the W came in that the database was grinding throughout the production Systems behind all that were not able to do any work that it is needed because the database pretty much in and a top aide of deadlock sometimes and has lost we were not able to scale the only which was most of the time the database because of the licensing costs and we were found in the same physical not well and the licence fee was prohibitively expensive expression because this was exposed to the way we could have done it possible in a bid was in the the licensing his that is a completely different storey when it exposed externally to the web said that with is why we we would not have the money sell something had
to be done and we have set up that is some some up mechanism had enough of that and which bought where we do not allow was made W request into a system that a reasonably symbol of self defence and but along with the increased capacity we have proved were so that it is working quite well and data debut in with the wrong the W must of the database for some reason they when the as well but taken out and we latched on to a replica that we've but in the chain off to the oracle database and and pudding that in place allowed us to trim the database structure as well but the table structure of the defendant and the USA were but not before so that looks like it could
a much this reusable model which is basically latched on this 1 and we still have this in place and this to be the working find which is not able to scale it as we would have liked but budding these and vice time
this is are 1st implementation of it would for a couple and what we did here is that we used to in the autumn database Bojado on to make an ugly copy of 2 posted must the database and played and on the same machines we have we had installed much of it and the notion that the 1st year that the debate would probably be Ironton and maps of the city and so I thought that a good idea and this less effective was very much so not in the long run so much efficient as the 0 ahead of of managing for these databases what mostly they take care of their on stuff but we still need to set it up and implemented naked run and so we have switched to would you could cause a more traditional set up a week we have the database behind the match of the those looks of applications without front and so that is as well to scale independently what it was before the new system architecture and that is to rely would we usually have to do it this themselves situations where we need extra before manteca capacity we need to put in more looked up front and not necessarily more database machines having the club service being as simple as possible and that the 2 of them up quick and easy when they ought to cut from the same database to all of the low slung having to rely on a machine that to take and a new don't of the data and the butt of the flavour of the month the database and all that and before the machine is a slight this rose
banque this where while yet and I'd did think remove these numbers that well someone American really in those are milliseconds on their for those conceded and and we have Sarah the about so it was bad and this is where a change in the December some to be the guy following which which 1 or services to the new database bag and so that when from what 7 to 10 seconds of of the response time down to a couple of seconds but which is still not lightning-fast as gone medicines and but also to be found and other for their yields database system was in need of constant during which we didn't get it at the end because we were building a flashing your system that we just want to get any job but we did not do
that is to complete a replaced the Oracle special database which is working find of the of the organisation and for us as well read it would we did which is just a week week good and now which that we could scale the things that we needed to scale the most which was at that point the database of data distribution deal database being off loaded with the task of 76 70 us with data is working very well but still things
we would like to do and tomorrow amongst load balancing the of the databases with the looking into a deep pool of as a when tried you have a bad word we would immediately impressed with the before man's over time show what a enjoyability more into initial freshens were not that could really as you agree the talking but will it in Malta them into a voice about that but if we get we need we do need to before for the time being the the load balancing it's really simple we may need is be to end the day in a system which is basically a simple round robin Load Balancing if you can you can call it that this is the husband running so also so stable that will not winner in the dying need of introducing more advance speeches we still want them to worry that the new account introducing Scintillo self-healing system again this is must be running find so we will not dropping databases broader left when you to take care of but to create a healthier system as possible that would be nice to have as well and that was pressed does offer some features South
need it is we have going through much some of us about along with lines and we've Freida self over the Oracle spatial licensing issue are which would prohibit installed fresh databases we need them which is very nice and we were able to in the process to switch off in a controlled fashion at least to the new databases some services that even ran from multiple databases at once from Oracle and from she fresh or period between into using it and reasonably small steps which is a very good idea of enjoys being able to roll back anything failed and and the and the way the the other fist and as and when of the said he might be preaching to the converted which is this 1 there was a lot of it is very much a database system so we shouldn't be afraid to use it to get out there and we were not at all experts when we saw the best of but the party has probably Roman high on man hours but into it and low on funds as in money put but into it but it in a good projects on really and as well on the way we learn that the purchase is while fully functional of sleek and a we would really not be afraid to put it into production used will not in a position to push it back necessarily into the rest of the organisation but the and us to work with it would not be afraid of doing such a week we do it for capable of doing that as well but turning up that fully integrated production database system is a big prices and a where the point where they were considering dumping backwoods combustibility with the with the Oracle facial database feel the system is running refined said
the group earlier this 1 and they were down by 0 now on humbly so but some would rather than that of the system is 1 question at the time with setting about 5 to 10 per cent of the request dating from the 1st systems that 3 to 400 thousand requested a comes through the system which it is and brought the rest is mostly W test and W from Ross the data sauces and we have some of the house to make a week old and you piece of what they take up some of the most W W and all the rest the worst thing computing would
launch of world your to the Duke of public His and P pause for thought processes well it isn't yet the questions were ProLogis is which bought this which would usage of a component but we have built in house but you can't of to measure rose behind you find his them after the which off yet Broadway yet the question is do we run into problems with the Jonjo John conversion yet in smaller things are would say there all that's yet emitted on to get across is what you will probably need Bunton the road but we have had no shows that at all the things we have to feel from the Oracle database with so many characters in it when I go through 0 down to guide somehow Fauces the same filling the actively in the coaches database meaning Adalgisa with 0 so blanks on this week to care to conventional datatypes conversion through your job to of young along those lines and but no show so that we but the you will spend time doing an integrated the that's for sure so that he could go all the way but there is no a all do will be a tough fight the study found the real thing so maybe I don't want to complete nightly dumped from from a database and we do not want 1 other Leoma sevices as well in this but the data is ought somehow we do dumped shake and then use that from 7 1 about services just 1 particular that it was taking up more than half the time it because the dump to shape of from purchase could really quite and maps of the really quite taking it from the 1st is somehow a bit of so I worked around that pragmatic fashion Williams the loss was not that good but it is the 2nd but the yet again I can approval for a your take a step the job was that because of budget said it was obvious that a the organisation the database of the Oracle special database with no longer up to the task and everybody could see upfront what it would cost to Scalability not where the something else to be done to the Seppo for the looking young streets from above return will not don't understand so not yet but near there was no understanding and well a common feeling that this could do the job and if we were to scale on the Oracle database it would police scaled the pretty much Llanelli with spending which via
which is now under going way in the long run but that so that would we have to do something else with the identify eyed and see him and identity are a think here we still pay and a reasonable amount of money that is going off at half of the following a week of the World Cup that I take the 1


  444 ms - page object


AV-Portal 3.9.2 (c7d7a940c57b22d0bc6d7f70d6f13fde2ef2d4b8)