Farm Maps Online: An Open Source Success In Norway
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
Title |
| |
Title of Series | ||
Number of Parts | 95 | |
Author | ||
License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
Identifiers | 10.5446/15528 (DOI) | |
Publisher | ||
Release Date | ||
Language | ||
Production Place | Nottingham |
Content Metadata
Subject Area | ||
Genre | ||
Abstract |
|
FOSS4G Nottingham 201328 / 95
17
25
29
31
32
34
48
50
56
58
68
69
70
82
89
91
00:00
Open sourceLevel (video gaming)Computer programExecution unitMetropolitan area networkPointer (computer programming)InformationSoftware developerForestMaxima and minimaLetterpress printingFunction (mathematics)Client (computing)World Wide Web ConsortiumDifferent (Kate Ryan album)Web serviceLocal ringFocus (optics)DatabaseLoginCategory of beingSystem programmingOvalDecision theorySoftwareServer (computing)Parallel portFrequencyGraphical user interfaceFluid staticsConstraint (mathematics)Value-added networkAxiom of choiceCodeBefehlsprozessorPlanningPoint (geometry)Phase transitionSanitary sewerComponent-based software engineeringMobile appFront and back endsGeometrySpring (hydrology)Java appletBroadcast programmingInterface (computing)Generic programmingTable (information)Random numberStress (mechanics)Software testingError messageTopologyDisintegrationEnterprise architectureLogicTomcat <Programm>Configuration spacePhysical lawLibrary (computing)Open setSampling (statistics)Computer-aided designOracleProbability density functionGrand Unified TheoryComputer clusterSelf-organizationComputer hardwareCycle (graph theory)SpeicherbereinigungStructural loadScaling (geometry)Stability theoryMoving averageWide area networkInfinityBoss CorporationPhysical systemSound effectTotal S.A.Data managementSoftwareBitSoftware testingCartesian coordinate systemLevel (video gaming)Error messageSoftware bugBefehlsprozessorWordPairwise comparisonSoftware developerProjective planeKeyboard shortcutTable (information)ForestServer (computing)Process (computing)Computer programmingClient (computing)Hash functionStress (mechanics)MereologyOpen sourceRun-time systemFunctional (mathematics)CASE <Informatik>Electric generatorSelf-organizationInformation technology consultingMetadataWeb serviceMobile appPoint (geometry)Open setFluid staticsSet (mathematics)CausalityINTEGRALPerspective (visual)DatabaseView (database)Physical systemSpring (hydrology)GeometryStandard deviationInterface (computing)Interrupt <Informatik>Communications protocolDifferent (Kate Ryan album)Slide ruleJava appletMappingCore dumpData miningObject (grammar)Enterprise architectureWeb 2.0Stability theoryWeb browserMultiplication signInformationCategory of beingConnectivity (graph theory)In-Memory-DatenbankComputer programStructural loadCodeMathematicsGrass (card game)Network topologyLatent heatAreaAxiom of choiceNatural numberResultantWebsiteAuthenticationGreatest elementUniformer RaumGroup actionLetterpress printingBlogTraffic reportingPlanningWindowProgrammer (hardware)QuicksortSystem callPrime idealWater vaporService (economics)Power (physics)Moment (mathematics)Building1 (number)Commitment schemeWhiteboardWorkstation <Musikinstrument>Bit rateFourier seriesProfil (magazine)NumberWärmestrahlungBoss CorporationVideo gameSemiconductor memoryProduct (business)Computer fileHypermediaEndliche ModelltheorieOrder (biology)Archaeological field surveyDaylight saving timeDivisorCoprocessorRight angleDecision theoryGame controllerProof theoryDialectGoodness of fitPolygon meshMusical ensemblePhysical lawMetropolitan area networkScaling (geometry)Computer animation
Transcript: English(auto-generated)
00:00
We are now going to talk about the farm app online for a shortcut called GPI and another program called Regional Environment Program called RMP. Both of our programs are a success in Norway.
00:20
They are used a lot and they work very well. Open source has been a main contribution to that success as we see it. A couple of words, I work at the Norwegian Forest and Landscape Institute and they provide information about soil, forest and do some research around that.
00:45
I am a developer and my name is Lars Apsal. First, I want to say a couple of words about the application so you know what we talk about. The first application is GPI, the Farm Map Online application, it's a web client.
01:03
It's the minor tool and it's a major tool for the farmers to collect information about their farms. That means area information, information about what kind of soil it is and so on. It also contains information about ownership and it has different reports for printing
01:23
etc. Here is a short picture of the application. You see here, you have kind of area numbers, you can probably see it, it's there. And there is the ownership information, is there anything to point to there?
01:41
Is it on a pointer or not? There was one, okay, I'll try this one. Here you see different ownership information, you have advanced printing of properties and you can do a search on any property in Norway. I said it was mainly used by farmers but it's also used by private persons because
02:04
it contains information about all properties in Norway. It's open so it's easy to use for everybody. It's also used a lot by community employed or governmental employed that work with agriculture. One thing in Norway is that farms seem to be bigger and bigger so you need to group
02:24
properties together because you rent a neighbour farm or something. So that is new functions that are coming now that you make, construct your own farms, you group together all kind of farms until you have the correct area that you actually are growing on.
02:45
The next application is called Regional Environment Program. That is also an application that takes the information from many different sources and presents it to the user through a web client.
03:00
This is a client only used by farmers and they used it to apply for subsidises. This application replaced all paperwork this summer so a farmer cannot use paper to apply for subsidises anymore, he has to do it on a web client.
03:26
That of course means if you're going to apply for money we need to know who you are so we have to integrate with mean either or as a national system for authentication. This system is used by external systems so external systems connect to this system
03:45
to get the information that the farmers draw on the maps. I can show you a short overview here. This is some of the complicated part. Like in Norway there is about 100 different ways to apply for subsidises.
04:03
And from region to region that varies. So in one region you may have 50, in another region 70. This application is the same application used all around in Norway so it behaves differently according to where your farm is.
04:29
So according to which region your farm is located in. You get different choices for applications.
04:40
Here I have applied for grass text one way. Everybody knows what that is? Yeah, that's good. That's a kind of subsidise. So you have many different choices there. Where you can apply for that kind of subsidise is kind of difficult to know even for farmers and even for the government employee.
05:02
So it was a problem earlier because they are really not actually saying can I apply here or here or here? No. You can only apply at this area. So the system creates a legal area for the farmers to draw on. So if you actually made a road like here, he draws the road.
05:23
And if he tries to draw on the outside, it's cut off. So that helps the farmers to make correct application and it saves the community a lot of work to control the application afterwards. And of course you can edit the geometries
05:41
and you can delete them or do whatever you want with them. And that until a certain date because after a certain date the application is closed and the farmers get their money back. Now back to two-test-on-file. We had a very old static HTML client and they wanted that fixed.
06:04
They wanted a more dynamic and faster and more functionality in their client. We had a requirement specification ready. We had four months of development time. And we had about four to five programmers available.
06:21
Some of them are sitting here. Software investments had to be clarified early in the project. And since the short time frame, we had to work on client and server in parallel. And at Google landscape where I work, we use open source if that's available.
06:48
But now we're going to turn this situation a little bit around. Let's say use only commercial software. Let's see what will happen with the project.
07:01
And we also wanted to install by only certified engineers. So we didn't take any risks on anything. The workflow would look something like this. First we had to use some time to figure out what kind of software to use.
07:24
And then we had to wait for price quotas from the vendors. And the price varies quite a bit from case to case. It's quite impossible to tell the price before you actually have the correct run time environment you're going to run on.
07:41
And we had to wait for internal discussions regarding prices. You really mean that you need 100,000 bucks to buy this? Well, in our organization that is impossible. And then it needs handling licensing misunderstandings. Like you mean you run this on VMware which has 64 CPUs?
08:04
Okay, then I have to pay for 64 CPUs even if you only use two CPUs on your host OS. And then we have to wait for the engineers to install the software. Then we can just start to code. Well, you know what the result would be.
08:22
Yes, it would be failed. And because one reason the coding would be late by weeks and another reason it wouldn't be money for it. We would just continue to run the old application with no changes and that would continue to run until it wasn't possible to use it anymore.
08:47
And another reason it would have been too expensive. Like we had to rob a bank to get money for that software we needed for that project. There is a silver lining which means a hidden benefit here.
09:02
As a developer I wouldn't be blamed for it, the project's failure. Because I was only waiting on external resources money so it wasn't my fault. I would have a nice spring, quiet spring and walk around. So, but some might say that is bad planning.
09:21
You should know that you needed like one million bucks this spring to software. But that's the opportunities you get when you work with open source. Because if you should stop in 2011 with the project you worked on, then you have to get a team together, start to plan this project and find out different, what kind of different software we needed, what kind of protocols we needed.
09:45
And that would mean an interruption in the current working project. But with open source we can do it like this. Say here's your project, hopefully you find open source that you can use and then we don't have to take all the budget discussions that they have already taken
10:04
or we have the amount of money we need because we don't need so much money. Let's do a comparison.
10:22
Yes. So we did go for open source and this is some of the server side components we used. We sent those for all servers. We use PostGIS, PostGresl, PostGresl and gear server and map server, Java Spring and Hibernate Spatial.
10:41
That is not very often mentioned. That's a very good tool, the Hibernate Spatial. It works very well and if you work on a set of tables, like if you work on 10,000 different tables and you never work on the same table, Hibernate Spatial is not the tool or the package.
11:02
But if you work on a set of tables, a small set of tables and you know you are going to do a lot of work on them, Hibernate Spatial is quite nice. It works very well. And another thing with Hibernate is that a lot of consultants know Hibernate. So if you like to take gear tools to consultants, they don't really know about it.
11:23
But Hibernate knows about it. The new GPU application was launched in June 2012 and the development was done on Scrum-based. We used SVAN and we used Maven. The Maven tool is very efficient, as you probably know.
11:45
It solves a lot of problems, creates some but solves the most problems. And people were developing on Windows, Fedora, Ubuntu, that was no problems. So it works very well. To get the system up, we had the client interface ready in about two weeks.
12:05
And that was done by using XSTs and Spring to generate web services. We use temp data in Postgres using generic data tables. And that works okay because we are storing tables, data that looks alike, pretty much the same, but is different kind of data.
12:28
So we didn't have to create like 10 tables, we created one table and we had metadata about what those tables were. We could of course use an in-memory database there, like MongoDB or whatever, or BerkeleyDB, like a hash table database.
12:45
But the reason why we use Postgres for temporary data is that those data are going to be viewed in GeoServer and MapServer later. So it's much easier to connect to Postgres by GeoServer and MapServer.
13:02
We did run quite a bit of integration tests and stress tests. And the importance of stress tests is of course, it's important. But we actually did run many thousand farms. So we got the system well tested before we went to production.
13:24
And since farms can be put together by many farms and by many properties, we talk about many hundred properties. And to compute ownership you have to check properties and get properties, so it's quite a complicated check, it takes some time. So we have to use a lot of currency to get it through, or else we have to wait in minutes.
13:45
And of course you have to handle topology errors in SQL or Java. Let's do a comparison between commercial and open source. Here is the commercial solution. If you go for Microsoft, Oracle Enterprise, ArcGIS, WebLogic and TopLink on the open source, you know it.
14:08
We go for this very simple solution to save money. It wouldn't work though, but it's simple. So we have two servers, one database, one application server and one VFS server.
14:30
This will of course cost something. And here's the price for open source at the bottom and commercial software on the top.
14:40
There's some percentage difference there. And I can't guarantee for the prices because if you read the S3 price side, you can't say what the prices are because there are so many different factors, so it's quite impossible to put the price on the web. You actually have to see your server and find out what the price is.
15:03
And it's pretty much that with many other systems because it's like if you want to find the price, ok, thanks. Here we go on. There's also one difference between governmental and non-profit and private organisations.
15:26
Like heavy load on private sites usually means more revenue. But on governmental and non-profit organisations, more traffic does not really mean higher revenue. Of course you may get some more hardware, but there's no automatic relation between traffic and revenue.
15:53
Was that the previous one?
16:03
How can we say that open source increases stability and performance? Because we could scale up the system with no extra cost. Like if you had gone for the first solution, you get to August, where the traffic is high, we're up about 500,000, more than 500,000 requests total.
16:24
And you have to ask for more money and you have to get the engineer to install the software, it will be a failure. So it's not difficult to scale up the system from the beginning because it didn't cost us anything. It's easy to install, just take the servers, put them up and we are ready for traffic in August.
16:44
Like if you buy an extra Oracle license for a 64 CPU, you hopefully use it. We also did a lot of horizontal scaling, it's easy on this system because of the nature of the data and the usage.
17:06
Here's the system as it is now, there's some more servers, but we won't go into details here. There's one more here. Another thing with open source is that it makes it easy to add new functionality.
17:24
Because you might buy a system and you cannot buy an enterprise system, so you have to add new functions all the time and that costs money. But usually here it's very easy to find a plugin, add it, dump it into your system and you are up and running.
17:43
Like we needed JSON support, it was very easy. We just took the Jackson package from FastRXML and we run that on the Java objects and we suddenly have JSON. It was very easy.
18:06
For integration we need to use OpenAN with the MineED. Yes, these functions are added after the GPI project.
18:25
This is the different servers in the system and the interfaces. One important thing in this slide is that standards. Without standards it would be impossible to do it. The importance for the open source community to support standards should be moral.
18:48
Because if proprietary software uses their own standards there is no way to get in there. Like this is written in LibreOffice, I tried to open it in what's called Microsoft.
19:03
It doesn't look like, it looks like hell. So you have to support that. Is there any drawbacks with open source? Yes, there is of course some drawbacks, but from my point of view there are not any real drawbacks.
19:25
Because like you can say that with commercial software you can call a guy and say, hey I bought some software from you, it doesn't work. Usually the answer is, it isn't my fault, it's a file via disk, your network or you used the software wrong.
19:43
But the farmer at the end doesn't really care if you run on open software or commercial software or if it's a programming file from your side. The cause or the fault doesn't care for the end customer. So if you see the perspective from the corporate or from the organisation's level
20:09
it's not important. You have to take some chances yourself, there is nobody to call to blame but usually there is a problem. There is much less fault in open source and of course you have to test it.
20:23
That's a part of your job to test it. And like some say that you may get a refund but as far as I know it's very silly to get a refund from Oracle
20:40
because there is a bug in Oracle. And you have paid so much money already so you will never get back the amount of money you paid. And who will help you to fix any problems? There is plenty of people out there that will help you and that works very well.
21:00
We have contributed some ourselves but there is also always an answer. And I have very seldom been the first person to have this error. Usually there is a person before that had the error and Google finds it for me. We have been through those. Hopefully you agree on that.
21:22
It helped us a lot on all those points and made it possible to develop those applications. I have the same question. You mentioned that you are running Postgres and Postgres on CentOS.
21:44
We have done that in our own organisation because it can be difficult to set up. Was that your experience? It works without any problems. It's easy to install.