Taming Rich GML With stETL, A Lightweight Python Framework For Geospatial ETL

5 views

Formal Metadata

Title
Taming Rich GML With stETL, A Lightweight Python Framework For Geospatial ETL
Title of Series
Author
Broecke, Just van den (Just Objects B.V.)
License
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license.
Identifiers
Publisher
FOSS4G, Open Source Geospatial Foundation (OSGeo)
Release Date
2013
Language
English
Production Place
Nottingham

Content Metadata

Subject Area
Abstract
Data conversion combined with model and coordinate transformation from a source to a target datastore (files, databases) is a recurring task in almost every geospatial project. This proces is often refered to as ETL (Extract Transform Load). Source and/or target geo-data formats are increasingly encoded as GML (Geography Markup Language), either as flat records, so called Simple Features, but more and more using domain-specific, object oriented OGC/ISO GML Application Schema's. GML Application Schema's are for example heavily used within the INSPIRE Data Harmonization effort in Europe. Many National Mapping and Cadastral Agencies (NMCAs) use GML-encoded datasets as their bulk format for download and exchange and via Web Feature Services (WFSs).
Loading...
Computer animation Open source Geometry Multiplication sign Open source Software framework Independence (probability theory) Local ring Resultant Independence (probability theory)
Computer animation Personal digital assistant Software developer Projective plane
Flow separation Context awareness Numbering scheme Mapping Polygon mesh Computer animation Mass Cartesian coordinate system Event horizon Window
Multitier architecture Numbering scheme MUD Mereology Event horizon Emulation Number Measurement Pointer (computer programming) Maize Identifiability Address space Metropolitan area network Multiplication Spacetime Scripting language Sine Building Element (mathematics) Interior (topology) Image warping Instance (computer science) Cartesian coordinate system Maxima and minima Computer animation Uniform resource name Element (mathematics) Address space
Computer animation Mapping Computer file Transformation (genetics) Scientific modelling Computer file Scientific modelling Transformation (genetics) Address space
Arithmetic mean Computer animation Object (grammar) Database Element (mathematics) Object (grammar) Transformation (genetics) Mereology Element (mathematics)
Computer programming Computer animation Computer configuration Personal digital assistant
Software engineering Numbering scheme Transformation (genetics) Geometry Projective plane Planning Bookmark (World Wide Web) Flow separation Inclusion map Process (computing) Computer animation Mixed reality Energy level Energy level Quicksort
Context awareness Computer file View (database) Shape (magazine) Mereology Scalability Degree (graph theory) Flow separation Computer animation Database Energy level Graphical user interface Scripting language Quicksort
Computer animation Observational study Equation Equation Curve fitting
Computer animation Streaming media Set (mathematics) Matching (graph theory) Row (database)
Context awareness Building Computer animation Mapping State of matter
Chain Computer animation Transformation (genetics) State of matter Scientific modelling Process (computing) PRINCE2 Quicksort output
Integral transform Degree (graph theory) Computer animation Infinite conjugacy class property Scientific modelling Bit Figurate number Streaming media Local Group
Degree (graph theory) Integral transform Latent heat Degree (graph theory) Computer animation Personal digital assistant Scientific modelling Scientific modelling Bit Right angle
Digital filter Building Computer file Geometry Shape (magazine) Field (computer science) Summation Chain Computer animation Dedekind cut Function (mathematics) Hausdorff dimension Process (computing) output Supremum
Digital filter Computer font Numbering scheme Scripting language Software configuration management Twin prime State of matter Mountain pass Connectivity (graph theory) Scientific modelling Point (geometry) Abstract syntax tree Grand Unified Theory Heat transfer Port scanner Counting Shape (magazine) Stiff equation Computer animation Uniform resource name Scripting language output Cuboid
Metropolitan area network Meta element Digital filter Empennage Interior (topology) Mathematical analysis Shape (magazine) Local Group Principle of maximum entropy Maxima and minima Pointer (computer programming) Fermat's Last Theorem Computer animation Telecommunication Function (mathematics) Moving average output Loop (music) Online chat Supremum Scalable Coherent Interface Newton's law of universal gravitation
Axiom of choice Computer programming Digital filter Computer file INTEGRAL Transformation (genetics) Connectivity (graph theory) Sheaf (mathematics) Translation (relic) Parameter (computer programming) Wave packet Integral transform Chain Social class Mathematics Process (computing) output Units of measurement Scripting language Computer file Shape (magazine) Computer animation Lattice (order) Function (mathematics) Quicksort
Digital filter Context-free grammar Computer file Scripting language Connectivity (graph theory) Machine vision Computer file Prediction Shape (magazine) Local Group Compiler Integral transform Chain Social class Latent heat Computer animation Function (mathematics) Configuration space Scripting language Process (computing) output
Installation art Keyboard shortcut Standard deviation Installation art Electronic program guide Set (mathematics) Subject indexing Computer animation Network topology Data Encryption Standard Conditional-access module Resultant Window
Read-only memory Digital filter System call Computer animation Data storage device Function (mathematics) Cellular automaton output Library (computing) Library (computing)
Digital filter Context-free grammar Computer file Constructor (object-oriented programming) INTEGRAL Line (geometry) Logarithm Connectivity (graph theory) File format Parsing Integral transform Social class Chain Network topology Information output Computer file Streaming media Bit Component-based software engineering Computer animation Function (mathematics) Sheaf (mathematics) Clef Determinant
Point (geometry) Dataflow Standard deviation Information INTEGRAL Connectivity (graph theory) Disintegration Element (mathematics) File format Streaming media Set (mathematics) Revision control Degree (graph theory) Component-based software engineering Degree (graph theory) Computer animation String (computer science) output Data structure Writing
Digital filter Transformation (genetics) File format Password Generic programming Table (information) Substitute good Chain Social class Degree (graph theory) Mathematics Database Personal digital assistant output Stokes' theorem Multiplication Electric generator Computer file Open source Parameter (computer programming) Drop (liquid) Mereology Element (mathematics) Computer animation Database Web service Function (mathematics) String (computer science) Quicksort
View (database) Programmable read-only memory Mathematical singularity Mathematical analysis Word Degree (graph theory) Computer animation Database Web service Process (computing) Volumenvisualisierung Matching (graph theory) Address space
Computer animation Feedback Software developer Translation (relic) Resultant
Numbering scheme Transformation (genetics) Code Connectivity (graph theory) Scientific modelling Multiplication sign 1 (number) Function (mathematics) Streaming media Scalability Workstation Template (C++) Revision control Infinite conjugacy class property Internetworking Computer configuration Software testing Scripting language International Date Line Validity (statistics) Element (mathematics) Content (media) Coma Berenices Extreme programming Line (geometry) Instance (computer science) Directory service Degree (graph theory) Flow separation Arithmetic mean Computer animation output Self-organization Whiteboard Data management Window Library (computing)
by tabling which has said that mean stealing from the rich human give to the poor results of could be taking the play but that is something that sustainable and which is likely to buy the framework for geospatial into that all
the good about myself from independent open-source you spatial professional and and and that like most of the secretary of the Ohuruogo chapter in the in loans and and a member of the Dutch opened you cope aeration of independence and professionals in a local providing sausages spatial superb but and this is something so I like to do a much better time playing with by and blessed but out of the show
stabbed and project because you want to solve problems in this case again we have a problem and the problem with
the rich Jinmao problem and and we say which G Amalita's wanted but would not decided to the developer of the Greek because they don't talk about complexity amount that some of the negative which may be sent to follow the early this disrupted which not
about original context mesh I could say about the big of applications scheme and out course the design for the new breed in tools like supplies are taken the the push of a button some scheme not is generated said
probably several of the scheme us and not the date defences against by a window and next N scheme mass but also many of the Dutch national datasets in several countries use and an application to the mass or some other form of complex it BigSim also in the knowledge that National datasets in Germany the mesh datasets an adjustment that you can be a last is also for what it Tumelty and apparently complex Mark events that are set up to give the
impression of a talk about this is that addresses this is the and Applications scheme and this is what I've would also my Jim Melott 6 2 with the Jews and then spaces and well lots of events and what you see also is a sort of arbitrary and now like you could be a multiple of those elements of of the same elements of nested elements Edmund implicit 6 weeks said and look at despite for
instance this part of the NHS by the playing of the elements here but this is the street name only opened via the Soviet by smaller 1 of the most more complex models amid and that would be like a Street number and and a place but some of it is actually discrete 9 and assault a over their consent up its said part of demanded but we want to to do something
useful with this blitzing make a map maggot Yokota of the addresses the and we have to deal with complex mobile transformations and not only sad models
complexes but that it's a a very huge files like talk about gigabytes of files files and
so this part of the debt that is really downloaded you get all the fictional files and that
means that millions of objects and later
millions of elements
said due to transport to do something useful like like them in database to making that we need
spatially so what are the options are this out
1 approach cost to write programs for each data try to answer in some cases of shame working works and but
was a good look at the open sought to special wealthiest several high level tools NYTimes flybmi tools for the we can sort of the said out the transformation Spasojevic Edelman admitting that we know to to some of you don't specialise and this week of recovered also about had a lot of projects from the book of the year to go to was which but it seems to me very much improved and if you sort of on a surge of pride the semi night at also tried but I'm a sort of power and speed
Unix on plans had contacted close to the island and the size of some of my favourite tools at so that says if you if you you have to confirm a shake post yet would use the job what may shape to be just growl and but did each of these stories salty someone not milieu which so what the police said that the strong makes a mountain of smell scheme hour anything else imposes with problem each of these tools very volatile but can go through the whole thing if you are in the mix and just use Bojado job me to do that has said in the transformation steps
said and this also came out of the house on the edge of the city this so so the question is how to combine each individual very powerful tools and that from now on and will be somewhere women the next just not talking and said in
this game out of the research in some of the year in spite but it said it in the best in Europe during the context several people a year in the UK from an uncertain and said that we would say that there was this multistep approach that saying to could prevent that was exploded into a shape Fulham at the end of the phone Gelato shut views simple feature Jinmao file and sympathy to the amount that could be translated by a and then we could be generated by makes 1 Jinmao and we use at this level which part of the degree to set loaded into by database but that was sort of script was but had he didn't scale-out said a friend that as it had to combine
these tools and the answer is basically and that applies into the equation and the UK which has written about so that the new study by the middle of sons and you spatial world because integrate told the existing of said and this is really what it
said was about in the sense of that it combines the basic tools
and the and the and the abbreviation is set about return like this thing in the expected with capitals and it's about simple streaming spatial speedy into that's what records
tries to stand for set and is basically from the and buckets of Jinmao France's into 1st yes and and was introduced to the beautiful match because she chose to make his special Conference or and you could standard not just about the only Jinmao into both she that's 1 of the scenarios said should show that
here is disguised amazing is also in the context of also be used some of that to lead to produce and demographic maps of the cities in the world that combining the demographic and and the state building data Standard Cuba's of cost so what and stable
concept accuse quite
simple if you have multiple transformations that you go from 1 sauce to a destination targets it said that like that money to meet boats and and several failed to press as the day that this is still quite a stretch the Prince an improved could be Jinmao following up but yes and then and will go for several 1 or more of those and and jumped into the site but that this is run by the
sort of triplets Opel also you could have some kind Jinmao leader model and then that would it sends out tumult your acuity and other model in the UK to the US he said that the state is in
spite of all of whom showed earlier and Princess that could also be already in the post yesterday based on Gelato justice Woolwich and command line that were which is integrated and which data data its but it produced a simple features such as the continuous at ICCS Simao streams and and a society would be used to add create complex figures showed that the group is now so it's not just about reading stuffing to boast at added that
specific it's still a little bit that cheques and said that
it's a little bit like Ant Lago you can just go next anything to anything at as crosses distinguishing up which are compatible so this right through June of how good the and the Prince and
replaced by the degree writers which should specific model model which right into in this case an agreeable Chtaura but it does also not put lactiflora W 1st published directly to the world to do so that the adjustment
and so this is really work in the field shoppe a 2nd
examples so we have some random file who will be applied to the silver and reusable judge what you're up to produce a should said
sometimes you get this kind of random makes a mouse it's not not feature by its just 6 Melott has dimension to coordinate and said it could run out Gelato geometry with them for a couple of clever command building but as the some Sunday to to convert that we have an excellent
model in the UK should stay component in instead and
then we produce an exotic difficult to create the strongest to simple features of juvenile so and it is up to the
idea that some criticism on only to stop the but this very very powerful when you want to add transfer 1 basically 3 occasional scheme to another scheme us this simple places at the script for the state the inside of and and produced
out and geography to collection which the 2 collections so we find saying that places Amsterdam but then this is it throughout the to
Jinmao so this into this
said that the comes out of the ashes
of the filter so once we have a simple features a Jinmao but the applied algae are Jodie are than the usual 3 anybody who do not its estate said analysis all group to get style is
based on over Gration file so you don't have to programs that would be nothing that don't have to consider that the transformation in the translation is so that train is a specified integration and it's a simple 5 from taking file and you can find it in a meeting with the US but also in price and is used a lot so basically the that saying that a such as recession called in to help him get began multiple choice and this is 1 changes to the England extremophiles going through to the society and the up to the judge and the sort of identified the switch to sections further up the file so it predicts about this section and as you can see that the specific components for assessing the unit is that identified by cost and then there are his parameters for their clubs and across the component Chodorov book like million
is a components and its
specified here as a prediction of file an issue to fight by pointing to a specific would file LeCrone Luciella compiler metallised that as well as society is a component here and at the script and the script is to make sure Stofile and protracted you can justify a regular but yokel to UK among which is nice because it said that this is not the basically group together to use a different tools and it should be open to the ocean Foulser local has could be more expensive this is vision Workshop at
around the same basic amount lined up to local stable and and the which was about configuration with my she options and that the producer of the new schools to
be is the results of a tree
set standards and the guides and so it's needs that its critic in the install feel Standard Life and the best place like by an equity index of that to instal staple it's not pedestrian or other packages and those of the US and the and the Linux that this is very trivial to instal these dependencies some of modern windows and said talked
about so speed as well or the about also as part of as the tools to streaming thing because you come out and say if you 200 in a year and the megabyte a commodified is cannot pasta memory and cost from something like documents cells medicines and and stunning without intermediate storage
and asset sales calls by native libraries she libraries of the Texas to fictional too far Standard libraries Linux so it is speed optimized automated and and each
of these include assaulted components search auctions now and then but you are also a able to write your own food
so from now that this is a little bit of 5 million should show some CODE but if you want
to add to on component here as filter you can specify the continent in the early integration file and then that is a particular trampled just potential stand up with the cost undertaking up so your for those who to implement new invoke methods and then get back in the thick of things data and the and the status
said OK but this exchange between East components we still does not do is to make a but it's a feature my internal future while its stationary close to feature information that comes out of the different tools and where necessary the along among them translated to another 4 months so that this is the trade off is that by the that version of a picture well-documented a document and stream as that of a large document and smell streams and then you can specified which points in the pound in the future should make a documentary in the huge amount all into multiple small documents but you could use it to the element history that individual features
and so and that's not to say that components to do a degree integration especially to write degree France's bookstore would or the flow of a full set of degree or standard to justice and that the 1st he
said as to what sort of new places were establish widest for and inspired transformation sought to generate harmonise data and that is through to the national team elevated mostly in both and this is
not expensive example Francisco and which is the national Bychkova datasets and is now extensified refugee multiple at change with his for initialisation setting up a database you can also use the and all these but the can be substituted on climate Stoke among so it's not talk about code of on again off should
show match and also
recently would be Jedi where he took a couple of hours and then we could really do Majedie into both he and this is what stands
Adam probably won't go into details about that this is as real as the using a should status and that is the day strike this which offered to effeminacy recently that about half year and still struggling so bad
said the status and so it's not a full fledged rather but still in the development of caution that why most presenting here to see the show some of the results of its own to feedback but you couldn't say already feel like by the documentation this would be to go and that the real translations so it is told
I'd like stood there from the about a lot of medals blue news of the
victory went had questions of the it stands with a local woman for it and that the board is not as much several but disappointing libraries and realist find this problem earnings following something with 5 and windows but and that several options starting think in documentation of pointed to a portable and that actually made by Joe Cook with around here in she in organisation of Kennedy was the stick with over at the end of versions of the basic Leo produced and you can run the without installing the and that's very very powerful taken Guppy that would directory for instance and initialised that ones and so that the 1st step and and then you already of G Dolangi built by combining said even of she of thinking Oh scripts both and so you don't have to instal a Cisco potable the the 1st half of all to the good news is you for Windows where the something called used the Jesus well the board will gesture with some of the players so there you have it all open it can also be but will it be to animals have full year by higher now here we have here you from Jim amount 2 the at ICCS assaulting on the it is about without W fastidious basically a container you know about the beauty of the the that had actually took the lead across the UK not is to 3 lines of code of despite the and has been and and in the other transformation she rejected large amount just like you would central file than and the last at the degree of 50 riders will will take those Gemayel features and but that in a template and that's the difficult content for the W of but is instead feature will a long 1 in search and had as 1 of the established players history and but you can't have the stream or stream petition extreme into manageable elements so you could say idle W Fast I for 1 am pretend features for or but you could also say that the stream to demonstrate of degree of overstated gigabyte of humour and so the several Chincha but that is another approach radio still come and in Seattle the dedicated to to and and not output to bed in the independent of the input of some of the best TV models and a where how the other entomologist produced but it gets to documents on all members but the document could be of lot the American double flying I'd didn't understand winter cost his not at by him now all that publishers scalable worries about the player but it is usually it's the station in the city or so to the question is if you would like to validation where we were Djibouti stepped into the strain of buses and and actually destination mouth validated component and the place offered a assaulting step that produce full of documents in the pocket scheme for instance and the nice thing about the way that it commercialises wants to to get all the existing so the from the internet and the it validates each but by that uses only done testing because it take child before means to validate each and every time the Mets echolalic this is allowed to see the goal worth allegations that bought can thinking the slogan for the
Loading...
Feedback

Timings

  778 ms - page object

Version

AV-Portal 3.11.0 (be3ed8ed057d0e90118571ff94e9ca84ad5a2265)
hidden