Bestand wählen
Merken

Fiona and Rasterio: Data Access for Python Programmers and Future Python Programmers

Zitierlink des Filmsegments
Embed Code

Automatisierte Medienanalyse

Beta
Erkannte Entitäten
Sprachtranskript
of 2 morning everybody but we picture coming out and others like there's a lot of different sessions going on at the same time there's lots of places be really appreciate your attendance here and I'm looking for to tell you about some new pi up for them right there's going to be some code in the slides and I I haven't exactly master of my presentation system so the code samples might be a little small you may wanna jump upper row if there's a seat in front of you I think it's going to be visible in the back it's going to be visible on the back but just very maybe of knowing in the keynote as kind squinting at some of the slides there and had done the same thing so apologies of so my name John Gillis 7 coming to the user phosphor G. conferences since since back when it was the map server usage and uh my working from box now on the talk a little bit about that suffer been writing there's some of the modification the motivations for writing new software that that box and it's Khanakul to follow up on my boss stocks talk because I think about some of the same things my design software if if no where near as deeply as like this so I think I'll a kind of reinforcing the take advantage of his talk to talk about some of the same issues the alright so
the is it necessary right I've been in Python package that I'm talking about here Fiona and Ross stereo is recalled in that box and use a new Python interfaces to DO G R and G Weber's and I think the distinctive things about these the software is that of they are a Native adjacent speakers right so that's kind of the native interface for the for GA as feature data from the 2 race the good parts of Python many of us who here is some scene or red JavaScript the good parts right so pipeline is good parts to in there are there are they're not the same that parts as JavaScript in they're not the same good parts a c and are not the same good parts as other languages so Python as on unique strengths and other suffers written to take advantage of the unique strengths of and any 2 the software on embraces the command line on a show some of the command line tools that I'm developing with these and how we're using them at that map out production so I look at this as being in suffer Goldman I look at it as being a double challenge and for myself I wanna help experience Python programmers get comfortable and familiar with areas concepts and be able to and take advantage of data and take advantage of of concepts like projections and and what not to do work to get them up to speed the people I work with on it that box are we're we're hiring people the kind of new JS right there the programmers 1st GIS people and I also want help GIS experts learn to be better Python programmers but how many people here are not yet Python programmers prior but you know right now on my seat was going around the things essential things the GIS professionals need to know the states in Python isn't at the top of the list it's you know the top 3 something that so there's of the I would like sub-prime writing I want to be software that teachers people could pipeline practices it increases the general quality of Python code that's out there that's my radicals on all of this work is coming out of
a of a project in that box to make a cloudless mosaic of several different datasets you of that's kind of the that's the work context for the sub-prime developing
and since since this problems been tackled before in I've actually done it done similar things other jobs or were doing some different approaches we wanna do it cheaply we wanna do it fast we wanna do without terms of staff and off the shelf solutions for this are scarce and trial-and-error isn't are only method but it's still it's a very important so we need new software to do this can work I and then the requirements for the new software is that we need to be able to have we build iterate rapidly right we need to be able to try things scale fast move on and we wanna use and scientific software we want a robust algorithms of dependable algorithms reproducible results we have to build a scale out to 0 to many many nodes and and with those requirements from the talk about how we can override or suffer traces and these are the these will probably fairly obvious lot of us that the 1st and 3rd those argue for open source of go back for the scalability of using open source software means not having to ask for permission war of paying voice to to roll out upward servers other than say the service your prior prior on the 1st of those requirements are used for a high-level language with a multidimensional array syntax 3rd arguing for scientific software lately packed in and proven Rasta format drivers in so if presses Linux in Python num pi in the scientific part of stack now there's some there are existing phageal Python bindings and I think probably all pretty familiar with the use of unexplained to explain why we're getting away from using these writing software these so you know it's service well they don't actually fit very well In my opinion with the good parts about the the so I think we can do a little bit better so what I mean by the good parts of Python and this is this is my 1st ever Venn diagram that actually published on the Web so bear with me this is this is a bad diagram of the unique good features of 3 common languages so this is the good parts of each of these languages and it's not exhaustive I'm leaving out good parts of we can debate whether I got these exactly right and I hope we will at some point so I think it's gonna be sort tho I does anybody recognize I what the purple languages yes and the yellow language but I'm alluded to that we see and then bonds so blue language which right so I'm what you get Geels existing bindings are generated by a a program called a simple simple wrapper interface generator and this is a tool for getting a multi-language bindings from C + + it will you get at the end of it is C + + like interfaces for your languages so you tend to give the you get an API that that emphasizes yeah so it's not this for its use outside part of the and here the overlap between the way around in the states of the word cards and select for a while and operators is lost that's swing doesn't really you of the newly access to the good parts of Python which are up in the left corner this so as an example here of this is from the the rather excellent Jegal anode your cookbook so this is kind of a canonical example of creating a a geometric object Sergio the it looks a lot like see right there's kind of there's there's methods to add things it's very but it's very imperative kind of you know scattering thirdly figuring out a point at a point of point of at point yeah so the classic those Giorgia gentry object and think I think this can be kind up a little I think that was a different API let's programmers already know Python well get into this a little bit faster than the start the so I have an analogy here to Python was construction and a top ensuring that I apologize Internet seeing well at the top and showing the most inherited see like way of making a Python was the 1st of the numbers OK so he created create an empty list and then you iterate over the numbers in and for each of those you can that I don't want lose right so that's that's probably the most see like Python you could probably and then after that is the idiomatically Python way of doing this operation so this is a kind of this is what I'm looking for a new Python spatial software less of the lesser the top or the bottom less code a more by end it's not just about less code it's about speed 2 so the pipelined the the core Python developers have made idiomatic Python run faster rate it runs faster than the imperative code so in it's actually in this case ridiculously faster if you would use time it to time the first one again what showing like a thousand times faster so you the idiomatic Python code is much much much much of yeah so I think we should do we should provide EPA is like this and other factors that you experienced programmers for are coming in wanting to find stuff at the bottom and should be there spatial which but and I'm taking a will that further to say well do we always really need to create a fully geometry objects with hundreds of geometric operations on them just to ship data around their applications how about of using Python literal of data syntax to carry out the carry geometries their applications so this is the this is my my take on that note your code it's the same geometry waste of its data and I benchmark that is well what time and this is for example this is 3 times faster than me that is anybody recognize that and that kind of geometry representation that students the and then ideally I think we like to I would especially in my application selectable pay only for what I so find shipping data around I can ship around as if I don't need operations on the if I do need operations than there other libraries strictly for example were then I could rapidly it up you get a full-fledged geometry object really
new operations like those of this so I think that I've got a bit of a the a recipe here for winning that the double charge related really which is that the Python programmers get GIS data access using familiar at the eyes so not GIS CPIs the built-in Python stuff like say dates iterators generators the good parts about yeah in the future by some programmers learn to do things in the fast and idiomatic of yeah so having an talk about the motivation mechanism design concepts here talk about the actual software design of your address rail rest stereo there's a it's a Python package you import the use that's the 1 of the top of the thing that most people interact with and there's extension model in the middle that's using some sci-fi whether it's iPhone users in here yes I found is is free OS it's a a translates Python like code into see very fast see any get things for free he gets a pipe on 2 and 3 compatibility practically for for free when you write your C extensions the quite quite nice it's got some so we use things we use this for fast loops for memory views on numpy arrays and we can release the icons global interpreter lock elements of places get faster and then underneath it all is and that you'll shared what now here's here's an example of the API for reading Rasta data with rest area and there's a and there's an open function which gives you a file like set object so this has a lot of the same a duels open method it's like I struck I struggle with a lot of struggle with things like having to go to the documentation to find out whether I need to pass a 1 or a 0 for the argument to the open and say that this this method here is runs just like Python's open method of gives you an object that has some of the same methods back so you open a dataset here and there's you can see that the name of it but you could it was applied on file you can find whether it's open or like a Python file rather it's close of the encoding for future data sets so it's really a made and try to make a very strong analogy to to types and yeah with with the data set object the read method then gives you all the bands of the data back in numpy arrays and this is like in this case if it's multi-band data that gives you a 3 D array right and it's like bands the role of for the um the reason why a mimicking Python's the standard Python software here is to kind of get some of the same did some the same properties in the software that Mike Bostock was talking about d 3 where the 3 embraces the Don embraces way the browser works right because for that you get and you free documentation the for you get free of free support right because you're the leveraging all this so the documentation of use of the knowledge about how that all works out there well and I want you the same thing in Fiona Mysterio I don't want people to say now what I want you to see this open method is 0 that's like that's appointments and reading vector data is on more or less the same way so was the same thing you get you open best of of fire a path to an open function you get a collection that this collection is an iterator so like a Python files an iterator for lines is an iterator for features their duties like dictionaries and then the right on the right inside kind of think as I try to make it as a the Matisse intuitive but as transparent as possible you you open it in w mode like you would a normal file there's keyword arguments in order to create a dataset for writing this is pretty new file and this you got to set up the way out of the the datatype of advanced etc. these are all in the function is keyword arguments and you can get these keyword arguments from another file that you've OpenCyc you wanna make a copy something the it you write iterated that so what does anybody recognize does anybody recognize and not recognize the of the with statement there so this is Python's context manager whenever block ends there's a exit function called on the object that's going like within that block you having a little runtime environment in at the end of it the context manager that the destination object x methods called on that and it cleans up after itself closes selfish that this what I think a weakness of the the dual roster library is if there's there hasn't been a deterministic way to write your data to disk when you want to be written to disk right the kind of disease Kluge is where you to delete the object or something like that I myself I like to you know what I write something I would like it to be written and I'd like it to be you know determinist divided be written when I write so we have this actually close enough and on the following individual right exactly the same way the more lets you write you Jason like features to it the and for georeferencing there's a great project pipe proj and I'm following the lead of that so it by proj embraces the approach for library and but instead of strings I think it kind of interest to me the concept of Willie's part for strings were just keys and values kind of concatenated into a string right but there's a there's Python data structure for keys and values so we do this we treat everything as a dictionary that that's that these these libraries don't have a spatial reference object a class where using dictionaries status information some and a talked sub-modules of terrestrial library the first one is features model this has some a function that gives the shapes that of your answers so this is all the features in your in your rate as duties objects as an iterator over and then there's a reciprocal operation can burn the same features into another right right so it all that all works off a Python fanatics Dixon iterators tuples Hindi there's actually no datasets were layers necessary so you can pull a puree out of thin air were created in a computation and you can run it through the shapes function and L. Paul features out you don't actually need to have a duty of war DRG or or whatever so but as
example of Attica Python logo pull the shapes out of it in and bring them back into another image this is just kind of round-tripping it through but not keeping the colors and it point in which yeah there's also work module that gives you some of the same features and but with all of the jewels of cartographic projection machinery in so there's no again there's no dataset need for these narratives of GOG pay whatever any kind of num num if you give it a a transformed and quarter references and you can work these into other reference systems it would be good and then as I said the software of embraces the command line so I do a lot of ash scripting at work and I work with people who are bash wizards it's crazy that kind of stuff that yeah 1 if you can see this summer stuff is super clever but there has to be a better way right there has to be something better than dumping the output of GeoInfo info and scraping it for the number of bonds in a file or for the datatype your file some of that that so who's who's done so the reason and this is not the case Mr. developed I mean it's not like it's not like Frank is going to change the friend another gonna change output of dual infrared but still scraping is kind of and what minister places great thing I don't I want be scraping stuff and production that's that's so I'm I'm writing some new command line programs using these to to get us into a little better places so this is of this is rats Ryo's command line interface which program called Rio again I apologize because this kind means you can actually bash script with this and your and a few other he part records or something like that the listing of a commonsense so the help I'm using the Python package called click to do the command line interface in it so quick is really nice it gives you a really nice helpful free and which just you know if few lines of a few decorators and function so real info imposes subcommand for this and instead of giving you a text output of file it gives you structure data so did you Jason out metadata about the file edges and this couple options for indentation investment and Indian public index single items of that out of there with that with another option so that was the Count option you can pull the the can bands out the you can pull Sierra so that itself so it is so this that was
getting into production work really could be the people of there's a the shapes commands in Rio yeah is can in my program to dump will it ones that achieves function in the features model dumps all the features in your data set out as duties and then I'm using this I wasn't as program I've the underscore CLI this library furthering underscore judgment so yet I'm using that to extract a particular feature from the duties and in time should be given to the duties and I O command line to get and yes
and it was OK human fish on top of it so I recently I think I have a lot about this but we get new releases the software so if you 1 . 2 was new so were stereo of 113 and then I over the past couple weeks have been working with people on getting back to the ship was version 1 . 4 out of these as well so try stuff out items in the back and working I'd like to think that not box satellite teen Christen Charlie in that renowned Miller I'm I wanted to thank my collaborators on the and roster cheaply so as the my toes Brennan Ward considerable name far Jacob Wasserman overtime offer Jochum false in that area because there is romance so and of course warmer down in the role that's the you and about
what so thank you questions among and yet there's the this demand a question that indentation so the idea default is like the Python's ergies models default which is not right so just speed you know all on 1 line spaces in between that of the default and which but for me later and focus on from the mean can I learn from small or but it's is thread-safe is dual and yeah I have I I've been thinking a little bit about threads I know that I the Python community is a little taught about threats you from core developers lot disdain for threads and saying well of the threads if you're using threats to doing it the wrong way of I don't I don't use don't use threads very much I think it's in production we tend use of multiprocessing things so to get to get cheap but she parallelisation we use use parallel a lot so these command line scripture usually run under the new parallel to to run them and in that way threads of it's no less thread-safe and you this is another basic question how away redrafted that enter to them numpy arrays L so that I use rosters open function on on a file surpasses the paparazzi have rested you get your data set object and then that object has a really method that then gives you a multidimensional array but of all the goods you everything you can actually kind of person you can ask for specific bonds we can ask for Windows is that of a as well because you know dumping of gigabytes Russia's people rest the it there 0 I I think would probably still be angels Python bindings so that we can develop the features as we needed in a restaurant table rule has come up I think that I would probably if it if it was of you love feature requests the and yet happy to work with people on implementing this stuff it's actually not it's hard to do this kind of programming as you could yeah if you'd like to see the future of the get involved good on all of the next week ago and army around for technical support install questions and things like that for you the
Bitmap-Graphik
Bit
Subtraktion
Quader
sinc-Funktion
Physikalisches System
Kombinatorische Gruppentheorie
Code
Mapping <Computergraphik>
Rechenschieber
Wechselsprung
Datensatz
Benutzerschnittstellenverwaltungssystem
Software
Stichprobenumfang
Server
Subtraktion
Programmiergerät
Mereologie
Quader
Formale Sprache
Schreiben <Datenverarbeitung>
Schriftzeichenerkennung
Code
Computeranimation
Demoszene <Programmierung>
Software
Mosaicing <Bildverarbeitung>
Radikal <Mathematik>
Hilfesystem
Schnittstelle
Expertensystem
Güte der Anpassung
Eindeutigkeit
Mailing-Liste
Kontextbezogenes System
Biprodukt
Mapping <Computergraphik>
Modallogik
Flächeninhalt
Rechter Winkel
Mereologie
Ablöseblase
Projektive Ebene
Aggregatzustand
Mereologie
Browser
t-Test
Iteration
Algorithmus
Trennschärfe <Statistik>
Gruppe <Mathematik>
Translation <Mathematik>
Gerade
Benchmark
Schnittstelle
Array <Informatik>
Shape <Informatik>
Befehl <Informatik>
Sichtenkonzept
Kategorie <Mathematik>
Güte der Anpassung
Bitrate
Kontextbezogenes System
Dienst <Informatik>
Generator <Informatik>
Entwurfsautomation
Menge
Rechter Winkel
Festspeicher
Lesen <Datenverarbeitung>
Programmbibliothek
Ablaufverfolgung
Ordnung <Mathematik>
Zeichenkette
Lesen <Datenverarbeitung>
Objekt <Kategorie>
Multiplikation
Zustandsmaschine
Klasse <Mathematik>
Abgeschlossene Menge
Maßerweiterung
Räumliche Anordnung
Open Source
Loop
Informationsmodellierung
Iteration
Determiniertheit <Informatik>
Datentyp
Programmbibliothek
Maßerweiterung
Datenstruktur
Algorithmus
Open Source
Gibbs-Verteilung
Elektronische Publikation
Chipkarte
Data Dictionary
Array <Informatik>
Schnelltaste
Softwareschwachstelle
Offene Menge
Minimum
Wort <Informatik>
Bitmap-Graphik
Resultante
Programmiergerät
Bit
Punkt
Selbstrepräsentation
Formale Sprache
Adressraum
Kartesische Koordinaten
Computerunterstütztes Verfahren
Element <Mathematik>
Internetworking
Datenmanagement
Skalierbarkeit
Prozess <Informatik>
Minimum
Multitasking
Analogieschluss
Nichtlinearer Operator
Schnelltaste
Zentrische Streckung
Konstruktor <Informatik>
Lineares Funktional
Interpretierer
Parametersystem
ATM
Euler-Diagramm
Ähnlichkeitsgeometrie
p-Block
Bildschirmsymbol
Teilbarkeit
Dateiformat
Projektive Ebene
Dualitätstheorie
Information
p-Block
Schlüsselverwaltung
Geometrie
Mechanismus-Design-Theorie
Aggregatzustand
Standardabweichung
Stab
Zahlenbereich
ROM <Informatik>
Term
Code
Benutzerbeteiligung
Software
Mini-Disc
Wrapper <Programmierung>
Softwareentwickler
Optimierung
Profil <Strömung>
Mailing-Liste
Vektorraum
Sichtenkonzept
Quick-Sort
Objekt <Kategorie>
Modallogik
Höhere Programmiersprache
System F
Diagramm
Druckertreiber
Flächeninhalt
Loop
Formale Sprache
Mereologie
Speicherabzug
Normalvektor
Bitmap-Graphik
Punkt
Mathematisierung
Zahlenbereich
Zählen
Computeranimation
Eins
Metadaten
Informationsmodellierung
Datensatz
Software
Gruppe <Mathematik>
Datentyp
Programmbibliothek
Skript <Programm>
Optimierung
Hilfesystem
Gerade
Bildgebendes Verfahren
Schnittstelle
Funktion <Mathematik>
Programm
Lineares Funktional
Shape <Informatik>
Axonometrie
Element <Gruppentheorie>
Datenmodell
Einfache Genauigkeit
Mailing-Liste
Physikalisches System
Biprodukt
Elektronische Publikation
Modul
Gerade
Konfiguration <Informatik>
Automatische Indexierung
Mereologie
Projektive Ebene
Dualitätstheorie
Information
Kantenfärbung
Summand
Satellitensystem
Bit
Quader
Versionsverwaltung
Raum-Zeit
Informationsmodellierung
Software
Bildschirmfenster
Petersen-Graph
Thread
Installation <Informatik>
Softwareentwickler
Optimierung
Parallele Schnittstelle
Default
Gerade
Array <Informatik>
Schnelltaste
Lineares Funktional
Güte der Anpassung
Schlussregel
Elektronische Publikation
Biprodukt
Weltformel
Satellitensystem
Arithmetisches Mittel
Objekt <Kategorie>
Kollaboration <Informatik>
Flächeninhalt
Offene Menge
Parallelrechner
Speicherabzug
Tabelle <Informatik>
Computeranimation

Metadaten

Formale Metadaten

Titel Fiona and Rasterio: Data Access for Python Programmers and Future Python Programmers
Serientitel FOSS4G 2014 Portland
Autor Gillies, Sean
Lizenz CC-Namensnennung 3.0 Deutschland:
Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen.
DOI 10.5446/31598
Herausgeber FOSS4G, Open Source Geospatial Foundation (OSGeo)
Erscheinungsjahr 2014
Sprache Englisch
Produzent FOSS4G
Open Source Geospatial Foundation (OSGeo)
Produktionsjahr 2014
Produktionsort Portland, Oregon, United States of America

Inhaltliche Metadaten

Fachgebiet Informatik
Abstract Fiona and Rasterio are new GDAL-based Python libraries that embody lessonslearned over a decade of using GDAL and Python to solve geospatial problems.Among these lessons: the importance of productivity, enjoyability, andserendipity to both experts and beginners.I will discuss the motivation for writing Fiona and Rasterio and explain howand why they diverge from other GIS software and embrace Python's native types,protocols, and idioms. I will also explain why they adhere to some GISparadigms and bend or break others. Finally, I will show examples of using Fiona and Rasterio to read, manipulate,and write raster and vector data. Some examples will be familiar to users ofolder Python GIS software and will illustrate how Fiona and Rasterio let youget more done with less code and fewer bugs. I will also demonstrate fun anduseful features not found in other geospatial libraries.
Schlagwörter learning
python
programming
raster
vector
gdal
lesscode
morefun
zen

Ähnliche Filme

Loading...
Feedback