Add to Watchlist

Extending Python, what is the best option for me?

6 views

Citation of segment
Embed Code
Purchasing a DVD Cite video

Formal Metadata

Title Extending Python, what is the best option for me?
Title of Series EuroPython 2014
Part Number 60
Number of Parts 120
Author Castaño, Francisco Fernández
License CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
DOI 10.5446/19975
Publisher EuroPython
Release Date 2014
Language English
Production Place Berlin

Content Metadata

Subject Area Computer Science
Abstract Francisco Fernández Castaño - Extending Python, what is the best option for me? Python is a great language, but there are occasions where we need access to low level operations or connect with some database driver written in C. With the FFI(Foreign function interface) we can connect Python with other languages like C, C++ and even the new Rust. There are some alternatives to achieve this goal, Native Extensions, Ctypes and CFFI. I'll compare this three ways of extending Python. ----- In this talk we will explore all the alternatives in cpython ecosystem to load external libraries. In first place we'll study the principles and how shared libraries work. After that we will look into the internals of CPython to understand how extensions work and how modules are loaded. Then we will study the main three alternatives to extend CPython: Native Extensions, Ctypes and CFFI and how to automate the process. Furthermore we will take a look to other python implementations and how we can extend it.
Keywords EuroPython Conference
EP 2014
EuroPython 2014
Series
Annotations
Transcript
Loading...
so hello everybody that's gonna come from this from just go look in the long way from moderate to present this talk about different approaches of extending pi thank you hello can you hear in the low end of the day and
month is the minimum and promoted work process of engineering the goal is that we are working on creating a dependency model of course of the last and around this season just classes American money and also a new quality assurance that I'm I'm very glad to be here thank you for calling and then on the a talk about the standard by the hand Python there are different approaches that we have to understand by and have their own not in this place I will talk about
the motivation why would you we have to do right to another language is then mostly some basic concepts that we have to know to understand how these expenses works and then explain a bit about that
if expensive and that the biphones give us that has claimed that we don't see that and then I'll see if the file is an external library and finally has plane you mind representation of some of the different approaches and
I would like to argue that there is a huge topic they sort of things to explain the sum of the thing to studying this
sort of thing the there's a resistance operating system these celebrities in 1 way or another so then this would be the so this book is intended to be an influx to to this topic and so on a lot time to spend the basic
concepts probably the the example that I give you that would be useless but I want to explain you the basic concept that you do have to to apply these mechanisms in the future you will have the tools so this is not my intention here 1 family unit you serve so I don't have the chance to test my my examples in windows so I don't know if they work or not under and I use mainly the less effective implementation to the sum of money this so
and what was the motivation was to go about writing C or even other languages like Fortran was so we want to do all these things which is
from from Python will ensue after the with memory we have to danger trade with
someone thing some things like point there would be hard to do with it was so like coupled but I think that there are mainly 3 reasons in this place we we
have probably we have some Python program that have the in some parts let's say that you have some product have very intensive calculation
that's because they have a lot of time by the we want to write this down the into so we will improve our performance although recent would be using the lexical decision that I have somebody we write in library in C or C + + important and I would think that good many of system writing in
summer the reason that we we going use these mechanisms and finally the reason and the other levels like for example I'm seeing of speaker of living a we can directly using expensive to use this library and provide some unknown reason could be that this is in and other hand broadly we want to integrate out I think that but there in an existing and current writing in C circuits about is out of the scope of this book but this could be a useful yourself by from severe OK and now I think you some some
concepts that we have to know of all have effects and binary libraries would have been 1 would what we static libraries on the other hand we have celebrities
as explained due know the difference between these 2 panels of
labor OK if the declarer is a
bunch of of the files us we can see here we have that the declaring and just to objects and find a book 0 unbelievable some with information and everything that I a link a link the slavery without binary all this new binary half of popular use these objects so
it seems that in this scoping context with don't 1 to to compile and use by finding but they're with this extensions so it's in the rest of the global doesn't well with this these mechanisms on the other hand we have celebrities and sensor libraries were seen in other way operating systems loads a celebrity once in memory on every program that use that and that library a is the same type of memory and every program to some copy of variables about they use the same missing
copy in memory and we can look at dynamic other on time this kind of labor so it seems that similarity is is that a way to to expand by OK they are all
the start with the 3 mechanisms for but we have not been sent and survive from the past artist lists
API and CBS is a Python API that defines a set of functions macro somebody will that provide us access to most of methods
that we have to to fluxes to buy from runtime so we can expect How Python works using Siegel OK the there's
lot a lot I thought that making available using that this this is simple and be very very useful so all I've been thinking how hard problem and I want to that later is very very helpful that the 1
edition of the number of true the
on and so OK on we have a model with a function that takes 2 arguments and and the true that this isn't very simple you can imagine another powerful function here that they can be used to find similar something
that takes too much time and we can serve the interest seam I'm not too suspensions look like this and
the and you see is what I'm going to
the bitterness of light but just is that we have our function and then we define a model in OK so in this
place we include of Python the of by the so we can have access to lose all of the themes of that finds and Mark and variables that
might only be a fast and so that is the uh we want to use some system includes we happen to have up there by from the plates because by phone right guaranteed find some instances them is about so we want to to be used by the ASR system includes we have been collected including by the be here we have our our conferences thing at the balance have pointed apply a by the past the noise and then the mean of the policy by from implementation and then it it takes 3 arguments in 1st place my comments and it takes on 1 point that every function in in that it has been since they are of pointed to set up a special event and then appointed to prepare arguments OK then we have 3 viewpoint this avian results then we get this the argument was that those 2 answers that up so I use these micro defined by Python API and I get the 1st argument and then there's the 2nd 1 and then use that knowledge function given by the media to plot these don't but they and I get this alternative candidates of very simple it is not is that this is not a toy example is quite possibly 1 argument wouldn't probably right because I don't think the past 3 arguments of word then we define development of methods that my model we have this is that this the name of the of my method of pointed to my as some the and finally find the 1st method that be called function for that would be called the 1st time that important these models and it does indeed my mother if it is unable to get my body functions on the question of his also by from funding fast in some so tools to their with compilation of mine extends and we have excellence in the last year we gave a name and the use of file it will take care about compiling Michael depending on the platform we don't have to think care about what they had to compile the forces in independent code because the central library it will take care of so a compiling we can try my unseen model I'm and from my function OK so we think that this is my 2 numbers but I like to load the and think about how this
work at sea level of so I did a bit of research and I finally ended in this Monday's the mandates of the
and this is a system called defined in terms of the of the of the month windows is available and this function with what does this you know about the incident light if you look outside the cell library into memory and on and then the number of of references for for these these library so I grabbed by this if I sample and they ended up in decided so we can see here that we have and are going to be a little bit and with without buffer name to my model that I'm looking is that it's no pencil and if you if everything is OK you will be used for a new reference to my to my hand and is an assistant below we assume that the symbol in my in my social life it will be discussed his looking for the Fed funds and that this the 1st time that the model is important remember function OK
because this function and that was just the my model into an online dictionary of form in
but volume 1st you know we
have to live with memory and see we
have to is we look at some
memory 1 the 3 after finishing With only we have released but we are lacking in Python we have an automatic about collective it uses the reference something knowledge so we can have access to mean so this our reference in using and is the macros finding right and by the right so if we have to do with Python objects we can you see that it will take care of the and it also gave us the faster cycle detector because you know a whole reference Watts and
basically takes an account of common external references have fun out there and
when you discount the constants 0 a free that basically it has a very very basic infrastructure but we don't have so I want and the means that we can have a reference to to that to the cell so and by the direct collective cycle detector and about what happens when we have to be with since he will have
exceptions as far as you know so by convention we after returning point that point there's under reduce their my variable that Python interpreter past and encode it
looks like that we will use the oxygen we some type in the spaces basic with assets and we're talking about and I was also there are differences between all of these buildings from Python to you we want to work with other Python 3 there are a lot of differences in the API but also the differences between and how can we find the models in Python to us and you are
since we have to call on the phone with some arguments independent competition it's an absence semantics in Python 3 we have to define this is that we a lot of more information we have this we can define more things but basically we use on into a multi best our friends to our and functions what we create unwanted with this structure this is all of that is expensive
and minerals that user types see that is an density for function interface for Python biphones it allows us to create fund to both function from some libraries and also allows us to create access and manipulate back that the pipes OK yeah we have
values from their so that and they
still not here we have religion between other types but basically what I found that underlying but has a mechanism for something like a flat do translate from these last 2 but
see that that let's say for example and there is a lot of idea so at some mechanism items admired by the people in the recycling what we see that the US and then we can
define also structure we have to know from the start that is defined see that we have to be clear the fields of the fact we all sit types that types we can use the various defined structure as officials again so I'll give you an example and you want to classify algorithm implemented in C and has some libraries and I want to map us by phone call so I can call these algorithms using a on I will also implement a similar theme using Python and I will measure the differences between these 2 2
implementation OK here we have all are within the
community here's the thing I have 1st of all I foresee types and then I don't mind satellite into memory when underlying his opening use in the you and I have a hundred to my to my level that this story might viable lead field and then make the final point 2 thousand people that takes an argument OK and basically what this function that these are out there that close my 1 of funds and implemented in seasonal I have a reference to my celebrity loaded in memory memory you see it and I thought my for my function implemented in see Figure on this
argument dates on type the you have to do the task to do 1 thing on the right we have the Python implementation social I
a massive there the times using Python is no doubt very very good spends most and we got them so that the differences used in this implementation we then we have since this is the point that microsecond for a little while when in Python implementation we have up to 3 . 2 Ms OK as was as the thing there is in the same but as I told you at
the beginning of 1 of the reasons for producing this dynamic and would be
the which using lexical so
I I thought OK when we used for instance a sample and we can take some gold randomly getting things in the internet and try to grab using using Python so I want to be found I want to 10 and positive there is a a need for that and there are a lot of of 4
so that not based the fact that I get this is for the family
that is this avalanche of mathematical stuff so you want to write using survived
and here we have quantum function that real time so reality is much too float in safer run it takes a of reality on what does that mean of this nature within and so I library using different than and I listed there ABI the symbols that are about and what is the kind of be able to good nice and simple use someone here most out to given a set of symbols used in different time of the uh so when tested in
the simple so OK let's see how this works with load also library that is only this but the space is at a library was compiled with different then I get the reference to the function that I want to rock but in this case the thing is something 1 would survive I can I can continue the in and that the function with In that case as I thought and my for defense and will they of of reality in C would be eligible and the right of of flow so I'm telling here to see even types OK this function with date of point there of Minorities the the dimension of the other in the case also 2nd year there was some type these factors on the front and every time flow there are you going to binary of 2 elements of floats and I passed my right to my function that the mean of these 2 numbers for this forward and so OK
that's looking to see that I want to look as if they both and I look for the open again so on the only half of local so the mechanism is mostly the same that is followed using that themselves uh and we get our reference to a set and with everything is something clearly not our reference tool to satellite and then becomes a status the same OK you now
we will study the
effect I see if the fire is so strong that bound for inference and the face form or by phone and in fact they both survive on see if the values of the that the fighting library about see by from been is sold uh into this solution because it seemed that is very close to the status of the missing but we are using the if if I have to start 1 was it also allows us to to you got to go function from some libraries and also we can create access to manipulate see that but
the difference between the that and see if the fight is that we can
work also maybe 11 and maybe as we can see is mostly the same as the died
by and a lot of see if if I want to do that let's say for example
we are working with 500 seat because we are working with of extension we have to let the whole know but a lot of stuff about Python API and a lot of the people that we were working with with the we have to learn the API for example we have to learn a lot of data types from Python all directly to do sequel and we have to learn how can we create an instructor so you we have to to we noted from the last Soviet we have to run things so the thing is also we want to use but aside from we have to learn a new language so they're trying to minimize the things that you have to learn to use their API PAC somebody minimum amount of functions a because all the common way to extend by by on OK so solutions of examples and here we have an example of further from the invention the of fight and yeah we will use it get a reference to print that is in ellipse in each same and so on where does it if a 5 OK I'll use the symbol of and use their functional printer reducing that you I don't have to build the whole the whole thing another differences actually get the compiler to to infer some something In that case I only know that print that takes a point at the time with the former of the things that I want to bring then I will open I opened salivary by conventional when we don't Boston any argument to development it opens itself we create an another
way of saying with water in that case the penalty is the difference between C if financing types yeah we are writing most directly use cold we only have to you learn that newly created new environments in in C a C code on their we have been using variables here we have the reference to lip-sync so we can call that with with my in my pocket but now it's in model the same as we have seen we've see by and becomes the the interesting thing we've see a fair fight but we can work out maybe a 11 the what the
man working at a high level OK we can the steeple directly right there divide in that case it works in the next 1 way OK in 1st place with defined a thing that we will use the case I am defined in the sense that you might want to a function of it economy together uh it takes on the name and then I closed the from somebody from with yeah we have money in a recursive algorithm right in see if if I would take about writing this inverse sequences file compiling and only person similarity to Michael so I don't have to take care us of so I have to do with the data to compile my Michael
and then the if 5 the multiple and then I call my in my function my little thing then OK summer so that it will be easier doing it would survive to all different basis and what happens if we pass sampling understand we have defined field where they can learn in OK so all its of break break then we get a better 1 when we when we tried to call the 1 and 2 without professor and we had a savings so if in that case and you remember the example of that that is that we have to learn and they have to in areas from from structure that is the financing died is it looks like a gas Python but we have to define something it's not like our regular classes and by from inside the termite's anything towards OK and see if the file we can define that case in black remains about and we can access directly from from by phone we have we can write the see and then we can access from cause we don't have to have this this makes between the 2 worlds what is still here and define announced that lead to the Physics and online that the fields so then I can create a pointer to these so these new type and notice that this is the fi when we are dealing with other race and so structure on point that develop into the memory for for the fact that you were in the point on the still so in that case 1 point of x employment
of light will have the seal values obtained From they will assign by the people that are the value we can use over the we note taking care so if you will different into
our and to see if if I go we we'll find that when I told you opening from we are using the same mechanism and then we have uh loathing of celebrities In to memory and if everything is OK In the case I tell you what it is is all so we can uh by 1 of the things to we use in CAPS unlike the story at the time the main reference to the minus so library OK so long as well as we can see mechanisms are mostly the same so mn OK
my conclusions we have 3 different ways 2 of them are very very similar if we want to extend helped by a lot of things we want to we will use by finding the the I would like to mention here we want to all external articles it depends on you and you have
to think that's effective use distributed with a standard library so that have to have a external library external dependencies cipher fights and external moment but it's more portable because and and then use as a fight for example we by by so it depends on the context as we have seen done 3 of the properties of the same principles they are based on the and open and the exams and when operating system cause and with we have differences in portability we have seen that there are also differences in implementing that could have sent in Python 2 1 3 so we are the type tools some bias and if we write down the the expenses on me with use used for example if if I was there will be more portable and in my opinion not if extensions would be hard because we have to to learn How Python works and that knowledge would be right because we it's nice to learn how the important we used survive if we have to to know but it things when we use and see if the final so we gonna have extensively I don't know we've can let go on in that case you have some questions which NASA
microphone over there next to the camera if you have questions the but
I don't have a question as much as the common in the beginning you should have the greatest native extensions of the C maybe I and when you use the post Python project this becomes a lot easier it's like and you just this and that's the also just this code and when you write to lines so you don't have to to learn about the internals what what about do you have to depend on board was that he nutrients libraries so it depends on your context probably do have often available the best practical and you will have to resume to include both to about compiling and interpreting posts will be really difficult and it's it's not difficult but of course you would you would also depend on post but in the end you also get a set library and then you just open item and stuff like that of the conversion is automatically handled for you without any of the coat was 5 so this might be a way to make of the annotations and have and time to or the helical systems so this is the main ministry ways of this could be a way to that thank such any more questions you think image there will be another
Berlin (carriage)
Computer configuration
Computer animation
Code
Subtraction
Maxima and minima
Berlin (carriage)
Standard deviation
Process (computing)
Computer animation
Scientific modelling
Social class
Metropolitan area network
Plane (geometry)
Computer animation
Computer file
Representation (politics)
Bit
Formal language
Library (computing)
Extension (kinesiology)
Metropolitan area network
Summation
Implementation
Sample (statistics)
Computer animation
Lecture/Conference
Multiplication sign
Auto mechanic
Quicksort
Family
Window
Physical system
Point (geometry)
Metropolitan area network
Read-only memory
Computer animation
Lecture/Conference
Formal language
Computer programming
Calculation
Product (category theory)
Computer animation
Multiplication sign
Decision theory
Code
Mereology
Writing
Library (computing)
Physical system
Ocean current
Digital electronics
Computer animation
Binary code
Auto mechanic
Energy level
Sound effect
Writing
Library (computing)
Computer animation
Wage labour
Computer file
Information
Lecture/Conference
Linker (computing)
Binary code
Object (grammar)
Subtraction
Declarative programming
Computer programming
Read-only memory
Context awareness
Wage labour
Structural load
Multiplication sign
Code
Auto mechanic
Similarity (geometry)
Variable (mathematics)
Single-precision floating-point format
Computer animation
Read-only memory
Extension (kinesiology)
Data type
Library (computing)
Physical system
Run time (program lifecycle phase)
Computer animation
Electronic mailing list
Set (mathematics)
Functional (mathematics)
Macro (computer science)
Flux
Extension (kinesiology)
Metropolitan area network
Computer animation
Scientific modelling
Functional (mathematics)
Extension (kinesiology)
Suspension (chemistry)
Modulo (jargon)
Meta element
Computer animation
Function (mathematics)
Multiplication sign
Statistics
Module (mathematics)
Variable (mathematics)
Functional (mathematics)
Extension (kinesiology)
Point (geometry)
Meta element
Implementation
Computer file
Code
Ring (mathematics)
Scientific modelling
Multiplication sign
Parameter (computer programming)
Event horizon
Table (information)
Number
Lecture/Conference
Hypermedia
Energy level
Module (mathematics)
Computing platform
Physical system
Suite (music)
Chemical equation
Software developer
Forcing (mathematics)
Independence (probability theory)
Core dump
Ext functor
Bit
Instance (computer science)
Functional (mathematics)
Arithmetic mean
Word
Computer animation
Function (mathematics)
Noise
Right angle
Resultant
Library (computing)
Extension (kinesiology)
Read-only memory
Asynchronous Transfer Mode
Structural load
Set (mathematics)
Scientific modelling
Multiplication sign
Data dictionary
Number
Video game
Term (mathematics)
Module (mathematics)
Physical system
Form (programming)
Link (knot theory)
Cellular automaton
Sampling (statistics)
Bit
Incidence algebra
Functional (mathematics)
Symbol table
Computer animation
Personal digital assistant
Function (mathematics)
Buffer solution
Window
Library (computing)
Read-only memory
Computer animation
Function (mathematics)
Set (mathematics)
Memory management
Volume (thermodynamics)
Right angle
Module (mathematics)
Object (grammar)
Cycle (graph theory)
Total S.A.
Macro (computer science)
Logical constant
Metropolitan area network
Arithmetic mean
Computer animation
Lecture/Conference
Cellular automaton
Direction (geometry)
Memory management
Cycle (graph theory)
Point (geometry)
Building
Spacetime
Ring (mathematics)
Set (mathematics)
Scientific modelling
Total S.A.
Data management
Summation
Pointer (computer programming)
Computer animation
Interpreter (computing)
Module (mathematics)
Data type
Subtraction
Exception handling
Multiplication
System call
Information
Independence (probability theory)
Interface (computing)
Parameter (computer programming)
Functional (mathematics)
Semantics (computer science)
Population density
Type theory
Computer animation
Module (mathematics)
Data structure
Data type
Library (computing)
Metropolitan area network
Type theory
Computer animation
Lecture/Conference
Auto mechanic
Data Encryption Standard
Binary file
Data type
Algorithm
Implementation
Mapping
Point (geometry)
Code
System call
Field (computer science)
Social class
Type theory
Computer animation
Lecture/Conference
Function (mathematics)
Data structure
Gamma function
Subtraction
Data type
Library (computing)
Satellite
Point (geometry)
Read-only memory
Implementation
Parameter (computer programming)
Functional (mathematics)
Open set
Field (computer science)
Type theory
Computer animation
Lecture/Conference
Energy level
Right angle
Figurate number
Data type
Task (computing)
Point (geometry)
Metropolitan area network
Implementation
Type theory
Computer animation
Multiplication sign
Data Encryption Standard
Subtraction
Loop (music)
Type theory
Computer animation
Lecture/Conference
Mathematical singularity
Sampling (statistics)
Code
Instance (computer science)
Metropolitan area network
Real number
Multiplication sign
Magneto-optical drive
Code
Median
Real-time operating system
Set (mathematics)
Binary file
Discrete element method
Functional (mathematics)
Symbol table
Mathematics
Arithmetic mean
Type theory
Computer animation
Lecture/Conference
Function (mathematics)
Quantum
Data Encryption Standard
Uniform boundedness principle
Family
Library (computing)
Point (geometry)
Satellite
Dataflow
Asynchronous Transfer Mode
Divisor
Multiplication sign
Auto mechanic
Number
Pointer (computer programming)
Type theory
Statistics
Curvature
Spacetime
Structural load
Magneto-optical drive
Element (mathematics)
Set (mathematics)
Functional (mathematics)
Arithmetic mean
Personal digital assistant
Hausdorff dimension
Right angle
Data type
Local ring
Library (computing)
System call
Type theory
Computer animation
Sound effect
Functional (mathematics)
Inference
Library (computing)
Form (programming)
Point (geometry)
System call
Sequel
Software developer
Multiplication sign
File format
Letterpress printing
Parameter (computer programming)
Functional (mathematics)
Ellipse
Formal language
Compiler
Maxima and minima
Type theory
Computer animation
Personal digital assistant
Extension (kinesiology)
Subtraction
Data type
Algorithm
Code
Scientific modelling
Similarity (geometry)
Water vapor
Variable (mathematics)
Functional (mathematics)
Sequence
Computer animation
Integrated development environment
Personal digital assistant
Energy level
Right angle
Energy level
Recursion
Subtraction
Data type
Metropolitan area network
Point (geometry)
Area
Curvature
Multiplication
System call
Computer file
Line (geometry)
Software developer
Computer file
Point (geometry)
Basis (linear algebra)
Control flow
Functional (mathematics)
Field (computer science)
Pointer (computer programming)
Computer animation
Causality
Personal digital assistant
Boom (sailing)
Energy level
Data structure
Integer
Data type
Read-only memory
Hausdorff space
Computer animation
Spherical cap
Structural load
Personal digital assistant
Multiplication sign
Auto mechanic
Portable communications device
Library (computing)
Data type
Metropolitan area network
Context awareness
Moment (mathematics)
Sound effect
Portable communications device
Category of being
Computer animation
Causality
Personal digital assistant
Extension (kinesiology)
Subtraction
Data type
Library (computing)
Physical system
Metropolitan area network
Context awareness
Code
Multiplication sign
Projective plane
Set (mathematics)
Line (geometry)
Open set
Medical imaging
Computer animation
Whiteboard
Data conversion
Extension (kinesiology)
Library (computing)
Physical system
Loading...
Feedback

Timings

  703 ms - page object

Version

AV-Portal 3.8.0 (dec2fe8b0ce2e718d55d6f23ab68f0b2424a1f3f)