Extending Python for Speed

Video thumbnail (Frame 0) Video thumbnail (Frame 2488) Video thumbnail (Frame 4808) Video thumbnail (Frame 7406) Video thumbnail (Frame 8506) Video thumbnail (Frame 9823) Video thumbnail (Frame 13206) Video thumbnail (Frame 15356) Video thumbnail (Frame 17168) Video thumbnail (Frame 18404) Video thumbnail (Frame 19779) Video thumbnail (Frame 21329) Video thumbnail (Frame 23888) Video thumbnail (Frame 24809) Video thumbnail (Frame 30001)
Video in TIB AV-Portal: Extending Python for Speed

Formal Metadata

Extending Python for Speed
Title of Series
Part Number
Number of Parts
CC Attribution - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license.
Release Date

Content Metadata

Subject Area
Python is a nice high-level language, but most graphic applications have some code that must run fast at any price. Everyone knows that it is possible to write C/C++ extensions for Python. Not everyone knows that it can be done cleanly, simply and DRY.
Keywords Libre Graphics Meeting (LGM)
Area Type theory Computer animation Code High-level programming language Cuboid Pattern language Online help Line (geometry) Object (grammar) Extension (kinesiology) Computer programming
Programming language Pixel Interpolation Overhead (computing) Computer file Multiplication sign 1 (number) Set (mathematics) Machine code Variable (mathematics) Event horizon Power (physics) Template (C++) Medical imaging Computer animation Internet forum Operator (mathematics) Network topology Speech synthesis Pattern language Object (grammar) Library (computing)
Field extension Email Building Computer animation Computer file Code Bit Directory service Extension (kinesiology) System call
Point (geometry) Pulse (signal processing) Functional (mathematics) Length Parameter (computer programming) Data dictionary Graph coloring Number Pi Hypermedia Integer Computer-assisted translation Condition number Form (programming) Social class Counting Bit Total S.A. Machine code System call Pointer (computer programming) Computer animation Normal (geometry) Pattern language Object (grammar) Resultant Library (computing) Arc (geometry)
Point (geometry) Pixel Randomization Information Keyboard shortcut Staff (military) Line (geometry) Cartesian coordinate system Medical imaging Field extension Centralizer and normalizer Numeral (linguistics) Computer animation Computer configuration Different (Kate Ryan album) Semiconductor memory Operator (mathematics) Right angle Pattern language Library (computing) Condition number
Point (geometry) Functional (mathematics) Pixel Code Weight Parameter (computer programming) System call Computer programming Field extension Computer animation Internet forum Visualization (computer graphics) Profil (magazine) Cuboid Pattern language Right angle Object (grammar) Position operator Resultant Library (computing)
Slide rule Run time (program lifecycle phase) Inheritance (object-oriented programming) Multiplication sign Binary code System call Symbol table Computer programming Field extension Medical imaging Type theory Computer animation Semiconductor memory Profil (magazine) Library (computing) Physical system
Windows Registry Slide rule Pixel Code Multiplication sign Mereology Number Semiconductor memory Computer configuration Speicherbereinigung Cuboid Programming language Dependent and independent variables Inheritance (object-oriented programming) Cellular automaton Projective plane Moment (mathematics) Content (media) Electronic mailing list Counting Bit Instance (computer science) Type theory Field extension Computer animation Order (biology) Video game Pattern language Object (grammar) Cycle (graph theory)
welcome to to talk about extending type this would be
a quite a technical talk and my name
is not in notes on develop might my can of course
is written in Python so python refund extensions written in C + + it's still a small portrait and only 12 thousand lines of code and it started so in this area so and it's slowly evolving and gaining some popularity recently ieee data sometimes at the director of the program is quite false so I hope to sharing there is you the technique used to make the pattern but 1st of all what is always a Python High all about it the pattern is just a very nice a high-level language so you can see that the way to where box through these encode and do it in C + + or also in a for example the object and you end up with the help of Croat so title makes this nice to keep complex
but there are also drawbacks of course as 1 of the drawbacks is speech and so it's time really fast enough so you don't have them my experience is anaphoric we stuff of course things happened artifice 2nd it just doesn't matter how fast it is it's it's really work and can also do well some stopping pattern for each time so usually with the images you have 2 times time image and the for each of those pixels in there you can't have evolved offered each pixel about walking to each time and compensated and that's perfectly possible as fast and it's even a fast enough to go through something for each motion events so for example when you have a graphic that there are lots of motion events generated per 2nd and then you don't really notice much overhead when you process those motion events in power and but this is about various and when you have to do things for each pixel like compositing or scaling or wrong some interpolation operation then it will be has had to be done in a sea of course is not to get up to speed
so now there are several tools available to extend patterns and their there now Dr. tree ones of those that their known at that have tried to sell and C is the most basic 1 1 I think you've stumbled up and it's very good for us right an existing library especially and you just take your seat possible set of files and it's generates go to make a there is also Pyrex that's the time and tried is a pattern like language and not say much about it the main reason for me from not using this is generates a special language and the idea relaxed to variety In see their I know how to optimize things and not on additional language and they might kind Bayesian use that those a cold generator used from the prior ticket very specialized codes generated but then got away from that later because sounded she object is that these 2 variables and you have a look at this is forum boost pattern if you are going to C + + and especially the more to want the stuff like templates and so every be handled that apply to those so
what I have settled have now is so as to be mainly at the bottom then using it a bit differently than when I started so here's a very simple example uh that's actually all the code you need for a patent extension and you just like the header file that implements you called you have defined as just normal goes and then he writes this I divide for and but really just a lot do is called out to find the file and included in the and includes frustrated and doesn't work so
how do you combine this these I recommend that to use the distutils I think this is way to work it's to right combined among the calls to the so you right he says it up and spruce and you executed with the building next command the legs for believe extensions and I'm a scientist tool called puts the resulting took the current directory so you can directly using start tightening imported and called coat which we have run so I think this is quite simple way to do this but it
and it's a bit more complicated that but the 1st feature of so I think all you need to know that because the city goes to great lengths do wrap existing libraries and it has those that's and that they are out there really getting complicated and things of that in complex than you would want to write something that is not the uh you pass arguments that are not float on integer or something but or a pointer to something you return on several values so but that is 47 what used to be but I use to see the pattern API directly and so we are actually supports this I think this is an optimal and so on but you can just use that together with that I have a shot what about pattern CAT in that 1st of all it's just C API is that condition of Titan that arc you'll find the post as 1 important concept you have to know and that reference counting so partners uses the reference count for each object but those are those are the pilot objects pointless and during the there is conventional who owns those references so you know when you have to release them or when you have to increment the number of references and it's quite a simple convention that basically when you call a function the call always holds the references so when the function returns the value it returns a new reference which the caller's responsible to really and the other way around and they're called pulses ennoblement eater will be owned by the color and the function does not need to release successive borrowed from so now here comes the
more advanced example this is of course also be personalized media and just ask In ahead of violence is of the form that you can do a closeup can make public members of the class and you can see below that you can just access dismember from patterns that could just like and here we have a member function that returns the Party object point this is how we use the pattern CAPM so they could not do anything with this point we can just call function pi value this is an example of of this is a the positive form of storing that we're passing entry individuals and it will return new reference to apply it in a Python dictionary and the dictionary of total of those 3 integer values so on in codes below you see you can and use this slightly would to a normal pattern original treatment use results do we need at this point all the magic the documents usually demands but now this is very nice but there's
1 important thing for a graphic applications especially and there you need a way to share the pixel data and you would not want to copy this you would just want check out the point and 40 so there is at least as I see this the central numeric pipe costs which just apartment option wrapping pointed with information and how it is organized and and you have those different libraries libraries pattern bindings often or at the library for example to decay peak staff can scaling PNG images or or the carousel faces can render of some vector and stuff strong nice underlines the lines of the PET image library they all provide a place to compare it to our memory into the marriage partner right and decelerating condition of excessive from C so and its 1st critical about America
numerical pattern is just extension and I pattern library that allows you to work do must operations so adding a whole array of of individuals for example and here's a little example how are we just can create an array of zeros and assign some values are using the patterns lies in the front and the put some randomness in it and below you see the picture like that of that the so now the question is
how do we in the face now we found our own extension of America pattern and you got a small example for it's a function called Randall something in its place the novel and it takes a pi objects in Python Everything is pi objects that but actually in there a numeric pattern and we positive moderate argument for example radios and then we can use this API call to get the height and weight of the right for example and we can make a simple costs to get a point to the actual data and then we can look to all the pixels and it will be faster speed of the use of the usual tricks to enhance OK that that's about what I
wanted to say about this topic now let's move it to that underlie saying that an existing program this is I think 1 on the right is a tool to profile Python code profile is pretty much known about the this she profit box and makes for a very nice and rich visualization of the results so I can only recommend you to try this if you are trying to get the feeling what is slowing your python code in a patent program it gets down to the level of of the library calls if you have an extension you will see that functions of your extension that are executed what you not see what this C functions they are calling themselves so this is very different now for
moral being that profiling maybe not your Python goes is slow but it's something else maybe some some library has a problems of decided usually do system profiling of all profile is quite nice for this it can analyze the whole system including the call even if you need to and it will tell you about it suspends the execution time so for example and here I got the profile of my parents and doing it scrolling and we see that the actually time is spent in or cannot in my hand and not even the Python extension of my time but x stock is using this the the peaks I tool that assume the image I think and we see below that bound those 20 per cent of the time and the spending in Python and in extension library at specific you to my pen and we also see how much time is spent in the C library for example feeling memory on coping memories stronger and we can also write this down to the individual symbols of course have on slide so what I recommend that produces where things are going slow you might still be surprised what actually is constant judgment what about debugging and very short well
it's just the same as the and he did not in Python library at another time and SC library that when you have a Python program you can't just start to develop for example or and the and on the Python binary and then you start type and it is going to be a library and you can get the best rise can set breakpoints and OK and now here is
final slide about the memories that's because I think it's quite important to know if you actually our writing an extension and it will be the last box of who we are and the good thing here is to know whether they're done those facts usually come from the most common and popular you were producing primarily because the is that you have a reference to some options and that is keeping the object of life so you started into some lists but you don't actually need and you just for example your parents to dismiss for example the pixel data of the layer and the it's the list was not meant to start and prominently but what well mistakes have been and it's a bit hard actually to find those attendance but I think that correctly an awful lot of if you know something but I think there's nowhere common tool that it is you can also heightened for a list of all the memory it is managing all the objects into account counts the number of instances of this is where is the trick here is to this and now common dropped piece and reform cycles for this I have to say time that uses this reference counting but then there are can be and an object to friends in and other 1 and that 1 reference in the 1st again the cell to break the cycle so there's the garbage collector and the garbage collector can only work if he knows how to call that destructive of those types of objects and is especially in what order so if you have a reference cycle and both of the objects have a destructive impact and doesn't know how much the orbital is too cold and how will it will not do anything it released a reference online and look at it is easy to detect you have just to call this detective garbage collector module there's this garbage member and when it is empty then it's good and then there's some stuff in there then you have a look at it and find out how to break it as a reference dangereuses the generalizes both the structural so watch out and you might have to do is and not just start would be also you forget to decrease the reference count in your seek out about this is quite rare it tells you don't have to handle talented well since it never happened to me that actually discuss the problem so that's about it to the content is available here in this world if you want to check out some personal examples and if there are any
questions at good thank you very the b times does anyone have a question there's always of course the chance to ask you questions center and maybe it's the moment to invite the people that arrived to a bit later to come and have a seat In the meantime people can think about of the questions they my eagle for example welcome back to the this is a question here and generally speaking when you you're this of we consider project to try to work on Python and having to see if something registry consider generally speaking but yes I think especially if the project is new and it's those existing code maybe there's a lot to seacoast based on what projects usually using to do is that they do not extend part both the embedded so they it from the C code at but I would suddenly recommender when starting a new project to it is if you want to do this in pattern then extend pardon not that time but for existing projects it depends of course must develop respond if there are some of the familiar with but the like the language and the battery right the whole world publication that might not be an option we have time for 1 more question I think mood of the people we can discuss later thank you very thank