Kung Fu at Dawn with Itertools

Video in TIB AV-Portal: Kung Fu at Dawn with Itertools

 Title Kung Fu at Dawn with Itertools Title of Series EuroPython 2016 Part Number 18 Number of Parts 169 Author License CC Attribution - NonCommercial - ShareAlike 3.0 Unported:You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license. Identifiers 10.5446/21239 (DOI) Publisher Release Date 2016 Language English

 Subject Area Computer Science Abstract Víctor Terrón - Kung Fu at Dawn with Itertools The itertools module is one of the cornerstones of advanced programming in Python. This talk offers practical advice about iterator algebra that can be put into practice immediately. Discovering the itertools module means taking the veil from our eyes, and once we use functions such as repeat, takewhile, dropwhile or product, there is no return — it is impossible to come back to the world of the mere mortals, where solutions are more complex and need more memory. ----- The itertools module is one of the cornerstones of advanced programming in Python. Part of the standard library, it provides an iterator algebra that allows us to elegantly chain abstractions, enabling solutions that are both simpler and more memory efficient. The goal of this talk is to offer practical advice and clear lessons that can be immediately put into practice. Illustrating it with numerous examples, attendees will leave having assimilated at least several concepts that will improve their code undeniably and irremediably. Emphasis will be on showing specific cases where a traditional solution can be overhauled over and over with functions from the itertools module. Let’s say, for example, that we want to alternate indefinitely between two values: -1 and 1. The novice would use a variable, updating its value at each step, and the average user would maybe opt for an endless generator. Both are worthy and honorable solutions, but they pale before the mastery of the martial artist who only needs itertools.cycle. Because that is the nature of the itertools module: once the veil falls from our eyes and we come across functions such as repeat, takewhile, dropwhile or product, there is no return. In this talk we will learn to identify when they can be used, accomplishing with a single line of code what for the mere mortals takes much more effort.
presentation building integrators states exiting time Ranges ones sheaf dictionaries Arm information single Em's box series monster exception classes area man generate mapping The list meetings orbit Types data management orders normal Right write point functionality files maximal Mass elements number terms string level focus key lines limitations system call Indexable words loop Spring Computer animation case life iterations Abstract
Actions code states time sources Ranges water prime number part Erzeugung information mathematics estimates memory square framework stable management generate capacity The list terms Right point track flow functionality factor similar fields number elements powerful Prime root naturally slices optimal conditions default noise Super expression counting division Symbolic frame words loop Computer animation case life iterations objects
Prime man functionality generate Computer animation states case string elements
Actions exiting time equal Ranges sets water part prime number dictionaries programs area Dogs mathematics rates different memory Forest square negative box series extent source code algorithm graded The list maximal staff wave hash dual chain input pattern Right cycle p-values Results record point flow track functionality server factor files print threshold events elements product number training second powerful MACH Prime versions period root causal string manifolds integer conditions default comparison key sin Ionic forces NET expression counting lines limitations system call frame words kernel Computer animation case predicate life iterations objects table Snake
functional programming pixel Actions building combination Contracts sets argument part information mathematics rates scratch framework Office model hill Recursive classes generate Blocks construction fit Types means communication input website sum pattern Right Free Results record point Slides functionality perfect distances fields elements product level conditions form modules Super surface lines Computer animation case Factory life iterations objects protocols
and this presentation is be that the on his side in reliability engineering and delivers and there's a good them and we that I did want and
this is not conform but it was super so I have use will make up for it later also have a lot of things to see if any of you disagrees breeding is going to happen and and will not how we
look over at at least in my right we're told these all the time we have to say for thinking in whatever I do think for example here we're looking over at the end and we are always told that's where they're doing these which we might be tempted to do when we come from other languages we say OK I'm going to generate all the indexes and I'm going to do stuff with the indexes accessing each 1 of the elements and even more so there was no we don't want to do this like even instead of using a range we could tried to increase the value of the index i have we don't do this so we know we have to use what I found for words which another language at this age for each and we know it that it that's going to give us a higher abstraction level which means we call this more readable leads to Europe so we like we have to think about what's going on we say OK move over these whatever it is that it works it's nice out so it's safer and this is important because we are not working with index of 1 how the think that's amiable like we just have to do that for example if we tried to work with index and how it could this is a off whenever forgot we we want the far and we tried to access the end of the last element it's not there when we use that for each state i from 4 this is not something that can happen and of course it's the only and were replaced we can use it for example with strings for each player the in the spring of whether it's the name of the variable we can access each 1 of the characters in the string we using sex we can look over the young although we don't know the order in which we are going to see that we can look over dictionaries for example here using items we can get the get the key and the value of each mappings and even with files we open the file and then we use the word lower over the final descriptor and we can move over each 1 of life and that's the when in the end we can use loop with anything that's itself and that's ended up we take it for granted not question what some it when we go to the box it turns out all the answers that are not only started flowing into the box and and it says OK only to not get capable of turning 1 element at a time blah-blah-blah what it means our class has to implement a methyl there down there or also get item focus on the 1st so OK we have to implement during these my method what what's this 1 this book on it says OK this method is going to be called when we need and and if so it should return and iterate OK so we know what we know we need to implement ITER and make it returning it so 1st of you at some point we will call either of the building and we will get the iterative it's so if we have list of numbers for example we can say OK agreement and on the right there on this it's there it's a mass of the least of and if we call it the bill and get the interior door yes it's not least iterator so the blind but now what's an intuitive Will it turns out that an iterator is something that conforms to the probably which means you have to implement 2 different so you the 1st employment title again and you have to return silver which makes sense but 1st it some look like but you have to do that thing so this is remember we're writing we are writing our own iterator and adults are right it makes sense in that case you can use the greater with the forward we face for an animal and states in which in a way we don't care we will build this we can follow instructions and then we have to implement and all the management of women next and next to the return the next element and when there are no limits left which should raise iteration I don't know what the 2nd sentence means it's something low-level and this was next thing thanks to knowledge of the 2 and so now can finally our can't see all work inter-rater and we are in in over a place set up in series of elements as the ITER is which which trying so OK and next is going to return the next element once we are no more elements we went far with index we're going to raise stop iteration date with it and we can see the ones that are in the world so I have stopped iteration now something important we noticed that when we work with it the way the next person to return the next 1 in but we can only use the next iterate so so if we try to call next directly only at least it's not going to work so to say OK no it's not an intuitive depending on the so we have to call it 1st eta and then we get the integrator and then we can call next as many times as we want until we hit this stop iteration iteration which means no more elements life and that's exactly what the for is going on with it's called it's using ITER it's getting and right there and then it stops going next forever and you it sees stop iteration so if we want to implement for a loop in pure Python for no reasonable we will have to something like that and that's basically what in low-level actual useful goes it's happening in the class who room not something nice we said OK you enjoyed on these monster there we wrote that racist iteration it has in the past 2 ways it's with an exception when there are no elements like but we haven't seen any exception when we work with my family for over we look over something we're to stay in section why because the for is actually listening for these stop iteration and that's sick and it is is to stop processing so it's actually going to be expecting a stop iteration in the in 1 of the next calls and when that happens it breaks now during nanoreactors area are a special type of Italy so that we all know how a normal function works we start in the function we do stuff and then at some point within each and we return from and the that's all use but sometimes we can think about it whatever we have to return from from our function has to be done once for example here we have things to simple function and it's it has a single exit point so we do something 1st line 2nd line and then we return a value we can have multiple single points this is 1 1 we can read from the functions in the in the 1st return used in a 2nd but once we hit 1 of the terms we haven't finished and we will never be if a function doesn't have the term that's a please not so actually functions are always return is something we don't say return something it's returning of known now how is that problem well it's not that for example to we want to work with a using numbers and what we want get even function with that that's going to give us a of it is numbers until the 1st meeting with right we the right the simple act the way orbit Python way using right which we passed clearly and it works with that could become a problem at some point we want to work with a lot of numbers because this function we said OK we have to return everything using time so if I want to some millions and millions those outside of the save numbers we can't do that because that's something defeating into so that's what we use generators we just generate our functions that are able to return values want which me is the state of the function is frozen and we wouldn't use which will used you and when we get to a deal that we return value and function phrases there until we call the following next 1 over over that's so in basically we will have 1 or more jails finds itself going to be normal function it's
going to be generator function and it's going to be it's going to we're trying to do it in a way which for short referred to us generate my so for example here simple generator using to using G which means we can call it up to to 3 times and it's going it's going to return value is only 1 of the last time there anonymous left so it's going to return as old iterator iterates iterations so in this case we say OK we want which all natural numbers OK I'm going to say that I'm going to do this I have found and was look it's it's never going to work but they don't care because only returning elements 1 1 in this way would keep calling these generators all the time and we the normal function during to try that if you try to find these people and can process because it's never going to work we can never return all natural numbers as far as I know what I mean so here we are I wasn't paying attention iterator object on which we call next that's similar thing we can and every generator is an iterator but not vice versa also and this is something greater Alex Martelli what it was before everything you can generator with the with with own an iterative class but generate also give you a basic framework basically all the states the people in the in state is already done for you frameworks there so if it's a single most of the time even for simple cases we can use a simple generated for extra points how how we measure a generator and we call when no attempt is we will we are generated numbers when they want so if we go away and it's going to say that there can we could call it would cost capacity to Italy's and then we could call and what's the parliament will listen for women to be knowing all the list in memory and then we can use so we can say OK I'm going to add a lot numbers 1 for each element in the generated so we have to to solve the director and in that way we can come to an end before estimation we use and restore a C away throwaway variable the signal we don't care about the actual money way time for some 5 minutes left I would want that were both is not inside and we have what we do have water stabilize thinking and the prime numbers and numbers and the way how we determine how you determine the fate a number is prime there are a lot of actually ways really mathematical ways but I like the that I'm going to do it by hand I want to try all possible member because prime number only have all have was only going to have to be the source but to and it's 1 itself OK I'm going to try all the other members and if there is at least 1 device or you're not try and otherwise you are applying so I'm going to do it this way it works now we know have to do that really and in the end we need independently said no no no you use right the right we can look elegantly over all the candidate might missing mandatory optimization although it's not strictly related to what we're seeing here we can we only have to check devices of the square there are a lot of groups I didn't understand them actually a really bad 600 flow again there was approved for humans there but basically assessed every non prime number will have any device noise on the square root for so OK that's a lot of don't pronounce so now you did your how how can we use of something remotely In the year factorization so the composing number into factors so for example in is 2 times 2 times 2 and symbols method again the trial division which means that we are going to try all the prime numbers and we are going to look at sissy OK can by number of ideas so did write my solution here I said OK I want what I want to get the 1st n prime numbers move come to it OK I'm going to be only in interlaced and I'm going to keep trying numbers and and the number of prime ideals then I put it the list and finally I will try it work again problem we're returning all the prime numbers and so on OK we're here so let's use the generator and in a way we the same that we don't use these memory in which we can get we can return the prime numbers we joined but now we are going to different things here and that's how cold can right because we are we are keeping track of how many numbers we have generated so far and what's the next number have to among books books and that's due to this kind because we are in the other people's pockets identical super simple keep it going it's going to make an iterator the returns number and the default we are going to come from 0 and default that these 1 In other words it's like range but we know actually we can call it keeps calling keep calling forever for example here OK next next next and we will get number and how is is usable well we can at least simplify are code a little later we can say OK for a number in it Interpol's count and that's never going to stop but still willing to think we're generating prime numbers and the same time we counting how many have generated so far so I'm going to do something OK I'm want to display in 2 different things 1st a generator of power prime numbers and then a function get crimes which is going to be seeing all the generator and it's going to say OK how many have returned so far these many OK example but we have not we stopped the but we can make it even better again we have written a function there for did before we can use a simple generate expression for probably maybe even better in this case we can say OK OK filter I'm going to feel all the numbers starting from 2 I mean going to take only is prime because that's what fields were going to the field was going to lazily 1 by 1 it's going to check the numbers everything we next and it's going to return the next thing that this condition filter by the way was heightened tools I feel there before but again that all history get priced now becomes this thing OK and counter and have released primes because then prime is going to be a it's going to be away from the friends I want and we don't count on when you about comment this thing is a common problem actually give me the 1st n elements of these intervals and it turns out we can use slice impatient because it's going to complain excellence when the signal generator it's not a subscript which makes sense because you're generating millions wondering what you're doing that horrible Lexus randomly on you can do that but you can use it to to a kind of tools is life which is basically like slice of patient about doing lastly we all frames which so for example if we have item pools counts which means all the numbers in the world and we say OK intervals is like as 5 take 5 it's only going to address the 1st 5 and then stops but we can use it not only with generators we can use it with for example with a brain and we are going to the 1st 3 letters or we can and the 1 that's usually can use steps for the while or we can start counting up from 0 from all right so if we want the 1st and the numbers we can say OK intervals is sliced crimes and as many numbers as many so let's with right or function now we have a k I have all the on going to loop over these thing is slices returning an angel and then and now that's OK we're being over the right and the only thing we do is given that's what you from was invented from from has what we it allows us to always have that the important part it allows allows us to delegate power of the work to a different generate change every time we call the next June found is going to last the 2nd generator OK in try next year in your value and it's going to return it so it's basically the same thing as we were doing before it's the same thing as doing for anything thing deal that thing we can do it with in a 2nd language you from so
here found using deal from food which means we're simply calling internal food and returning the on
so it's called found goes to fill and food yields an element and deals state into turn and it works I
can not only with generators we can assume that we've anything that's internal for example here are dealing from a string the given novels so we can rewrite our function as states we're simply OK we're taking the waste of primes and we're taking how many of the same name and it works for example in this case far and it works 18 minutes so far and I have not