Logo TIB AV-Portal Logo TIB AV-Portal

12 years of Pylint

Video in TIB AV-Portal: 12 years of Pylint

Formal Metadata

12 years of Pylint
or How I learned to stop worrying about bugs
Title of Series
Part Number
Number of Parts
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license.
Release Date
Production Place
Bilbao, Euskadi, Spain

Content Metadata

Subject Area
Claudiu Popa - 12 years of Pylint (or How I learned to stop worrying about bugs) Given the dynamic nature of Python, some bugs tend to creep in our codebases. Innocents NameErrors or hard-to-find bugs with variables used in a closure, but defined in a loop, they all stand no chance in front of Pylint. In this talk, I'll present one of the oldest static analysis tools for Python, with emphasis on what it can do to understand your Python code. Pylint is both a style checker, enforcing PEP 8 rules, as well as a code checker in the vein of pyflakes and the likes, but its true power isn't always obvious to the eye of beholder. It can detect simple bugs such as unused variables and imports, but it can also detect more complicated cases such as invalid arguments passed to functions, it understands the method resolution order of your classes and what special methods aren't implemented correctly. Starting from abstract syntax trees, we'll go through its inference engine and we'll see how Pylint understands the logical flow of your program and what sort of type hinting techniques are used to improve its inference, including PEP 484 type hints. As a bonus, I'll show how it can be used to help you port your long-forgotten library to Python 3, using its new --py3k mode.
Keywords EuroPython Conference EP 2015 EuroPython 2015
Computer animation binding
link time sets analysis Code static Em's Software logics system testing data types area rules programming computation total Code instance Types processes errors Computer animation orders system testing form
system call views analysis inverted Code fan programs variables effects Software system functions form Blocks DNF programming share computation Code instance variables Modular processes Computer animation SoC system statements testing WPAN form reading
exiting gaps inner analysis argument variables static Computer animation case orders Video classes objectives errors Cats classes
point constantly states gaps moment states branch Code functions Computer animation statements conditions pattern functions sort Right Cats conditions conditional probabilities
Virus response Computer animation closures case Ranges pattern print Right variance
components gaps time schemes clients Code orders of magnitude maintenance Geschwindigkeit pattern functions Abstract sources parsing Fourier building Development inference engine Continuation AST static testing Right pattern Representation structure point abstract syntax flow statistics lines real component life AST analysis versions topology Google string graphs patched Abstract interpretation basis print lines Code Symbolic Indexable Computer animation case topology PEP versions interpretations verification cfg
parsing sin gaps AST print rules Modular attributes AST versions mathematics Computer animation topology versions interpretations speech box man record
Super gaps high resolution AST fan orders rates Em's testing vertices statements functions input classes form data types parsing Meta high resolution Abstract interpretation print instance Semantics Modular Schlussfolgerungen Computer animation case topology quads orders versions testing classes Right objectives
acidication water argument Semantics rules automatische Ableitung depth-of-field different operations objectives functions Partial form classes Super binaries inner effects Code instance Schlussfolgerungen Types Computer animation case calculations interpretations classes sum Right Results record
point tuple Density functional theory implementation Super plots instance The list argument rules Semantics mathematics Forest functions classes data types rules part system call Schlussfolgerungen particle Types data management Computer animation functions generic since Results kernel
hard tuple system call Super time The list instance argument rules elements orders Dogs operations string logics Indexable comparison man data types rules Context Operational Research Super comparison multiple Super high resolution cellular moment operations print bin Schlussfolgerungen subset Indexable data management algebraic Computer animation case string logics Blog orders classes sort
Context system call Super gaps AST argument Code elements orders Pointer structural topology verification pattern integer vertices position domain complex analytic matchings mapping high resolution inner lines Indexable Types data management algebraic Computer animation environment case classes verification pattern sort
Slides factor load lines time directions instance Modular argument clients rules fields attributes configuration pattern model extent category errors classes default default multiple help bases DNF Abstract interpretation argument attributes lines Modular connections Schlussfolgerungen Types category words errors Computer animation PEP orders statements interpretations framework classes Results
Context mapping time inverse favourite Pointer data types Gravitationsgesetz Context default mapping Guide Amsterdam Ordnance Datum attributes staff print Code Modular Computer animation URN Forum orders sum matrix reduce source
tuple Slides attitudes gaps similar analysis multi-process Content proper Code metadata variables structured data static evaluation logics logic box model localization libraries exception data types man constantly proper flow projects Abstract interpretation analysis attributes Continuation variables similar errors static Computer animation PEP exception
words Computer animation time part
Computer animation clients product
Computer animation configuration Flickr sort
Computer animation time
Robots Red Hat Computer animation Rolling
thank you got it's really nice to see so many people's interest
in the 2nd license here and this year is especially for finding because it is a it's a lot of what they so gap basically it's the all the static analyzes to invite and that is the main thing which is really nice so in this talk i'm going to talk all to be about binds history and the so they need to or in static analyzes and we'll see where where that goes so what is this the inn
think I'm talking about basically link is like a tool you use to analyze the code in order to find blogs or but they should all areas or style or stuff like that but saying this about finding is an understatement since it can detect a lot more than normally in the total this logic which enforces debate the roles of it's a structural analyzes looking at your all classes and and your special methods and checking if they are implemented correctly and so so type don't find works by using a technique called static analyzes which is basically the act of on analyzing cold without actually executing it on and if you don't use static analyzes right now should use the following you should use any other not just finding all because it can really help
you in your day-to-day job all instance you can use that analyzes if you have a lot of tests and it takes a lot of time for and you just want to check that for all there are no obvious areas no called stuff like that or if you have been legacy systems which no have this test set
of infected as my 1st job experience of I work in a company where we have a lot of big legacy systems but they do not have this so all using static analyzes was of paramount for for checking that every everything works before going in the introduction and if you don't use it as a form of linking views but of course it's not equivalent manual review but is better than no review at but so all of this of of called read
share has a couple of problems following I'm going to show all it takes this kind of stuff so for instance if you can see imported always but I use it across my program and I also define a variable which is just there is not use of course this is a this is not the problem per se but that it leads to all of the code or unmaintainable called or stuff like that and more important is this block here from 8 to 10 all this is just saying that I invented that called incorrectly and that would now we're not because it's often statement so basically it's useless gold also you have mind then I had little called the execute method but I didn't so that's also a problem and as against the dollar down here to find the things this kind of stuff and properly it means that there is something fishy in your code but can you think of more serious bonds and things like using undefined variables or trying to access members that don't exist for going functions that on functions like
going an integer and as against the here find the takes this kind of stuff and it's really nice for a static analysis tool tool video that's you or you made a mistake in a program and as it was saying go along before why in is also structural analyzes checking that all your special methods are properly implemented for data classes that are our recent correctly and in this case the down exit methods all needs to be parameters in order to to work properly and finding things this kind of stuff you use it then it's that up to that warning right so Moon and other you might
error find can detect is this 1 physically sure I have
a function and then I have an if statement to read it found as as a condition of the conditional for for the statement and what point in the future do initially was long but I forgot to write a paper the patterns and this statement will always be true and that the state is the statement if branch will always be taken so issue right this for this for this piece of gold you might and how to say that if you like this for this for this piece of code you might overlook of rocket usually you didn't this problem is solved by an can can help you in this kind of situation as well pick a couple
of moments to figure out what's the here Fisher to answer you do not want anyone so at higher values mean changing the so what's the output of this code who said that yeah that's it so sort of basically what we did is we
created a closure here and the virus was looked up independence schools when the when called that doesn't
involve are was was on the up independent scope and gold and for this particular case the patterns school was these comprehension at this by the 3 sold when the formal was executed and callable was called basically what was the last value from the formal which was 9 so that's about and then you should try to do any of you you might have missed this stuff and why can help you get there would the sound of situation going on the
middle world history about finding because created 2003 by a friendship company called modular already these maintain it for a while for 10 years but there no they're not so involved in the morning thank him not involved in the morning once the development from even though many is still seen that about about them which is wrong I I know that we will use is a modified version of wine you internally for the use cases are because they had a maintainer for large that was also maintain a for fighting and they use to push upstream a lot of changes from and some statistics so according to of the document we have over 55 over 2 thousand lines of code which is a pretty big but not enough because we can you take all the problems in your and unfortunately unfortunately applying his GPL license because there detailed was of the order of magnitude intervals and it's there is yeah but then she is my involvement the client basically in I started
working on it in 2013 went by 1 . 0 was released and opted by 123 II become a committed maintain that doing maintain and the dialog which is a good thing for 1 and then introduced by way 0 in 2016 from my time velocity and I'm going to have some way of some advanced techniques in it but it's still a little right now pretty advanced but is going to be even more exams and that all basically I'm going to use a technique called passive interpretation where I'm going through all these interprets statically your goal and not actually executing but statically all no so I'm going to add support for big 484 I want to all we just talk was about and so changing the things internally like having control flow graphs and having the symbolic execution engine for Fourier yes right there and at this point you might ask yourself book 8 this is cool but how it works and why does it work static 1st of all widened is fitted in actually into components real checker which is finally and and it's inference engine and the component that understands quite and which is estimated on the following the general pattern of building only interim basis review using abstract syntax trees but even though all Essex index trees are augmented with a couple of flow of functionalities that have to have all seen them in building the inference engine and internally Astrid uses the dividend module a scheme which kind of looks like this when using that you import os and you give it a string them off from their you get abstract syntax which basically looks like this so you guessing
at the top of the tree we have a man which has a a couple of comments and the rule of the blue stuff of that the blue thing is is unknown and the the rest of them are just the attributes of those nodes so all the speech is structured in you can just by looking at the if you can reason about cold even though great it's built in you don't
have to write own boxes is not perfect because the it's not backwards compatible not even tho for minor versions of Python releases like 50 . 14 2 . 5 this the going to add new nodes or remove nodes or change things over there and the aspirated strives to be a backwards compatible long record over a speed up you can use for Python 2 . 7 2 5 and 3 . 5 and as well for my 1 giant or any of the interpreter which supports abstract syntax trees and DAPI is quite similar with the release thing busy doesn't differ much we have basically the
same functions yeah so if you're using a steel right now and you want something more capable of you could you could use in and as it has a mention of our as signals are quite similar with 1 from AST but the really old men them with the a couple of capabilities that are specific for for our purposes all like in this example for instance you can actually reduce the patent of of over and you can you can walk up in the tree starting from 1 0 also we don't walk down by using the yet to get children methods right was a you know can achieve the
lexical scope of a variable but for instance in this particular case the lexical scope of form is the function the rate is defined and the down there and miscomprehension basically and from for for you in Japan and the the scope of 4 of full at least on Python 3 is use comprehension itself so full doesn't occur outside now in while can actually nodes and the stuff and that I some notes are augmented even more more than that all like in this example we have a couple a couple of classes and the using a metaclass using ABC ABC made them and defining sent lots for all of our class and you would find in what again called so all that we can achieve the lots of 2 classes using slots method hold the method that using the metaclass method or using the immoral method you make it we could retrieve the meta the resolution on the review of classes which is actually the method is elution order when running them over that of scriptable or or that test which is nice yeah um that the most important thing what about the asteroid here is not the case the itself with the capability of living in Florence basically involves means the act of resolving what an old really represents like if you have a name and that means represents something like a variable a function called you want to for that and you want to see what's at the end of the of the tree for that particular node and Our nodes basically implements the
fighter semantics for instance the court found for us will really would be determined values of the coffin all this water these qualities comprehensions it's impossible would be gas actually say that use of the donor when evaluating these comprehension power and only vessels of the sums partial of interpretation but it's partial because it's not looking good for all of the cases in your code right
and in this particular example where we have a function which adds and 2 values the argument of form and as you can see in this particular case when informing what this function actually goes effect when find what's the result of 2 function called abstract knowledge that it's 48 because 24 + 24 and 48 yeah this is a more topics example involving binary operators and if you know the the rules so awful binary operators by Python and goes like that if you have tool different objects and you try to add them 1st of all what the down the ads method of the left hand side would be called and if that doesn't are not implemented then I think the the right hand side of the object no I think that the idea and the on and methods of the right hand so the object will be called of any particular examples that have been in the left hand side a support by which is a as you can see a is a super type for the because of the has as its base class and in in the on the right hand side we have the which is a supertype of 80 and the rules of the that the difference in this particular case because of the 4 in the semantics would be the same as for the for the all different classes are all then the 1st row then the 1st method a 8 at done that add to always be called which is wrong and in this particular example of what happened is that 1st all down there are admitted of the because and this but for example if just not implemented so it will fall back toward the other 1 which is will add the down there and methods from a and you should do the calculations you can see that acid described below the result of that operation is actually 45 the OK
no as great find is great but they can't understand your called I mean the Carnegie understand your excellent called sold we have to deal with this kind of situation and we provide you provide of providing all a couple of HDIs that could would help you like her having Nolde's transforms so basically we're we denote transform begins modify a plot of the ACT tool be some something else like we have a function call and instead of that function call you want to do in aligning glial and replacing the base the node with the result of the function or anything you'd like and begin to adapt to this the basically that API is a function which should accept the 1 parameter that being the only original knowledge and should be determined by the you know the modified where a new and old and you just register that function will be done in demand manager and so long implementation details but anyway you just register your class or function and when doing as what's that the density function called for for that particular type of knowledge In this example I'm registering very transform for the class node and also you can apply a filter function because you don't actually want to change all classes or generic or maybe you want to change something in particle and has this year I'm just I'm giving you the filter functions for filtering in other classes that it's not seeks and but the 2nd thing can be used for in forest fuels because at some point here you want to have different in foreign semantics then fighting despite and offers you sold using the same API you can you can provide a custom inference rules for I
don't want to pay for his comprehensions differently on for function calls differently so you're using this involves a universally and you're changing basically the somatic cells widen it in your face the and really eating our own
dog food in this example and that in this example the all about food in fossils because we have inference rules for remember this as a sure you immediately basically reading fossils all of your understanding is a stance is subclass well you understand pottery etc. died callable least say whatever other understanding not in particularly in fossils all binary arithmetical operations who would logical operators sort of comparisons also the understanding pundits manager these same string indexing slices yeah but as to
what you think a couple of moments to see where blogs in this particular college there should be like the box but if you can find more John violence also should not be assets reality lounge and as a hint man basically a what the 1st element of support specified in the last that the last argument of support specifies the object for of throwing reach the method is ocean or the validity and in this I all yeah that's good because in that particular case surpass in cells would would be hard demetallization order in this particular case I don't know what it is but 1st pursue self will call on the because this is a doesn't have rules yeah and it's multiple multiple of cooperative inheritance if I'm not mistaken and the ball will be called and people doesn't accept 2 arguments anyone how yeah that there is no spot and that is if you put all sorry so that there are too many ways 1 the time for example exactly so the binding
Texas stuff because asteroid knows quite and and uh and as again see what what Biden says there are too many position elements applying for just actually all full is not going about of course because food is an integer and lot of supernovae has no spam and 1 which is actually true because it's spam or not spam yes uh is a more complex example of of pastoral capabilities and has been see here's a strange understands the this indexing understands has after all of all get out of the and a particular line that match is retrieved what would happen is that has not the recall reported done true because they test methods of coal maps and the aim at the School of all and get not from get to the from a military don't need that object so it would be like through and through and they got mad and in this particular case the last value which is also true will be done so that would be a that and then the context manager is involved and the context manager returns reviews found which has no argument whatsoever and we're going to call that environment see that they use domain positional arguments not enough method called should change so this kind of stuff that astronauts aboard white and gold the opposite finding knows knows about it as well yeah but
and aligned with the fast 1st and I have but I think you know if I and is not so complicated as it is the most complicated thing this it it's like the fancy Walker of the and it has a couple of patterns more than the problem because qualifying can detect almost 180 type of sort of verifications on and basically you using these that patent will visit each non because that of patterns KDD Cup was your little structures from the it's and and a small example of how this is to this is pattern works let's see the implementing our own checkered qualified you and in this particular example the important
connections and obtaining the default attribute from collections there's no default activity directions by the rates and for the and what will happen is that this is your option would be called with the original which be get off knowledge afterward so you for 1 that knowledge to the represents 1925 and of the statement here represents the benefit of the the name the name collections and we need you to euphoric because really should know what that nor the represents at the end of the stick after words literally according to the yet out from and read the result from the and if that doesn't raise are any area such as not that year elections has the fault of the default category else of eventually to imitate no members of the bond but before that is going to have flaw in a couple of field or a slide it's to all nodes a mixing class because you might not have that after reading the mixing class for a little the only as a last with unknown bases like bases from extension modules and we regard the understanding social models by the yeah I'll go to you 1st from only because I don't have much time left for basically or absent interpretation would have widened for for this kind of gold that is going to give the example of the done that D is updated with the dictionary of idle what kind of arguments be passed and the headline 5 if you recall or some other and on the or whatever other attributes and that that you but my not exist and that what happens right now is of finances hey you don't have that much of it but did actually having that attribute because he just he said that if you have lying to sold that's what I I have said interpretation is going to fix it doesn't just interpret every line at every logical line in in your cold and it will not at the end what side-effects each line had obtained from have multiple type of checkers and the errors also just conventions for rules or a factoring and various warnings which unnecessarily Monseigneur called of an actual vaults like but no member or not above things like that and we have 2 types of checkers based and the based the top and base generous or for and all of them in the line is too long or bad indentation where are the snows examples all clients
has an overview of the vibrant community you're a lot of floor external packages for improving fossil when proving Bystrica so there of you could write your own if you want to on as the problem is that and their purified and so you don't need to write might encode in order to have a custom chip my guess is it
comes to the the couple of features suffer just like you could generate you you know UML diagrams from your code particularly you could spell check your docs sinks of comments he desire is enabled by default order you could use the bite importance of items you put in check which is a checkered when when activated all other Chickasaw disabled and it will end its staff that is not going to work and by constraining anymore was like using a syntax automobile and so special methods all on of my favorite is using MAP filter on reducing monitorable contexts as it is the inverse of a particular example of items the map of user lazy evaluated and is the in this particular example Donald you you're all will never be called because I the sum because you should be evaluated 1st yeah it
it really time generator stuff like that of although there are
a lot of similar tools like I fixed my In chicken by Jefferies the forefather of finding and I should say a couple of things even by chickadees not defunct and that for a couple of years it was for a more advanced than many of the static analysis tools that exists quantity in Python as it is in this example of unpacking 3 items into variables on live having the constant check a are attaching an exception which is not really an exception to and by fixes as against the detected almost nothing what I want you to take at all all of these terrorists down there's also jeopardize and my wife can the internal node is that uses of the support of them actually expects is kind of cool to be understood but the that's not actually possible so the EU Phuong said the nicest tools to understand this just yeah yeah I and this is this is the future of of abundance of we would have liked employs 0 next year that have will flow-control analyzes of metadata model like understanding descriptors score on having the proper attitude access logic is not the same thing as it is in Python and I'm interested in bringing more contributors to the project and my final slide of they but the euro booktitle is the homeless employing a some of the box and so on all will find helps if you if you're going to use it and if you were going to write as many tests as possible so thank you and Misiones
this for many questions that you would then there's 1 question at higher
I actually have a common words or word of caution and the question in that the common this if you're going to stockpile and it will take some time to configure it to make it shut up a lot of things which don't interest you but it is well worth the the time to do so and and the question was how did you get into and this kind of passing pipe they used to to use an I mean at tried to write a patent plug because there isn't really much documentation for part in the history of find it's very hard to get into this topic I II was
a user of finding before and it had of a lot like really didn't detect of stuff like that didn't take on but unbalanced government like and making the items into 2 items and I had the bottom production with that stuff and I just want the clients to pay their borders so I implemented that check and something from the that thank you 1 of the questions whether
your thoughts about bilateral of i llama yeah I found
not mistaken is a rapid over over multiple check all the more fully like fighting Flickr 8 years so it just takes violent and others that I never use it and can have more questions there on here
expenditure what is more visible the other from the and also regarding the deed 1st question the most common complaint the complaint about binding to that would be the has to to many checks active so maybe awardees he there will be like some sort of configuration we assert that will show you examples then must you you want this for not the out of it in the world that
would be nice to have what I don't have time to write it myself actually so if you of to yet that is the question is does Sprint and biting there is that it is Saturday and Sunday that's it and then Chinese they can find any giant printed so they can yes of course more questions sick is
fine of and they think you're going to