Identifying Bugs Before Runtime With Jedi


Formal Metadata

Identifying Bugs Before Runtime With Jedi
Title of Series
Part Number
Number of Parts
Halter, Dave
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Release Date
Production Place

Content Metadata

Subject Area
Dave Halter - Identifying Bugs Before Runtime With Jedi Finding bugs before runtime has been an incredibly tedious task in Python. Jedi is an autocompletion library with interesting capabilities: It understands a lot of the dynamic features of Python. I will show you how we can use the force of (the) Jedi to identify bugs in your Python code. It's not just another pylint. It's way better. ----- Jedi is an autocompletion library for Python that has gained quite a following over the last year. There are a couple of plugins for the most popular editors (VIM, Sublime, Emacs, etc.) and mainstream IDEs like Spyder are switching to Jedi. Jedi basically tries to redefine the boundaries of autocompletion in dynamic languages. Most people still think that there's no hope for decent autocompletion in Python. This talk will try to argue the opposite, that decent autocompletion is very close. While the first part will be about Jedi, the second part of this talk will discuss the future of dynamic analysis. Dynamic Analysis is what I call the parts that static analysis doesn't cover. The hope is to generate a kind of "compiler" that doesn't execute code but reports additional bugs in your code (AttributeErrors and the like). I still have to work out the details of the presentation. I also have to add that Jedi I'm currently working full-time on Jedi and that there's going to be some major improvements until the conference. Autocompletion and static/dynamic analysis as well as refactoring are hugely important tools for a dynamic language IMHO, because they can improve the only big disadvantage compared to static languages: Finding bugs before running your tool.
EuroPython Conference
EP 2014
EuroPython 2014
Condition number
Computer animation
Range (statistics)
Torus Point (geometry) Trail Musical ensemble Open source Software developer Mathematical singularity
Focus (optics) Word Computer science
Video game Code Projective plane Bit Complete metric space Subtraction Pie chart
Performance appraisal Goodness of fit Network topology Open source Code Multiplication sign Basis (linear algebra) Complete metric space Data type
Computer animation String (computer science) Complete metric space Object (grammar) Wave packet
Apache Forrest Video game Iteration Scripting language Line (geometry) Complete metric space Functional (mathematics) Position operator
Open source Multiplication sign Projective plane Species Line (geometry) Complete metric space Functional (mathematics) Social class
Subject indexing Video game Constraint (mathematics) Computer animation Natural number Multiplication sign Electronic mailing list Thermodynamics Complete metric space Functional (mathematics) Data type
Point (geometry) Electric generator Mathematical analysis Electronic mailing list Bit Complete metric space Performance appraisal Fluid statics Word Statement (computer science) Quicksort Recursion Data type Local ring Exception handling
Pairwise comparison Fluid statics Word Computer animation Multiplication sign Mathematical analysis Resultant
Computer animation Compiler
Computer animation
Slide rule Focus (optics) Personal digital assistant Multiplication sign Parameter (computer programming) Functional (mathematics) Error message Proper map Traffic reporting Inference Typprüfung Attribute grammar
Area Natural number Code Mixed reality Multiplication sign Cuboid Software testing Line (geometry) Error message Social class Library (computing) 2 (number)
Presentation of a group Closed set Error message Position operator Social class Attribute grammar
Sign (mathematics) Touchscreen Computer animation Quicksort Time domain Error message 2 (number)
Computer animation
Computer animation Multiplication sign Mathematical analysis Energy level Complete metric space Recursion Limit (category theory) Recurrence relation
Mathematics Computer animation Functional (mathematics)
Computer animation String (computer science) Parameter (computer programming) Function (mathematics) Subtraction
Suite (music) Web crawler Constraint (mathematics) INTEGRAL Mathematical analysis Hecke operator Function (mathematics) Computer animation Interpreter (computing) output Software testing Data type Library (computing)
Exterior algebra Multiplication sign Mathematical analysis
Mathematics Open source Multiplication sign Universe (mathematics) Software testing Client (computing) Library (computing)
Intelligent Network Lecture/Conference Text editor Plug-in (computing)
Goodness of fit Multiplication sign Data management
welcomed Dave That's the stage for his talk on using I want you you jet these conditions and his talk how to use effectively here on the positive fj
will for yeah that stupid thing
will go away while there's a lot of about 5 to 3 and we're going to try 5 major questions again so as you are you going to leave relatively quiet again what people are asking candidate answers alright and so it's really greater role here it's great to see so many people
and I'm really thankful for all of the just quick range there raise hands please
and who doesn't know generally all right so who does alright like this like 10 person you something that but whatever and of sorry this thing is going to look in the book anyway and so on I'm I'm days and this is like my CD and I'm not getting is
really this is really my CD like I was sentenced to companies and so and this is done on track points and uh it kind of shows what I what I love doing and it's it's like being creative all over and I like the music him like played a torus singing and so on and so on there's like 2 things that are not on their and that's basically and I started West Coast swing dancing and I went to Afghanistan in
2013 because it kind of stops there so and what would happen from 2013 until now is desired in open source development which is Chad I basically and I did and I went to Afghanistan and this again is the kind of like this and so
this is a fighting Taliban or something and I went there and with like an education focus I teach
Africans titles of consists in in computer science and that kind of stuff so but I want to kind of get away from Star Wars and Star Wars and 2 words chat
ID the deal to completion and so on I hope everyone understands what all the completion is here right so it's it's like this thing that pops up when you write code and so chat ideas this is all the completion for Python and so it's it's kind of a new approach like there has been road before and like there's a pie chart the bit but it's kind of different and it's it's it's a project that has grown quite a lot in the last few and months even if you was like to have a thousand to other stars and get up and it's it's well tested and it has only been around for 2 years ready and so we we kind of have an emphasis
on Python 3 and that's because at least I think is the future of life on so it works in Python 2 but still I and we we we do a lot of coding in Python read and so
that's what it's like the basis of JET I like what would which we important is to have a good time deduction and that not as easy as you might think because python and it's really dynamic and it's a different approach and quite shockingly high charm has this kind of approach for a date or at least I think they have an approach because their code is not open source so sorry is it is open source now like the whole completion things alright so that has changed and so anyway what I what I thought it is is is like they they kind of built this and Type tree wherein they just know everywhere at the pipes and like they do it this way and I do it on a lazy about evaluation approach where I'm and like just moving from place to place and I tried to keep all the values as well to have as as much information as possible and I can show it to show you how
I did that and that and this is a small example is
like this string objects down there and goes like which I would not work if you have you all right so you have this completion here you have capitalized center and so on and that kind of needs this training object if you want me to know what the string of is so it it goes back and looks where where history not defines it's likely that that that get extra thing so it is again after on the string and and executed so basically what that actually does it is the major so it goes up there and kind of puts me maiden in a market together and then worked so that's pretty cool because it's that's something you would expect out of the not commissioned to do why like the the so
but and generalize then
there's like Chennai Forrester which is fault-tolerant life it it it basically has to be because all the completion doesn't really work like wrote the all the all the all the all the completion thing they kind of use an AST just lines but that's been the have to kind of iterate that and if if the fine is really bucket they are not able to to give you any hints and so basically with each other I can just might give
it anything like you can give it a PHP script and as long as there's like 1 python function there's little work and so the wait we were important is the false positive that's kind of a
strange thing and that's splits the final into functions and classes
and the idea behind it is that while you're doing what to completion all the time and you're only changing like 1 line so if you change only 1 line you should be reparsing that whole trait it just it's just a big waste of time and so on 1 which had I does it is it's it's basically is this basically to get rid of that and just just parts that 1 function again and sometimes that body like there's that that's why people sometimes think jeopardize not really good or it's as fucking or whatever so if you find that with the way to reach the reproducing species like we would be really happy and and there's like 1 funny thing about this is like if you have Japanese guys as contributors they will they will fix them like the most also people to to have around that as an open source project because they always they always kind of go for performance and and fix things and so that has happened a few times a year and generalizable and then that the
dynamic and nature of life on is like we understand that by by doing well by by by basically understand get captured stuff and so on but that doesn't really help a lot because you're not using that that the biggest issue actually of of of the dynamic stuff Python from this is list things that because he pushed into these kind of containers and you expect something out of it and it's hard to understand it because the press from somewhere and kind of skills which she has to handle it and which is as far as I know again and not handled by for applied at road and so on and so this is an
example like features kind of Chennai understands it even if it's in the form of poor like in a different function and so you have an next that is defined as an array you you pushed times and you kind of in the end you end up with completion proposed In constraints like you could now say well that the index is not right like it should just the union well we might work on that but I don't think it's very important because typically you only have 1 type in there but you need to know that 1 time and so what we end up with is this tonight and I really
understand a lot about tri-phones so this is this is kind of the introduction of a 2 words static analysis because like if you don't understand all those things like decorators generators list comprehensions and so on and so on like if you fail to understand just 1 bit 1 statement standing failed to understand the whole kind of things because jet recursive and unlike goes from 1 place to the next so if you don't understand the list comprehension you cannot know the typing and an generating understand descriptors like that others like almost nobody understands those because they're pretty complicated and there's an there's like exceptions that we just from the beginning on several we were not we're not going to do that next metaclasses set at and I think if few others like locals and globals modifications and ICT modifications and like that crazy stuff and which is just too hard to understand and so
this led to the idea of of creating a lecture or static analysis and so this this came up when I was writing a proposal for from February or something and so I was like 0 well to completion is not going to be that interesting to talk about like it's just all the completion everybody wants it nobody really cares how it's done and so and I cannot fault well static analysis could be really nice to do with each other because we understand the points and we can just do lazy and evaluation on everything so basically evaluate everything so you could put push sort and with you push the
lazy whatever and the 1st I thought it was slow but it turns out if not that slow it's not as fast as timing toward pi flakes but it's it's OK like I'll show you the results afterward and it yields very good results just like I I worked on it for a month or so not not a lot of time like you i in comparison to what I work on chat iron general it's it's been like word in 2 years so it's just a month and then it was really fun because there was a talk like the question that the the Haskell keynote so
and this was found for because he just said static analysis is primitive and I'm sitting in there and having a talk about static analysis and how how we will do that in the future in Christ and so on like he says well that's not going to do it and this
like here and piling compiler like the Python compiler and so on and their face or whatever and and so
Chennai world it doesn't but but they but will be disease
so anyway and it's but we have to be fair there's like and it it wasn't really working on Monday because I have I work on the
inference I I worked most of all on attribute error and had to be at errors and type errors from function where you would call like the function with too few arguments or whatever but so still like I had acted together in 2 days and as children year and like so here's question it didn't take a lot of time and so we will work we will definitely work on that like to to have a proper and proper type errors for all that kind of cases but and
and here here's what I worked well worked on like this this is the slide that I wanted to start with originally but then like I a kind of changed it and so this is what what which had I really does well it's it's like 0 we have stirred upper and well we forget P or something and any kind of reports it to us the same for like a function that is called with was like an argument it with like 3 arguments instead of to like that kind of stuff uh is is working really well now and that's that was my focus in that kind of in a month before you reply from which is which is pretty cool because and this is
not like this is a very simple example it it works on like pretty crazy stuff I I cannot really showed examples because it's just like very long Python code and like but you can just take the example from before with get natural like play around with it and it will still find some and some on so we would also like to
speak giving example I like I thought well I don't really have time to test to test against all the libraries or whatever but I just like a weak goal or so preparing the talk I thought well I could just take the most well known library that probably doesn't have any thoughts and and so I took request which is which is like 4 thousand lines like you were late for reuse not included here and unlike scanning it takes 15 seconds the yield 51 errors and like 20 errors of those spot areas that you want like its compatibility code like imports from I don't know like Python to imports and because I can't in Python and so that kind of stuff and so that's fine if you see those errors and there's like a few boxes and pretty much everything else is related to mixing classes classes no mixing classes is something that I didn't even think of about before but mixing classes in
and chat I is gonna be kind of hard to understand because you you have this mixing class and the attribute is defined in a class that is different than a than the 1 it's cold it's Houston and so that's kind of special but we might work around and I don't know but it's pretty cool like you get 50 errors what were not let's let's say you get 22 errors that's not a lot of false positive and to show you
that still get errors if there's a a mistake I kind of prepared this no no prepared this but the so if my presentation but close yeah so I don't see
on my screen so I have to find a so we can just like 1st let's let's just
scan it sort of very small I forgot to change the sign and so scanning it takes like a few seconds there's not much and doesn't yield an error we change something like this here kind see time-domain coat remove something so we
have an inherent right like eat it really it so it's not just a joke is that it works and now we here
yeah you can have that it's the worst thing ever like thank it's even worse than OMP applying I think because chat to so-called like the hold the idea of of and Running Chad ion channel I this is basically that you run recursions on recursions and that's very complicated to understand but it works like that I'd ask that kind of an the limitation in recurrent recursions where like so if you're like 10 levels deep just ignored arrest and light because you cannot see you cannot fall everything there's just no time for that in all the completion we might change that for static analysis of the and so let's just do 1 the but
yeah it's just to another 1 but what didn't change I don't know let's just think change this function I hope works
yeah like we we we get some errors and like you can also
like now you would also get pi flakes also yields the worst down there so let's do it and let's removed this parameter and so apart from that username thing that is not there you will not see an error but
let's see the output output here so it will tell you more because it will tell you well you have a user name is not defined like high flights the and but they can also tells you kind of in a in a typical price from way 0 basic all string takes exactly 1 argument but toward different things and that's pretty nice I would like to that were undone now with that as so those and the of the 1 of finish here and so forth
what are we gonna do in the future last working we brought is for the win so and we we will improve
static analysis that's 1 of our goals now offered like the lecture and the next if you uh weeks at least and then there's like to go off adoption and mainstream ideas which is already on going like spider is is doing it it with a guy cost Sylvester and there's as part of his interested and so 1 of the things we're going to do it if we have enough manpower is refactoring and and then there's um 1 thing I'm really excited about it and it's the integration of the Boston crafts by some type annotator which answer kind of a way to understand not and and built-in types way better he kind of runs and that a test suite and heck supplied from interpreter to kind of make every input output that type and name so you kind of know as what goes in what comes out in built-in libraries so that idiot you apply constraints regarding going to do that were 3 people so if if you'd like to join your very welcome and and there is like
my small future and I think I've been working on it for quite a long time now and it starts to get financially unsustainable because it's just my savings there vanishing and but if you're a company like you you want to have really good static analysis well maybe you want to hire me or something like before that and yeah alternatives are either kick-start 4 or more of the Japanese guys some of the
and here so the here's my contacts and there's
2 things I want to say the 1st is like test well and often there is and talks piconets coverage client like for open source stuff there's also Travis and which urges all some tools and if you have like like library that that has a been test coverage that's just really cool and and create libraries if they don't exist
so that's that's really the strength of the of the Python ecosystem if you ever thought and something should be better changes and so yeah we might have time for a question but probably not we have the be universes the
Thank you got the wall questions think the 0 I the the
question that think yeah so he said he asked what how well it wouldn't work with women and then and like I wrote in a plugin for Vin cultured either which is working pretty well and there's another plugin called and you completely which is used using chat and so they're are working really well there's like we plug ins for you next there's to plug ins for self-aligned there's plug ins for a lot of editors and so yeah I guess that's it so go to Have
a good time manager for economy and remember Afghanistan is beautiful and affect thank you Dave


  572 ms - page object


AV-Portal 3.10.1 (444c3c2f7be8b8a4b766f225e37189cd309f0d7f)