Best Practices for Debugging

welcome everybody when I was 15 years old I was writing

a computer game a little bit like this 1 I wrote this honesty 64 computer in the assembly language is anyone here in the room who has tried doing that actually not a few people of great congratulations while I was writing this was my 1st figure program I wrote in assembler and the way it was like I wrote a couple of lines try to run the program usually the computer would crash and ice was 0 i switched it off wait a few seconds which is on again loaded the computer of the compiler node of my source code and the en tyre process repeated needless to say that you won't program very fast doing this because my average d'Holbach run cycle was about 10 minutes and this led to a lot of frustration at the moment my game is 24 years behind schedule only later I

learned that on the C 64

we had devices like this 1 a small cartridge that you could plot in the back of the computer it had gotten on it you push it and you are jump right in to the compiler can edit your code and continue where you stuff a few seconds ago but I was totally unaware of that and basically the aim of my talk is to prevent us from pitfalls like this 1 in Python projects now I'm not going to talk about anything fancy or totally new here because what I want to address is a common problem that I have observed happening when people are new to Python or are a little bit more advanced than Titan after

you master at the Python basics you pretty soon figure out that there are plenty of libraries for instance if I want to do data analysis than I need to find a library for data analysis like pond us if I want to do I don't know signal processing of Fourier transformation than I Google a library for that and find sigh pi for instance or if I want to do that development and I will find GenGO and so on these things on the left side they are easy to find even if I don't know them I may have a hunch but something like this must exist but the ones on the right side if I have no idea that something like inter active debuggers exists then I won't be looking for 1 the same goes for automated testing and the same goes for all the tools in the piton ecosystem that help support and maintain our code and I want to shed light on these dark spots so if you walk out of this room and and see I haven't learned anything new then consider this an additional safety checks pretty much like if people start to an airplane pilots they have a checklist that they go OK are on the we have clearance from the tower of any other plane's on the runway do we have enough fuel for an emergency landing and so on and so on and I think it's good to have something like that in a programming project to and this is why I call it best practices and my talk is split into 3 parts I would 1st like to talk about debugging then show an example of 10 about testing and then about maintenance the debugging when you

talk about debugging and tighten the 1st thing that comes into your mind might be pretty print now print it is something that I consider a bit problematic even though I do it a lot because it's like shooting holes into a building to see whether there is a fire inside every time you add a print statement to a piece of code there is a risk that when deleting the print statement if you delete 1 line too much without noticing and this is why it's worth keeping in mind that there are other of

debugging techniques for instance the the interactive debugger for instance blogging with loading model this is really an excellent way to produce diagnostic information if your program is bigger than a couple of screen pages than logging becomes superior to print after a while you need to know your introspection functions like year and type is instance and a couple of their cousins and I put code red you here as a best practice that helps with debugging a lot we had a tutorial on debugging yesterday that if you wear attending some other part of the conference the exercises out there and get help you can try them out by yourself but what about the really tough parts if something does not work and does not work if you stare at the code and go on staring at your code without any progress I'm sure that this has happened to most of you in the room at least that happens to me still after 28 years of programming and I have figured out something very nasty over the years not that I'm negative letting some additional best practice for debugging here because 9 out of 10 times the problem is inside my head rather than in the computer and fortunately there are ways to fix that so I would like to add to those best practices of debugging for very elementary things sleep

talk to read and write much most of the time if I spend more than 15 minutes on a box and I don't find the solution then I'm probably tired often it helps to talk to another person explaining what you do it to realize what a problem really is maybe I'm looking in the wrong spot and explaining helps usually sometimes my knowledge and my knowledge is limited and reading the manual of the library this time for you helps and if all of these failed writing down what the program is formulating a couple of hypotheses on or at least ideas what the problem might be could lead to progress most of the time I'm lazy and take a break and this has solved a lots of parts in the past for me testing the check I can

here is actually a bit of a provocation because this suggests something that automated testing does not do testing actually does not prove correctness of your program I know that many Python developers they laugh automated testing I like to use all the bright automated tests and random it gives a feeling of achievement and but there's a pitfall and the pitfall is that 1 in the bottom right corner if there's always a possibility of my tests passed it could be that both the code and the tests are incorrect even if I try hard to keep my tests as simple as possible so test by themselves do not prove correctness of the code but they have the potential to prove the presence of Boggs so if I see a failing test I know that somewhere something is wrong now how can we write good tests actually let's imagine we are

writing the game the the this time in Python not an assembly this group would tire some we had a figure that is pushing these blue boxes around it can only push 1 box at the time cannot push any boxes through the wall and so on until it reaches until you reach the exit here in the bottom right now how could they test for this situation look like the the 1st thing that you can think of is a fixture in

PPI test I learned today when talking to the tightest core developers that it's a good idea to place fixtures in the fire Conf test of P Y and because they get automatically imported into all your test files now fixtures are actually pretty straightforward you decorate a function with the higher test fixture decorator and then in the name of the function will be available as a variable in all your test functions if you put it there is a parameter so in that case we would get we could have a level parameter available in our test that then contains a parsed version of our example game situation actually I did something additional here I currently parameterize the test so there are 2 versions of the playing field supplied to the test 1 with the empty spaces and 1 with thoughts on the playing field so I can have 2 fixtures for more than 1 by parametrization and then we can use this in a test function i like to group by test functions into classes with high test this is fortunately a lot easier than it used to be with

the unit test I still have str unless boilerplate code so I can write a normal test function with with just an a search that is self sufficient or I can use the level parameter here note that I'm not importing level anywhere this gets ultimately automatically filled in by pi test and this is test function will generate 2 tests for me 1 uh 1 for each of the variance in the fixture what else can we do the 3rd most

important thing that I would like to emphasize about automated testing is test parameterization so we can have 1 test function right multiple EC try multiple examples like here we say by this parameterized decorator we would like to try out all the examples in the list like having a move that goes 1st up and then left and afterwords the plane figure should end up on square to x and to y it's even possible to build failing tests with this of tests that we expect to be failing so we have still right only 1 test function but with this 1 we would generate 8 tests in total so it saves a lot of code the code becomes actually very readable and if we end up in a situation where our test cold is ridiculously easy to read much easier than the code we are testing then we're on the right track so if we

execute this code by writing test this is another thing I learned this morning on that the adult inside pipe test has been deprecated we can use pipe has developed about in the middle so we see that all the tests

executed this test actually uses a window and we see 34 past tests in out of the entire test set that not only the ones that I showed on the slide there's a couple of more than a few more running in the back plus 2 that are expected to fail because I mark them with the x fail decorator the

now how much testing code should you right In my opinion this depends quite a lot on the size of your project if your project is small and prints and obvious result anyway then maybe a manual test is enough unless you want this to be continuously integrated sometimes I still write test code in the main block of a small Python module I can write make automated tightest functions all of this quite easy later if the project grows at some fixtures as the thing grows further and if the program keeps growing and growing then at some point in might be helpful to switch on testing tools like Jenkins Travis for continuous integration are talks for testing multiple Python conversions when I speak about size of the project this can mean different things that could be absolute volume in lines of code but it could also be the expected lifetime of the project so if I expect code to be maintained for 2 weeks and then I would not worry too much about testing if I'm writing a throwaway program if the program has a high dependability so if it needs to be extra safe then doing more tests and reviews and things like that is also a good idea In the final part

of my talk I would like to elaborate on maintenance python has a fairly sophisticated ecosystem of maintenance tools and and they serve as the single purpose of keeping your code in a good shape like pet 8 being of layer of paint on your program like many of you probably have heard in the talk of unknown a while ago you to making beautiful code is a virtue and Python has a nice tool support to help you with that

instead of picking a few must have 2 worlds I tried to throw in some that keep reoccurring with it did being in the middle of being not a coincidence so if there's anyone not using did or version control at the moment moment it's a good this conference is a good the starting point to learn that because you won't be getting anywhere without version control but others are there are many other tools as well as some of them are interchangeable like pi scaffold recently are In my personal ranking surpassed by cookie cutter the there's things for documentation virtual and if and pi and for our managing your Python installations and libraries Thailand and pi flakes for and other for watching your coding style and so on and so on now what can you do to keep an overview of all of these tools now I would like to just mention 2 possibilities here 1 of them is on modeling of what is going to give a talk this Friday and where she's going to preserve give an overview of all the different configuration files that are that you can find in a well-maintained piton project so this is on Friday afternoon if this is still too far away in the future I recommend you to take a look at the

Kuala want you to some of you may have noticed that there's a flyer in the conference bag I visited the quot up yesterday and the developers actually gave me a quick introduction and I was able to run quality within our within 5 or a 5 minutes or 10 minutes so quality is a framework that basically hosts many leading to was that means codes of tools like payment or my pi or other tools that analyze the quality of your code not just for piton and I thought how also is that I can put I can check not only my Python 5 I can answer check my HTML templates then JavaScript code and whatever has have been accumulating in my project and get everything from 1 tool that tells me how good it is so how does that work Kuala brings its own configuration files that are mainly contain list of all the tools that you want to switch on for a given type of 5 for some reason acquire like all of these different Lintas bears so you have a list of pairs in this configuration file which is in my opinion it's kind of kind of cute and I need to say thanks to the developers for a for doing that I really appreciated and I want you can put in some additional parameters what you can do when you have disqualified set

up is that you simply write Kuala minus minus the CIA and crawler starts scanning your entire code base recursively analyzes all the

pipe and 5 and comes up with a huge

list of on the number of

comments and suggestions for opera potential improvements this ranges from sorting imports to spy checks or even our using a static type-checker on Titan so please feel encouraged to try climb out I'm not as brave to call this a best practice yet because it's kind of a new a new tool

but on I hope to make that says that statement next year now this is an overview of testing debugging and

maintenance practices that I wanted to give here the and there's 1 more thing that I would like to do I got into this topic and liked it so much that I wrote a book about it and I got 3 copies with

me that I'm happy to give away so after the Q & a session I'm holding a bag open here so if you put a piece of paper with your name inside I will draw 3 lucky winners at the end of the session and are you can read more about this practical pressed is best practices there and some of the content like the debugging tutorial you will find on my give did have profile that's the for the for my talk thank you very much for your attention and

fj August we have also some devil questions any questions the in the right I want the NIC didn't really mention was talking about and virtualenv and and of the mismatch of packages with preposterous posters impediments of you compare themselves so could you repeat the question please as is since since that in uh about and flight show that's a good practice to use but and so you have the correct vision packages so is so what's the best practice for long before getting there the versions of packages right so user requirements yes that's the lenses file at the we have requirements that we we have requirements

text on the talk in the cloud so yes this is the number 1 way to go off foreign getting there are getting the right versions because it can deal with it condo should be able to deal with it and our faces some trouble the Chairman

questions structure so so 1 thing whilst have other requirements is there so I was of where the where the version number is not fine-grained enough so we have used it commits for the exact than in their actual check out a book there let's to if in your requirements file the don't then exact um virgins it in the future leg debts in From the deprecated it it ties in CR maintenance issue is that future users will be able to use your phlogistic keypoint there that if you're not really and put it out the foreign talks earlier book it's so interesting 1 this this is 1 of the this is 1 of the tougher problem so on IPA I'd be a bit careful with armor with leaving are leaving a certain version number in your dependency like for ever I wouldn't be that on I'm rather a rather checkered far from time to time check if you're different once if a if a new word neuro 1 comes out at because I would not feel confortable with leaving the version number of um empty all the time especially not if you are planning to automatically deploy the code if you're running the program manually then are OK I'll I'd I'd feel confortable with that but not if you have any automated pipeline running in the back of is your book also available as an evil I yes but I'm unfortunately you have to pay for it to happen but it is yes for enough came on questions then so the debt so it