Practical Debugging - Tips, Tricks and Ways to think

Video in TIB AV-Portal: Practical Debugging - Tips, Tricks and Ways to think

Formal Metadata

Practical Debugging - Tips, Tricks and Ways to think
Title of Series
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license.
Release Date

Content Metadata

Subject Area
Practical Debugging - Tips, Tricks and Ways to think [EuroPython 2017 - Talk - 2017-07-14 - Arengo] [Rimini, Italy] When we write code, oftentimes things are not working as expected. We followed the tutorial and yet we got an error. We introduce a new feature but something else broke. Having to deal with bugs is inevitable. Trying to ""debug"" what happened or what caused the problem can be really frustrating and timewasting. The aim of this talk is go give ways to think & steps to take when we are faced with the process of debugging. The language for the examples is going to be Python & the tools used - from the python ecosystem. The talk will be practical, with a lot of real-world examples. The goal is to cover the following scenarios by showing different ways to approch the problem: You followed a tutorial but it's not working. What to do? You introduce new feature but things broke somewhere else in the project. What to do? You are using a popular 3rd party library but something breaks. What to do? A bug occurs and you have no idea what or who caused it. What to do? You can't fix or find the bug. What now? You want to generalize your debugging skills. How to do that? We will be talking about critical changes, binary search, problem isolation, interactive debuggers, printing, testing, greping and other interesting things
Covering space Process (computing) Term (mathematics) Software developer Gender Self-organization Student's t-test Hacker (term) Mereology Twitter
Point (geometry) Frustration Cellular automaton Weight Code Mereology Tracing (software) 2 (number) Digital photography Arithmetic mean Different (Kate Ryan album) Cuboid Energy level Pattern language Energy level Exception handling Error message Exception handling
Logical constant Greatest element Existential quantification Building Code Multiplication sign Sheaf (mathematics) Frustration Stack (abstract data type) Mereology Tracing (software) Heegaard splitting Coefficient of determination Arrow of time Software framework Endliche Modelltheorie Error message Exception handling Spectroscopy Linear regression Message passing Order (biology) Right angle Asynchronous Transfer Mode Spacetime Web page Slide rule Functional (mathematics) Real number Graph coloring Number Product (business) Wave packet Software testing Standard deviation Information Cellular automaton Poisson-Klammer Projective plane Code Basis <Mathematik> Stack (abstract data type) Line (geometry) System call Word Function (mathematics) Game theory Intercept theorem Exception handling Library (computing)
Functional (mathematics) State of matter Debugger Interactive television Letterpress printing Interactive television Disk read-and-write head Computer programming Heegaard splitting Term (mathematics) Interface (computing) Video game console Right angle Software testing Spacetime Form (programming)
Slide rule Line (geometry) Real number Cellular automaton Multiplication sign Software developer Online help Line (geometry) Computer programming Arithmetic mean Loop (music) Software testing Software testing Error message Exception handling
Context awareness Computer file Multiplication sign Sheaf (mathematics) Electronic mailing list Stress (mechanics) Line (geometry) Mereology Affine space Element (mathematics) Heegaard splitting Data management String (computer science) Single-precision floating-point format Gastropod shell Exception handling
Logical constant Computer virus Existential quantification Pixel Confidence interval Source code Archaeological field survey Open set Shape (magazine) Disk read-and-write head Tracing (software) Computer programming Dimensional analysis Neuroinformatik Inference Convex set Error message Physical system Exception handling Social class Arm Search tree Binary code Control flow Arithmetic mean Data management Metamodell Process (computing) Ring (mathematics) Drill commands Order (biology) Cycle (graph theory) Writing Spacetime Point (geometry) Slide rule Computer file Open source Control flow Online help Streaming media Student's t-test Rule of inference Code Number Goodness of fit Latent heat Term (mathematics) Operating system Software cracking Data structure Focus (optics) Weight Total S.A. Line (geometry) Directory service System call Error message Software Personal digital assistant Network topology Universe (mathematics) Video game Collision Musical ensemble Exception handling Library (computing) Pulse (signal processing) Building State of matter INTEGRAL Multiplication sign Sheaf (mathematics) Materialization (paranormal) Frustration Parameter (computer programming) Function (mathematics) Mereology Bookmark (World Wide Web) Formal language Mathematics Spherical cap Oval Matrix (mathematics) Software framework Endliche Modelltheorie Scripting language Area Curve Algorithm Linear regression Parallel port Flow separation Entire function Connected space Degree (graph theory) Proof theory Type theory Vector space MiniDisc Quantum output Software testing Right angle Clique problem Whiteboard Pole (complex analysis) Functional (mathematics) Link (knot theory) Observational study Virtual machine Web browser Graph coloring Field (computer science) 2 (number) Power (physics) Wave packet Programmschleife String (computer science) Operator (mathematics) Ideal (ethics) Software testing Task (computing) Domain name Validity (statistics) Gender Projective plane Subject indexing Voting Digital Equipment Corporation Local ring Abstraction
there was really happy to see you all here on Friday on Europe item it's really nice conference do you like the covers of our terms of them's down here it's for me is my 1st conference onset Bulgaria so I'm blown away by the organization by the venue by the city by the heat up from like about and runners have Georgie if saying all Eastern European these are hard to pronounce so just Colorado and by can congenital developer doing this every day and I'm also founder of softer company called hack self is based in Sofia Bulgaria we're developing mainly with Python and Django and on Twitter github whatever if you care 1 thing that's are highly contextual for the speakers that we have an academy in Sofia where we do Python Django courses something like 9 in total so far where we prepare people for their 1st job and were quite good this most of our students stop working as widening gender developers and I am a part of the teaching the for almost all the courses and this is highly contextual because the goal of this
talk today is to share my experiences and to be practical and helpful and I have I believed to have a lot of experience because when you're doing the course there's always an people or learning new things there's always someone raising can't insane nothing's working can take a look in go there and you look at the difference of people writing different kind of biting cold and the at some point to start seeing the problems for 2 seconds in your cell you have an error here already have an hour here and while looking at a lot of gold I started seeing some patterns repeating partners and this is actually the the point of this talk to be practical to helpful and to see different after in the budding not just tools I will mention tools in bike and of course this is your item but most of these will be partners and of course this is by no means expensive farming in that your photo we need an entire week of the book to cover everything about the blinking it's a huge topic so I want the extensive don't hate me for not mentioning something that you are passionate about alright I will talk about 3 different things that can happen while developing any colds and flu produced all 1st we stopped we will start with there is an exception everybody has seen that traces servicing exceptions so we'll start with this that will continue at the point where nothing's happening nothing's working day and things are starting to get frustrating the and we will finish weight we gotta on cancer this is maybe the worst of all of course we'll say some common things after all this because there's always come on things to say but all right let's start but this is my beautiful boxes of 4 stration level when we have an exception and we started the beginning it's usually not that frustrating can we understand why so this is level 1 of 4 stration are the running example that i'm going to use because it's easy to use example here is that you have a fire lowered song they time interval singletons trace and have a bite gold that checks if there are overlapping intervals inside it's not really it doesn't really matter what I'm going to show you this is going to be the example and we have yeah we have reception the and what have
an exception the 1st thing that we see we're the problem there is an exception and we suspect trace and stack traces are actually really really helpful this is what this is a friend of the developed world while developing the call stack traces they tell you what actually happens before that exception and not if you have ever red stack traces from top to bottom I love doing of what I love reading subtraces unless there are like 5 pages of scroll then you just scroll them in between the model but when people start calling they tend to not treat stack traces his like boredom exception dinner please help me in something's happening so stature is our friend because they contain a lot of valuable information it's like on the bottom is the exception value there whatever and on the top I don't know if you have read this but this most recent call last which means that the the 1st thing on the stack traces the 1st function that was called and the last thing comes back traces the last function that was called Oracle called executed before he that exception and I have some red arrows to point that up to point that's the information you have the mode you have a function name had the line number this is really have a because you have an exceptional line 45 and his goal to my 45 don't look for the function of the mind so this is really helpful and mn almost there and had the M had exception of spectroscope basically
said the exception if there is an exception we can have stack traces without exceptions this is also nice if you want to try something the function method called back to that exception it tells us where where we went to and what functions recalled in order to get the and so we can use this to actually the book what's happening in this is easy because it's all the information is there all right the 1st thing that people are do when are there are the building history deception and I showed you an exception that fits in a flight but you will believe me that real world exceptions fixed especially the frameworks are like 5 scrolls long and if you have a big exception it's hard to read it because you have to scroll and most of the cold the snow on your side is not your goal this on the framework side if you're developing jangle in something's wrong with the model you're going to have at least 2 scroll so far general DB models y all calls all words something's happening can you get a value error again and this is not really helpful so my 1st advice when reading an exception is to find the line that separates your call from a framework called because most of the times new the framers problem sometimes there their books but also times it's your call that actually broke and if you're the BN can read in general DB models stack traces of really helpful because each time it's the same thing and then you get in there and if you have a long trace it can be your called framework your call framework the line that separated is usually if you start from the bottom and traversed up is that the 1st piece of gold that's on your side is usually the goal the triggered exceptions so this is like a sick for reading long stack traces just find is line for example here are interceptions fuite sandbars they time this is the line that separates after this i'm going came Python library and not interested there I don't know what's happening and problem this is not the book because it's Python library does been tested that in production and so on so on so if you find the line you will find quickly the piece of code that actually broke I yeah this is the allowing is seen some risk comprehension and doing stupid things just passing the entire line is all the time from out which is not actually right so there's a governing if there is a book and you can write a test for the book you should write this for that but because otherwise you're going to face this is something guess is going to break and I call this constant regression and I've seen this in many projects is and this can make you crazy sex fixing things and breaking color thinks it's phone if it's a game but if it's a production called this funnel so if you find a book right at this for it and and that's what we're doing here is for the slides if someone was to look after this it's a standard that the writing a best and this is failing so we're fixing the cold we're rewriting the pass line function that's that does the naive approach of passing by splitting can taking the parts so good and we're running the gold again the and as you can see we started with an exception that some kind of a brackets scan of the bars now we see some kind of a date sentinel parts and if you have a trained eye you'll see the space in front of the day time and this is the problem but if you have untrained by you can start getting frustrated cell yeah when this started happening there is an exception you fix the book you think you're fixing it to run all section it's about our time it's probably the most popular to invite island going to tells for PDB and IPTV because they're really good dogs and on your bike for this but I prefer using might the basis it drops nicer Sheldon PDB
and so the thing that we can do is set a breakpoint I call this breakpoint and the truth is the correct term but you just put this on a short continue right book by PDB Vulcan medicine get it breaks builds and once during the cold
for from other states so we interactive Python Buddha which 0 interphase and stops the world and lets you look around this is the nice thing about it you ask the values of the locally and globally defined names and you usually find the but this way or you start jumping on breakpoints or start continuing these ecution of the program and helps you look closely at what's happening and is the better print function everybody starts with print and then most likely to be so when we run this program we're dropped tonight shell I'm sure all of you have seen this or is going to see this in you use it and if I has the 2nd value I'm going to see that here is a nasty little space so we have but we should not forget to out for we're out in the vast we're fixing the function by splitting by comma in a space because this is a form of were filed and the head tests are
passing not if that's passing does this mean that we have a guarantee that running the program is going to be OK yes no did you about it now yet tests are a good 1st step but we need to run in real world and Of course running the program bloom another exception In this out the French it as you can see the exceptions themselves are not really helpful it's value error below error in the cell the French and this is by far the these helpful exception for me in this out the French not that did there's something really wrong with this and I don't know why so more the Boeing time I drop a PDB in this sparse slide because I know this is the only place I using this as I learned the calls and
this is something that happens through our junior developers that start understand and use IPB usually drop by IP to be in the loop where there's opt top 100 thousand light and that's going to be built and you start hearing cont enter cons and cons and cons enter and after 10 minutes cons and cons enter here because you need that you need to get to the line that actually breaking and this is also for straight again by wasting it's good to do it once a twice or when there is no other option and you're not sure what's happening but IPTV there some nice
context manager Colts and lunch IPTV on section which if we're the functional and
run the cult is going to drop us PDB shell at the line that calls the exception saves a lot of time such a pretty nice and if I ask the value of parts I'm going to C and the string and it's 1 element array do know why this is for someone that if you have passed a lot of files in all its files tend to have a new line at the end that if the split is going to result in the single item array with SentiStrength so this is pretty nasty 1st time affinity the budget 2nd time just know attempts articles for it in the list comprehension with stress that if it's empty we don't want and it well good yeah
fixed fixed as thousands oval burns running best surpassing we're happy the and it is so the approach was we started them with an exception we fixed this exception which gave us an our reception we fix this exception would which gives an hour section and this is usually step you're going to in the steps you're going to follow when you're debugging it's either you see the problem in general in you fix it or you just freeze the small problem in order to get to the 2nd problem In order to get Interpol problem and if you have tests you can knowing the ready so this is kind of every data but if you don't try to fix everything at once you just fall although stack-trace and exceptions and in the end if you're lucky enough you're going to have a working for for at least some time hence there some exceptions dives that the whites are not so useful and helpful I like improperly configured validation their permission denied this is when the frameworks are the folks guys and girls that are developing the frameworks are nice and want to help you and in general there is like DOS of exceptions like properly configured you messed up something they will raise you an exception and actually tell you what to do you're missing the metamodel of field here on this class or you cannot do this all you need to do this which is really helpful and that's why reading stack traces is good sometimes it can be helpful sometimes it can be it can be index all the French error which at least in Python is good because if you're writing in simplest plus if you're going to get segmentation voltages have you heard to the back alright so and fix should here pictures here are easy just read it of and to summarize the 1st part stack traces are your friend if you were learned how to read them you're going to the but faster they can be huge so looking for the line that separates the Oracle from the framework called this really helpful because it will save you a lot of scrolling and often the problem is not in the framework sometimes this and if you learn how to cite PDB of like it's a brittle over your Python 2070 unified at some materials for short for IPB if you learn how to use i PDB you're going to be really good for the body like seeing stock trays jumping around so and if you hit a book right 3 I cannot emphasize more on this and the more I develop software the more I realize that tests are really important tests constant regression in I don't want know 1 to so for phone calls his regression the all right so the 1st part I have 3 more minutes there's good 2nd part we learn how to about with exceptions but now nothing's happening during your program there is no exception there's no local but sorry nothing's happening and it starts to get frustrating because you're into you don't know what it if you have an insecure happened if not the 7 income he he you don't know what's happening as the case and you need to find some source of flights this is a detective work you need to quickly find a way to break things of the you need to know what should happen and what is the expected behavior because our oftentimes in students I see how they're struggling with something is working and they don't know that it's working they expect something else to happen and this is really bad doing something that's actually working so you should have good idea what should happen thing and what is the expected behavior of the cult and of course if you are in this case got everything because there are some pretty nasty gotchas that we're going to cover that can make your hair white and if there is no exception the quick fix for this is to look for uh what's sometimes you just open web browser it says 500 this is just a number probably not the winning lottery number and all sessions in that when you launch an exception so we need to look for error walks in order to get an exception and this is actually up for me this is the more the most important thing when and the something that small multi-sample across something's art I want to break it on purpose I start typing random syntax errors I start to raising kids exceptions left to right in order to get an error to get the program to fail so I can fix it and if I can break it is good but if I cannot break it then I think I need to isolate the call to this of working for example you're passing Sunday Times in language of raising curves resemble BHP instead of passing they times in your entire big project just stay this piece of gold create a new file this like my moles given advice to everyone greater new file and paste the text in there is this a is this call that you're the budding that's actually wrong or is it something else open your file and things there evidence sprinkle with the burgers and trace every step before institutions he you need to know the value of everything can lead to know what's happening cannot to see why nothing's happening this kind of sound smart should should insights so we called isolation is really your friend because before working on our 59 by script devotees pretty easy you have 15 50 lines you can read them but if you're working in the 50 Kb general project sometimes you need to isolate the cold fix for is the court and then go back to the framework an integrated called inference so more easy to work on something small How what did of mean in integral part of everything which Jan going frameworks and there's some common gotchas that's actually happening by 2 the 1st thinkers module hiding I was the budding why you cannot import they time from the from the time for about an hour when I realized that students think his filed the tangled PY it's that we were reading the Python call trees starting I turn the opening what's happening quite at don't wrap pole on my computer and reported help Honourable his computer I cannot imported and in in the frustration I kind of change directories I mentioned for daytime polar we're very stupid is they cannot exception handling my favorite thing someone catches the exception passes and nothing can all put most trace nothing's happening just really really bad if you're writing convex managers you can hide your exception really easy I did this last of endured argument that section in check and we raise it the 2nd where is to this kind of in Sofia that alright and other common duchess's calling something by stream can making good cycle and not having the cold that's going to raise an exception if there's something wrong this is also ready for certain because there's nothing wrong you the type which is a string that you're going to a classic thing here the budding why this function is not working well in the 1st place we need to call this function it's like from alright income a metal by conscripted forgetting to out if name equals main called called domain at really really classic classical thing acts editing the wrong file or running around 5 this is also the buying something that's working you're editing something in 1 place and running the same thing copy of it on workplace there's no way for it to be working here letting it on purpose what's really well if you ask around them rubbish syntax error in the broadest you're running you're most likely the most likely here you're were rotating around 5 and of course the body something that's working this is hard to catch because you need to know what means for a program to be working or not sometimes you just know if this is OK end example on on Monday ideal Gendler salary training here on Europe I can have some orange but nice there was some like ours linger life coding without preparation which always goes as planned and as you can see I have not registered tasks here a salary it was working just a while ago I did something final test and I don't know what happened everything seems working I start the panic and when I start to panic I did this thing called binary search the voting which is and also really nice and it looks like this I commented everything that about it in uses will user you why it's working right so here's the problem i'll get to the probe I can't comment that the task and selling parts still working uncommon that's the AP that I might I broke it commented a specific impart working and commented that in part not working and from the book and I like this is a natural viruses but are I like calling this by usage of because you're not putting your pulsator toss it in the process you're just common think in checking case like the bisected go pretty much doing what by signals and the generalized version of by the shoes serves a billion years come in some parts of the call that you just added to solve the problem starts working in your happy uncommon a smaller portion of the cold that you have just commented if if this is working then you get uncommon some more if it's not working then you go to step 1 but only for the call that you have just 1 comment that and things broke and by this you kind of our narrowed down below search area of in your in your project so you know what's happening and I have to think about this we just need to comment and commentary on comment and common trunk and you have a small enough coal to look at in order to find a problem and in the end you're going to find breaking lines and I find is very very helpful because sometimes you're tired frustrated you don't want them something breaks that should not break you start doing this define a poem quickly you're happy you go is some there the beach or whatever you like to drink and a summary for nothing's happening try to get an exception that this is the most important thing try to get in there so confused Sarah isolate the cold so you know that the working on is actually working and then try to integrate it in your project because it's really bad to try to integrate something that's old working in a project that actually can break the thing that you're integrating the yet make sure you know what this to happen string part on the but working things because you will find but or if you find a book is going to really strange and try the biases the blame think if the case is suitable for binary search the booking wages comment and comments and you you find a smaller portion of gold where connections for the book what type of I had now we know how to handle exceptions realize of exceptions in famous thing that if you're cold rules from the 1st time it's really bad try to get exceptions so you know that your is wrong can you fix that and now it's right if nothing's happening there severity that you can do but for me the worst thing of all s everything is working there is no exception but to get the wrong answer this is like the most frustrating thing because it's OK but is wrong it's you and is the algorithm that actually needs the building which is hot this starts to get hot and you need to be focused and need to be thinking about and when this happens I use fall this spice fights that drew the drill down I don't want this song clique Beatty or something like this but is just 5 studies that I tend to do when I have a wrong answer and the 5 steps so like from easy leveled through I should that change my job no from question the 1st thing I always do is check test because I usually best my thing with best and if I get a wrong answer most likely might this is wrong and if you're doing gender and all of how many times it was the case that you need to refresh from DB your arm all the way is because you have a different states in your local test tract and this is like fixing color from 1st 1st check your best writer at good good tests are actually pretty hard to write what I find is that you can write called actually pretty good with good abstraction right and the best for me is even harder need to be both more structure more thinking ends not to make tests new tests yeah so take addressed that this is correct maybe you need validation maybe you've passed some wrong input maybe of past something that the outward and should not take in mind so take this is the value are the values that you're passing actually OK for you maybe or passing on values that the validation this is good this is good because you add validation and your algorithm because more proof it's going to work on more without breaking now if you don't the validation and your tests are OK then probably the next placing need to look at this at your algorithm and I'm using an algorithm is a general Our purpose cold of a piece of gold not of binary search tree or something specific that's called hour at the I'm using this in the most general sense is your algorithm correct In order to catch correctness how written you can either get a computer science degree and try to perfect the formal which is nice thing but takes off time or you can put of the burgers on each 1 of your algorithm and put your headphones on or go somewhere else where can focus and start looking good values start looking at the facts is from this that to this day do I have to do doing those values or find a place where values get wrong and this is quite hard to do and requires focus but for me this is this is working really nice diverse everywhere and a focus not and start looking about and most likely you'll see where things get wrong especially the for loops in this there's a race that's whatever and this way you can fix your work this is good but things start to get worse if your algorithm is correct maybe ops system designers of correct maybe you designed this and forgot for a whole dimension of the problem it happens quite often it's like you you do something for a time eyes in the future is suddenly you realize and it something for time as in the past you have 0 confidence you don't have the most in place for those who have nothing so here starting fresh in going to the white board like the the way to go you need to and step away from the cold think again am I doing this right and maybe call it again or change the color this is really really but when it happens it happens from time to time and probably the worst thing that can happen is if you don't understand the problem space well the owners if I if I start doing science if I get a or a wrong answer I have no idea what to do there there will be something about matrices and vectors and 1st I need to learn about matrices and vectors before I The but my matrices and vectors I need to learn about the problem space and so this is but what happens because someone else to feature called for a to so we candid hat and is also will 20 happens because it opens opportunities for new learning and this is always good and the those were the 5 steps that mirror cap DEC ask good generalization J. Khoury trace every step FIL check system design are you actually solving the problem that you need to solve and finally check your understanding of the entire problem space are you doing math and long thank for example and finally some practical tips to have been more minutes so I can go to my last 2 slides 5 minutes each all right some practical tips explain your problem problem to someone I'm sure you've heard of the term rubber did the bullying sometimes there is no need to have a rubber did you have coworkers you just you can use your caucuses rubber ducks but when someone asks me to help the 1st time I approached I all I always and as the question what's happening the those of the answers usually itself working and then I ask why it's not working and then the person starts to explain to me why some working I'm not sure I listened carefully everything that he's explaining and suddenly the person is like old yeah yeah wait a minute I I found my program to locate I will that it sounds funny but it actually helps a lot and so if the person does not find the problem from his 1st nation class again consider the given and then I start listening so I can help but the dreaded it really really really helps our explaining what you're doing to someone else because then you're going to find if you have some small problem right if it's OK for a total of ductile to rapidly whatever you find suitable something I really like if you're frustrated and have deadlines you have a problem and you're you're not in your best more than shape to fix the but now what it needs to be fixed you can practice the so called power parallel the building which is asking questions like off all open and you could be to ask a co-worker and continue working on the book is like Oh dear from or more inputs to your to your problem because sometimes you're struggling with something can someone stuck awful actually answers the question and says you're doing this entirely wrong take a look at this new year and their red or sometimes someone give could be she says I had the same problem here is my call to system in whatever you look at it and you find the problem that you also high when a co-workers thinking about it's also good more heads in the probe itself disk this is not actually nice because you're like firing from the questions will stuck or or when the could be issues and I know people who were supporting global source the don't like this very much because it's like fix my problem at my work because I'm using cure apostles Oprah open-source library and so it's important when you do borrow debugging to the output of that will prevent that contribute back to the community answer questions with back-off if you see someone asking the question that you be struggling with 4 are we just sit down for 2 minutes and him so all the people can see this make poor quest for documentation I love doing this when I struggle with something that's not really nice nicely described in the tutorial documentation I do it encounter better documentation so all the people won't waste time because I'm not wasting time when there is a good documentation and people before me have done this and the the yeah as review issues but help help other people because this is the way it it it can get so far better for everyone and waste not smaller amount of amount of times in that fixes and 1 specific differ stock over 4 read the small comments next to the answers because sometimes they containing the better answer for the answer there like really a pixel fonts comments but you're also and don't token a conference this month to share experience and to be helpful to other people evidence I guess this is it for me thank so and if you have questions we have like 10 minutes what do me 9 minutes 20 seconds for questions arise hi uh later the year I just want to add to this part of programming 1 crucial that why we don't like it in open source and why how you can make it make s love you for doing this when you sort of go back to the places where you asked about you can write this in and call duration yeah this this phosphate everything and we were up to long the I agree so if you gotta give up and you solve your problem erasure right how you solve the issue make up a request not just leave it hanging there it's the back in we don't think it's I'd like to know for example if you have read this it's something that were in the summertime the food what is your topic of the day is thought to be the yes so that's actually good question what do we do when we have this test that's working hard at times not working yeah how the type so we have such a test on a month ago now the thing that we did this run the best 100 times like right about script run the best goal to build will rather lunch no background the locks see where it's failing against audible you get it was 1st imposed so faker generating the names that were not compliant with our capitalize function and tests were failing confront them and it's and for me is like roommates are lost see where it breaks room of all the point that breaks angevines to what's happening why why would the specific value it's it's failing if you have to that failing is that this may be fighting because you have a different helps a system operating system we're income non presented ink is there some ring I spoke a survey about Doc arising by test like having the same operating system on your machine in and same thing running on Jenkins sees the need to do this is like what is happening can lead the detective work fine what's happening can't fix it it's an accident happens university the the because you to use even more visualize visualize the models are about to be or beauty in the abundance of my talk about the I I think this is like a personal taste of for me it's whatever works for you if you like visual the devoting devil of tools for this I'm not person collision becomes also kind but he consults personal based whatever works for you should be the tool that here is and I can I can coming from John a world where FIL the bonus starting like next next next how they the link thank you questions not enough something not related related to the body but most users mentioned something about the liquid Academy yet in Sofia yet and years I know you know how long he's and how long does it usually the people who learn and current in the high or it's also the thing that you're doing in Sofia called skull have Bulgaria and now it's being free branded to cracks softer economy for whatever reasons uh but we do courses that they're named programming quantum 1 weights by item which is 0 4 months long course we require some programming skills before you enter and from for like 4 months we do quite a lot of stuff it's like 3 times a week for hours scolding on place is not like just lectures and it's working isn't as if it's on him is because it's bands affected and is the way to learn for us OK think around in Europe there