Writing good error messages
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 132 | |
| Author | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/44900 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
EuroPython 2018105 / 132
2
3
7
8
10
14
15
19
22
27
29
30
31
34
35
41
44
54
55
56
58
59
61
66
74
77
78
80
81
85
87
91
93
96
98
103
104
105
109
110
111
113
115
116
118
120
121
122
123
125
127
128
129
130
131
132
00:00
Message passingError messageSoftware frameworkDialectFormal languageSpreadsheetWell-formed formulaTime domainLatent heatInteractive televisionSoftware developerStapeldateiProgrammer (hardware)Library (computing)Module (mathematics)Stack (abstract data type)Source codeComputer fileAttribute grammarLie groupSingle-precision floating-point formatKey (cryptography)TrailFile formatLogarithmProcess (computing)Euclidean vectorType theoryCodeGroup actionException handlingLetterpress printingData typeChainOpcodeForceLine (geometry)MereologySoftwareIntegrated development environmentRule of inferenceFunction (mathematics)DatabaseMathematicsObject (grammar)ExistenceOffice suiteComplex (psychology)Coding theoryPhysical systemFatou-MengeInformationRun time (program lifecycle phase)Military operationDerivation (linguistics)Level (video gaming)Event horizonGraphical user interfaceRule of inferenceSystem callLinear regressionOrder (biology)TouchscreenPoint (geometry)ResultantInheritance (object-oriented programming)BitProgrammer (hardware)Data managementCASE <Informatik>Decision theorySoftware bugTraffic reportingError messageExpert systemOnline helpLine (geometry)Uniform resource locatorBound stateCurveProper mapPoisson-KlammerGreatest elementInterpreter (computing)View (database)NumberSubsetFormal languageDynamical systemSheaf (mathematics)CompilerPhysical lawArithmetic meanClosed setDivisorInformationWave packetQuicksortStatement (computer science)Variable (mathematics)Client (computing)Level (video gaming)Exception handlingWordCartesian coordinate systemExpected valueSlide ruleLibrary (computing)Squeeze theoremMultiplication signDifferent (Kate Ryan album)Attribute grammarGroup actionVideo gameBuffer overflowSoftwareLeakWeb pageLetterpress printingTracing (software)MiniDiscConnected spaceStudent's t-testInternetworkingTranslation (relic)SpacetimeRoundness (object)1 (number)Source codeFilm editingSocial classMacro (computer science)Validity (statistics)Session Initiation ProtocolDataflowTwin primeMedical imagingState of matterWindowReal numberIntegrated development environmentMessage passingCodePhysical systemComputer fileType theoryStack (abstract data type)Functional (mathematics)Module (mathematics)MereologyLie groupEmailInterface (computing)Row (database)ChainComputer programmingSoftware frameworkReading (process)DatabaseFundamental theorem of algebraDirection (geometry)Product (business)Basis <Mathematik>Mixed realitySet (mathematics)Process (computing)Insertion lossElectronic mailing listCrash (computing)Function (mathematics)Video projectorCodeGoodness of fitSoftware developerObject (grammar)Software design patternStapeldateiProgramming languageRun-time systemOffice suiteWritingCarry (arithmetic)Identity managementArrow of timeSquare numberComputer animation
Transcript: English(auto-generated)
00:05
Good afternoon. Good afternoon, everybody. Thank you all for coming this started out because I went to a new client and I looked at their code base and all I wanted to do was rant. This is wrong. That's wrong and
00:22
I thought the best therapy for getting the rent out of my system was to turn it on its side and say I Am NOT going to say what's wrong and try to try and say what you should be doing and what is right? And so this talk was born as a result of that first I'm going to say a few words about my background and then comes the meat in the sandwich and the message there is
00:43
Good error messages are useful error messages And finally a little framework that I put together that puts all these ideas into practice Although I have to say don't have great expectations It's actually too big to be it's not big enough to be a framework even a little one
01:00
And it's not abstract enough to be a design pattern, but I didn't know what else to call it Okay, first of all Me I'm Paul Keating. I've been programming in Python I've been programming in Python for a living for well since Python 1.5 point
01:21
And I've been attending your Python for nearly all of that time. My first euro Python was in Charleroi and Belgium in 2003 that wasn't the first European Python that was the previous year, but it was my first year of Python. I support an application that has embedded programming languages Python one of the most prominent and
01:43
I write tools for programmers applications for end users and practically everything in between and it's one thing to be angry about
02:01
The bad error messages you see but it's much more important to say what is a good error message And this is what this this talk became Good error messages are useful error messages. So the first question is useful for who? And this is just another way of saying that as a writer to get through to your reader You have to know who your audience is
02:23
Look at what you're writing. Is it understandable to that audience? Now if your audience is if it's an interactive application, then your audience is probably an end-user If it's a batch program, then your error message is being written to a log So that's going to be a support person or a programmer and
02:41
If you're writing a library or an API then your support person, you know the person your audience is a programmer and you have to speak to that programmer as a programmer and pop-ups are not helpful there and sometimes you do have two audiences if you're writing a library module then
03:05
one audience is the programmer who is calling into your library module and The other audience is the end-users of The application that that programmer is writing But for the person just let's consider the fact that let's consider the situation where you're you have just one
03:26
One audience because two audience is hard. There'll be more about that later on Consider first of all is your question is your message
03:40
understandable Now this is an over the top example As far as I know it says error file not found in Punjabi It's an over-the-top example, but there's no getting away from the fact that if you show a stack trace to an end-user It means no more than that. If a stack trace leaks out to your end-user. That's a failure
04:13
Stack trace is indispensable to a programmer. It may or may not be a value to a super user depending on their background
04:21
It's gobbledygook to everybody else and Even if someone can read a stack trace if they don't have access to the source code It's still not very useful. I Don't know if any of you here answer questions on stack overflow Over and over in a comment, you have to tell you the people who ask questions on stack overflow
04:43
Please post your stack trace. I think I have to do that Two or three or four times out of ten for every question I try to answer and that is because People don't actually know how to read stack traces without learning how to do it
05:00
And these are beginners. Usually they're very very very close to being end-users and If the person who sees your stack trace doesn't understand it, then you're going to have to do some translation Now this came from a tutorial
05:21
Student called me over and said I don't understand this message. It says dict isn't callable I know a dict isn't callable who said anything about calling anything and his problem was really a syntax error He should have used square brackets and said around once and he lacked the he lacked the background to make sense of
05:44
What the error message was telling him the error message was telling him you think you have a callable in that variable But you don't you have a dict but he knew he had a dict and It wasn't any help to him to say that he that he was trying to he was trying to call it
06:03
Now fixing this is not a question of fixing the error message The interpreter and the people who write the error messages and the interpreter have to assume that the programmer has at least grasped the basics of the language and That's really what training courses are for. Oh Lord, but
06:29
Even if the interpreter can assume That the programmer has grasped the fundamentals of language if you're writing a library You should not assume that the programmer who is using your library is an expert
06:41
Most programmers use libraries on a casual basis. They can't be all be experts in every library module. There are dozens of There are dozens of things There are dozens of libraries in the standard library It doesn't in the standard library and there are thousands on PyPI. You can't know them all
07:05
Nobody can so to show you what I mean. I was using beautiful soup and I made a beginner error and the library would have been entitled to issue this error message Results that has no attribute prefix, but it didn't what it actually said was this
07:27
This not only told me what was wrong with my code. It made a shrewd guess at the concept. I had failed to grasp That shows concern for supporting a beginner For seasoned programmers fixing such an error is quick
07:42
You've got the error because you wrote food up prefix so you go to where foo was last assigned and You see it was assigned by a call to find all and you look up find all to see what it actually returns And there's your problem solved that four-step process for a seasoned programmer is so automatic. You don't even notice you're doing it
08:01
But it's hard to appreciate sometimes that for a beginner They Don't even know where to start Now when I was writing this actually went to read the code in beautiful soup that issued that error message And there it is It's a simple subclass of list. All it has is one dunder method
08:26
The only reason this subclass exists is to give that nice error message That was why it was put there. I Think that's really great because it shows real concern for the needs of a user who may well be a novice
08:42
The next question is is it explicit this code is from an end-of-day batch process It writes its message to a log that log will be read if at all the next morning Now the thing about that batch processes is they can't just raise an exception toss their toys out of the cock out of the cart
09:05
and stop Because in a batch process there is some client system downstream. This is expecting the output from this program It can't just stop in midstream. It has to Deal with the error as best it can and carry on Now the question here is why is anybody reading in this log? Well
09:24
It's because the manager of a downstream system put in a ticket or sent an email saying look I was expecting 5002 rows in my interface file And I only got 5,000 and the person who actually reported the error can probably even tell you
09:42
The numbers of the trades that he didn't get that he was expecting so your application supports person already knows that something is wrong and he already may already even know what what trade was giving the trouble and then he reads an error message that says Something went wrong with trade one two three four five
10:02
Yeah, thanks a lot guy. I knew that already If an error message is a call to action. Well, the only action this makes me want to do is put my fist through the screen Now, sorry about this in this situation traceback is your friend
10:26
Traceback allows you to put all of the information that would have been in the exception that stopped the program and then carry on So because this is being read by an application support person its tracebacks are allowed just write out the traceback and carry on
10:41
It's a very very simple thing to do and it's so easy You only have to lie add two lines to your program import traceback and print and trace bracket dot print exe and that's it so The thing there is that if you don't have a sufficiently explicit error message then the person who's reading it doesn't know what to do
11:07
The next thing is is it unambiguous? Now we have a situation where the message is going to an end-user The user is being told they're using the wrong payment type
11:20
If they don't understand why they're getting the message They'll not want an explanation and it's really helpful in that situation to know which validation rule they have broken Now in this particular case you see an exception being raised But that is actually being tracked by the application The application puts a pop-up and says you can't do that. So they don't see a stack trace all they get is the message
11:42
the trouble with this is that There are two identical texts in the message in the program if your programmer he's all he's got to go on is The text of the message and if he searches through the source guess what he's going to be pointed at two different places in the code
12:00
So the rule is don't put two identical text into error messages Because the error message there the text of the error message is performing the same role as a line number would in a stack trace At the very least put in an extra full stop on the end or put one and two or a and B or something So that when the person who's trying to follow up this problem knows what could go for it to find out what to break
12:26
What it is, he knows where to look if you point them at two different places in the code It's likely going to double the time to resolve the issue I'll Talk more about this issue of later on and the next question
12:43
I'll leave you to read this for a bit If I can get the thing to stay on this try except look at what it's doing
13:07
It calls a function It traps the error. It changes the exception type and it says the error happened in that line and Furthermore the error message that it gives you
13:21
Error occurred in call to that function tells you nothing that you couldn't have to learn from the stack trace It is doing absolutely no good whatsoever So in this situation don't point the user at this point in the code because that's not where the problem is The real thing to do here is that just call it
13:41
If you're going to raise an exception anyway, well raise the proper exception Sub substituting your own error message here just misleads the person trying to find the problem and the final question is Does it work the difficulty about exception handling is that it very often doesn't get well tested
14:06
You have it doesn't get well tested. Look at that code and let's assume
14:21
That the exception is deep and far inside that first call if that happens then the assignment to curve name doesn't take place and So the print call at the bottom will fail now this is showing
14:40
Variable reference before assignment because you mentioned the variable in the print statement But it hadn't been assigned yet there are other possibilities very often you'll see an attribute error because something wasn't initialized or You'll get formatting errors, but all told this is actually pretty good It shows you what originally went wrong the original exception and it also tells you there's something wrong with your except clause
15:05
That's terrific The only thing that bothers me about this is that it is pointing me To two different errors at two different places in the code and that takes tends to make me go a bit cross-eyed but This is in Python 3 as it happens
15:21
The code where this is taken from is actually running in Python 2 and in Python 2 there are no exception chains and so all you get is the except that all you get is the error in the exception and the only way to find out what's really wrong is to go fix your Error in your exception and then rerun it to get the real thing that went wrong in the first place
15:50
So sorry about this So you have code like there and you want to make sure that your except clause really works
16:04
Well, one thing you can do is just force it a zero a zero divide at the top of your try clause And that will force the exception and then you will know whether You have all the information in your except clause that you want to print to the screen. I Thought everybody knew this dodge, but apparently not so that's why I'm mentioning now
16:30
This little framework that I'd like to introduce to you I'm going to have to talk a bit fair about about my working environment and The challenge was to apply these principles in an easy way that could be mastered by people who are super users and not professional coded
16:49
I'm going to talk about a little about the software environment and the people who write the error messages And what sort of validation rules they are our application vendor defines hooks These are points at which the application will call your code
17:03
In GUI programming they would be called in vent handlers or callbacks in your callback You get Your function callback function gets called and you and there are three things you can do You can silently return you say I think this data is cool. You may save it in the database
17:22
The second thing you can do is fix the data you don't like and then silently return and The save will get saved in the database. And the third thing you can do is raise an exception if you raise an exception Then the application will see the exception trap the exception and give the user a pop-up to say you can't do that because of this validation error
17:43
The trick here is that if you raise an exception because there's a validation problem you'll get the pop-up But if there is an unexpected exception that you weren't expecting and you didn't trap for then the user will also get a pop-up and That's something you really really don't want it very often happens when you move a new validation rule live
18:05
that There's a regression error in it that wasn't properly tested and So you get a new exception in that has got nothing to do with the validation rule necessarily But stops the save from happening and that can bring the system to a grinding halt and you can't do that
18:21
You're in a situation where you're validating a trade and somebody presses the save button and now he can't save trade any trade you really want to avoid a situation like that because it's dreamy embarrassing if it does this is what the Application does if it's if it gets an exception It doesn't give a stack trace it just takes the exception text and puts it up on the screen
18:44
And this is something I deliberately did in order to get that pop-up to put on this screen I just named a module in a way that the validation rule didn't like and That's what that's what the result is, but it doesn't have to be a Python module. Of course
19:00
It could be anything now secondly a little bit about the people who write the error messages They may be professional coders, but they frequently are not they may be back off a super users They may be risk managers. They may be accountants Why aren't programmers writing these well because it works better if the people who know what they're doing write the code directly
19:22
Python permits that that's why it was a really really good decision to put Python in as the embedded programming language Now that would be fairly an obvious thing to do today But the vendor did in 1997 that was foresighted and also rather courageous so we get complicated corner case validation and
19:45
The end users very often don't understand why they save as being rejected They're often regarded as a bug and the validation problem may be so subtle that the developers don't understand it either So developers responding to a bug report may need to get a subject expert to go to the user to explain to them what?
20:03
They did wrong and that means you have to identify the rule that was broken and Maybe you have to run a blame on the code to find out who wrote the rule and then you go to that Person or his department and you say hey explain to me this Actually better than explaining to me. You just go to him and explain to him what he was doing wrong
20:24
So the requirements were it had to be simple cut-and-paste coding that non programmers can use quickly and accurately It must be possible to identify the rule where the problem took place even if even if there are duplicate messages and
20:45
Finally unintended exceptions mustn't bring the system to a halt Now super users very often do attach the same message to two different situations They may do it accidentally or they may even do it deliberately because they don't understand why it's a bad idea
21:04
and the solution consists of one simple class and also a way to distinguish between The exceptions that we want to propagate to a pop-up and the exceptions We want to do something else about because we don't want to bring the system to a halt now
21:20
This validation error class is really really simple its sole purpose really is to identify the exception as a validation error and not some other error and Secondly to identify the point at which the exception was raised That function underline on there over there
21:40
All it does is pretty well the same thing as the dunder line macro and see it tells you what line number the problem was And there are two backs in there because it has to go back one stage out of the function and about one stage out of the exception call to get back to the line where the actual exception was raised and
22:05
When you when you do accept raise that exception the validation exception, that is the thing you see If you look there You'll see the validate text object is the name of the validation function and the 9 7 7 is the line number With the raise statement then you can say to your programmer
22:28
Well go and look in this function It's on that line and he knows exactly where to where to find the problem and then he will very soon know who to go And talk to to find out what the error is
22:41
What's really important here is that we raise a validation error? So that the application rejects the save and suppress we suppress all the other ones you see accept exception The rule say never do that Well, we do because if there's an unintended exception that we weren't prepared to trap We cannot allow that to leak back up to the application level because it will stop the save
23:01
And you really don't want that to happen now it gets written to a lot on the screen Not all users will see it and some of them won't report it even if they do But super users will see it and you'll find out about it eventually Now this is one case where I haven't been true to the source. I
23:27
About 15 lines to determine which validation function you call when And that depends on what object it is you're trying to validate So it isn't really called my validation function now
23:44
I mentioned earlier that it can be difficult to have To handle messages that have two audiences and this little example here illustrates two things you can do Firstly have different channels for different levels of messages
24:00
So an end-user message comes in a pop-up But the stack trace gets written to a lot and the second thing you can do is raise Different kinds of exceptions for different things if someone's calling into your library, then it may be an error on the programmers part So that's one kind of exception It may be things that you expect the application programmer to anticipate and do something about
24:27
and The third kind of exception is things that neither of you can really do anything about unless the user actually gets involved suppose for example that your library needs a resource a web page and network connection enough disk space if
24:44
The users lost his internet connection or hasn't not enough disk space There's not anything very much that your application writer can do to fix that But you can make the application writers life easier by having different classes of exceptions for the different things that can happen and the different things that can go wrong
25:02
so To sum up an Error message is a call to action What do you expect the reader of the message to do with it? And the first question is is it understandable does he understand what is wrong? That's why you mustn't give stack traces to end users
25:23
Secondly is it explicit does he know enough about what was wrong to actually go and do something about it? Something went wrong with trade one two three four five Doesn't tell you enough to fix it Third thing is is it unambiguous? Does he know exactly where to look and the fourth thing is does it point in the right direction?
25:47
Do not tell the person who's trying to solve the problem that was an attribute over here attribute error over here When actually it was a key error down there Now While I've been saying this, I'm sure a lot of you have been saying to yourself. Yeah, I knew that
26:05
okay, but that's just common sense and Pretty well, all of what I've been saying in here has been just common sense But I didn't make up these examples to put them on slides and then throw stones at them These examples are real messages taken from production systems that were written by professional coders
26:27
So Although what I've been saying is mostly common sense Well, I won't be the first person to remark that common sense isn't nearly as common as one might like but The good side of that is that the answer actually is simple. That's good news writing error messages is common sense
26:46
It equates just to thinking about your users needs think about the people who are reading your messages and What you expected them to do with them and
27:00
If you do that and you do it well Then you really really can't go all that far wrong now I'm afraid my pacing here has been a little bit put out by the fact that I've been had the projector go off on Me a lot. So a lot of the sort of intervening intervening chitchat between the slides
27:24
I'm afraid was lost in the lost in the frantic clicking And so I've come to this rather as at the end of this rather earlier than I had planned or rehearsed So at this point, I think I can call for questions because I've run out of things to say
27:40
Raise hands for a question Can I go back to one of your throwaway comments? Yeah that Ten people's divided by zero that you thought everyone knew. Yeah, I don't know. Yeah. Can you go back to that?
28:04
Because Surely it raises an exception. Yes, that's all therefore. Yeah, but therefore you crash at that point Yes, but you will if you get a zero divide exception Then that's fine. But if you get another exception because what you're printing in your except clause fails
28:20
That's what that's what it's meant to test I may not understand Handling properly here. So no, the thing is that That is a bad expert that that print statement is a bad print statement because it can actually It can actually itself throw an exception if curve name isn't defined at the point where you do the print
28:44
You're going to get an exception in the except laws Okay, yes, but you don't want an exception of the except clause You don't want your you don't want you or your stack trace to be muddled up with two exceptions one That was the real problem source of the problem and one that was the book at the except in the except rules
29:02
The except clause must actually print out the problem without failing Okay, okay so the idea of putting the divide in there is you force an exception and then you can that will allow you to see Did I did my except clause tell me I had a zero divide or did my except laws tell me I had an undefined variable in my print state Okay, so it's giving you
29:22
Something in the log file to find just to give you an index. It's temporary It's only there to test the except laws to make sure that you haven't your coverage is good Okay. Okay. Thank you Any more questions? Raise hands Okay, everything seems clear. Okay, one more
29:44
Thanks very much. Just like Do you have a vague idea of like the longest message you should ever give because I've seen some huge ones very descriptive but Well explicit is better than implicit
30:02
I Can tell you that in in this particular environment if your error message is more than 512 characters instead of getting a pop-up you get a scrollable window I've never got that far But
30:21
If you know, you're going to get a pop-up You don't want the pop-up the message to extend off the edge of the screen and so on so that really more Depends more on the situation. I think though that if it's really complicated What we do here is we put line numbers in and that will that will go to a subject expert who will explain
30:42
The maybe then a URL or something that will give a give a help message would be an appropriate Way of keeping the message within reasonable bounds Okay, any more questions, okay, it appears it all was just common sense
31:07
Sorry, like thank you for the applause, but I think we still had one more question for Sure, all right, thanks In the case you mentioned of the dicta example Would you propose to change this arrow message saying you try to call a dictionary but maybe you should use the square brackets
31:28
No, I don't think we should change the error message and the reason why is because you have to You have to take account of the fact that the real error from the point of view of the of the interpreter is
31:42
that You thought there was a callable in that variable and there wasn't now To the beginner. He knows it's a dict and he expects the Interpreter to know it's a dict and he doesn't understand that it could in the meantime have been reassigned to something else Very often beginners don't understand the pitfalls of dynamic languages and the fact that he knows it's a dict
32:05
doesn't mean that the compiler knows it's a dict and And so the I don't believe that it would be possible for the compiler interpreter to actually give him a message saying you've got your syntax wrong and An actual fact you only have to explain that to a beginner once or twice and after that they don't have the problem anymore
32:25
So that's what training courses are for Okay. Thank you It all okay. Thank you so much