Value of unit test
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Alternative Title |
| |
| Title of Series | ||
| Number of Parts | 150 | |
| Author | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/51457 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
00:00
Software maintenanceSoftware testingHydraulic jumpPoint (geometry)Local ringDisintegrationSoftware testingTask (computing)Product (business)INTEGRALGoodness of fitMereologyGraph coloringIntegrated development environmentMultiplication signWeb pageMassSoftware frameworkInheritance (object-oriented programming)Right angleRange (statistics)Object (grammar)Different (Kate Ryan album)QuicksortLocal ringWeightConstructor (object-oriented programming)MathematicsProgrammer (hardware)Software bugRoundness (object)Functional (mathematics)Projective planeExecution unitInterface (computing)Dependent and independent variablesInformationNumberBridging (networking)Point (geometry)Data managementCondition numberWritingIterationPlanningSoftware developerSlide ruleBit rateTelecommunicationPhysical systemOvalDescriptive statisticsSystem callProxy serverArithmetic meanConnectivity (graph theory)LogicDivisorEmailBeat (acoustics)Statement (computer science)Chemical equationCausalityFormal languageChainTerm (mathematics)DatabaseMilitary baseForestFocus (optics)Line (geometry)Shared memoryMonster groupInterface (chemistry)Computer programmingGame theoryMultiplicationSocial classSoftware maintenanceData miningHypermediaTotal S.A.WebsiteSpacetimePrice indexTime zoneCategory of beingDegree (graph theory)Decision theoryCellular automatonMetric systemFreewareEntropie <Informationstheorie>Stress (mechanics)Message passing1 (number)Form (programming)State of matterFormal verificationLevel (video gaming)Context awarenessSoftwareService (economics)Online helpDrop (liquid)Process (computing)GodData conversionCodeMixed realityInsertion lossTorusLink (knot theory)Electric generatorComplex (psychology)Thread (computing)Perturbation theoryPeer-to-peerWordView (database)HoaxUnit testingPattern languageBlogDistanceSuite (music)Latent heatLoop (music)Computer fontBitBasis <Mathematik>AutomationEndliche ModelltheorieCartesian coordinate systemTest-driven developmentUniversal product codeJava appletSurfaceModulare ProgrammierungDigitizingVisualization (computer graphics)OrthogonalityComputer animation
Transcript: English(auto-generated)
00:02
Hi. Before we start, I wanted to ask a question. Who is writing unit tests? Super. Many people write unit tests. People using C sharp. Many people use C sharp. Java. C++. I guess two
00:21
people. Right. So, did you ever have a problem maintaining your tests and trying to read it and not understand it? Have you ever spent hours in understanding the tests? Did anyone have this problem? A few people. So we'll talk about that. About how we can
00:43
write the test that is easy to maintain. That it's easy to work with later on. My name is Kaisa Mustafina. I'm a developer. I developed software for about 17 years in different companies and roles and countries. And I worked for six years in Schlumberger
01:04
Information Solutions Norway Technology Centre, which is part of bigger Schlumberger company that produces oil services. And at NTC we produce software solutions for oil industry and the flagship product that we produce here in Norway is called Petrel. And Petrel
01:22
is a software that helps people find oil. That is a tool for surface modelling. And if you're an oil industry you can do with Petrel basically whatever you want. And the coolest feature of Petrel is like 3D visualisation. That's why I have this cool picture on that
01:41
slide. And that is considered to be one of the largest software packages in the world. It has hundreds, developed by hundreds of developers and has millions lines of code. So this is a successful product that is being sold on all the continents in Europe,
02:01
South America, North America, Russia, Asia and all the continents. And the key component of that success is automated testing. We have started developing Petrel in 1995 here in Norway when TDD wasn't really around and we had to adapt to it. We had to start
02:20
using it at some point and we have gone all the long way from total hate to total love. So I will try to connect the good practices of writing a unit test to our experiences and to what worked and what didn't work for us. So TDD and unit testing,
02:40
it's became a religious issue. Everybody is talking about it. Some people believe it, everybody believes it, but some people are deep fanatic, some people follow it to some degree and there are some people who even think we don't need it. There are still some people like that. So but generally as a programming community we have adapted to unit testing. We are all
03:03
committed to spend some time to write tests for our software and that's generally happened because people tried it and they saw the value of it and they actually realized that there is a benefit for us. We started spending less time on bug fixing, less time on testing
03:21
and bug fixing has now been our favorite activity. So we recognize the value of it and we really like that and that is a value that we all share these days. Like 10 years ago it would be normal to talk to someone and realize, okay, they don't have an automated test at all. They don't have an automated test suite.
03:42
But today it's nearly impossible to have that kind of conversation and I think everybody raised their hands when I asked that people write the test. Yeah, I had this kind of talks recently with some peers who don't have unit tests but that's actually immediately smells that they spend a lot of time bug fixing. They spend a lot of time testing probably.
04:03
It's probably not that exciting of what they're doing on everyday basis. And we have done a great job in using tests as our helpers and this talk will be very much about there is more than we can do about that. Better quality with a less effort.
04:25
And let's outline our problem statement here. So we're using automated tests at our safety net and it was well spotted by Martin Fowler some years back who invented that acronym. So it's so easy to refactor. It's so easy to change the code that we have a test for.
04:43
So we can easily detect the problem and that is all very good. But then we write more code and more tests and more code and more tests and there are several releases we have gone through with these tests and sometimes they start failing. We have to go back to them. We have to understand what is happening and that brings us to maintenance problems. If we cannot easily
05:07
detect what is happening with the test. If readability is bad then we spend a lot of time on that and I spoke to some colleagues some years back and they just started scrumming their team and that iterative development actually made it very visible what they are doing
05:24
immediately. And they were complaining like we spend an iteration just fixing the unit test and next iteration we spend just fixing the unit test. We're not doing anything good. We're just fixing the unit test and that is situation that we want to avoid. We don't want our safety net to bring us to this. We don't want that. We want to spend less time on maintaining our
05:45
test base. We want to make it easy and at the end we are not paid to do tests. If we could ship the software without the testing suit that's fine and no one would ask us that Do you have automated test suite? That would be fine but this is our own internal tool
06:05
to do that. So let's talk about what makes this hard to maintain. What we can do to make things easy to maintain. And here it brings us to the question what is a good test?
06:22
And looking at the good test, what we can think about? Can we think about how it looks? And can we think about what it tests which is coverage? Can we think about coverage? And we talk about coverage all the time which is our main metric of the quality. But generally should
06:40
we think about coverage or should we think about other things? Should we think about when we run our tests? Do we run them all in the same time or should we probably should run some integration tests afterwards? Or where we test? Which part of code we choose to test? Or we should blindly follow the coverage and gain for the coverage? We'll talk about in a minute. And the first thing
07:09
I wanted to talk about is readability. We'll talk about how it tests. Anyone had that situation? Some people had, right? Why we care about readability at home? That is not a production
07:27
code. And the thing is that we want our tests to fail at some time in future. Anyone had a situation when someone told okay I had wrote that I had written that test and that was so good because it saved me so much time afterwards. Yeah unfortunately we cannot
07:43
predict it but we actually want them to fail sometime in future. And when it fails someone need to go and understand why it fails. What happened actually? And we want to minimize that effort because at this point we are doing something else. We are developing software that
08:00
is not related to maybe what this test is doing and that part of software that we have written. So when we do that and if you have to spend a lot of time actually understanding and maintaining the test, that's a pain for us. We don't want that context switch. We want to
08:21
continue to do it. We want this to be a light activity to go understand the test. Ah that's because of that and then fix it and then go on. And I really like the Roy O'Sheriff phrase that he wrote in his book who said that readability is the connecting thread between the one who
08:42
wrote the test and the poor soul who will read it a few months later. Tests are stories that we tell the next generation of programmers in the project. And that is so true. We want to tell that story and we want to tell a good story. So let's look at the pattern that would actually help us to achieve readability. How would we get
09:06
there? To write a good readable test. And let's look at the pattern that is called arrange act assert. Anyone is using that pattern when writing their tests? Yeah a few people.
09:20
And that has been blocked I think in 2003. So I have a link below that. And it's very simple it's just it couldn't be simpler but it's very actually explains what data being set up. What data being executed. What concerns we are interested in this test and what kind of verification we are. So when we actually
09:45
organize our test with arrange act assert that is becoming so easy to achieve readability and to help people those poor souls who will read it a few months later.
10:02
So before we go and look at each step of arrange act assert let's look at the example. That is an example that is trying to test something and I called it messy test example. Is it easy to understand what it is doing? Can we easily identify what is actually happening
10:25
in this test? It's probably not that easy. So let's look where arrange and act assert in this test. Let's look at that. So if you try to understand and read it carefully and it
10:41
took me some time to actually color it and understand it. So you're setting up something and then you're doing some mix of assertion and execution and then you're setting up something else then you're doing something more and you're doing all of that stuff. And I think if we look at that test we see several arrange and sometimes arrange act assert is
11:03
even mixed up all three together. So someone who will read that test will be actually confused what is this setting up? What is happening? What's going to be? And he's going to spend a lot of time to understand it and he doesn't want that. So maybe one day he'll say okay I'll look at that later on. I'll put this test to hold. I'll mute it until sometime and that actually
11:27
never works because there is a tendency this test stays there forever. So that is a violation example so what we can do to make it better. Let's look at the arrange part. Arrange part is responsible for data setup.
11:45
And the important thing here that is all consolidated at the beginning of the test. There is nothing that you can set up something then execute something then set up something more execute something and execute something. We are setting this up at the beginning of the test and to do that there are several actually partners that I used to
12:06
clarify of what you are setting up and two of them for example are object, mother and fluent interface. Those are patterns that we can use and I'll go into example in a minute just to clarify what is being set up. There are some links on the mountain folder blog
12:26
what you need about them more. Let's look at the example again. So this is another example and that's quite simple. So let's look at the test on the left. What it is actually doing?
12:43
That is trying to set up some points. What is this for? That is trying to create some plane so probably point three is a class that is a point in a three-dimensional coordinate system and then plane three it's some plane in the space but
13:01
what is this one zero zero zero one zero zero one one zero. We have to kind of think about why those digits are here and stuff like that and that is an arranged part. That is consolidated at the beginning but we can't actually read what is happening here very well and just at a glance. So let's look at what we can do. What we are actually doing?
13:25
I will color it. We are creating xy plane. We are creating xy plane here by those numbers zero zero zero one zero zero one one one. That is just xy plane the plane perpendicular orthogonal to the z axis. This plane is well known and people understand what it is.
13:47
So we can just create a method a helper method that would just create xy plane and just name it that we are create xy plane and if you go to right hand side the one who is reading he's actually understanding this is xy plane created okay I understand that and what is the point
14:03
where is this point we create the point at the axis so we understand there is a point exactly on the axis that we are creating and those are matters those are data that we are going to test. So here is the helper methods and we could call them object mother so that those are methods that
14:25
we consolidated to actually make a better readability for this test with simple example and we emphasize what data are being set up and we could actually abstract from the constructor details as well and we can reuse it in several other classes and we could create objects that
14:41
are we are familiar with with in for example we can create diagonal plane and we could create horizontal plane and we could work with those those objects in our test framework in our communication framework so we understand what kind of dummy objects we are creating to test and those helper methods actually serve us as a mother for those objects that there are many
15:10
methods to create xy plane but we only use those points but we don't want to know what what kind of points we use just to we want to abstract on those details that makes our tests
15:22
readable and our setup readable and fluent interface I don't have example but it's very similar if you have a long constructor you can actually set up special methods like with I have a girl with a balloon or go without a balloon and then you can build that that
15:43
particular object and return it so you create sort of a specific number of dummy objects that you're working with in your test framework and that is very handy and this is very good for readability so that that was arranged and there are more patterns to for a range that
16:01
there is this book x unit test patterns that Geraldo Mercedes has written and this is very good this explains all it in details it's 900 pages if you want to read it let's look act what is actually act it is code execution so what we can say about it we just execute the code
16:21
but there are some things that we can think about as the best practices we could make it visible yeah some people insert asserts in act part that shouldn't be there we should assume everything works here we are only executing what we are trying to execute we don't know anything else and if you want to actually isolate ourselves from the environment we can
16:43
use stubbing and mocking and use a different dummy fake objects for those that our object depends on to make it isolated let's look at this example again what is this act part it's not quite clear it's actually here and it's mixed up with the third so the one who's
17:06
reading that he has to understand he has to read it like a just one line which is not very handy and i think it's it's just a mixed up but on the right hand side we can see a call and we can actually understand what is happening here we actually make a plane and calculate a distance
17:26
to a point and we're going to get this number that's what the one who is reading the test would be interested in and this is much more readable than we have it here on left hand side so the act is separated here and it is clearly visible and those are some other examples of act
17:46
that are what we do here we just execute some code calculate some context do do the work that we need to do and now we come to a third and the third is the verification stage it could be state verification and it could be behavior
18:05
verification and state verification is where we verify the state of our objects what happened to our objects what are stated and behavior verification is a little bit more tricky where we verify actually how our object influenced them as an object have it been called
18:25
that is also important and we should do one or another or not mix of it and the best practices here is to have one assert so and why we should have one assert mostly it's just because you will never execute multiple asserts in the test the test framework
18:44
wouldn't let you to do that whatever a test framework it is and you need the microsoft framework g unit they they will just stop at the first assertion that's how they design and they will never you'll never be able to execute all of them but you want to focus
19:02
on one thing what you're doing and you want to write a clear message for that assertion what actually happened this is only achievable with one assert and if for example some people just try to write an assertion letter asserts that a certain different properties of one of one
19:21
object that is easy to do just comparing objects and just one assert there should be example here so here we have two asserts so that is testing completely different things so we actually can divide this test in two different tests but on the right hand side this is clearly
19:43
visible what is being tested on the assertion so that was arranged after third a pattern that if you use it you actually almost achieve readability for free because i used to write the messy test myself and when i was trying to understand what this test is doing and someone
20:03
else is just what are concerns being tested what are data being set up and what is being verified and once you reply to yourself on those questions that is easy to make a decision what actually happened in a code but what else we can think about when we think about how the
20:22
test should look and what it should do let's look let's talk about the test scope the scope of the test it should think it should test one thing it should focus on one thing to test not on
20:41
multiple things on multiple aspects or maybe the problem multiple properties of an object an indication that we are testing several things would be that we are having some logic in the test if you're having if statement if you have switch statement if you have for loop we are it's very high probability that we are testing different things in one test if that we are
21:06
trying to test multiple aspects of that and of course if you have multiple asserts we are almost certainly testing different things if you are wondering about that you can try to name your test
21:21
just by looking at all the asserts and just to write a sentence that this test is doing this this and this you get a long sentence if you have asserts it will be several things so that's a good practice to have one thing and that will help people to understand it later how actually test is how test is working what test is testing the size of it what is a good
21:47
test size that is a religious issue again so I believe in five lines some people believe maybe another amount but generally I think the perfect test is a five lines maybe six lines
22:03
I don't know the exact number but again it's a religious issue I think six I can't imagine the test that would have 16 lines and would still testing one thing I've only I've never seen test 25 lines testing one thing ever haven't anyone seen that okay and that was
22:25
testing different things yes so it should be small and I believe this function should be small but the test should be small to test one thing to focus on one concern and let's look at a good test naming that is very interesting issue so we have that
22:44
concept don't repeat yourself in coding that means we are we want to remove duplication duplication is a maintenance for us we don't want that in a code we don't want to repeat ourselves but from the other side we have the the idea of writing everything in descriptive and
23:05
meaningful phrases which is coming to us from the main specific languages and there is kind of a balance between it because the more we write in the descriptive and meaningful phrases we could actually violate that don't repeat yourself principle and I think test lives in
23:27
its own little world it's isolated it shouldn't have any knowledge from the outside world and because of that sometimes trying to remove duplication we are actually sacrificing
23:41
the business logic and the meaning of the test so in testing being descriptive and meaningful is more important than removing duplication sometimes that's fine to have some sort of duplication in a test than actually trying to force people to read some other code I was actually
24:03
struggling to get example for that I tried I've actually pulled out some some code I've written a while back and that was a good explanation but it was so difficult to understand in such a short time so I would just I just removed the example but think about that and the main descriptive is much more important in tests because they live in their own little world
24:22
and that's fine if you repeat sometimes it's it's a test code at the end it's never executed in production good test coverage that is my favorite slide how much we should test we talk about coverage all the time
24:44
management use it as a metric to detect quality is it really true so I will tell you a story three programmers asked the great master of what coverage should I aim for him the first one asked
25:00
and the great master said don't worry about that just go and write some good tests and the second one asked about that what coverage should I aim great master said how many grain rice I should put in that pot that depends at the second programmer it depends because
25:21
how many people you have to feed what other food you're serving what actually you are what is this how hungry they are and those things are how I can tell you that's what he said and the master said yeah that's right and that programmer left and third one came what coverage
25:45
should I aim for 80 percent of no less said the great master and then followers came to the great master and asked him I we heard that you did the three different answers to the same questions to different people why it is and he said I the first one was new to unit testing
26:02
so he had to just focus on writing some good tests and the second one was experienced so he understood and realized with my answer that there is no simple answer to that the third one just wanted the simple answer to the question that doesn't have any simple answer so he got it
26:21
80 percent and no less and um in coverage it is very important that we should balance does anyone has a hundred percent coverage nobody okay but we wanted we are saying that we just have to do it do we have to blindly follow the coverage do we have to blindly aim hundred percent
26:44
oh eighty percent or whatever number simple number I can't believe that it could be some tests some projects with a hundred percent coverage they still have bugs in that we have to focus on something else and just test coverage we have to focus what we are testing
27:00
and where we are testing and what we are testing and we can talk about also coverage duplication here and this more brings us to the questions when we are testing so we have isolated tests that test local things and we have tests that are integration tests without stubbing and marking the isolated tests the unit tests referred to so we are
27:25
we are focusing on one class one object it's very local and then we have end-to-end tests and we have to have end-to-end tests because there is no way that we can test the system we don't have end-to-end tests but however anyone had a situation on one change in the code caused
27:40
hundreds of tests yeah few people that happens because we have some coverage duplication in our tests and the good practice that we found here is just to run the isolated tests local tests before and make sure that our local systems work fine and then run our integration test
28:03
end-to-end tests afterwards and this should be the way less integration tests than the local tests it's a it's a normal situation and then when we run the integration test there's just a few they they test the actually components communication they don't test the local things and we call that smoke test that we run and we run them first and we make
28:25
sure everything is fine and then we can allow execution of the integration test so that that is a very good practice that we found for ourselves good tests fail and they fail and they want them to fail but they want us to fail just for one
28:42
reason one good reason is that the functionality code change we don't want tests to fail randomly due to some isolation problems we don't want to them fail because there are bugs in them we want them to fail just because we change the code and it fails and there is a dependency on
29:01
that but we don't want random failures but the appeared that that random failures and isolation problems and test bugs they're quite easy to fix but to find the tasks that fail where we want it it's quite difficult we want to write a task that will save us some time in future but we cannot predict the future very well how we can predict the failure information
29:25
where should i test each test is associated with a cost i have to spend time writing it people have to spend time executing that and we have to think carefully how we are going to write the test where and here is ken back and he said i should test where i'm likely to make a mistake
29:48
not the things that always work that's fine but how do we know where we'll make a mistake that is probabilistic investment we don't know that's a bet we we just but we can think about it maybe and he's saying for example i'm tend to make mistake around conditionals so i am trying
30:04
to write a test around conditionals because i know i'm making mistakes there but what else we can think about when we talk about the good test value can we predict the failure information we normally don't think about it very much but we can we if you have tasks and we have a code
30:22
and everything works can we think about when it will fail we can try comment out production fault and watch it fails we can try to change production code in a way that we think it will evolve in the future and watch it fails and that is a very good exercise in determining when
30:42
the test will fail and we normally never do that exercise when we write the test we can look at tests when as an api as a documentation to our code and we this is a very good practice as well because we have we are seeing what we are trying to achieve and we are and here we can
31:01
predict maybe how it will evolve how it will change so we want the test when we feel like the odd value but not before and that is a i think that what we can think about when we write the tests which tests to write and which tests not to write
31:21
so what is a good test so the question that we ask ourselves when you write a good test is how easy i can understand it and more important it's not like i can understand it and how easy someone else would be able to understand it it's very very likely that you will not be reading that test you're doing it for your peers you're doing it for the one the poor soul who will
31:43
come to the project few months later and say ah and then when it will fail will it fail when the what what kind of mistake i should make to it fails and where i can predict the bug and we can think about coverage of course and we should think about coverage in
32:01
but we should understand that this is not this is a simple answer to the coverage question so we figure that the good test when we look from maintainability point of view it's more important to think about how it looks and where it tests then thinking about when it will it will be run and what it's actually
32:21
testing and focusing on coverage that was it thank you very much
Recommendations
Series of 3 media