Mocks, fakes, dummies, stubs and spies: Successfully isolating the snake
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 132 | |
| Author | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/44921 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
00:00
SoftwareActive contour modelSoftware developerSoftware testingControl flowCategory of beingParameter (computer programming)Function (mathematics)SimulationEmailPatch (Unix)Module (mathematics)Exception handlingSoftware testingDoubling the cubeDifferent (Kate Ryan album)MeasurementPatch (Unix)Social classGame controllerEmailSlide ruleLevel (video gaming)Object (grammar)CASE <Informatik>Physical systemAsynchronous Transfer ModeAttribute grammarVolume (thermodynamics)Parameter (computer programming)Interactive televisionFunctional (mathematics)Data management2 (number)Wave packetOrder (biology)Default (computer science)Unit testingWhiteboardGoodness of fitCategory of beingMultiplication signString (computer science)Right angleIterationException handlingModule (mathematics)Sound effectBitLibrary (computing)Real numberUniformer RaumSystem callCase moddingSoftware bugInterface (computing)NumberTrailElectronic mailing listMatching (graph theory)Ring (mathematics)TouchscreenInternet service providerInterior (topology)Link (knot theory)Context awarenessSubstitute goodMereologyContent (media)outputComputer animation
09:38
Dependent and independent variablesSoftware testingPatch (Unix)HoaxExact sequenceSystems engineeringDefault (computer science)Module (mathematics)Letterpress printingPhysical systemTouchscreenCodePatch (Unix)Execution unitFunctional (mathematics)INTEGRALHoaxParameter (computer programming)Different (Kate Ryan album)Slide ruleSoftware testingFreezingModule (mathematics)String (computer science)Type theoryEndliche ModelltheorieSet (mathematics)CASE <Informatik>Expected valueSelf-organizationUnit testingMultiplication signPhysical systemExistenceObject (grammar)System callAttribute grammarEmailComputer wormKey (cryptography)Category of beingInteractive televisionContent (media)Direction (geometry)Exception handlingLibrary (computing)Wrapper (data mining)SmoothingData dictionarySampling (statistics)NamespaceGame theoryMatching (graph theory)Case moddingText editorLatent heatDemosceneComputer fileRhombusProgram slicingRight angleLoginMonoidal categoryWeb crawlerMereologyAsynchronous Transfer ModeComputer programmingFigurate numberUniqueness quantificationVideo gameDatabaseGradientPattern languageFocus (optics)Degree (graph theory)Mach's principleRevision control
19:10
Active contour modelCodeTouchscreenSoftware testing2 (number)Computer animation
19:33
Game theorySoftware testingTouchscreenCode2 (number)Social classComputer animation
20:24
Point (geometry)Module (mathematics)HypermediaHacker (term)System callMach's principleLine (geometry)Multiplication signCase moddingFamilyNumberException handlingType theoryPiAsynchronous Transfer ModeModule (mathematics)Functional (mathematics)Code1 (number)RecursionSystem callSoftware testingCASE <Informatik>NP-hardMusical ensembleRight angleLatent heatComputer animation
24:07
Point (geometry)TouchscreenCodeLink (knot theory)Metropolitan area networkSlide ruleSoftware testingAsynchronous Transfer ModeEndliche ModelltheorieLink (knot theory)Patch (Unix)Multiplication signModule (mathematics)Functional (mathematics)Computer animation
Transcript: English(auto-generated)
00:00
Hello everyone, my name is Mario Croceira, hard to pronounce for a non-Spanish speaker and I work at Bloomberg and I'm here today to speak about testing doubles and the different kinds that we have and how to work with them in Python. So first of all, the clicker doesn't work, so first of all, so what are testing doubles?
00:24
So testing doubles is just an object that looks like the original one but where the creator is in control of its behavior. So as in the movies, you probably want your baby to be there so you just put a testing double that you have control over just for the system and the test that you are testing.
00:44
And you might be wondering why do we need testing doubles? So you just leave uni, you go to a company and they tell you like, hey, you need to test out this function which is, it's a really important function, we have no test, we want
01:00
you to write some tests for it and see if there's any bug and this is going to predict the department expenses so you say like, okay, I've started this thing, I'm going to try some values, you know, its cases, blah, blah, blah, then you say, oh, whatever, I'm going to drop an array of numbers, I'm going to test a lot of things with different budgets, then your phone rings and they tell you, hey, Mario, the whole
01:20
management board is receiving emails about the department expenses for next year, can you please stop doing whatever you do? So testing doubles are here to help you isolate the system under test that you're testing. They are great to simulate the scenarios, let's say that, for example, you want to try out, what happens if when sending an email which this function was doing and we did not notice,
01:42
what happens if when sending an email something fails and it's also good to be able to scope your tests. So, first of all is how do you create them in Python? So, because we just said that testing doubles are just objects that can be used to replace other objects, which you can just create, for example, here, this is Mario double,
02:03
my name is Mario, so you can just create this object that could behave as a real Mario. But in Python, we are lucky that we have the mock library within the mock module within unit test and by just using, for example, the mock class, you can create this kind of object much easily as we'll see in the following slides.
02:24
So, but this whole talk is around testing double and sometimes you might not need mock at all. For example, in some situations, you might have a class, well, you might have a function that you want to test and you might have arguments that you just want to fail. In those situations, a good testing double can just be simply known or a string.
02:45
This is known, so there's a lot of naming conventions around testing and testing doubles and the kind of testing double. If you're interested, I'm going to put a lot of links at the end. For those of you quite familiar with testing, I'm going to put how this is referred by Mesos and other philosophers of testing that you can find out in the wild.
03:04
This is known as a dummy and it's just basically a testing double that you just use to basically fill a parameter. There's no behaviour you want to control. There's nothing you want to do around them, just basically fill a parameter. But things get more interesting and it can happen that you have this function that is
03:21
checking if, so this function is now taking the email sender, so we won't make the main mistake as before, and it's just checking if the email sender is enabled and then just sending the email. So, because as we said, we have this mock class that helps us create these kind of testing doubles, we can use it in this way, which all it's going to do is create this
03:45
kind of object as we saw before that we can create manually, but in this case, it's just going to create this object where when you call it, it's just going to return success and when you pull this attribute, it's enabled, it's just going to return true. So with this way, we can easily create a mock. And I want to ask, because I want to know if this is too basic, how many people already
04:04
knew all this? Well, is there anyone here who didn't knew this? Okay, so I'll speed up. Don't worry, there is more advanced contents later in the slides. Okay, so this is all good. When you're creating mocks, it can also happen when you're using what's known as a
04:22
stop in the testing terminology. You might not even care what's inside, you just want to fill a parameter but still the object is being interacted within the function we are testing and you can just basically create a mock. This is something really cool about mocks in Python. Mocks will automatically create child mocks whenever you access a property of them.
04:41
So what this means is that in this function, when you call email sender, which is a mock, and you call dot is enabled, this is going to return the mock or when you call the mail sender, it's also going to return your mock. Every single time you call any method in a mock, it's just going to return you a child mock. Okay, so this is all cool but then it can happen as well that within your function,
05:04
you might be using what's called magic methods. And in that situation, if you run this with a mock as we saw before, this is going to race because the mocks don't have implemented magic methods. So basically, you use a magic mock which is a mock with magic methods. The only magic method which is actually implemented in mock is call.
05:22
Call is implemented both in mock and magic. But for the rest, you need to use a magic mock. Let's speed up because we said all of you already know this. Yes, so really quickly, what can you do with mocks when you want to define behavior
05:41
and you want to instrument it to do more complex behavior than we saw before? We said that you can use, for example, return value to say what's going to return whenever you call it. You can also use side effect. You can pass a list of values that are going to be returned in order. So here, if you call, for example, email sender four times, it's going to return first two, 10, 20, and zero.
06:00
If you call it again, it will raise the stop iteration. You can use side effect to raise exceptions. And you can also use side effect to call a function. Has anyone seen anyone, something that they haven't seen before so far? Okay, it's going up at least. That makes me feel a little bit better. Okay, but what happens if, as we saw in the first example,
06:24
the object that we want to patch, well, sorry, the object that we want to use a testing double is an internal dependency of the function. Here you can see that the function doesn't take email sender as an argument, but it instead, you know, it can be a class that has this as an attribute, or it can be some kind of a global object.
06:42
So in this situation, we have the mock library has a patch function that you can pass a string and it will just replace whatever you have in there for a magic mock. This can be used as a decorator or as a context manager or manually, and then basically returns you the magic mock that it substitutes the object with,
07:03
and then you can control how it's going to behave. If we have time, which I'm sure we will because I'm going to go fast, we'll see how this actually works, which is quite cool. The patch function has a couple of arguments that allows you to, so as we said, by default, it's going to create the magic mock. You can change what it creates.
07:21
You can also change the object that you want to pass in, if you want to put your own object, and we'll see some more afterwards. You can also verify the interactions that you have with your mock. So here we said we are passing a mock that's going to iterate and provide like internal inputs into the function.
07:41
And we can also do this like assert call width, which is going to check that the mock was called with those arguments. Does it make sense? Has anyone noticed the bug? Because this is going to fail. So this last function is going to erase. This send email has never been called,
08:01
because here what the function is actually calling is send mail. This is the thing I hate the most about mock. So when we are working with mocks, which are testing doubles that try to verify the interactions, the mock class in the mock module of the unit test library
08:22
sometimes can play against you. So how do you solve these kind of situations? So you can use spec. Spec allows you to make your mock have the same interface as the object that you are mocking.
08:41
So here if you pass mailing, which is like an existing class, the mock is going to have the same interface. And if you try to use something that's not part of the interface, like send email, because we saw that the original one was send mail, this is going to erase an attribute error. When you're using patch, there is an argument that you can pass which is called auto spec.
09:00
And because you are patching a real object, well, because you are patching an existing object, it will be able to infer as well the interface from here. Okay. Now, this is all cool, but then you have this, right? So you go to the training and they tell you what's spec. Yeah, you always have the spec, it's so great. But, you know, Python and back typing and how is spec going to figure out what the mock is going to return?
09:23
Because if you use spec here, for example, on your mock interface, when you call send email, well, send mail, what that's going to return is just a mock, a magic mock. And that magic mock doesn't have a spec, right? So how do you fix that? How do you work with that? And the thing is, because we have the test like that
09:43
and we're working with a mock, what we're doing is we create the magic mock here, right? We then, sorry, so this email sender is actually mock. So here we define the expectation and the behaviour of the mock. We are then going to call the method here.
10:00
But if this function that we are testing calls anything in the mock that's outside of the specification, it's going to fail. And you can see here there is another problem, which is that we are calling .data here and we are doing .payload. And again, this assert call is going to fail. But this is going to go through smoothly. And if you're working with a mock that we want to verify how it works,
10:21
you ideally want it to fail here, right? Because this is not what you define. So you can see that when you're writing a test and you're using a mock, what you really want is, okay, I declare the mock, I declare its behaviour. Here is ultra useful, the property of creating child mocks. But here I don't want child mocks.
10:41
If I'm working with a mock that I'm going to verify the behaviour, I don't want it to create child mocks. If you have worked with gmocks, for example, in C++, they have different type of mocks so you can handle this. And this is the good news. From Python 3.7 you have a new function called seal where you can freeze a mock.
11:02
Who has seen this before? Who has seen seal? Hands up. Great! Yes! So now you can do seal, and seal is recursively going to freeze your whole mock. Okay? This will allow you to effectively now define your mock, seal it, freeze it, that's the behaviour it's going to have.
11:22
And when you pass it to this function, if there is something that doesn't match, it will like the real object, well, like the object you defined, so it's going to raise an exception. Yes. Okay. Now, something I find extremely useful is if you have done JavaScript, you know what spies are,
11:41
basically sometimes you have a real object that you want to pass to a function, but you want to verify the function, how the function interacts with it. So with mock, you can have the wraps keyword, and what this is going to do, this is going to say like, hey, create the mock, okay? And I want the mock to whatever function that's being called,
12:01
just pass it to the real object. But I want you to record all the direction that it's doing, so then I can assert on them. So not only we can, like, as all the mocks, we can then verify the interaction with them, we can get the arguments that we're calling, so it's basically recording everything that happened, and you can test, you can verify how the function that you're testing
12:21
is interacting with others. This can happen, this just can be useful, for example, if you have, like you're calling, if you have some integration test where you're calling outside, you still want it to call outside, you want to call it to a fake database, or something like that, and you don't want to re-implement the whole thing, but you just want to be able to record what's going on, this is really handy.
12:40
Okay, so basically, what I've seen, I haven't flown through the naming, sorry about that, I shouldn't have explained you what the names are, this is on the slides online. So there are dummies, fakes, studs, spies, and mocks, which is basically namings for different use cases of the mock. We see how UnitTest.mock can help you create them, how Patch can be used to testing
13:01
doubles or internal dependencies, how you can use Spec or Seal to free your mock, and how RAP should create spies. Now, if there is something I want you to take away from this talk, is there anyone from the organization here? They are not going to like that, but if there is something I want you to try out in this conference,
13:25
in this trip to Edinburgh, is anyone here from outside Scotland? Okay, amazing. So there's something you have to try before you leave your Python, and it's a deep fried Mars bar, okay? I'm hooked to them. Okay, but anyway, we have time for more content.
13:41
So I'm going to show as well some other features of the mock libraries. So this is what's called Sentinels. Basically, there are some situations, like the first sample we saw, where you might have an argument that you might just pass through an internal dependency of the functions, and you want to verify. So you can pass a string, for example, a lot of money.
14:02
I have a problem coming with strings, because I always wonder like, hmm, is the function actually returning what I tell him to return, or is it internally going to be like that? So mock had what's called a sentinel, and basically it allows you to create unit objects, by just calling, for example here, you see unittest.morgan.sentinel, and you can do sentinel.budget,
14:20
and that's always a unique object, so you can assert on it. It's just handy. But I'm sure many people have had problems with patch. Who has had problems when patching things and they don't work, right? Great. So these three slides ideally will solve all your problems from now on.
14:41
So how does patch actually work? The thing is, when you try to patch something, what's going to happen is, the mock model is going to go to, well, you're passing this string, right? This string has to set, an attribute to set, and all that mock is going to do, it's going to do a set utter on it. So, as we saw that the patch has function, has different keyword arguments to do different magic,
15:02
but here we're just going to focus on the basic one. So this is just going to do a set utter. And the way it works is, so here you have the package module target, and you say, hey, I want to patch package module target, right? So what's going to do? Magic mocks on it, right? That's easy to understand. Now what happens?
15:21
When you have a real program, and you have this myfile.py, and it's import target, if you tell unittest.mock to, hey, can you please patch package module target, what's it going to do? It's going to go to package module target, and put a magic model there. Is that what you want? Nope. What you want is to patch that one, right? But because what it's doing is,
15:40
it's going to the namespace here, going to target, and just doing a set utter. It's not what you expect it to do. So what you want it to do is, you want to patch target in myfile. If you do that, mock is just going to do, and put it there, which is what you want. So whenever you are patching anything, remember to patch where you are using.
16:02
That's usually the easiest way to understand. If you understand that all that patch is doing is literally going to the dictionary of the namespace, or whatever you are telling him to go, and then doing a set utter on the target, it will be much easier to mentally understand what's actually going on.
16:21
Also, as a bonus, when you do patch, you can pass the new, and if you're trying to patch something that still doesn't exist, patch has a keyword argument called create, that instead of doing the set utter, will actually put something there, so you can do a patch even if that doesn't exist.
16:43
The key on how that patch works, and if you have problems with that, is always patch wherever you're using it. And then all problems go away. Okay. How were you doing with... Okay, great. So, next thing, something I loved. How many people here use PyTest?
17:01
Great. So if you combine PyTest with mocks, you can do something really cool, which is you can create fixtures that are just mocks. And if you have really big mocks, which can happen and have gone through, you can just create, for example, this is a super simplified version, but you can create this, for example, or system mock, and you can just depend on them, and you can define the behavior in a single fixture,
17:22
and this will be quite useful, because you won't have to repeat them. Something that happens quite often when you're patching, if you have multiple mocks that you're patching in a function, and you're patching it as arguments, is remember that because of what patch is gonna do is, it's gonna go, it's gonna patch the oldest thing,
17:41
and then when you use it as a decorator, it's gonna add an argument at the end that gives you the mock that you're working on. So if you have two patches, remember that the first patch, I should have put a slide about this, the first patch will be the last argument, the second patch will be the second argument. Just think on how it works, which is that it's appending at the end
18:01
when you declare the decorator. It's a little bit weird at the beginning, but if you see it as it works, which is probably not the best answer, you can understand why it works like that. If you use a lot of mocks, you say, wow, this is amazing, right? I can mock all my stuff. So if you use a lot of mocks,
18:21
it can happen that when you are debugging, you're seeing your logs and everything, you see things like this, which is rather unuseful. So mock has a keyword argument when you create them called name, and you can name your mocks, and whenever you use wrapper or a string because you are printing them or checking them in PDB, you can see which mock you were referring.
18:42
If you then use the child mocks thing where you are accessing other views, you will also see the whole name of it, which is quite handy. And we have time for Kahoot. So as I told you about the mocks part, I really like sweet. And how many people have tried this? This is a UK sweet, which is absolutely amazing.
19:01
So we are gonna do a quiz, and the person who wins takes this, and it's about mocks. So you can, if you have a phone, you can go to kahoot.it, and let me bring that up. Also, you'll get all the glory of being the person who won.
19:25
One second, internet. Come on, you can do it. Okay. So you can put chug, chug, chug, chug. Has anyone played Kahoot before?
19:43
So there should be a code somewhere here. So if you go to kahoot.it, you can put that code. Names will start to appear, and we'll start with four questions. The fastest and correct one will win the teacups. I love them. I have like five of them at home.
20:01
Okay, let's give 10 seconds. I have five minutes? Okay, plus questions? Plus questions? Great. Okay. Let's go. Ooh, still people joining. 74377. Easy to remember. So there we go.
20:22
Uh, full screen. So the first of all is, who is the original art of the mock module? Which is around the conference, you can back him. So Michael Ford, Victor Steiner, Ron Obius, or the Black Knife?
20:40
Hey, hey, hey, hey. It was not that easy. It gets harder. Uh, there's a stop. Which of the following allow us to create a spy? So, yeah. Which of those ones are we going to create spies in Python? I have to assume music.
21:01
Cool. That was easy. So you were actually paying attention. Wow. I should tell my family this. Okay, which of the following lines will raise? You have some more time here. So you can choose, so this is the code, and it's either A, B, or none of them.
21:20
So we're using seal here. Remember, seal will recursively freeze all your mocks. And here we see that we have mock, callable one. So, it was the blue one. Why was that? So, there's something I didn't tell you, so you failed this question, specifically.
21:40
Which is that, when you seal a mock, it will seal recursively as far as the mock is a child mock of the mock you are sealing. So what that means is that, here when you do mock.callable, this mock that is being automatically created is marked as a child of the first one, but as a way to be able to disable this recursiveness on the seal.
22:04
If you pass your own mock, it won't seal it. So it's basically going through the child rather than checking all the mocks inside. Ha, ha, ha. That was tricky, right? And the last one. Another, I like exceptions.
22:21
So, there you go. So you have mock.called. This is a normal mock, no seal, no anything. Mock.call, mock.assertFakeCall, mock.assertUnscalled. Which one will fail? Is any of them going to raise an exception?
22:42
This one was, whoa, are you serious? This one was really hard. Okay. Wow. I'm impressed. You should come and give the talk instead of me. So for those of you that either hit all of them by mistake or like, hey, let's try it out.
23:02
So called is raising an exception because called is actually an attribute of mock and it just returns a number and if you try to call the number, it's going to raise an exception. Now, assertFakeCall, if how are you, I would have expected that to create a child mock,
23:20
but it doesn't because there is a check in the mock library that if the name starts by assert on this call, well, by assert, it's going to raise because there were so many mistakes about people mistyping something and then you are in this situation where like, is the test passing because the assert actually worked
23:41
or is the test passing because the assert is created in the child mock? So there were so many types of typos with that that there is now a check that if it's assert, it will raise if it starts with assert. And the third one, which I'm impressed you knew about it, there is also a check in case you mistype assert
24:02
and you do assert. Wow, you're amazing. Okay, so... Philly, who's Philly? Oh, no, are you serious? Come on, man. Please, a hand for Philly.
24:24
Okay. So with that, I'll upload the slides. You have a bunch of links here. The mock module itself is kind of easy to read if you're there and then some other slides about testing in general
24:40
and a cool talk if you want to know more about the patch function and, well, the module itself. And now should be time for questions. When anyone of you has questions, you can just come in front to the mic. Either a question for Mario or for me, the graceful winner.
25:00
Yeah, you should take them all. So, any question for Mario? Any question for him. Okay, so if there are no questions, then let's give a thank you for Mario.