Advanced Uses of py.test Fixtures
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Part Number | 14 | |
| Number of Parts | 119 | |
| Author | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/19973 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language | ||
| Production Place | Berlin |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
| |
| Keywords |
EuroPython 201414 / 119
1
2
9
10
11
13
15
17
22
23
24
27
28
41
44
46
49
56
78
79
80
81
84
97
98
99
101
102
104
105
107
109
110
111
112
113
116
118
119
00:00
Software testingTexture mappingLecture/Conference
00:26
Coma BerenicesInjektivitätSoftware testingSicDesign of experimentsPredictabilitySocial classScalable Coherent InterfaceThermische ZustandsgleichungUniform resource nameInversion (music)Letterpress printingHill differential equationMetropolitan area networkInfinityTunisParameter (computer programming)Cloud computingGraphical user interfaceCAN busFlagEstimationEmulationCodeSatelliteSet (mathematics)Form (programming)Process (computing)Function (mathematics)LogarithmTrigonometric functionsReal numberModule (mathematics)Statistical hypothesis testingSet (mathematics)Control flowQuicksortComputer fileStatement (computer science)Level (video gaming)Function (mathematics)Different (Kate Ryan album)Graph coloringSystem callDatabaseLetterpress printingWeb browserFunctional (mathematics)Distribution (mathematics)Software testingService (economics)WeightEndliche ModelltheorieMathematicsBitInternet service providerInjektivitätBuildingParameter (computer programming)Multiplication signOrder (biology)CASE <Informatik>Social classCache (computing)Module (mathematics)Power (physics)CodeSuite (music)XML
03:41
Distribution (mathematics)QuicksortLecture/Conference
04:03
Table (information)Hand fanSoftware testingCloud computingCorrelation and dependenceInflection pointCAN busComa BerenicesSoftware testingStatistical hypothesis testingDatabaseInternet service providerTable (information)Connected spaceInterior (topology)XML
04:46
Graphics tabletTable (information)Software testingCAN busCloud computingControl flowServer (computing)RoundingHill differential equationClient (computing)Infinite conjugacy class propertyStatistical hypothesis testingEstimationError messageReal numberSummierbarkeitMaxima and minimaStrutInterior (topology)ASCIIState diagramNetwork operating systemRing (mathematics)Bus (computing)Monad (category theory)Uniform resource nameArc (geometry)Asynchronous Transfer ModeFile formatPhysical lawSurjective functionWeightMenu (computing)Computing platformPhysical systemStack (abstract data type)Form (programming)TorusSet (mathematics)Repeating decimalOrdinary differential equationWide area networkFunctional (mathematics)Table (information)Distribution (mathematics)MultiplicationSoftware testingMereologyCASE <Informatik>Exception handlingQuicksortMultiplication signObject (grammar)DatabaseElectronic signatureInternetworkingRepresentation (politics)Control flowCodeMessage passingStatistical hypothesis testingSystem callInformationOperator (mathematics)Single-precision floating-point formatEndliche ModelltheorieConnected spacePrinciple of maximum entropyComputer fileAttribute grammarConfiguration spaceSlide ruleMoment (mathematics)Computer configurationBitView (database)Hazard (2005 film)VotingLattice (group)Execution unitSoftwareData structureParameter (computer programming)Variety (linguistics)LogicArc (geometry)Regulator geneCartesian coordinate systemService (economics)Sheaf (mathematics)Remote procedure callSound effectWebsiteClient (computing)Scripting languageArithmetic meanPlug-in (computing)Server (computing)Right angleSoftware developerDefault (computer science)Electronic mailing listUnit testingLecture/ConferenceXML
14:03
Arc (geometry)Newton's law of universal gravitationSoftware testingNetwork socketInterior (topology)MultiplicationUniform resource nameWeightFamilyRaw image formatFinite element methodMetropolitan area networkMagneto-optical driveOracleArmParameter (computer programming)Level (video gaming)Normed vector spaceReal numberDatabase transactionInformationStatistical hypothesis testingHookingElectronic mailing listParsingRight angleEndliche ModelltheorieComputer configurationOnline helpGroup actionServer (computing)Mathematical singularityExecution unitMenu (computing)OctahedronEstimationControl flowVarianceRoundingSkewnessLocal ringSupremumComputer clusterCAN busCloud computingNetwork operating systemSoftware testingDistribution (mathematics)QuicksortDatabasePower (physics)Projective planeMultiplication signMereologyCASE <Informatik>Insertion lossMedical imagingBuildingStandard deviationAttribute grammarComputer configurationConfidence intervalStatement (computer science)WordParameter (computer programming)Video gameMathematical analysis1 (number)Electronic mailing listRepresentation (politics)Complete metric spaceMarginal distributionContext awarenessPhysical systemRevision controlObject (grammar)ResultantCartesian coordinate systemSoftware developerVotingDifferent (Kate Ryan album)Functional (mathematics)Scripting languageInstance (computer science)Ocean currentSource codeOracleParsingConfiguration spaceSet (mathematics)TelecommunicationConnected spaceView (database)Office suiteString (computer science)Point (geometry)Directory serviceCombinational logicComputer fileInheritance (object-oriented programming)Network topologyService (economics)Server (computing)Internet service providerInstallation artPlug-in (computing)ParsingHookingVariable (mathematics)MultiplicationBitLine (geometry)2 (number)Suite (music)Entire functionLecture/ConferenceXMLUML
21:49
Software testingElectronic mailing listPoint (geometry)Combinational logicDistribution (mathematics)CASE <Informatik>QuicksortSystem callProgrammable read-only memoryStatistical hypothesis testingEndliche ModelltheorieEscape characterForm (programming)InternetworkingState of matterCategory of beingBranch (computer science)Physical lawWordParameter (computer programming)Plug-in (computing)Parametrische ErregungLevel (video gaming)View (database)Group actionCache (computing)Lecture/Conference
Transcript: English(auto-generated)
00:15
Please welcome Flores.
00:24
Good afternoon. So, yeah. Advanced fixtures in Pyretest. Pyretest fixtures are sort of a quite unique and powerful feature in Pyretest. And they allow you, they basically allow you to create your fixtures using dependency injection.
00:43
And it kind of creates a kind of isolated and quite composable as hopefully you'll see. So I'll try and go through, like, so I've been using Pyretest and been contributing for a few years now. And so basically based on that experience, I'll try and build up some more of the
01:02
interesting things to build a big test that we find useful building a big test suite. I'm going to assume some knowledge about Pyretest itself and basic, and you get the basic idea of how fixtures work. So how dependency injection sort of works. But a very quick reminder of that.
01:22
So a fixture is basically, so any test function can request a fixture by just taking a parameter, a named parameter. And the named parameter will then be looked up to a function that's decorated this fixture marker, which is just a function that returns a value. And that value is then injected by Pyretest into your test function.
01:42
You can create fixtures on different levels in your files, so if you do it in a class, it will only be visible to members of the class. So that's sort of very quickly the basics. So the first thing to start sort of extending that is caching of your fixtures.
02:05
This sort of, so basically by simply giving another keyword argument to this Pytest fixture decorator, you can change the scope, the lifetime of how long that fixture lives.
02:22
Normally it's being, if you don't provide an explicit scope, it will just be torn down straightaway after the test function has run. But you can change the scopes into session scope, you also got module scope and class scope, as well as function scope.
02:42
And that basically means that you'll only call that function once for the scope. So in this example we got two differently scoped ones, one function scoped, one session scoped. And we got two tests using both of these, this is fairly contrived I guess. But if you run Pytest in the output, I'm using minus S here, which kind of stops
03:03
the, make sure that the print statements I put in my test code actually get shown up. Because normally Pytest will capture that and only show it in case of failures. But with minus S, you can clearly see the order that things happen here. So the session setup happens first, and then the function setup happens, then the dot, which is basically the test that's been run.
03:23
And then the function finalize it there down kind of happens, and then that happens again. So you can see how the caching sort of works there. So if you got expensive fixtures that you don't want to recreate the whole time, like populate a database with schema or something, or create web browsers or those sort of things, this is a very common thing to start kind of using.
03:49
Yeah, and you have sort of available scopes there. Next one, you can sort of, fixtures can just, can be used in other fixtures as well, not just in test functions.
04:04
And that makes it really composable, because it means you can have a, so in this example, creating a database connection fixture, and that might be used in some tests directly. But then, for example, if this was a functional test, and one functional test needed a table to be
04:25
there or something, I can just build on those existing fixtures, and you can puzzle them together in that manner. One of the things here, the request fixture that's being asked there is, it's the way to add finalizers to places, so you've
04:43
probably already seen that, but it's essentially no more than just another fixture in pilot tests, it's just a built-in one that's provided. So that kind of happens quite seamlessly. The test function here simply uses the latest one, the DB table fixture, but it could have been using both as
05:05
well, so if it needed both of them for some reason, there's no reason not to combine it that way either. Another thing you can do in fixtures is, so normally when you're on tests, in pilot tests, you can mark testers, oh, I want
05:25
to skip this test or something for whatever reason, but you can also do this based on, trigger this from inside the fixture, basically. So if you do it out, it means that any test that will be using that fixture will automatically be skipped instead of, so you can make this depend on something else.
05:48
So this example is something that we use quite commonly when we connect to a remote service sort of thing. So basically, first try if the developer is running it on their own local host, if not,
06:00
see if they're in the office, try connecting to the server on that network sort of thing. But, you know, if neither is there, then whatever, let's go and skip it, and combining this with the session scope, for example, means that you don't keep doing that again and again, because that might be a slow operation or something. Another thing to note here is that when you call pytest.skip like this
06:23
with a message, it's basically pytest.skip will raise an exception in your code. So for the control flow of your fixture, you have to realize when you execute pytest.skip, you're basically raising an exception in there, and pytest will interpret that as, but no code will be executed afterwards anymore.
06:43
Just like you've got pytest.skip, there's a pytest.fail. We'll do exactly the same, but failure test for whatever reason, if you want to do that. And this is sort of a slight side step, talking about introducing marks a little bit.
07:04
Hopefully you've already encountered marks, but pytest is a very flexible marking system, so it's basically just a decorator, pytest.mark. And then a name you choose yourself, sort of thing. And you can just apply that decorator to your test functions, and then your test functions will be marked.
07:22
That on its own doesn't provide you very much. You can sort of use that on the command line to select your marks, but it doesn't provide you that much yet. One thing, I'll get back to that on the next slide basically, but one thing to note here is that
07:40
you can mark multiple marks, you can obviously apply multiple marks onto a single test function or anything like that. But another thing is that because marks are so flexible, they're allowed to be made available on the fly basically.
08:00
And one sort of side effect of that is that if you make a typo in a marker, you may not notice that, and they may kind of hurt you later on. And that's why you sort of have two camps of people, I think. Some people like me prefer to use the minus minus strict option to pytest so that you get caught out and you have to basically declare your markers up front and you get notified of any mistakes you're doing.
08:27
And the obvious way to just always enforce that is to write it in your configuration file. So that's basically the example of how you write it in your configuration file. Adults basically always add the command line option when you invoke it, and then you declare your markers. In this
08:42
example now, if you try to run this, it would fail because only one of the markers has been declared. So that's all the markers. Markers are used by plugins as well if you've used plugins. They often make
09:01
use of that. And one way to make use of that is inside, basically detect them in your fixtures. So in here, the test function wants a Mongo client. It's a little bit contrived because PyMongo doesn't quite work like that, but almost.
09:21
So basically it needs a Mongo client. And we're also declaring basically the markers, declaring the database to put in the URI to be used at connection time. And then when you actually look in the fixture, basically what it tries to do is it tries to look, is there a marker? Yes. Then I'm
09:47
going to use that as the database name. If not, I'm just going to use the default database name and then create the client and return it. And to actually get this marker information, you use request.node. So request.node itself is basically the test node, which is an internal py.test representation of your test itself.
10:07
So one of the attributes will actually be the function that you're actually testing. And it has quite a few methods. I think you can look it up in the documentation. At least parts of them are documented. But getMarker is basically how you get hold of a marker.
10:29
The only thing is you then basically get this sort of marker object. And in the case, so either you get this marker object or none, if you get none, we just use the default one here. But in the case of a marker object, the way it passes, because as you see
10:44
the mark takes an argument here, and the way it passes on the arguments that you've specified, it just passes this on as this args and kwargs attributes on your marker object, which is a list and a dictionary, which is a very common representation. But the problem with that is that it's not how Python signatures sort of work.
11:05
So I use this little helper function, which is like call API phone in this case. And it doesn't do anything useful other than it uses Python itself to pass a signature for me. So it means that if someone, if I wrote the decorator here differently,
11:23
MongoDB equals users, it would still work, because Python would just pass a signature and return my database back. So that's why I call that function with star args and kwargs. So that's sort of a little trick to get native Python signatures working there.
11:49
I have to add that in the future, I think probably one of the next releases, that there is going to be a slightly different way, or a new way introduced of declaring markers,
12:02
which will get around this little hacky signature passing stuff. So that will improve, but this is sort of the way we do things currently, and it works very nicely. Another commonly used thing is fixtures can also be automatically used, basically.
12:24
So normally fixtures are always dependency injected, so your test function requests the fixture by name it wants to have. But sometimes that might not be suitable, so there's this auto use equals true argument to the fixture decorator.
12:44
And in that case it will basically be called for every test function automatically, whether it's been requested or not. So this is kind of a lot closer if you're used to the unit test kind of way of things to set up and tear down, because that's just called every single time.
13:01
One of the nice things here is as well that you can combine this with the scope argument as well. So if you create an auto use fixture with a scope of the session, that basically means that right at the beginning of your test session you'll be doing set up and at the end you'll be doing some tear down.
13:26
The auto use fixtures, in this case I sort of use it with an underscore, sort of indicating that I don't expect it to be used immediately or directly. But there's nothing stopping you from returning a value as well and explicitly requesting it,
13:45
so if someone actually requests it you can just mix those two together as well. So in this case you'll sort of use it to create this kind of custom skipping logic for something that's only supposed to work on Linux.
14:01
There's various ways of doing that I guess, but it's an example. Then parameterizing fixtures is sort of another very powerful feature that you can do with fixtures. So in this example we have a test function, the first test function there, TxN, which just uses basically a URI to connect to your database.
14:28
The problem here is I want to ensure that this works with PostgreSQL, Oracle and SQLite for example at the same time. So instead of having to write three tests or write a test with different fixtures or something, you can basically parameterize your fixture itself.
14:46
So by parameterizing this fixture, PyTheTest will call my test functions three times, once with each parameter. So the arguments in the list to the params keyword in your fixture decorator can be anything, it's up to you.
15:05
So you can use direct values there if you want, or in this case I use simple strings which I'll probably use later with an if statement or something. But you get access to the parameter being passed in by this request.param attribute on the request fixture.
15:21
And that's how you access them. Another sort of building on top of that is if you want multiple fixtures with parameters you can combine them, which is basically what the last function does. So in this case, because both fixtures are parameterized, PyTheTest will basically call the last function six times because it requests both of these
15:46
and you will get each combination tested automatically. So in this case, the example suggests that I want to test the connection of both IPv4 and IPv6. But you automatically get all combinations of your fixtures.
16:06
Sort of slightly building on top of that is you can optionally sort of mark your parameters again in your parameterized list. So you can, if there is a fixture, so in this example I don't assume that every developer has their Oracle DB API installed.
16:27
So I just try to import Oracle if not. And I sort of introduce two things at once here. So we've already seen the markers basically. So this is PyTest mark skip if is a built-in marker that PyTest provides.
16:45
But instead of kind of using it directly on something, I just assign it to a variable now. And that's just to make my line a bit shorter later on basically. Or maybe I want to use it more than once or something like that. So you can just assign your marker to a variable. And now you have that marker object there which you can use later on.
17:00
In this case, I apply basically on because my parameter is not a function so I can't really use decorator here. So I just manually apply that onto my parameter. And it means that if I don't have Oracle installed here, despite that the test function should be parameterized,
17:20
only one version of them will be run and the second will be skipped. This is sort of an example taken roughly from an early version of the PyTest Django plugin. But it shows kind of how you can see basically the thing here is the request.fiction names part of the fixtures.
17:44
And it allows you to see all the other fixtures that are being requested by the current test function. So you can, in this case, I'm just stopping two fixtures that are mutually exclusive being used at the same time by simply calling pytest.failon if I detect that case.
18:05
But you can sort of peek into what the test function is asking and sort of have a look at the other fixtures around, which sometimes could be a useful thing. Then sort of a very, another sort of step aside, I guess.
18:27
You've probably, if you've been using pytest, seen this conftest.py files. So this is sort of a typical directory you'll get. At a simple level, conftest.py, so you can declare your fixtures in there and then you'll see them in their entire directory basically.
18:44
But conftest.py is basically, from pytest's point of view, it's just another plugin just like any other pytest plugin you can install. But it's a per project kind of plugin. And essentially when you start building a big test suite, you're going to be building a per project plugin kind of thing.
19:05
The plugins work with this hook system. So basically in your plugin you find function with your hook. Pytest will at certain times call these hooks then if you defined one. These are just a few common ones. There's a much longer list in the documentation.
19:23
And I'll be showing basically the adoption one here, which is sort of adding a new command line option. The parser object you get in that hook is basically just an outpass command line parser. So you can just use that documentation to add your options here.
19:41
I'm here adding an option saying, so basically the idea here is that for my script that invokes pytest on my CI server, I'll be just passing in that minus minus CI option so I know if my test suite is running on the CI server. And just very quickly, you can get to command line options both from inside fixtures as well as test functions.
20:05
Again, via this request object which has a config, which is basically pytest config representation. So you can get access to both command line options or configuration file options in there. And basically the both request.config and pytest config are basically the same instance.
20:23
There are different ways of getting to them. And those two, combining all of that together sort of, this is sort of extending on an example I used earlier. So again, so here I have this external service, in this case a ready service,
20:44
that before I checked, oh, is it on my developer's laptop? Is it in the office or something? And I was just skipping those. But in this case, by using request.config and checking the option, I can make sure that when I'm running on my CI server, it really is going to execute.
21:02
And if not, I actually get a hard failure so I can see. So if the server is down, I don't just suddenly stop testing part of my test suite sort of thing. So that's another thing that we use quite a lot and it's pretty handy.
21:22
Yeah, that's basically what I had so far. Yeah, thanks for listening. I hope it was useful. I know.
22:04
Okay, so the question is, I showed how to get the names, the other fiction names that are being requested by the test function. Is there also a way of getting the value of them? The answer is sort of twofold.
22:22
Definitely yes, obviously, by simply requesting it yourself. But that's not dynamic. There is an escape hatch to do that dynamic, although it's discouraged. So basically on the request fixture itself, there's a get funcag value call, which you can then use to call another value.
22:45
So that's how you could get to it. But generally that's discouraged because if you use that on one that's already requested, it's probably okay. But if you use that on one that hasn't been requested yet, PyTEST sort of loses its view of all the combinations of things
23:02
and it won't be able to do its caching and scoping properly as well, etc. So you have to be a little bit careful with using that escape hatch, I would call it. I wasn't familiar with the parametrization that you could do with fixtures.
23:22
Does that play well with the other parametrization that you can apply two tests directly? Yeah, so the question is, does the parametrization on fixtures, that you can do on fixtures, play well with the parametrization you can do on tests directly? So on tests directly, in case you don't know,
23:41
it's basically another marker, so you can decorate the test at pytest.mark.parameterized and then give it a list of its parameters. Yes, those two basically work together. When you do, so when you're requesting, so when you use the decorator parametrized,
24:04
you have to use the request and request.param again to get hold of the value that's being parametrized. So when you're requesting request, which is already going to be parametrized, and other parametrized fixtures, you're just going to get all the combinations again.
24:22
So the question is, when pytest and coverage are used together,
25:00
if you import stuff at the conftest.py, in the conftest.py file, it tends to be skipped by coverage.
25:08
And someone is nodding yes in that. The problem is, the plugin is at the bottom, the coverage plugin. What I'm here just doing is using coverage groups to run pytest
25:21
and then there is no problem at the import level. Because the problem is that if you do something in the fixture, then that's not in coverage. When you use the plugin. Yes, so the plugin itself kind of can't catch that as well.
25:50
OK, so the new pycoverage plugin should be fixed.
26:04
So it should just work then, basically. All right, well, if we're done then, thank you again, folks.