Proper Django Testing
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 132 | |
| Author | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/44918 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
00:00
Statistical hypothesis testingExecution unitData structureSoftwareStatistical hypothesis testingCodeSoftware developerDigital filterObject (grammar)CASE <Informatik>Reverse engineeringEmailSynchronizationSystem callDigital rights managementTask (computing)Service (economics)Dependent and independent variablesClient (computing)Group actionEndliche ModelltheorieNP-hardModul <Datentyp>Data modelFactory (trading post)Meta elementInternet service providerDisintegrationLinear regressionVisual systemFacebookLink (knot theory)Shared memorySet (mathematics)Projective planeGroup actionPhysical systemView (database)Point (geometry)Line (geometry)Context awarenessCoefficient of determinationCategory of beingDot productForm (programming)Lattice (order)Drum memoryMultiplication signNetwork topologyData storage deviceCASE <Informatik>Open sourceDescriptive statisticsSystem callDatabaseCAN busBoundary value problemDivisorExecution unitBitSheaf (mathematics)ArmProduct (business)Object (grammar)Utility softwareData structureInformationFunctional (mathematics)Computer programmingService (economics)Cellular automatonFilm editingAreaError messageComputer clusterLogicMereologyPrisoner's dilemmaBit rateFitness functionForcing (mathematics)Roundness (object)PRINCE2LoginStatistical hypothesis testingDistanceHoaxIntelligent NetworkMoment (mathematics)WebsiteEstimatorParameter (computer programming)Water vaporQuicksortCodeInsertion lossDifferent (Kate Ryan album)Computer-assisted translationOrder (biology)Source codeArithmetic meanDependent and independent variablesProcess (computing)FamilyPhysical lawTelecommunicationEndliche ModelltheorieLink (knot theory)Data miningBridging (networking)Library (computing)Chaos (cosmogony)Keyboard shortcutState of matterCore dumpInternet service providerFactory (trading post)Business modelSoftware developerSoftware engineeringMixed realityFile viewerTask (computing)Connectivity (graph theory)INTEGRALSpecial unitary groupDirection (geometry)Goodness of fitSampling (statistics)GenderAnnihilator (ring theory)Limit (category theory)ResultantShift operatorHuman migrationVideo gameLevel (video gaming)PlanningFerry CorstenMachine vision2 (number)Instance (computer science)Metropolitan area networkValidity (statistics)Coma BerenicesFormal verificationAdditionParticle systemMixture modelImplementationUnit testingMobile appDirectory serviceSoftware maintenanceStatistical hypothesis testingClient (computing)AuthenticationSlide ruleData managementLinear regressionString (computer science)Serial portVarianceRandomizationSoftware frameworkProper mapSinc functionAbstractionFront and back endsDatabase transactionVisualization (computer graphics)Software bugScaling (geometry)WritingNP-hardVariable (mathematics)Attribute grammarIntegerComputer animation
Transcript: English(auto-generated)
00:00
Hello, everyone. I'm Martin. I want to introduce myself first. I'm Martin, I'm from Bulgaria, which is this small country over there, if you haven't looked at it. My home town is called Burgas, and it's on the coast of Black Sea, so if you don't have any plans
00:20
for the summer, welcome. I work at Hotsoft, which is with these awesome guys. We are a sourcing company, and we mainly write Django and React. We also organise a hack-off, which is a conference for programmers, yearly, and it's taking place in the end of September,
00:41
so if you want any other information, meet me outside. Also, I've just graduated as a software engineer in the last two weeks, so make some noise! Yeah! Thanks. Okay, let's
01:01
dive in. Today, we are going to talk about testing in Python and Django, and the talk is mainly about unit testing since it's a beginner's talk. We're going to talk a little bit on Django substructure, and, in the end, we're going to finish with some other testing methodologies. So, before we start, I just want to move you a little
01:25
bit better, so how many of you have written Django? Raise your hands. Whoa! Fair amount! And how many of you have written a new test in Django? Cool! I can go out now. Okay,
01:40
let's start with unit testing. What is unit testing? We have some principles and advantages in it, and the first one, of course, is that one test should test only one unit. All our unit tests should be easy to read, and fast to execute, and should be run in isolation. The main advantage about unit testing is that it helps us modify our business logic,
02:06
and they serve as documentation from developers to developers. Of course, it has some trade-offs, and the first, of course, is that one test should test only one unit, which sometimes is impossible, and I will tell you why in a few more slides. Of course, when our
02:26
project gets bigger, our test code becomes larger, and it can be hard to maintain. I'm sure all of us have ever had a random failed test in CI, and that's why you can
02:41
push the production. But the mocking, which is the main part of the unit test, to me is a disadvantage, because it may lead to a regression, and it may cover some nasty bugs. The other problems of unit testing are the developers, because we are lazy,
03:03
because we are developers, and we tend to break its principles, which is bad and leading to non-exhaustive tests. Talking about unit testing in Django's context, there is one main thing. Proper unit testing in Django is bound with proper Django structure.
03:24
If our Django structure is bad, we can't actually unit test properly. Let me illustrate my thoughts with an example. Here is a simple invoice create view. We just create an invoice. When we get to the view, we just render a form, and when we submit a post request, we create an invoice. Pretty simple, right? Here is how the unit test for this
03:46
view looks like. We have a set up, then we test if we can access the view, and then we test if we actually create a new invoice, and direct after it. Okay, cool.
04:01
But the main question here is, is this properly unit tested? Let's raise hands again. Who thinks this is properly unit tested? Okay. My bad. Is this properly unit tested? Yeah.
04:23
Okay. Cool. So, the main problem here is that we have to assert that it's related to HTTP response request cycle, and we actually test the status calls of the response, right? And then we have a unit test that tests if an object is created in the database.
04:44
Is there anything in common between these two things? Except the CRUDs, I guess. Your database is an external thing, so you're now not testing just that unit. Yeah, that's the main thing. And the problem is that Lanyon does too many things,
05:01
right? We have a view that's capable of the response request cycle, and calling the ORM, which has nothing common between them. And in the real world, actually, this example is pretty much more complicated, because when you create an invoice in 2018, you don't just create an object in your database. You send emails, make some validations,
05:24
call third parties, and so on, and so on. And the problem here is where should this logic live? And here comes the services concept. A colleague of mine, Rado, gave a talk about it a few hours ago. It was pretty awesome. I'll put a link to the
05:41
slides if you haven't been there. So, what's a service? A service is actually a functional object where you put all your heavy lifting business logic. And why should you do it? It's because it gives you reusability. You can use your services in APS, in views,
06:01
in other services, and so on, and so on. And the next pretty good thing is that it gives you a unit distinction. Let me show you the next graphic. Now we have view, which just communicates with the HTTP response request, the client calls our view. The view on its side calls a service. And the service calls a Django
06:24
or a task, calls other service, makes some validations, and so on, and so on. This maybe looks like this unit, the service unit, actually do a lot of things. But if you look more abstractly, it's just our business logic. And now we can put the line
06:41
between them and define actually a view unit and a service unit. The view unit is responsible for the HTTP response request cycle, taking the request, running a template, and calling a service. And the service on its side is responsible for actually heavy lifting logic. Here is how our service look like. This is pretty
07:05
dumb example, but it made me think. We validate them out, create an object, and kind of call a task, for example. This is our view now. It's pretty much the same, because of the sake of the example, but we just, in the form valid, if you see, we just
07:25
create, we call the service now. Now I can create an API view, which really easily, again, just calling the service and giving a serializer response. That actually allows us to unit test properly now, because we can distinguish the units.
07:46
Here is the invoice create view test. Now it just tests the response code. If you can access the view, if it redirects, and if the service is called correctly by mocking
08:02
it. Here is, for example, a test from the API view test. It's pretty much the same, but I use the client from test plus test case. And here are the tests from the invoice service. Here I don't have enough place in the device, but the main point here is
08:28
that we have, we have distinguished the units. We now test if we create a new invoice, we test if we call the task by mocking it again, we test if validation is not correct
08:46
or validation error is raised. We test if we, we shouldn't create an invoice, right? We test it. Going away from the example, we should know, we should ask ourselves, what did we achieve? We just defined the units in our system actually, which gives us straight
09:07
forward boundaries between these units. And this actually allows us to properly unit test them. And this actually gives us maintainability and modifiability in the system. That's
09:23
since we have distinguished these units, we have now purely testing groups. For example, we have test reviews, APIs, services, models, utility functions, and maybe others. Or this is actually how our tests directory in the app looks like. Now I want to point
09:43
out some common mistakes I do as a developer and I recently seen in my colleagues work. The main part about unit testing is that sometimes tests are non-exhaustive. And this is the biggest problem because let's take a look at this example. We have a service
10:02
that is used in a lot of places in our system. One day we decide to change its implementation. We check if it has unit tests. Yes, it does. Are they correct? They are. We run them and everything is okay. We change the implementation, run again the tests, and
10:22
everything again is okay. The CI tells us that actually everything is working. We push the production, of course. And here is where the most nasty bugs come from because we have a non-exhaustive unit test that didn't catch the bug that we introduced.
10:42
And the problem here is that these bugs are usually caught by our users. The next common mistake that we do is to actually overtest or put too many asserts. For example, if you have the create invoice service tests and we have ten different test methods in
11:04
it, we shouldn't put the assert true invoices created every time. This actually breaks the principles. We tend to hard code values because it's just simple. Sometimes it's just simple to hard code one. For example, if you need an integer or some random string.
11:25
But it's bad. The more we get better in Django and in Python specifically, we tend to add too complicated logic in the tests. In my opinion, we should have better have two inner force in the test method instead of putting some abstraction over it because
11:46
it's just easier to read. And when I go to the unit test, I see okay, now I have two force, two loops, and it's just easier to read. The next bad thing is to have misleading test names and actually test something that is not in the test or not telling
12:05
in the test. And messing up units or not mocking. As you may have noticed, I've mocked the validation service every time I test it in the service. And that's actually how you should do your unit tests. You should just if you don't mock the services or
12:25
the utilities that are caught inside your testing service component, this actually breaks the isolation principles. And the question is how to get along with these problems. The first thing is to keep up with the principles of unit testing. This is how
12:43
our project should be how our work should go. The next thing is when you code review the code of your colleague. Don't only review the parts of the code that he's written. Also review the unit tests. That's the place where we actually caught a lot of bugs. And
13:03
again, unit tests should be simple. They should look like a documentation in my opinion. So keep them simple. Now, let's talk a little bit about the tools we use. Of course, the first thing I want to mention is the test models. And the first thing is the
13:22
unit test of Python. It's such an awesome model. It actually gives you everything you need to create the unit tests you need. The main part of it is the mock model which gives you the decorators, magic mock instances and a lot more. And the second main part
13:41
in my opinion is the test case object. From which you inherit and it gives you the setup method, the tear down method. Another nice feature of test case model is the subtest context manager which I really like. And we'll show you an example in the next slide.
14:01
The next test model we use is the Django test. It doesn't give a lot more, for example. But it's a nice wrap around the test case. And the test case from Django test is actually the thing you should use if you want to communicate with the database. Another nice thing is that it gives you some nice shortcuts with the search so you can easily search your responses,
14:28
for example. And the next thing we tend to use is the test model of Django. It's a site package, Django test, but it has a test case wrap around the Django test case.
14:40
We really like it because it has such a nice API to use. Now let me show an example for the subtest context manager. Here is how two tests actually look like from the previous example. And with subtest context manager, we actually can combine them and it can lower
15:05
the code variance. Here is how it looks like. It's pretty much the same. You just post and check if you're directing, if you created a mock, if you called the service.
15:21
Another nice thing is that it gives you these strings in the context manager as an argument which can serve as documentation, as I tell. Something to note here is that don't overdo with subtest because this may lead into too long and hard to read test.
15:43
If you, for example, enter a subtest with a subtest in it, and the subtest in it, you'll get three levels of going in and to me it's just hard to read this. For factories, of course, we use Factory Boy. I think most of us do. It has such
16:03
nice integration with Django models actually, the Django model factory. I think I want to mention here is the lazy attribute. This actually will create different variables for the same test run of your object. For example, if you want two invoices in one
16:22
test case, if you don't put the lazy attribute, both of the invoices will have the same amount. Another cool thing here is that it gives us a lot of customization because if you want, for example, to use the create invoice service here, we can just predefine the dash create, underscore create method of the factory.
16:47
For fake values, we use Faker. Here is how we can customize its methods. You just give another provider, the so-called providers in it. For example, I want all my
17:02
pience to be positive, so I just change the Faker pience to my pience provider. For test runner, we use PyTest. We don't have enough time. I don't have enough time to tell all the features that PyTest give you. Go to the documentation, check it,
17:21
but let me just, here is how most of the my tests runs look like because of these arguments. The create dash dash create DB creates a new database and puts all your, runs all your migrations again. If you use the dash dash reuse DB, the migrations won't
17:41
start again. This actually really fastened the developer process. The dash dash stays for last failed, and it will run your last failed test. Another nice argument I've used is the dash dash duration, where you give an integer and it will give you,
18:01
for example, 10th most slow test you have. That's nice for monitoring. You can now use the PyTest markers that I won't mention now, but you can just put a marker in it over with the decorator. It's such an easy to use and well documented in the documentation. Marker tests are slow and don't run it every
18:21
time in the CI. Now, I have a couple more minutes. I want to go more further and talk about, talk a little bit more on some other testing methodologies.
18:40
Okay, unit testing is awesome. We should do it a lot, but it has its limitations. When you go to the boundaries of unit testing, we should take a look and go with some other different testing methodologies. As a developer, I'm not really experienced with them. I'm not a QA guy, but recently we needed such tests in our project because
19:07
it went quite big and we wanted to be sure that some crucial parts of the system are working. Well, the crucial parts of the system were actually bound up with third particles,
19:21
so we used some kind of a mixture of mixing end-to-end tests with validation testing. For those who don't know what's validation testing, it's actually a testing methodology that verifies or validates the cost from your system with the third parties.
19:48
Let me tell you the approach we took. We decided, as I said, to use our end-to-end tests with validation testing, and the one thing here is if this approach works
20:02
for you, it would be awesome, but if it doesn't, and you decide to take something else, the first thing you should ask yourself is in what state is my project. In our case, the front-end is now heavily developed, so it didn't really make sense to actually do
20:20
the whole end-to-end test part or create a visual regression test because we will end up with deleting them because the design would be changed after a month, for example. What we did is to actually create a new project. We call it the client because it just calls our APIs, and its job is to, as I said, call our APIs. These APIs should call some
20:47
third parties, and the client asserts that the response is from the third parties and everything is synced. That's pretty much everything it does. The thing that we need
21:02
to do now is to put it in the CI because it's fully dockerised and easy to use, and I think that's pretty much everything I wanted to tell you for today, so thank you.
21:23
Sometimes for questions. If you have any questions, I'll ask you to come forward to and talk to me. Any questions about testing?
21:42
I'd just like to come back to the different frameworks for testing, like the different libraries. Are you really using... Distinct? Yeah. Okay. Yeah, it's like unit test plus Django test plus PyTest inside one project, or can you re-explain briefly what is better for what? Which one is better for working with database
22:10
and Django ORM, and which one is better for... To me, Django test case, for example, the Django test case actually scales from simple test case and transaction test case. The transaction test case is actually capable
22:26
for all the ORM stuff, and I'm not 100% sure, but I think the test plus test case actually don't give you any other abstraction over this. So it's the same, actually.
22:50
Thanks for the great talk. From what I understood, in order to have good unit tests, you need to have a really good project structure. Is that right?
23:02
100% in my opinion. So what if you don't have a good structure? How can you write good unit tests? You can't, and that was the point of the talk, actually. Okay, thanks.
23:22
So maybe a little follow-up question. You were talking about writing the unit tests each time for a single unit. So supposing this structure of a Django preg, where you go from the views on one side through API services models to the database, what would
23:42
be the... Can you give some examples of the components that you're testing? Yeah. Actually, what you saw there with the test of the views, if you have this structure, one nice thing is that all of your views, this would look something like
24:05
this. If we have... We actually put our permissions and authentications usually in a mix-in. So you can test if you, as an admin, I can access this view or something like this, but it's pretty much everything. And when you have mocked the service call,
24:25
that's the only thing. Another thing that your test for the views should look like. Actually, the heavy lifting business logic is tested only in the service, and that's the main part. Again, here comes the problem that you mock every communication
24:44
with the outside of the function, but this is how you should unit test. For anything in case, just use end-to-end testing or some kind of integration, if that answers your question. Okay, we have time for two more questions. If anyone has a question, please
25:02
come forward. If there are no further questions, then thank you, Martin, for your talk.