My name is Miloslav Poyman. I'm at home here in Prague. I work for Pugh Storage, which has a large and nice R&D center here in the city. And I have been using Python since the version 2.4. So I would like to share with you a piece of what

I have learned since then. Today, we will talk about testing. I discovered Python thanks to Django. And I mention that because a lot of patterns that we use in Python today, including those I will talk about, have their origins in the Django framework.

So we will adapt them to other frameworks and Python. But first, do you think it's worth to write tests? Of course, there is only one correct answer, especially if you are at job interviews. Which means that people are sometimes

expected to write tests, even when it might not be clear to them how to test something properly. So it's an honest question. Is it always worth to write the tests? And don't get me wrong. I'm a big fan of automated tests. But I want tests that I will understand that won't

be a pain to maintain. A well-known recommendation is to test the interface, not the implementation. And I believe that this is the most important advice for understandable and maintainable tests. The issue is that it's not always so easy to split code into units with clear and stable interfaces.

That's why I like to test either the lowest layer of my code or the highest one. You know, if my was a burger, it would be pretty clear that there is a bun on the bottom and a bun on the top. What's in the middle? Nobody knows.

It's more options there. So today, we will test the top bun. For web apps, it's usually some API endpoint or RPC function. And such endpoint has to have a stable API. Because if you change it, you will break your clients. And the dependencies are quite well-defined too. It's usually some SQL database.

And even when an app is a little bit messy, looking at the data as they are stored is usually self-explanatory. And if there is some other API that you are calling, that's also something given not changing often. So the whole endpoints are a great unit to be tested.

They have clear interfaces that do not change often. You may complain that this won't be a proper unit test because the whole point is too large. Maybe. OK. I won't argue with you. The point here is that I won't test something

well-defined in isolation. And the patterns I will talk about will help you to test high-level code in isolation. This is essentially an outline of my talk. We will approach this from the front. How can we call the application?

Web frameworks typically offer you clients, usually with an interface similar to the famous request library. In PyTest, we get the client from a fixture. If you came to this talk, I'm sure that you know what PyTest fixtures are,

so we won't talk about it. And you will see many examples of them here. I will use fast API in my examples, but that's not really important. The patterns are quite generic. It would be very similar in other frameworks. So my client fixture takes an application instance as an argument.

And if you know PyTest, you can expect that this argument will come from another fixture. I recommend having a fixture returning an application instance, not because of the value it returns, but because it's a great place how to customize

your app for testing. Whenever some test is calling my app, I can replace dependencies by mocks or something similar and restore everything afterwards. And all these overrides are at one place, so I won't forget to patch something.

I will get to this later. Now I would like to talk about authorization. When we call our application, we often face the issue that the endpoints need some valid credentials. So I like when my client has a login method, which

I can call as part of test setup. Still, maybe one more note, here is how to write that client. Just take the client from your framework,

extend it, and write method. This example sets an HTTP header, but it could be cookies, HTTP out, whatever your application is using. This login method is nice, but it can be still annoying to call it in every single test, especially

if a group of tests is not related to authorization at all. In such case, it would be nice if we got some valid credentials automatically. You probably know that pytest features can be auto-used, but that's not something I want, because I want them auto-used only somewhere.

That's where pytest mark.use fixture can be useful, because I can auto-apply a fixture to selected classes or selected files. The login fixture might look like this. We create a new user instance, pass it

to the login method we saw before, and that's it. When that auto-use pytest mark.use fixture is there to enable the fixture, it's not really important what we return from here, but I still return the user instance, because it can be useful elsewhere. Like here, it allows me to customize permissions

of the logged in user before I do request on their behalf. See how pytest gives me the user instance I return from the fixture. So I think we got pretty far here.

We have a fixture returning our application as a point where we can customize the app for testing. We have a client that allows us to call the application. We have a login fixture that logins the user. And in our tests, we can customize

permissions of the logged in user before we make requests using our client. I promised you to talk about isolating dependencies. And the most common dependency for an application is definitely some kind of SQL database.

You can find recommendations to introduce some kind of wrappers, hexagonal architecture, something that allows you to mock the DB out. But let's face it, a lot of apps are essentially a wrapper of a database.

So if you mock the DB out, there won't be much left for testing. And I really like how SQL is powerful. I don't want to replace it by some stupid create, update, delete methods. That's why I believe that it's usually best to test apps with real databases. For proper isolation, we can start a new database

in Docker, for example. And it should take more than a few seconds. And few seconds are fine if we pay this overhead once per the whole, like, for all tests run. So to isolate us, we can utilize database transactions.

If we roll back after each test, our database will stay in clean and predictable state. Here is an example for SQL. I have a fixture that connects to a database, begins a transaction, yields so that it allows the test to run,

and rollbacks the transaction afterwards. And this gets even better. Because if you use SQL alchemy, you probably use it because of its ORM layer. I can initialize an ORM session with the connection

from the previous fixture and tell the session to emulate transactions using save points. That's a built-in feature of SQL alchemy so you get it for free. So now I have a pytest fixture returning an SQL alchemy session, which behaves like any other session,

but nothing is actually persisted in the database. So the tests are isolated. The last step we needed is to configure my application to use this special session. And if you remember my app fixture, that's a perfect place to do it. Whenever my app will need a DB session,

it will get this one before the transaction. Of course, there are no server bullets. I recommend you to start with these transaction bound tests because you configured them once and suddenly everything touching the DB is much easier to test.

Sure, if you want to test something related to persistence, you might need a proper integration test. Or on the other side of the spectrum, if something can be decoupled from the storage, fine. It might be good to have some abstraction. But I will talk mainly about the transaction bound test

because first, I like them, and second, I have one more pattern to share. What concerns me is when the empty database is not really empty. It usually starts with an innocent idea. Just place there some test data to play with to test them. Unfortunately, every test needs like different data

so you add more and more rows there, and suddenly you have no idea what's in the database. And it's difficult to reason about such tests because I don't know, without looking at the global data, what should be in the expected result, whether something is there because it's testing something

important or because it appeared from some other test completely unrelated. And it can happen to you that you will be afraid or maybe lazy to write more tests. Because if some special case needs some special data, those special data aren't there, you would add them, it will break some existing tests. You don't want to go into there and better not

write a test at all. So the proper approach is to prepare new data for each test. It's a little bit more chatty, but I promise you it's really worth the effort. Just write a factory function, a helper like createUser here

that will create whatever object you need. Explicitly pass it every argument that is important to your test and let the helper use some reasonable defaults for others. When I see a test like this, it's appearing to me why Alice is in the result and Bob is not. So I know what the test is supposed to cover.

A small issue here is that you will likely have to pass a database object to all these helpers, mostly our special DB session. Explicitly is better than implicit and globals are evil, you know, you know.

But if this explicitness has to be repeated in hundreds of tests, it's a little bit annoying. So how could we get rid of this DB session everywhere? A pattern that you might consider is to define factory functions as pytest fixtures.

My test can have a parameter called createUser or createOrg and pytest can give me such function if I define a fixture returning that function. Here is an example. The end goal is to create a user, a row in a database.

That's what's at the most in the level. The user, the row, is created by a factory, by a helper, by a function. That's what's at the middle layer. And at the outer layer, there is a pytest fixture function.

It returns the factory method so that the factory method will be available in tests. And this is useful because pytest fixtures can depend on each other. And that's how we get the DB session inside without having it to pass from every single test.

It can be useful also for models with relations between them. If, let's say, every user has to belong into some org. But I'm testing something unrelated to orgs. Then I would like if my createUser fixture gave me some anonymous non-important org for me.

In such case, I can make my createUser fixture depend on another createOrg fixture. And if no override is given, I can call createOrg and get the value.

With this approach, you may end up with factory function for each of your models. At some point, your test might be asking for many, many different factories if it depends on many objects.

If this get files, I recommend replacing functions by objects. I can have one object with method for every object I want to create. And this is also much easier to type and annotate. Which approach is best?

As usual, it depends. Do you need something ad hoc in one test only? I would not bother with fixtures, helpers, whatever, just created in that test. Is the test too repetitive, too long? Introduce a helper. Are you annoyed having to pass everything again and again

to all the helpers? Convert it to a fixture. Do you need a fixture elsewhere? Move it to Conf.py and maybe convert it to object to make it more official. And don't remember that not all fixtures have to be in one Conf.py. You can have local fixtures for directories, for example.

And finally, if something is really needed in most of your tests, maybe it wasn't that bad idea to place it into the empty database. Testing with the SQL database was possible because I was able to run it in Docker

and I used that trick with transaction. So I was cheating a little bit. So what about other dependencies? For services, external APIs, strange databases, LDAP, Hadoop, AWS, you name it. What I would like to avoid is monkey-patching

and the use of magic-mock from the standard library. And I will be completely transparent with you. I'm afraid of monkey-patching. It's so fragile. It depends on the exact way how your code calls something that you don't even own.

It can break even if you change how you import your code. And when a patch is not applied properly, you can end up calling production APIs from your tests. And the magic-mock gives you false confidence. You can see 100% coverage, but the magic-mock silently ignores all the garbage it's getting.

And I should not forget, it's simply ugly. And once you patch once, people start copying this ugliness to more and more places and spreading. So what to do instead? Books say that you should introduce an interface.

The interface specifies what you need from a dependency. Then you can have two implementations, a real one for runtime and a fake one for testing. From my experience, wrapping dependencies usually helps with readability.

So even if we didn't talk about testing, I think it would be useful. Services and APIs tend to be generic. So when you wrap an API, you document what you need from it, and you have a place where you can hardcover your stuff or fix various stuff, everything related to that third party.

And when you own the interface, it's much easier to write some fake implementation. The fake implementation can return something hardcoded or use some simple in-memory storage, something simple. Check the example on the slide. It seems trivial, but if I get a joke about Python somewhere,

I know that my dependency was called and what the argument was. Whatever fake implementation you choose, it will be much easier to write it into a proper class than it was when you were trying to squeeze it into some ugly one-liner when monkey patching.

With the fake implementation, I can get rid of magic mocks. But I haven't still addressed the issue how to avoid patching in tests. What I usually do is that I define a pytest fixture returning my fake implementation

so that it's available in tests or other fixtures. In the peripheral world, the code would use some kind of dependency injection, so I would just inject this fake instead of the real object, and everything would work automatically. In the real world, some kind of overload might be needed.

I have already shown you the trick with a fixture. We can discuss whether this is a dependency injection or patching, but the key here is that this happens at one place globally.

So even if you had to fall back to patching, it might not be that bad if the patching stays in one place, usually a pytest fixture, and if the patch is applied to our wrapper, meaning to code that we own and that we have under control.

The fakes are great because you can test most of your code without real dependencies. Yet we should still ensure that real API's calls made properly are working.

An answer to that are integration tests or contract tests, but these tend to be slow, often require network or some credentials, can be flaky. So we don't want to run them as often as other tests.

Pytest.skip can make some tests optional. In the example, when a configuration for the test for my API is not set, I skip the test. And because pytest.skip is in the fixture,

I can skip all tests that depend on this fixture. And I like this pattern because I don't have to argue whether something is a unit test or integration test or something in between. I have everything at one place and whatever is configured is run.

The last pattern that I would like to show is my favorite one. I can have two fixtures returning two different implementations of one interface. So I might want to test something with both of them. And that's where parameterized fixtures can be useful.

You likely know parameterized tests which run multiple times for multiple values. Parameterized fixtures return different values for different parameters, so all tests using them run for those different values. The syntax is a little bit cryptic.

Don't worry about it. I don't remember it either. You can always copy paste it. Important is what you get. You can write a simple test function which expects some implementation of an interface. Looking at this test alone, I cannot tell that it will be run more than once.

Pytest will do the wiring for you and will run it with all the values returned from the fixture we saw at the previous slide. And this even works with the pytest.skip. So I can have tests that run with the fake implementation by default

and if I export some environment variable, I can run them with both the fake and real APIs. There is so much to talk about. I have skipped a lot of slides or had to delete one. So consider all of this as hints where to start, like doing cursor search, what could be in your tests.

And my recommendation is do not bother yourself too much whether something is a proper unit test or something else. Don't rewrite your app just because somebody told you. Use tools, whatever, what tools help you to test something.

Of course, the best tests are the fast one running in one process, what most people consider unit tests. But when testing real apps, it can be more practical to use a real database even when other dependencies are mocked or replaced by fakes.

And I have shown you tools how to assemble everything. And we should have some tests testing the real APIs, other services. And we saw how to run them conditionally.

If you want, like, if you are taking photos of the code snippets, you can find them at the link that is here. And the better the audience is, the faster I am. So I think I have five minutes left for questions. Thank you very much.

Okay, thank you, Miloslav. So if you have any questions, there's a microphone in the middle. Please go behind it and ask away. In the meantime, I'm going to ask one surprise question. So how to make tables in those databases?

Yeah, I'm surprised about this question. That's something, that's a good question. I haven't time to, like, include it in the slides. Yeah, there is this connection fixture that, like, runs for every test. What I usually do is that I create a session, scope session, meaning that it's running only once. And I make my database connection depending on the whole database fixture.

So whenever something touching database is running, the database is created. If not, the database is not created. Brilliant, thank you. And we have a question from online sources. Did you consider libraries like VCRpy to record network interactions instead of using fakes?

If yes, what's your opinion? I don't know that library exactly. And I'm sure that this has its place in testing, like, something specific. But it's, like, when I'm testing, like, business logic of my app, depending on some exact network traffic,

it's like doing, like, asserts that you get the exact SQL that you expect. So I'm sure that it's useful for stuff, but I don't use it for, like, most of my apps, because it's kind of low level for it.

Okay, thank you. We've got a question from the audience. So you said that to test external sources, we could either mock things, but, like, is it always that straightforward? Because, for example, if you're testing OAuth2 kind of thing, where you expect that there's going to be a redirection to your service and everything else, and you expect a certain format of the JWT token that you get,

do you think it's worth the time to invest in testing those? Maybe OAuth is a little bit different example, like, different case that I was talking about, because OAuth is, like, from client side, and essentially the client is doing something,

and it's, you know, it seems to me more related to some, I don't know, Selenium test or browser test, which definitely have their place there. I mean, like, depends whether my application is authorizing itself to some other API, that it could be covered or somebody is being authorizing to my API.

Yeah, but, like, so you mock the front end who will make the request to your back end to authorize, and in that situation it's quite complicated because you also have, like, an identity provider, like, I don't know, Azure, AWS, or someone else.

Okay, maybe what I should have made more clear, like, when I was talking about this authorization, it's when I want to address something that's not related to authorization or what's related to authorization but not to authentication. Essentially, I want, like, you know, you have one endpoint that does,

I don't allow, I don't know, users without passwords in. That needs definitely some special cases for that one endpoint for authorization. I haven't talked about it. That should be there. On the other hand, there is, like, hundreds of other endpoints, possibly, that just test, like, yeah, admin user can do this, employee can do this,

outside user can do something else. And at that point, I'm fine with, for example, hard-keeping secrets into my test because I'm not testing that insecure passwords don't get there. I'm just testing that whenever, if authorization is working, if authentication is working, then I want to test the authorization now.

Okay. Thank you. We've got one more from online. Do you have a pattern at hand for testing an async fast API client route requests? Lately, I tried to research for one but ended up with a solution that seems deprecated partially. Async?

Async fast API client route requests. Like, you can write async tests, but what I usually do, and I saw that there are people who recommend to do it the other way around, but even when my application is using async endpoints,

I call it, like, synchronously because I, like, in tests, I don't need to do the asynchronous stuff. I just do the request and, like, an example for a fast API, I know it's working. I write, like, async dev some endpoint, call it from the client with, like, client.get default await, and it just works.

Brilliant. Thank you. So, this is it. A big hand for Miloslav.