BDD (Behavior Driven Development) Testing for Django Apps by
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 50 | |
| Author | ||
| Contributors | ||
| License | CC Attribution - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/44048 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
DjangoCon US 201838 / 50
2
5
7
8
13
18
20
21
22
35
41
44
45
47
00:00
Binary decision diagramMobile appStatistical hypothesis testingTelecommunicationFocus (optics)SoftwareSoftware frameworkTable (information)String (computer science)Data structureDescriptive statisticsIntegrated development environmentStatistical hypothesis testingPresentation of a groupDifferent (Kate Ryan album)Hecke operatorProcess (computing)SurfacePrisoner's dilemmaWordSoftware frameworkMultiplication signGoodness of fitVideoconferencingGreen's functionMetreFormal languageTelecommunicationStatistical hypothesis testingTable (information)Software developerLevel (video gaming)Product (business)Speech synthesisInformation securityMereologyAreaGender2 (number)MassBinary decision diagramMobile appComputing platformReal numberTask (computing)Inheritance (object-oriented programming)Computer animation
03:17
Binary decision diagramThermodynamisches SystemWeb pageData structureDevice driverGraphical user interface9K33 OsaWeb browserCodeBuildingStatistical hypothesis testingComputer fileReduction of orderGroup actionPasswordLoginData managementData modelObject (grammar)Element (mathematics)Linear codeMultiplicationGroup actionIntegrated development environmentWordDevice driverWeb pageBinary decision diagramGraphical user interfaceData structureStatistical hypothesis testingSoftware frameworkComputer fileDatabaseWeb 2.09K33 OsaElement (mathematics)Web browserSlide ruleRevision controlImplementationCodeLoginPiObject (grammar)Data managementFactory (trading post)PasswordQuicksortScaling (geometry)Level (video gaming)Casting (performing arts)Order (biology)GenderException handlingPopulation densityINTEGRALDirected graphData storage deviceComputer clusterTask (computing)Process (computing)Service (economics)WeightComputer animation
07:28
Web pageVideo gameLoginWeb pageIdentifiabilityStatistical hypothesis testingElement (mathematics)DebuggerGroup actionoutputPasswordOrder (biology)WordArithmetic meanLevel (video gaming)Object (grammar)Message passing
08:18
DatabaseCodeDemo (music)Binary decision diagramData managementStatistical hypothesis testingWeb browserStatistical hypothesis testingElectronic mailing listStack (abstract data type)ExplosionSystem identificationClient (computing)HTTP cookieLogicNP-hardComputer configurationMotion captureHTTP cookieBinary decision diagramClient (computing)Point (geometry)DatabaseMobile appEmailLimit (category theory)Memory managementActive contour modelStatistical hypothesis testingStatistical hypothesis testingDemo (music)CodeUser interfacePasswordTable (information)DataflowCondition numberAsynchronous Transfer ModeOrder (biology)Descriptive statisticsParameter (computer programming)Row (database)Web pageMessage passingWordThermodynamisches SystemDifferent (Kate Ryan album)2 (number)LogicComputing platformData managementMatching (graph theory)MappingShared memoryWeb browserInteractive televisionSoftware bugVotingStack (abstract data type)Group actionComputer fileSemiconductor memoryServer (computing)Finite differenceMereologyProcess (computing)RepetitionDiscrete element methodUsabilityQuicksortKey (cryptography)GenderArmDevice driverDemosceneForcing (mathematics)Complex (psychology)Multiplication signGreatest elementStaff (military)WritingCASE <Informatik>Task (computing)Inheritance (object-oriented programming)FreewareTrailFamilyWeb 2.0Control flowComputer animation
14:53
CodeStatistical hypothesis testingSource codeSynchronizationWeb pageRevision control2 (number)Software frameworkMultiplication signElement (mathematics)Binary decision diagramActive contour modelVulnerability (computing)Set (mathematics)Combinational logicUniqueness quantificationQuicksortINTEGRALLatent heatCovering spaceComplex (psychology)Software design patternUniform resource locatorDebuggerLoop (music)Link (knot theory)GUI widgetWeb browserBitError messageGraphical user interfaceEmailSocial classSlide ruleCross-platformUnit testingStatistical hypothesis testingMixed realityInformationMeasurementDescriptive statisticsSubsetUser interfaceFront and back endsMetropolitan area networkCuboidHome pageFreewareComputing platformFamilyPrice indexVirtual machineDistribution (mathematics)Computer virusProcess (computing)Message passingMedianSurfacePattern languageFlow separationProper mapSoftware developerFile formatLecture/Conference
21:28
Roundness (object)Web pageStatistical hypothesis testingRight angleTask (computing)Lecture/Conference
21:55
Coma BerenicesInternet service providerData typeXMLComputer animation
Transcript: English(auto-generated)
00:16
Thank you so much, thank you so much, Dolan. So, my name is Lure, and this is my first time
00:23
to give a speech in Django Core. I'm super excited. So, today's topic is about a BDD test in Django app. So before we head into our first slide, I'd like to ask you guys a question. So how many of you have no BDD, or even say, applied BDD in your development? Just put up your hands real quick.
00:43
Good. Looks like a San Diego BDD has a green market in San Diego. So let me do a quick introduction for a BDD. So, BDD represents for behavior-driven development. So, it is a methodology which is designed for improving the communication
01:00
between the tech people and the non-tech people. So it's a purely business-oriented, and it's user-focused and can be easily understood by the non-tech people. So, with this mass storage, why we need a BDD? So, because this is behavior-driven,
01:20
so it's user behavior-focused, and it focuses more on user behaviors than just functions, and also it focuses more on software development rather than just testing itself. And also because this behavior-driven development comes with the description, that means a description can be applied
01:40
to different platforms. I'd like to break down my presentation into four parts. So first, I'm going to introduce how we set up our BDD framework in the real production. And the second is on how we integrate our BDD with our CI-CD process. The third thing is how we do the refinement,
02:02
so how we just speed up execution for this testing. And the fourth thing is, we are going to describe what's an invitation for the BDD. So, before we set up our framework, we need to collect our requirements from our non-tech business people. So, we are using Gherkin language
02:20
to write down your description. So, Gherkin is a language built on the keywords. So, as you can see here, we have three levels, feature, scenario, and step. And each level will have its own sub-keywords, so which can make it executable.
02:42
This is a small example. So, as you can see, the feature is at the very top. You have your feature name. And then, below the feature, you're going to have your scenario name. And within the scenario, you're going to describe how you're going to execute your test. And here, I also like to add some tags for scenarios
03:01
so that I can just skip some tasks I don't want to run. And also, you can use the data table to have your own specified parameters, or your own predefined data. So, now that we have our requirements ready, we need to set up our BDD. So, we need to get some tools for our BDD.
03:23
So, you already, except for your feature files, you also need a scenario, which is for manipulating your browsers. And also, we need a thing called Behave. It's a design framework for BDD in Python. And thanks to our Django committee,
03:41
we have a Django test integration, which is called Behave Django. And the first thing is, we need the same kind of page objects. It's a methodology for just gathering all your DOM elements in your page, and it's easier for you to do the management for your elements.
04:00
Now, we now can get our hands dirty now. So, first, install all the dependencies I mentioned in the slides, and Selenium, Behave Django, and also PDB for debugging. And then, you have to install your web driver. So, because Selenium needs web driver
04:21
to control the browsers, and we need a Chrome driver for Chrome, and for Gecko driver for Firefox. And make sure you have already installed your web driver correctly. Make sure the path is correct, the version is correct. Otherwise, you're gonna have some compatible issues with Selenium.
04:41
And for the Mac users, you're already lucky. You can use Homebrew to install all of them. It's very good. And the third thing is, we need to create our features. So, following the Gherkin style, you create our features. And then, I want to emphasize, we also applied our own step convention. I'm going to introduce that in the following slide.
05:03
The convention here is, we are doing it this way. So, with the keywords you see in the Gherkin style language. And then, somebody do something in the page of the action words. So, the page objects, as I mentioned before, is where you're going to have all your elements ready.
05:20
And then, it's easier for you to maintain and simplify your UI. Of course, and you can reduce your duplication of the code. And the action words is where we have all our implementation for the steps. And also, it's because we want to manage those steps easily.
05:41
So, we're just using action words to represent them. Now, so we can see that there is an example for logging. So, when I log in with the user name and password in the log in page of the log in action. So, that's how the step looks like. With all those predefined stuff,
06:01
we can now build our code structure like this. And now, I just want to point out, if you want to have some predefined data to load in your test, you can use fixture. And second, if you want to do some quick ways, you can just use factory boy, if anyone knows that, to generate some fake data. That's very quick.
06:22
And I'm going to go through those files very quickly. So, first, you need environment pie. So, environment pie is where you configure your test running. So, based on the Gherkin style feature file, you're gonna iterate your running through those levels. So, before the beginning of the running testing,
06:43
and also the feature scenario and the steps. And by default, Behave Jungle have already set up the database for you before all, and then they will flash out to your database after scenario, and they will tear down the database after all.
07:02
For the action words, you need to load action because one feature can have multiple steps, and that's why you need multiple action words. And you need to tell a BDD framework, say, which action words you want to load in. So, based on our convention for the step definition, we just tell the BDD, if we find the keywords
07:22
inside the step, we're loading the corresponding action words. And for the action words, that's where you implement your steps. So, basically, you're just importing your page objects, and then tell the BDD, say, what do you want to perform?
07:40
So, here is an example, I'm importing a login page, and then I tell BDD, say, I want to log in, and input my username and password, and the login. So, for the page object, this is one of the examples. So, you just select all the elements you need to perform your behaviors. So, you have your user field, name field,
08:01
you have your password field, and the selecting method will be ID, cost, name, and expense. And so, if you can talk to your front end designer, just tell them, just give you some identifiers that make your life easier for BDD testing. And now, here is a step height.
08:21
This is where you are gonna match your action words with your step definition. So, that means, actually, BDD has no syntax meaning, just they will execute all the steps in order. So, just have to find a matching between your action words and your steps. So, it's a mapping file. Now, after completing all of them,
08:41
you are good to go, and you can run commands. So, this is how the command looks like. So, python manage pi, it's like you run short server in Django, and then add a behavior option. So, you also can keep your database after running the testing. It's easier for you, the next one, or even, say, you have some same database.
09:04
Now, I also did a quick demo for you, and I created an app based on a Django tutorial, and I run BDD testing against it. So, the demo is about, I just voted for my presentation, and I said, ah, it's pretty bad,
09:20
and I increased the votes for it to increase by one. Okay, so, the whole running just last for three seconds. Remember this time, it's very important later on. Now, we're going to talk about our code delivery process. So, everything will be online, and we need some tools to help us.
09:41
So, first, for the BDD, we need a feature file management tool, which is, we are using heap tests, and for a CI platform, we are using Travis CI, and therefore free for the open source, and also, we need a cross-platform test tool. Here, you can see a bunch of them on the market, and we are using browser stack.
10:02
And also, you are more than welcome to use headers mode of the browser, so NetKend is running very fast. So, PhantomJS, Papader, whatever you like. And then, I'm going to talk about the working flow. So, first, you're running all your tests in your local machine, make sure they're passed. You can run them in Chrome, you can run them in Firefox.
10:21
After they're passed, you can push the code into GitHub, and GitHub is already integrated with Travis CI. So, on Travis CI, we're going to do the parallel testing because we really want to have our testing passed in different browsers, different operating systems. So, this is called compatible testing. And after running all of them,
10:42
we're going to have our reports ready on our management tool. So, once we see our management tool reports, we're going to check out what scenarios I have already filled. I filled, we're going to debug those testing code in the corresponding platforms.
11:04
And then, you do the same thing again, repush it to the GitHub, and close the loop. Now, so, we just did a quick review of the BDD. So, we now just come to the refinement of BDD.
11:21
As you can see, BDD is very costly. Just one scenario costs us three seconds to run. So, we don't want to run everything for BDD. So, first, the principle here is you have to identify your critical test. And the second thing is you have to minimize your unnecessary UI interaction. We're not using BDD to test against the UI designing.
11:46
Also, the third thing is we can play a little trick with that to accelerate your execution. So, we do some prerequisite step by using the test client from Django testing framework, and then pass on the cookies into WebDriver.
12:02
I'll share the code. So, the code here is we are using the client to log in your app. So, you log in with the user name and the password. And then, after completing all your prerequisite steps, you're just passing your cookies into your WebDriver,
12:21
and then use the WebDriver so that the WebDriver will have the session of the server, and then just keep doing the user behaviors. You're gonna perform our page. Now, remember that we have the previous demo, three seconds for run BDD. Now, let's see how fast it can be now.
12:41
So, same thing, log in with the client, and then do the voting. Yeah, two seconds, or around two seconds. And we say one second. Okay, one second really means a lot. If you, especially when you run the testing on the cloud, and you're gonna run those testing on those cross-platform tools, they charge you by minutes.
13:01
So, save time, save your money. Now, we are having our debugging tool. So, somehow, we don't have debugging code. We have our bug in the testing. So, especially, you need to know how to debug it. And by default, behave will capture all the standing out, so maybe you're not able to do some break point.
13:25
And we just need to trace back the stack, the memory stack, to say where it failed. Here's a code for debugging, and as you can see, we want to have this in a debugging mode. So, when you set an option for it, so we say behave debug on arrow, it's turned on.
13:43
And then, if it's failed, here it is in your step. So, after step, you're going to trace back your stack and see where it failed. To check out variables, check out your logic. Yeah, so, that's how the command looks like. So, you need to customize your command to run this testing. So, make sure you have a no capture option on your command.
14:04
Otherwise, you are not able to just capture break points. Now, here's the last part, the limitations of BDD. So, BDD is very good, and it's very user-friendly, but I'm not selling snake oil here. It does have some limitations.
14:22
So, first, the logic of the description is linear. As you can see, they have many keywords, and they execute keywords in order. So, if you want to have some conditions to run BDD, like if-else, so one way for you to do that is you just have a data table ready and iterate the records against your BDD testing.
14:44
And second one, the solution is you just create multiple scenarios and passing different parameters. The second limitation would be it's very hard for you to interact with those complex JS widgets, like a data picker,
15:02
because you need to tell Selenium to click the very specified button, a specified link to use the BDD. So, I do suggest don't bother yourself to how to code interacting with the complex JS widget.
15:25
The third thing is the errors raised in the different, in browsers can be various. So, somehow you may face an error in Chrome, like the element is not available in this page. However, when you do the same testing in Firefox,
15:41
it may raise an error for you, like this element is not interactable. So, I do suggest if you want to test against whether you have this element on your page, don't use errors to raise up. So, just create some, just selecting multiple elements with the same class name or same ID
16:02
and see whether the array is empty or it's not empty. So, that can be very good. Okay, that's all I have today. And this is my email. And feel free to send me any questions. And how many minutes do we have?
16:21
Okay. I had a question. I wanted to know if you could elaborate anymore on one of your biggest gotchas. I know you had a slide dedicated to it, but what kind of vexed you the most with starting? You mean, sorry?
16:40
You had a slide on some of your gotchas or things to note. You're not selling the snake oil. So, I just, I wanted to know one of your biggest gotchas and one of the biggest challenges you had. So, for running BDD, actually, you are just, you are manipulating the elements on your page. And it can have some gotchas when you do the BDD testing.
17:02
Like, the page is already loaded, but your BDD framework didn't catch that elements and it threw out an arrow. So, you do have to set up some waiting time for it. So, that's the big challenge because if you do it on your local machine, it can be very fast. But if you do it online or even push up your CI platform,
17:22
somehow it may not, the elements may not be catch. So, you need to check your network and check out whether this has been captured and you set up your waiting time, like one second or two seconds to help you with that. So, that's the thing. Do you usually use asynchronous waits
17:41
or do you just use sleep or something like that? Oh, yeah, time sleep. So, when you check about this framework they are using for behave, actually, they are setting infinite loop and then if the running time just surpass 10 seconds, they're gonna stop and it's sort of upper narrow. Can you say anything about any sort of design patterns
18:04
between you and UI or front end that makes it easier for you to not test like the UI design while you're making these page locators and et cetera? Yes. So, if you have a very complex landing page,
18:21
you have many animation on a page, it's very hard for you to use BDD actually, yes. So, that's why I'm saying BDD is a methodology for you to have integration testing between your front end and your back end and you just need to see whether there's exactly same on your page,
18:40
the data from API, for example. So, when we do this thing, we usually talk to our front end developer, say can you add some specific cost name, ID, it's easier for you to do the testing because even though this BDD can support X-Pass,
19:00
the X-Pass can be very slow if you do the BDD testing so I do suggest you can use ID or use a cost name in your front end and can be very helpful for BDD. So, we use a little bit BDD but a lot of times there's problems with the versions between Chrome and Firefox and whatever we're using for CI,
19:23
how do you handle that? You mean how to handle the errors in Chrome, Firefox? Like, that regards to versioning, like how do you either keep everything up to date or keep everything into one version or like the whole sync with Selenium because it's really...
19:41
So, that's why we use the cross platform too. So, like a source lab or browser stack, they will handle those settings for you. So, usually when we do the CI at Travis, we set up our versions, usually we use the latest one, the latest version for the browser and if it's not compatible with the Selenium,
20:02
they will give you the indication. So, like a browser stack or like browser stack, it has a page for you to set up your browser version. If the Selenium is not compatible with this version, it won't show up. So, that's how we reference our settings. You mentioned some weaknesses of BDD,
20:21
but what are some unique strength of BDD? Do you guys do both BDD and then also unit testing or what combination is one? So, like what I said, BDD is not everything. So, you have to do this any testing, functional testing before you can do BDD. BDD is a complementary testing
20:41
and it didn't cover everything. Usually, we say something, you may not see your front end, but you have to consider it, like some error information gonna throw up in your page. We may not test that with BDD, but you have to picture it. There are some proper showing up in your page.
21:01
Just continuing on the last one, how do you measure test coverage with Bave? So, yeah, so that's why we need a description. So, the description will be measured by the user. So, actually, BDD is kind of like a mixing with integration testing and a user acceptance testing. And with the description has been proved,
21:22
you can make sure that your testing is covered, especially for the user behaviors. And they say that, okay, that's what I want for performing the behaviors on your page. Then they say, okay, it's covered. That's what we say. So, it's purely user behaved and the way owning tests against the requirements
21:41
we collect. So, that's the coverage. Right. Well, let's give a round of hand. Yeah. Okay. Okay.