Common pitfalls in mobile testing
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 46 | |
| Author | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/47156 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
droidcon Berlin 201544 / 46
5
7
10
13
16
18
20
21
23
26
31
34
35
36
41
42
45
00:00
Mobile appMachine codeStatistical hypothesis testingLecture/Conference
00:16
Mobile WebDisk read-and-write headWeb-DesignerMobile appSoftware developerMobile WebMonster groupCross-platformComputing platformStatistical hypothesis testingLecture/ConferenceMeeting/Interview
00:42
Mobile WebStatistical hypothesis testingSoftware frameworkRow (database)Computer animationLecture/Conference
01:11
Statistical hypothesis testingMachine codeSoftware frameworkMultiplication signProjective planeLecture/ConferenceComputer animation
01:42
Mobile WebExecution unitStatistical hypothesis testingUnit testingMachine codeMeeting/Interview
01:56
Machine codeStatistical hypothesis testingStatistical hypothesis testingMachine codeArithmetic meanJSONXMLComputer animationLecture/Conference
02:24
Mobile WebMachine codeStatistical hypothesis testingMultiplication signMeeting/InterviewXMLComputer animationLecture/Conference
02:46
Mobile WebExecution unitGoogolStatistical hypothesis testingOvalStatistical hypothesis testingSocial classStatistical hypothesis testingResultantUnit testingExecution unitCASE <Informatik>CalculationJava appletMachine codeLogicUtility softwareString (computer science)Multiplication signSocial classNumberReal numberExtreme programmingGoogolComputer animation
03:37
Statistical hypothesis testingBoolean algebraFront and back endsSocial classDatabaseView (database)Machine codeStatistical hypothesis testingFront and back endsSoftware developerView (database)Projective planeOvalYouTubeGame controllerExtreme programmingDatabaseUtility softwareLecture/ConferenceComputer animation
04:29
RepetitionString (computer science)Statistical hypothesis testingCalculationSocial classProjective planeUtility softwareMereologyMachine codeMeeting/InterviewLecture/Conference
04:52
Physical systemSystem callToken ringAuthorizationFront and back endsParameter (computer programming)Client (computing)String (computer science)OvalSocial classIntercept theoremSampling (statistics)Machine codePhysical systemFront and back endsSystem callStatistical hypothesis testingClient (computing)BitRow (database)Token ringOffice suiteAuthorizationUniversal product codeSoftware developerLecture/ConferenceComputer animation
05:47
Statistical hypothesis testingImplementationStatistical hypothesis testingOvalFactory (trading post)Client (computing)Client (computing)Factory (trading post)Machine codeToken ringParameter (computer programming)AuthorizationLecture/ConferenceComputer animation
06:12
Machine codeBitCode refactoringLecture/ConferenceMeeting/Interview
06:32
Client (computing)String (computer science)OvalToken ringStatistical hypothesis testingSystem callParameter (computer programming)Front and back endsStatistical hypothesis testingFactory (trading post)Client (computing)ImplementationStatistical hypothesis testingMachine codeError messageSource codeCode refactoringFactory (trading post)Token ringComputer animation
07:19
AuthorizationDependent and independent variablesPhysical systemFront and back endsAuthenticationLecture/Conference
07:35
Statistical hypothesis testingToken ringParameter (computer programming)System callFront and back endsStatistical hypothesis testingOvalClient (computing)Factory (trading post)ImplementationStatistical hypothesis testingUniform resource locatorComputer animation
07:55
Client (computing)String (computer science)OvalSocial classIntercept theoremMachine codeImplementationReal numberStatistical hypothesis testingClient (computing)MereologyFormal verificationLecture/ConferenceComputer animation
08:35
Statistical hypothesis testingMereologyLogicStatistical hypothesis testingLecture/Conference
08:51
Code refactoringStatistical hypothesis testingMachine codeMereologyStatistical hypothesis testingDesign by contractInstant MessagingWordGreen's functionLecture/ConferenceMeeting/Interview
09:45
Statistical hypothesis testingSoftware developerStatistical hypothesis testingImplementationMobile WebMachine codeStatistical hypothesis testingImplementationFlow separationInstant MessagingLogicMachine codeMessage passingReal numberComputer animation
10:24
Mobile WebStatistical hypothesis testingLine (geometry)Software developerRevision controlMachine codeMereologyMeeting/InterviewComputer animation
11:00
Mobile WebAndroid (robot)Content providerSocial classAcoustic shadowSoftware developerResolvent formalismAuthorizationMachine codeStatistical hypothesis testingContent (media)InjektivitätClient (computing)Intercept theoremActive contour modelLine (geometry)Lecture/ConferenceMeeting/InterviewComputer animation
12:02
Statistical hypothesis testingMachine codeUniversal product codeComputer architectureProduct (business)Functional (mathematics)Lecture/Conference
12:22
Mobile WebMachine codeWritingStatistical hypothesis testingStatistical hypothesis testingFlow separationStatistical hypothesis testingSimilarity (geometry)Machine codeConnectivity (graph theory)ImplementationThumbnailPattern languageDrawingComputer animation
12:58
Mobile WebStatistical hypothesis testingMachine codeWritingStatistical hypothesis testingMachine codeStatistical hypothesis testingEvent horizonSocial classStatistical hypothesis testingMereologyRotationFlow separationLecture/ConferenceMeeting/InterviewComputer animation
13:28
Statistical hypothesis testingoutputOvalSocial classStatistical hypothesis testingStatistical hypothesis testingObject (grammar)Process (computing)Machine codeWeb pageMobile appSelectivity (electronic)View (database)Web 2.0Field (computer science)RandomizationRotationoutputCASE <Informatik>Pattern languageLecture/ConferenceComputer animation
15:02
Machine codeStatistical hypothesis testingPattern languageReal numberCASE <Informatik>Fluid staticsStatistical hypothesis testingMeeting/InterviewLecture/Conference
15:23
InjektivitätJava appletFluid staticsMachine codeAndroid (robot)Pattern languageState of matterMechanism designMobile WebMachine codeStatistical hypothesis testingSingle-precision floating-point formatJava appletParameter (computer programming)MereologyUtility softwareInversion (music)PlanningSocial classGame controllerInjektivitätInstance (computer science)Pattern languageFluid staticsExecution unitLecture/ConferenceComputer animation
16:02
Machine codePower (physics)Fluid staticsUtility softwareCASE <Informatik>Statistical hypothesis testingMach's principleState of matterLecture/ConferenceMeeting/Interview
16:20
Mobile WebInjektivitätJava appletMachine codeFluid staticsAndroid (robot)Pattern languageState of matterMechanism designStatistical hypothesis testingStatistical hypothesis testingSet (mathematics)State of matterComputer animation
16:36
Revision controlService (economics)Universal product codeCuboidQuantumImplementationWordGame controllerMechanism designState of matterMereologyInversion (music)Flow separationInjektivitätControl systemLecture/ConferenceMeeting/Interview
17:12
MereologyInstance (computer science)Lecture/Conference
17:29
Mobile WebPattern languageMachine codeParameter (computer programming)Unit testingStatistical hypothesis testingImplementationExecution unitComputer animation
17:56
Streaming mediaLecture/Conference
Transcript: English(auto-generated)
00:09
Yeah, what I want to talk about today is Come pitfalls. I saw when I did code reviews on test code of our apps. I Work at a lesser mobile. It's an
00:22
company based in Dortmund Munich and Munster we do all kind of mobile development web development app development for all Bigger platforms and And cross platform development native development all that kind of stuff
00:42
And we are writing tests just brief history of our travel this tests prior to 2012 we didn't wrote tests that had to do with all kind of reasons mainly the test frameworks
01:00
prior to 2012 fact This was prior Roboelectric to that was when robotic tests were really really really really slow And so in the early days of 2030 we started writing tests That The frameworks have evolved and I thought great. We will reach great cop a growth code coverage in no time
01:26
but The reality got me wrong proved me wrong and we stick with coverage around five or six percent that was our biggest coverage in in some project, but the coverage was still pretty pretty low and
01:42
Then in 2015. I said yes, we usually great coverage about 90% The unit tests were running the automation tests were running and I thought everything is fine but again code coverage is not equal to
02:00
test quality Because coverage just means somebody had called this method to a test. It doesn't mean he had tested this method Was just called and then the code coverage went up without any test Just call the method code coverage went up everything great
02:23
But I don't want to talk more about mean generator and start with some test code At the first one I saw was Sticking too much to assertions on return values Especially from people who write tests for the first time and that has something to do with the hello world tutorials for
02:47
writing tests For example, that's the first Google result for Java unit test example and then you find something like that Some kind of calculator and then you give in two numbers and
03:01
Asserting the value this case you multiply with zero. So the result should be zero The hello world of J unit. The problem is all hello world examples show more or less helper classes some kind like string util or a calculator or something like that not real logic code and
03:26
All time I didn't see a hello world J unit example with for example mocks All on assertions of on return values and something like that So in the extreme Example we had a test
03:42
Boyd just called the backend and When the developer went in and write the test he said hmm. I can't assert Avoid return value. Sorry, he just added a return true at the end and then said yes, sir true. So everything worked It's some kind of extreme example
04:03
Didn't happen a second time for For any developer I have ever met But that was rooted for in 2013 for nearly all our project We had a more or less Coverage in the controllers or activities and database and very poor coverage in views
04:24
But 100% coverage in the util package because then again string util add two strings and assert the strings are concatenated or calculate examples Util classes are easy easy easy to test but very unimportant for a project
04:43
another one I Saw very often if just have one example for for this part Tests that can be food easily because so they are still green except the whole code is broken To explain that just met the requirements of the sample code. We have a call to our backend system and
05:06
It works without an auth token It works with an auth token But when the auth token is set it has to be appended to every call to the backend that has to be implemented and the developer implemented it
05:21
so you can see he used retrofit and Here we have our init client. It takes a talk token it's an interceptor and the token is added to every request and then the client is created and
05:41
That's the whole code It obfuscated but that's very productive code and that's the test of the code he creates a spy with moquito on the client factory and gives in the auth token and then he verifies that the inner client with the auth token parameter is called one time and
06:06
That the client is not now who can tell me what's wrong with this code, right?
06:30
Let's do a little bit refactoring of our code and just delete everything inside in decline and then we just initialize the empty client and
06:41
the test code will still remain green and The reason for this mistake is he's testing the implementation details So he wrote the test after he wrote the source code and then he knew what he did and said Yes, when this method is called everything is fine But he totally lost the requirements. He wanted to test and just test what he had implemented
07:04
So let's have a look at the refactored test method and Then oh, I see there's a copy paste error This spy is useless. Just ignore it. It's just new client factory with the auth token
07:24
then I'm here just that empty response in Robolectric and Then he just called the Backend system and asserted that the authentication token is depended To the last sent HTTP request URL
07:42
Then again now the test tests the requirement to append something to the URL and not the implementation detail to call the method Which appends something to the URL and that is something that happens very very often And when When the test code is written after the real implementation code
08:04
Let's get back to our implementation another thing we see is that the client and the init client method are protected and They actually mean to be private but to assert to make us an assertion the verify in the
08:20
The mock they had to be protected so we can assert that it's called So then again, he just changed the implementation to write the test on the implementation details. It's also wrong and It brings me to the next part testing private methods and making them protected therefore
08:42
That's again something which happens when? The logic is written before the test because when you think of Just forget everything about clean code and write a public method which does something that it's very very long some if some else and
09:01
A big fat spaghetti code and then you write a test for that Everything's green. Everything is fine And then you do the refactoring you put some part in a private method another part another private method and the test is sufficient Or it is still working and it still tests what the public method promises to do
09:22
to anyone uses this API and it's completely uninteresting if there are one ten or one hundred private methods because the Contract is alone in the public method But when you do the refactoring for the testing then you think yeah That's it that private method and I have to call it and so you write a test calling this private method
09:49
And again thinking of coverage the method the private method is called during testing the public test so the coverage doesn't went up if I test the private method separate and
10:01
Again, it's just testing the implementation details not the real test logic There are two reasons why you don't need it On the one hand every private method has to be called by a public method which has to be tested Or the private method is not called by any public method and then it's that code
10:22
Can be deleted so there's no reason for writing tests for private method and make it protected know your tools Mostly when I talk with two developers about Rob electric they know that part they know yeah the test runner
10:44
conflict annotation and we can create activities with Rob electric and Then for example, I saw Many many many lines of code and hacking with mokito to achieve to create mocks inside The Android container just because the developer didn't know that for electric
11:04
always or he provides mocking for example in the shadow resolvers for content resolver, I just can Register a mock of a content resolver with my authority Through the shadow content resolver in Rob electric and there are many many other methods in the content resolver to inject
11:27
test code or mocks Which are not so good in documented in the Robolectric documentation, I think that's very common again
11:40
Robolectric can mock HTTP requests if they use the HD the Apache client and not the Apache connect Because or when he can't intercept can intercept only the Apache client next pitfall
12:02
It's only test code At the beginning of all evil it's just the test not the production code the production code is nice and clean and it's separate and it's the architecture as well, but the test code is a Getting cold again just giant functions and methods who do everything again and again and again
12:27
So write clean codes also for tests We have to change tests often as often as you change the implementation maybe not as often but
12:41
There's similar If your test code has to follow suitable patterns, you have to think about how to write your sales as your test code how to create components for tests how to create packages for tests and something like that and Otherwise the efforts to take to keep your tests running but increase and increase and some then you will abandon the tests and
13:07
there will be test dead code in your project and Of One very important thing is you have to separate automation and instrumentation code from the test code Because it's two part of things and as you know for clean code every class should do one thing not
13:25
instrumentation and testing but instrumentation down the one class and Testing another class Have one example for that based on a p.m In this example, we are testing a web view inside a native app and
13:45
What we want to do is we select the target in a destination in a booking process and then Look again if the selection is present in another input field some simple example
14:02
And that's a test code We get some random targets or test data and then we just called select targets that's the instrumentation and then we look if our target value is inside the input field and that's
14:20
Instrumentation code we select target. We click the select target button Then we click every target We want to select and then click confirm and then we have select the target So if you have one class for the test and One page object. There's a common pattern in when writing selenium
14:44
tests a page object Which encapsulate everything you can do on this? Particularly a particular view and for that case we can select target in Every test we like without rewriting all this rotation the clicking and the selecting again
15:02
Well, that's very very common and nice Pattern to separate instrumentation from the real testing code So I'm now to the last and in my opinion most important case for testing static coupling
15:25
You can't test code which is static cup which is coupled static Therefore follows a inversion of control pattern that can be dependency injection like dagger but it hasn't to be
15:41
Inversion of control the most important part is don't initialize classes unit inside the class But outside the class and give them in as parameters and therefore avoid singles the plain Java singles Not the dagger invitation single but the plain Java singles get instance and something like that avoid static methods, that's
16:02
Very strange because of the big utility package but avoid static methods because you can't test really really good static methods you can use power mark, but that's only a tool for that design code and In every case static code must not be stateful
16:21
Avoid global states at the real real real problem because you can't reset a global state in every test that only leads to some strange methods like reset or Set for testing or something like that in your single or aesthetic Global states, but it's again code. You just write in your productive code to test it and that shouldn't be done and
16:48
Have in mind that Android itself provides some some inversion of control mechanisms like registering services in the manifest registering Contemplate in the manifest all that is a version of control. So you set the implementation in another part then you use it
17:05
So you can inject mocks through Separate manifest for example. Yeah If you're in quick example
17:20
This part is completely untestable because we can't inject any Mock or something like that when we just call the static get instance and then do something Where would you inject or replace that implementation? through a mock on the other hand if you just give it as a parameter for example, or
17:44
Within it inject or something like that then you can inject some mocking example Or some stop and then you can do all the testing inside unit test Okay, and I'm through