Ruby on Rails on Minitest
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Part Number | 61 | |
| Number of Parts | 94 | |
| Author | ||
| License | CC Attribution - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/30700 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
RailsConf 201561 / 94
1
4
7
8
9
10
11
13
14
16
17
19
21
24
25
29
30
33
34
35
36
37
39
40
42
47
48
49
50
51
53
54
55
58
59
61
62
64
65
66
67
68
70
71
77
79
81
82
85
86
88
92
94
00:00
Exception handlingIndependence (probability theory)Group actionInformation technology consultingTwitterObservational studyParameter (computer programming)Computer animation
00:29
Statistical hypothesis testingExpected valueSlide ruleSet (mathematics)Computer animation
00:54
Function (mathematics)Statistical hypothesis testingInformation securityCodeBlock (periodic table)Exception handlingResultantLatent heatSound effectExecution unitInformationRight angleAsynchronous Transfer ModeError messageComponent-based software engineeringDifferent (Kate Ryan album)State of matterMultiplication signParameter (computer programming)Slide ruleSocial classMereologyPlug-in (computing)CASE <Informatik>Binary decision diagramUnit testingWater vaporInheritance (object-oriented programming)Projective planeLine (geometry)ExpressionSingle-precision floating-point formatDesign by contractSoftware frameworkExistenceWordMetric systemMetreVideo gameRow (database)Hydraulic jumpPhysical systemDialectBenchmarkKey (cryptography)Differenz <Mathematik>Computer animation
06:26
Process (computing)Statistical hypothesis testingTexture mappingDirection (geometry)Expected valueOrder (biology)Default (computer science)Execution unitSoftware maintenanceImplementationSoftware frameworkMaxima and minimaLevel (video gaming)Line (geometry)Social classPhysical systemMetaprogrammierungMereologyNormal (geometry)CodeFormal languageObject (grammar)Coordinate systemGoodness of fitOcean currentSimilarity (geometry)Negative numberReal numberSet (mathematics)Point (geometry)Digital rights managementDependent and independent variablesError messageArtistic renderingBlock (periodic table)CASE <Informatik>Mathematical optimizationDampingRandomizationThread (computing)Absolute valueLibrary (computing)Statistical hypothesis testingForm (programming)Plug-in (computing)Different (Kate Ryan album)Heat transferRule of inferenceCyberspaceMultiplication signFreewareAutomatic differentiationLie groupFluidEquivalence relationParallel computingPairwise comparisonShape (magazine)Computer animation
11:55
Rule of inferenceStatistical hypothesis testingNumberGame controllerINTEGRALWritingFlow separationProcess (computing)Order (biology)Mobile appRandomizationInternet service providerTraffic reportingRight angleExecution unitLine (geometry)Patch (Unix)CodeSoftware developerHuman migrationComplex (psychology)Statistical hypothesis testingStandard deviationLibrary (computing)Database transactionRevision controlFreezingImplementationTask (computing)Electronic program guideSheaf (mathematics)Group actionWeb 2.0CASE <Informatik>Server (computing)Computer architectureRow (database)DatabaseSoftware frameworkValidity (statistics)Software bugMultiplication signReading (process)Level (video gaming)HoaxSocial classFraction (mathematics)Film editingDegree (graph theory)Goodness of fitPower (physics)MeasurementUnit testingComputer configurationParallel portSlide ruleFitness functionReal numberSynchronizationPlug-in (computing)Extension (kinesiology)State of matterFunctional (mathematics)Condition numberMusical ensembleOcean currentKey (cryptography)Form (programming)Structural loadDifferent (Kate Ryan album)Computer animationLecture/Conference
17:56
Statistical hypothesis testingSoftware bugPatch (Unix)RandomizationSoftware frameworkDegree (graph theory)Arithmetic meanResampling (statistics)Type theoryLibrary (computing)CodeOrder (biology)BitPoint (geometry)TrailComputer animationLecture/Conference
19:42
Multiplication signEntire functionCyberspaceSoftware suiteStatistical hypothesis testingSubsetSoftware repositoryOrder (biology)Fitness functionComputer animation
20:43
Statistical hypothesis testingStack (abstract data type)Directed graphObject (grammar)Multiplication signLibrary (computing)Software developerStatistical hypothesis testingPlug-in (computing)DataflowBuffer overflowMathematicsProjective planeOrder (biology)Revision controlWordComputer animation
21:46
Parameter (computer programming)Statistical hypothesis testingDifferent (Kate Ryan album)CognitionIdentical particlesMultiplication signDependent and independent variablesCausalityPoint (geometry)Real numberBitStatistical hypothesis testingStandard deviationBuffer overflowStack (abstract data type)Functional (mathematics)MetadataProcess (computing)Video gameGoodness of fitGenderDivisorAmenable groupParallel computingRandomizationState of matterSoftware frameworkOptical disc driveComputer animationLecture/Conference
24:11
Expected valueObject (grammar)Social classBlock (periodic table)Boiling pointFocus (optics)Multiplication signPerspective (visual)WordFlagInheritance (object-oriented programming)Constructor (object-oriented programming)Statistical hypothesis testingRight angleMetreStatistical hypothesis testingTask (computing)Graph coloringGoodness of fitTableauControl flowGroup actionRevision controlComputer animation
26:13
Social classObject modelFlow separationModule (mathematics)Multiplication signBlock (periodic table)AnalogyStatistical hypothesis testingRun time (program lifecycle phase)HypermediaModel theoryObject (grammar)Constructor (object-oriented programming)Line (geometry)CASE <Informatik>Computer animation
29:07
Run time (program lifecycle phase)Line (geometry)Expected valueLevel (video gaming)NumberStatistical hypothesis testingMessage passingDifferent (Kate Ryan album)Slide ruleScaling (geometry)Metric systemCodePhysical systemSolid geometrySemiconductor memoryMathematicsLibrary (computing)Multiplication signPotenz <Mathematik>Software bugProjective planeComplex (psychology)Complete metric spaceComputer fontPlotterAbstractionSpectrum (functional analysis)Computer animation
32:08
Multiplication signStatistical hypothesis testingTask (computing)Computer animation
32:59
Computer animation
Transcript: English(auto-generated)
00:12
All right, let's get this started. My name is Ryan Davis. I am known pretty much everywhere as ZenSpider except for Twitter, as you can see.
00:21
I am an independent consultant in Seattle and I'm a founding member of SeattleRB, which is the first and oldest Ruby group in the world. So setting some expectations, this is an introductory talk. It has very little code in it. I'm not gonna teach testing or TDD in this talk. I'm gonna be talking about the what and the why,
00:41
not so much the how. I have 218 slides, which puts me at just under five and a half slides a minute, so I have to go kind of fast. So let's get started. The simplest thing that we can ask is what is Minitest? Minitest was originally an experiment to see if I could get test unit replaced
01:02
for about 50 of my projects that I had at the time, I'm up to about 100 now, with as little code as possible. And I got that to happen in about 90 lines of code. It's currently available as a gem. We didn't have Ruby gems back then when it was originally written. It now ships with Ruby as of 1.9.1 and up.
01:22
It's meant to be small, clean, and very fast. It's now about 1,600 lines of code, which sounds like a really big increase over 90, but still that's very small. It supports unit style, spec style, benchmark style, very basic mocking and stubbing, has a very flexible plugin system,
01:41
and et cetera, as we'll see. There are six main parts of Minitest. The runner, which really kind of is nebulous now. Minitest, that should say test, which is the TDD API. Minitest spec, which is the BDD API. Minitest mock, pride, and bench.
02:00
I'm only gonna be talking about two parts, Minitest test and Minitest spec. So let's jump into Minitest test, which is the unit testing side of things. So test cases are simple classes that subclass Minitest test, or another test case, and tests are methods that start with test,
02:22
and they make assertions about your code. It's just classes and methods and method calls all the way down. Everything is as straightforward as you get, and it is magic free. That slide is two years old,
02:40
so this wasn't my only barbit knoll. So Minitest test includes the usual assertions that you would expect from X unit, plus several added beyond what X unit and test unit usually provide. Methods marked with a plus are new to Minitest, and methods marked with a star
03:01
do not have negative or reciprocals, as we'll see in a sec. Unlike test unit, Minitest provides a lot more negative assertions. It doesn't provide some that you might expect, which I'll go into later, but one of the questions is like, why are there so many pluses? I really want my code, tests included,
03:22
to communicate to me and other readers as clearly as possible. Not only does the code communicate better, but the error messages are much more customized, so when there is something wrong, I get better information about it. And finally, assert equal is enhanced to do intelligent diff mode.
03:41
It allows you to see what's actually changed instead of a huge blob on the left side and a huge blob on the right. You get to actually see what's different between the two. But I said that I would describe why some negative assertions are missing. This is something that I hear a lot more than I'd actually like to hear. The question of where is refute raises,
04:01
or assert not raised. It's the same place as a refute silent. So let's look at that. Let's highlight the key components of refute silent. So refute silent says that this block of code must print something. What it is, I don't care. That is a valueless assertion.
04:21
What you should be asserting for is a specific output that you want. In the same vein, refute raises, says that this block of code must do something. What it is, I don't care. It's a valueless assertion again. Instead, you should be asserting for the specific result or side effect that you actually intend.
04:41
I've heard the argument, but it's useful. No, it isn't. It implies side effects and or return values have already been checked or aren't important, which is always false because you wouldn't be writing code otherwise. It falsely bumps your code coverage metrics, and it gives you a false sense of security. I'm an ex-lifeguard.
05:02
I have a lifeguard in my high school days at various lakes in my county, and one of the things we really feared were parents with kids with water wings because the parents think they've got flotation devices and we'll just talk to our friends and ignore our kids and watch them drown like this.
05:21
So in other words, this only makes it look like something has been tested when in fact it hasn't had any tested applied to it at all. I've heard it's more expressive. No, it's not. Writing the test itself was the act of expression. It's an explicit contract in every single test framework out there that any unhandled exception is an error by definition.
05:42
The test's mere existence states there are no unhandled exceptions via these pathways. And I've been having these arguments for years. In fact, I had this argument last month, and I know that some people will never be convinced, and honestly, that's okay.
06:01
You can't win them all, but it doesn't mean I can't try. So hold on your hats, stand back. I'm gonna try one more time to convince all of you. It's like if you call it, it must be okay, right? So I wrote all these extra assertions to verify that everything is okay,
06:20
and if you'd like to license this code, come please see me after this talk. If only that were possible, our jobs would be so much easier. Next up is may-test-spec, the example testing side. In short, where may-test-test is a testing API, may-test-spec is a testing DSL.
06:41
Instead of defining classes and methods, you use a DSL to declare your examples. Test cases are describe blocks that contain a bunch of tests. Tests are it blocks that call a bunch of expectation methods. Here's an example that is equivalent to the previous one. So we have describe instead of class, we have it instead of def test.
07:02
But in reality, describe blocks really are classes, and it blocks really are methods. This same example one-to-one transforms into this code where describe makes a class, it makes a method. There is no magic. All of your normal OO design code tools exist
07:21
and they work as normal, which is really, really important. It means that include works, def works, everything is as you expect, you're just using a slightly different language. Similar to may-test-test, may-test-spec has many expectations defined and a similar set of negative expectations
07:40
and a similar set of missing culprits. And all of this is gained for free because each one maps from expectation to assertion directly. Underneath may-test-test and may-test-spec is the infrastructure to run your tests, and it does so in a way that helps promote more advanced and robust testing. May-test has randomization baked in and has always been on by default.
08:02
It helps prevent test order dependencies and keep your tests robust and working standalone. By rule, every single one of your tests down to the lowest level should be able to run by itself and pass. If it requires another test to run before it, it's not standalone, it's not a unit,
08:20
and it's wrong, it's buggy. And as far as I know, may-test was the first test framework to have randomized run order. There's an opt-in system that lets you promote a test case to be parallelized during the run, and that takes randomization to a whole other level. It ensures thread safety in your libraries
08:42
and absolute robustness, because if it can handle the parallelization, it can handle anything. The original may-test was a tiny 90 lines of code, and over time, features have been added that you get to choose to help you enhance your tests. It's still really small in comparison, but it's incredibly powerful.
09:02
So what was my reasoning for may-test design? May-test is not special in any way, shape, or form. All of my usual tropes apply. If you heard me up here ranting and raving, may-test is no different. First and foremost, it's just Ruby. It's classes and methods and method calls.
09:20
Everything is as straightforward as you can get. I believe that less is more. If I can do something in less code, I will, absolutely. Method dispatch is always gonna be the slowest thing in Ruby. More importantly, less code is almost always more understandable than more code. So let's take a look. Here is assert in delta, which is the equivalent of assert equal, but for floats.
09:43
Never, ever use assert equal on floats, and never use floats for money. There, I've done my usual caveats. It's as simple as possible with minor optimizations. In this case, we use a block to delay the error message rendering.
10:02
In the case that we don't have an error, we shouldn't have that cost. And really, this just boils on up to assert. So you really only need to know about 15 other lines of code to understand how this works as a whole. Indirection is the enemy.
10:21
I want errors to happen as close to the real code as possible. I don't want things delayed. I don't want layers of indirection in between. I kind of feel like Noel is making my point because he just kept talking about the layers of indirection that RSpec adds over and over. I want to strip all of those out, as many as possible, at least. I want the responsibility to lie in the right place.
10:41
No managers necessary, no coordination going on. I want objects to be responsible for their own duties. This may not be the best example. I don't know how to get a good example of indirection as the enemy because it's rather hard to show what you don't do. Must equal is an expectation that directly calls assert equal on the current test context,
11:02
and assert equal is three lines long, if I remember right, and that's it. No magic allowed. Even test discovery avoids object space. It has minimal meta-programming in it, and it uses plain classes and methods to do all of its work. I originally wrote many tests in part
11:20
to see if I could because I was the maintainer of test unit at the time, and test unit terrified me. But I also wrote it because I was working on Rubinius, and I wanted Rubinius and JRuby and other implementations of Ruby that hadn't finished being a full Ruby to have the simplest implementation of a test framework possible so they could get feedback quickly.
11:41
And finally, it has a thriving plugin ecosystem. I designed many tests to be extensible so that many tests itself could remain minimal. Here are just a small amount of the popular plugins for many tests. Okay, so what does this have to do with Rails? Well, the official Rails stack uses many tests.
12:01
Each release, it peels back the testing onion, encouraging better testing practices, except that peeling onions makes you cry, right? Hopefully not in the case of many tests. So, Rails 4.0 was the first version to cut over from test units to many tests, and they did so on the many test 4.x line.
12:24
At the time, I had already declared that I wasn't going to keep updating standard lib many tests to many test five. I was only gonna maintain it at version four. And that's because test unit is built on top of many tests and has a lot of hooks into the internals, and it just made it really hard for me to ever update
12:41
and not break their stuff. So, I put a freeze on that, so Rails updated many test four to remove a layer of complexity and indirection, but also to allow them a migration path to many test five. Because test unit was already wrapping many tests, there was basically no impact on anyone.
13:02
Arguably, there may have even been an almost imperceptible performance improvement, but I'm not gonna claim that there was one. Rails 4.1 switched to the many test five line. This was painful for Rails itself because of crufty tests and the number of monkey patches that they had on many tests itself.
13:22
This got Rails onto the newer code base, the active development line of many tests, and it made things easier to do, like exec-based isolation tests. There's a number of tests in Rails, where each test actually forks a process to run a separate Rails app by itself, so that if they're running in parallel,
13:41
they're not gonna infect each other. As painful as it might have been to get Rails switched to it in passing, you hopefully never noticed, though. However, Rails 4.2 turned off the alphabetical order of the testing and to remove a monkey patch, and that started to run tests in random order.
14:02
This is solely to improve the quality of Rails and your tests, but it may have had some impact on some people. We did get some reports that after updating, people started having tests that previously passed failed. I had to isolate a number of test bugs in Rails itself
14:22
because of this, and I'm gonna say honestly, despite the pain that it causes, this is a good thing. Test order dependency bugs are problematic, and they're incredibly hard to track down. I'm gonna talk later about a tool that can help identify these bugs. And hopefully future versions of Rails, they should be tracking many tests. Aaron Patterson and I keep those things in sync,
14:42
and he lets me know when things are coming down the pipe. So as a Rails dev, what does all of this mean? Hopefully, if I've done my job right, it means nothing. You shouldn't even have to see many tests most of the time unless you want to enhance it with some plugins. And that's because you're subclassing Rails's test cases, like active support test case
15:01
or action controller test case. There's about six of them if I remember right. There might be more now. And the basic architecture looks something like this. You write your own test class, that subclass is active support test case, and active support test case subclasses may test test. It provides things like per test database transactions, so you don't have to clean up.
15:21
You just add a bunch of records. They're gonna be gone on the next test. Aaron and I wound up adding before and after setup and teardown hooks to make it easy for Rails and other libraries or frameworks to extend many tests to do extra wrappings that they needed to do. It provides things like fixers to load test data.
15:42
Declarative forms, if you like that instead. If you don't like those, you can just use def. And it provides extra assertions like assert difference, assert valid keys, assert deprecated, and assert nothing raised. Wait, what? Don't worry, this is the actual implementation of assert nothing raised.
16:00
It's only there for compatibility's sake, and personally, I think that it should be deprecated and removed. Maybe that should be for Rails 5. And all this means that you can write simple tests that describe your current task. Things like database transactions don't have to clutter your test code, and you can focus on what the test is really trying to do. This is a very simple example,
16:22
and I think all of these test examples come from the testing section of the Rails guide online. Action controller test case is another Rails extension, except that it subclasses active support test case, so you get all of those goodies layered on.
16:40
And it extends it to include more, like all of your usual HTTP verbs, simulate web server state, and provide assertions specific to handling requests, which lets you easily write clean functional tests like the following. I love air conditioning. I'm so dry right now.
17:02
I'm just gonna leave the lid off. Active dispatch integration test provides full controller to controller integration tests. As usual, it subclasses active support test case, so all of the usual stuff is there. It provides a ton of assertions that I cannot fit onto one slide, and allows you to write comprehensive integration tests
17:20
that span multiple controllers. If you want more details about all the stuff that Rails adds on top of many tests, you can get that here, and it's described in pretty good detail. It's actually a really good read. So Rails' approach to subclass min test test leads to a very simple setup that remains very powerful, providing you with everything you need for unit tests
17:43
all the way up to integrations. It leverages min test power, providing randomization, optional parallelization, to provide better testing, make your tests that you write better and more robust over time. But what happens when something goes wrong?
18:01
Perhaps you want to use spec style. Turns out the DHH disapproves of RSpec so much that he wouldn't allow us to switch the test framework in Rails to min test spec. He reverted that commit. As I understand it, he simply doesn't want people to submit patches using spec style,
18:20
so he didn't want it easily available. But that doesn't mean that you're stuck. If you prefer spec style, that's totally fine. You can use Mike Moore's min test Rails, or Ken Collins' min test spec Rails to do varying degrees of this type of code. The two libraries are not the same. This suggests slightly different styles. One tracks RSpec's style a bit more than the other.
18:44
Or perhaps you upgraded to Rails 4.2 and now you have failures. Also known as Ryan, you broke all my shit. I'm sorry. Unfortunately, that's not as simple to deal with. It is a bit harder, but again, it is a good thing to catch and fix this sooner rather than later.
19:01
Why are my buttons not working? It is a test order dependency bug. And quite simply, that simply means that tests will pass when they're run in a particular order, say A before B, but not when B runs before A. And if it was only three tests, that wouldn't be a problem. It would be pretty easy to find and fix. But you probably have hundreds and hundreds of tests.
19:20
That's not so easy, until a few months ago. I wrote minitest bisect to try to isolate the problems that we were having getting Rails onto minitest randomization. It helps you isolate and debug random test failures. In short, it intelligently runs and reruns your tests, whittling it down until the culprits are minimized.
19:42
And here we're gonna see a simple example of it running. Ignore the fact that I'm pre-specifying the seed. Pretend that I'm running this anew and we get this failure. We get the failure. We grab the random seed and we rerun the test using minitest bisect instead.
20:04
That's gonna rerun the entire test suite to ensure that it is reproducible. And once it is, it starts to bisect the culprit space down to the minimal subset.
20:22
And you can see that it speeds up dramatically as it goes on. Getting it down to two tests run in a particular order that will cause the repo every time. I see I'm not alone with this problem.
20:44
Or maybe you're just used to kitchen sink development. For starters, just try it. It might work. Things like Mocha and a lot of testing libraries already work in minitest and they're just fine. Otherwise, you're not alone. And someone probably beat you to it.
21:01
So look for the existing plugins that are listed in minitest-readme or search on RubyGems.org or Stack Overflow or whatever. But my suggestion, and I know this is going against the flow here, try less complicated testing. Only bring in plugins once you've decided that you really, really need them. Otherwise, start fresh and clean. Try it.
21:21
You might like it. But the problem is that change takes time. Just remember that you might want to measure the before and after in order to be more objective about this change. I've only got anecdotes of projects speeding up when they switch to minitest. I would love more data and if you would submit that to me, that'd be great. I have heard that people have halved their test times
21:41
by switching from RSpec to minitest, but again, I don't have anything objective. So, all of this minitest stuff sounds interesting, but why should I bother? The first argument is this. I'm not gonna bother with that. To me, you're a lost cause.
22:01
There's plenty of data showing the benefits of testing and if you can't get past that, I'd rather help other people. I'm gonna pick my battles, thanks. Like this one. This is a battle I want to pick every time. Obviously, not everyone uses it. The official Rails stack uses it,
22:21
which means DHH uses it. Tenderlove uses it. Jeff Kazimir and his cohort teach minitest at a touring school. Nokogiri, Hamel, God, New Relic, Sequelite, and a bunch of other very popular gems use it. In fact, there's more than 4,000 gems that declare dependencies on minitest and unfortunately, because minitest ships in standard lib,
22:44
there are plenty of gems that don't declare their dependency on it. So I'm sure there's plenty more. So what are the real functional differences between minitest and RSpec? Being test frameworks, there's plenty of overlap, so I'm not gonna go over that.
23:01
Where they are unique though is where it gets interesting. To be fair, RSpec provides a lot more than minitest. Things like test metadata and metadata filtering, more hooks like before and around, implicit subject described class, fancier mocking, et cetera, et cetera.
23:21
Basically, it's fancier. Minitest by definition doesn't offer as much. Some of the stuff that made it unique has been adopted like randomization, but benchmarking, parallelization, and speed are the main distinguishing features. Basically, it's simpler and more pragmatic and snarky.
23:43
So it's the cognitive differences with RSpec is where things really start to diverge and this is where I think that Noel's talk actually proves my point quite a bit. Myron Marston, a few years back, wrote a really great response on Stack Overflow comparing RSpec and minitest. It was a bit biased,
24:01
but honestly, I think that it was rather fair. The problem is that it's really long and the meat of it is in this first paragraph here. But even it's pretty long and apparently, I've been working with Tenderlab for too long whose attention span is that of a ferret on methamphetamine.
24:24
So there's so much there that I have a hard time dealing with it all at once. So let's focus on that one paragraph. I'm gonna color code it red for the RSpec points and blue for minitest. I'm not gonna read that crap. I've read it too many times.
24:40
So Myron thinks this is why RSpec is great and I think that this is everything that's wrong with RSpec. And we're both right. Philosophically, we're both right. We have different goals and we have different perspectives on what good is. So back to that paragraph.
25:00
Let's try to boil this down even more for the ADD. Even that's pretty long. So let's break it down to a simple table, example groups. Minitest compiles blocks down to simple classes where RSpec, and I'm paraphrasing this here because it didn't say what it did, reifies testing concepts into first class objects and that first word I think is the big red flag for me
25:21
with regard to RSpec. If you're using that word, you should be coding in Haskell. Examples, Minitest compiles, it blocks down into simple methods whereas they use first class objects. Minitest uses inheritance or mixins for reuse where they use shared behaviors for first class constructs
25:44
and simple methods for making assertions for expectations versus first class matcher objects. Even that's pretty long. So let's boil it down further. Minitest uses a class, they use a first class object.
26:01
Minitest uses a method, they use a first class object. Minitest uses subclasses or include, they use a first class object and Minitest uses method calls, they use a first class object. First class simply means
26:20
that you can assign something to a variable and you can use it the way you can use any other value and it just so happens that everything that Minitest uses is Ruby and nearly everything in Ruby is first class so that's not a good distinction. Everywhere that I could use Ruby's mechanisms, I did. And everywhere that RSpec could reinvent, they did.
26:43
So let's try to take a look at this and see where this starts to get more cognitively complex. I think this is best illustrated by examining how RSpec works. Here we have two nested describes. Each one has a before block and each one has a single example. Yet the before blocks seem to be inherited
27:02
and the examples are not. They'll be exactly two runs here where the first one uses one before and the second one uses two befores. So our A and B classes is nesting like subclassing. What's the analogy we can use to understand this?
27:20
Well if they're classes and nesting is like subclassing, then we need this undeaf method to ensure that we don't inherit any tests from our super classes. This is the approach that Minitest uses and it sucks. But that's the runtime behavior that RSpec users expected out of Minitest spec
27:44
and that was something that I wound up having to put in. What about this analogy? Are before and after like included modules? If that's the case, then we don't have to undeaf any methods, but instead we need to generate a bunch of inner modules and have a bunch of includes and setups need to intelligently call super
28:02
and because of the complexity, this sucks too. This is basically another object model. To effectively use RSpec, you need to learn a whole separate object model that sits on top of Ruby's object model. This is doubly confusing if you haven't already learned the Ruby object model or if you're just trying to learn both at the same time, you're gonna get overwhelmed.
28:22
What winds up happening is it encourages users to hand wave. Noobs are just learning Ruby. They don't have the time or the ability to dig in and learn both object models and it encourages users to hand wave the oddities away. What else are you gonna do? So it encourages them to not know
28:42
what describe and it or actually are or do and if you didn't see the previous talk, there's a lot of stuff that goes on inside those describes and its. Basically says, here's the magic incantation to do x. So when you're a beginner,
29:01
any sufficiently advanced technology is indistinguishable from magic thanks to Arthur C. Clarke. So now I need to rename my talk. Sorry, Noel, but RSpec is magic. So from that previous post from Myron, he said that many people find it to be overkill and there is an added cognitive cost
29:22
to these extra abstractions. Indeed. Here are the raw numbers of that added cognitive cost. Don't bother grokking these numbers. We're gonna visualize them in the next slide for both the Flog line and the comments plus code line. Flog is a complexity metric proportional to how hard something is to test,
29:42
to debug, or even to understand. And here RSpec is 6.6 times bigger. And this is the combined meat of the projects. This is code plus comments and it's basically how much you're gonna have to read to understand each library. At 8.5 times bigger, it's akin to reading Dr. Seuss versus James Joyce.
30:07
Do you like my Dr. Seuss font? I think I faked that really well. So back to that added cognitive cost. As we're gonna see in a second, it's not just cognitive cost. There are performance differences as well.
30:22
All those abstractions, reinventing the wheel, it has a real cost. Here we have some fairly complex plots. This shows the run time in solid lines and the amount of memory allocated in dashed. The green lines are 100% passing. The red lines are 100% failing
30:41
and the others are in between that. And as you may notice, there's a slight problem to these charts. Can anyone guess? They're not using the same scale. So now you can see that there's a severe and painful difference between RSpec and MainTest. Failures have exponential growth in RSpec.
31:02
Run time is always near zero in MainTest. Memory is linear and it's always lower. But Ryan, who cares? Passing RSpec is fast enough. Indeed, I actually agree with that. Everything is bread and roses as long as everything plays nice. But what happens when it isn't?
31:20
You have a bug or you refactor or anything else goes wrong. Often times when I'm refactoring stuff, I'm gonna make a change and I'm gonna have 100 tests fail. All of a sudden you pay. For completeness, the speed of the actual assertions in both systems are purely linear.
31:43
The speed of running those tests at the method level is also linear. And because they're linear, please do not try to regain any speed by reducing examples or expectations or otherwise reducing the quality of your tests. If you wanna speed up, MainTest will always be faster than RSpec
32:01
pretty much by definition. So you can switch to MainTest or you can never ever refactor or have any bugs. So in summary, at the end of the day, as long as you test, I don't actually care what you use. Use the best tool for your job. Hopefully I've shown the technical merits of MainTest.
32:20
Choose what works for you, not because it seems popular. Often times I hear that they chose RSpec because there's more documentation available for it. But maybe there's fewer articles about MainTest because there's less need for them. MainTest is much easier to understand. You can read it in a couple hours
32:41
and understand it head to toe. So maybe MainTest users aren't missing. Maybe they're just busy getting stuff done. Choose what works for you. Who knows? Try it. You might like it. After all, it's just Ruby. Thank you and hire me.