"RSpec no longer works with ActiveRecord"
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 69 | |
| Author | ||
| License | CC Attribution - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/37794 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
Ruby Conference 201723 / 69
2
5
8
11
13
19
20
21
32
33
34
35
37
40
43
44
52
54
55
56
57
58
62
64
65
67
00:00
EmailWeb pageCartesian coordinate systemSpeicherbereinigungSoftware developerNumberCodeRow (database)Line (geometry)AreaGodSoftware maintenanceRight angleMultiplication signComputer-assisted translationXMLUMLComputer animationLecture/Conference
01:42
Social classSocial classNumberRow (database)Hyperbolischer RaumComputer animation
02:11
EmailSoftware testingExplosionSoftware maintenanceControl flowEmailDisk read-and-write headDifferent (Kate Ryan album)Multiplication signEmbedded systemPoint (geometry)Software testingInformation securityCache (computing)Revision controlComputer animationLecture/Conference
03:20
Projective planeOpen sourceSign (mathematics)Connectivity (graph theory)Capability Maturity ModelMoment (mathematics)Expected value
03:47
Open sourceProjective planeProduct (business)Software maintenanceExpressionFocus (optics)Meeting/InterviewComputer animation
04:20
Bit rateException handlingSoftware testingCASE <Informatik>Core dumpSoftware bugObject (grammar)BitExpressionRow (database)QuicksortTraffic reportingGoodness of fitComputer animation
05:11
Revision controlBacktrackingInformationProjective planeSoftware bugTraffic reportingSoftware testingPoint (geometry)Task (computing)Exception handlingMultiplication signTerm (mathematics)Revision controlStandard deviationInformationLink (knot theory)Configuration spaceCartesian coordinate systemPatch (Unix)Repository (publishing)CASE <Informatik>Order (biology)Open sourceComputer fileCodeVarianceSoftware maintenanceFiber bundleExterior algebraInterpreter (computing)Universal product codeProduct (business)MultiplicationPattern languageObject (grammar)Touch typingNP-hardReverse engineeringExpandierender GraphComputer animation
08:26
Error messageHierarchyComputer fileComplete metric spaceOpen sourceInformationPlatonic solidIdeal (ethics)Order (biology)Computer animation
09:00
CodeState of matterInformationWhiteboardOpen sourceSoftware maintenanceElectronic mailing listPosition operatorCollaborationismReal numberSoftware frameworkLecture/Conference
10:02
ProgrammschleifeData modelIntegrated development environmentCloud computingFiber bundleInformationLink (knot theory)Turbo-CodeLocal GroupCryptographyServer (computing)Installation artCore dumpOpen sourceComputer fileEmailEndliche ModelltheorieCodeError messageEndliche ModelltheorieSoftware testingComputer fileCartesian coordinate systemOcean currentRevision controlRepository (publishing)Insertion lossBit ratePoint (geometry)Order (biology)LaceRational numberOpen setObject (grammar)MathematicsLatent heatSoftware developerBlogCASE <Informatik>Statement (computer science)DatabaseGoodness of fitQuicksortVideoconferencingBitWebsiteMeta elementSoftware maintenanceLetterpress printingDefault (computer science)Software bugInstallation artComputer animation
14:03
Order (biology)Repository (publishing)Linear regressionCASE <Informatik>Series (mathematics)Software bugPoint (geometry)INTEGRALSoftware testingPhysical systemMessage passingResultantGenderBitRange (statistics)WebsiteData managementCasting (performing arts)MathematicsMoment (mathematics)WhiteboardComputer animationLecture/Conference
16:24
Disk read-and-write headComputer fileSample (statistics)Configuration spaceRevision controlFiber bundleType theoryPoint (geometry)Disk read-and-write headCartesian coordinate systemCore dump
17:07
Core dumpDisk read-and-write headRevision controlFiber bundleBranch (computer science)Disk read-and-write headQuicksortNumberBitSingle-precision floating-point formatDialectPoint (geometry)Projective planeCASE <Informatik>Computer animation
18:43
MathematicsSample (statistics)Disk read-and-write headSoftware maintenanceComputer fileRevision controlType theoryRepository (publishing)Order (biology)Revision controlCartesian coordinate systemDisk read-and-write headMathematicsPoint (geometry)Open sourceSoftware testingLoop (music)Multiplication signFiber bundleRight angleCASE <Informatik>
19:41
Computer fileRevision controlScripting languageMathematicsSample (statistics)Software maintenanceInstallation artCompilation albumPrice indexInformationElectric currentFile formatOpen sourceRevision controlVideo gameSoftware testingType theoryMultiplication signSoftware maintenancePatch (Unix)Row (database)Physical lawProjective planeSheaf (mathematics)LoginObject (grammar)
20:38
Computer fileMathematicsSample (statistics)Disk read-and-write headOpen sourceMultiplication signSoftware testingRepository (publishing)Revision controlSheaf (mathematics)Office suiteMultiplicationComputer animation
21:09
MathematicsSoftware maintenanceDisk read-and-write headRevision controlComputer fileSample (statistics)Block (periodic table)Configuration spaceGoodness of fitCartesian coordinate systemMultiplication signRaw image formatSoftware testingType theoryWebsiteFunction (mathematics)Object (grammar)Structural loadInsertion lossComputer fileRight angleComputer animation
21:46
CodeComputer fileMathematicsDisk read-and-write headPrice indexRepository (publishing)Object (grammar)Computer fileGame theorySocial classSoftware developerMultiplicationCodeLibrary (computing)Computer animation
22:13
Asynchronous Transfer ModePrice indexCodeDisk read-and-write headComputer fileBlock (periodic table)Sample (statistics)Fiber bundleMathematicsSoftware maintenanceRevision controlSoftware testingObject (grammar)Commitment schemeNumberType theoryQuicksortCartesian coordinate systemSoftware developerProjective planeDisk read-and-write headGoodness of fitRevision controlProcess (computing)Message passingSheaf (mathematics)Computer animation
23:12
View (database)Normed vector spaceConfiguration spaceInheritance (object-oriented programming)Information privacyCodeAttribute grammarMultiplication signDatabase1 (number)Line (geometry)Object (grammar)Concordance (publishing)Row (database)Arithmetic meanGame controllerCodeExpected valuePatch (Unix)CausalityContrast (vision)Video projectorComputer animation
24:36
Computer fileSoftware maintenanceIntegrated development environmentAliasingContext awarenessInheritance (object-oriented programming)Block (periodic table)Attribute grammarSynchronizationHierarchyAttribute grammarVideo game consoleRow (database)Radical (chemistry)CausalityExplosionTable (information)Computer animation
25:12
Inheritance (object-oriented programming)Context awarenessPartial derivativeFormal verificationConfiguration spaceExplosionPoint (geometry)Dependent and independent variablesClosed setRow (database)BitSoftware testingSoftware frameworkTask (computing)Computer animation
25:49
Computer fileConstructor (object-oriented programming)Electric currentProxy serverChainRow (database)Patch (Unix)Multiplication signComputer animation
26:15
Strategy gameSoftware maintenancePhysical systemMultiplication signSound effectBlogLevel (video gaming)Software bugTraffic reportingData storage deviceRepository (publishing)InformationRevision controlStrategy gameCartesian coordinate systemRight angleTracing (software)Point (geometry)QuicksortSoftware maintenanceLecture/Conference
28:27
SynchronizationBlock (periodic table)Proxy serverLevel (video gaming)InformationCloningInformationLine (geometry)Revision controlCodeCASE <Informatik>Physical systemMultiplication signLinear regressionInfinityCartesian coordinate systemType theoryBlogPattern languageQuicksortSoftware bugSoftware maintenanceTraffic reportingStandard deviationPatch (Unix)Software repositoryFiber bundleComputer animation
30:25
Revision controlOpen setSoftware bugBitTraffic reportingMeeting/Interview
31:22
CountingServer (computing)Software testingSoftware maintenanceDisk read-and-write headOpen setStudent's t-testDistortion (mathematics)SoftwareComputer animation
32:25
Coma BerenicesInternetworkingAddress spaceComputer animationLecture/ConferenceXML
Transcript: English(auto-generated)
00:11
We're not quite starting yet. We have a minute before the timer officially kicks in. But before I do start, I just want to say thank you
00:20
to everyone for coming to this talk. Instead of going and watching Aaron Patterson talk about garbage collection and his cats, that's absolutely 100% where I would be right now. As an RSpec maintainer, knowing how the garbage collector relocates pages is actually like super important
00:42
to my day-to-day work, but I suspect most people in this room are actually application developers and being able to actually test your code is maybe more important than knowing what shiny, new, under-the-hood features the Ruby VM has, I hope. Or maybe you just like RSpec a bunch, or you care about open source.
01:03
Anyway, so yeah, I guess they've closed the doors, which means I should probably start. So this is RSpec no longer works with active record. My name is Sam Phippen. Just to be absolutely 100% clear, this is not a feature announcement.
01:23
Sometimes when you wake up in the morning and you're working through your email drinking that first cup of coffee, a line comes in, a subject of the email that fills your heart with such dread and terror that everything else stops, and you must immediately focus on it.
01:41
For me, that was this RSpec mocks issue, number 972. Allow no longer works with active record classes. And so while the title of this talk is slightly hyperbole in that RSpec was not completely broken with active record,
02:00
this basically is one of the most core and important functional pieces of the RSpec gem. And if RSpec doesn't work with active record classes, then you know, there's much gnashing of teeth, maintainers get sad, everything kind of breaks and explodes, and you know, no one is happy.
02:21
Another way of saying this is that when RSpec doesn't work with Rails, my email inbox suddenly explodes. And I think nowhere in my head is there a more poignant example of this than a different RSpec issue, undefined method cache in tests for Rails 4.2.5.1, which if you zoom in, had 37 comments come in
02:44
overnight while I was sleeping. You see, this version of Rails got released late Pacific time while I was still living in the UK, and that just means I was in bed minding my own business, and I wake up to discover that all of your tests aren't working.
03:01
Every single person in this room who did the security upgrade had non-functional tests, and I had an inbox full of very sad email, people sending pull requests, issues, debugging, trying to work out what was going on. I'm just like, everyone, please calm down. It will be fine. And it may sound like I'm complaining, but I'm really, really not.
03:21
I'm glad people care that RSpec works. I think it's the sign of an incredibly mature open source project that people care enough about how the RSpec gem works that the moment there's even the slightest tremor of an issue in a very specific component that applies to them, be it mocking, stubbing, expectations, or Rails,
03:43
that they will come in and file an issue. But this is a gif of me at the end of the day after work most days, and you have to remember that open source projects are mostly worked on in evenings and weekends, and maintainers have to divide their attention
04:00
very, very carefully amongst their projects, or all will be laid to waste. And so support a maintainer, hug a maintainer, all of that good stuff. So anyway, let's get back to our issue and actually focus in on what we're talking about here. The Rails issue that I've mentioned affected expressions like this,
04:21
where you say allow user to receive, and then some kind of scope method. In this case, we're using confirmed, but really this could be anything, and return something. Executing this simple test would raise an exception, and I don't know about you, but when I look at that, I face screaming in fear. That has to work.
04:41
That is a very, very simple expression in RSpec, and active record objects are one of the core things that people want to mock on. This is really bad. Like, when I saw this issue, I was like, oh, this is not gonna be fun. Before we get too far into debugging, I wanted to actually do a little bit of an examination
05:00
of the bug report, and think about whether it's good, bad, how it might be improved, and what sort of things we could do as contributors to actually make this issue better. If you look at the full text of the issue as submitted, it says, while updating a Rails 4.2.1 project from RSpec 3.2.0 to 3.3.0, I got a lot of failing specs.
05:22
I noticed the issue can reproduce simply, and then the user provides this test that we just looked at. There are a few really, really great things about this bug report. The user provided a Rails version and an RSpec version, and as a maintainer, that means that I can hone in very quickly on which exact things I'm going to need
05:41
to begin debugging what's going on here. And so, if you're filing issues on projects, if you're having a bug report with an actual problem in an open source project, these are really, really great things to provide. In terms of things that aren't really great, we don't have much in the way of steps to reproduce. We only have a failing test. We don't have any of the production code.
06:02
We don't have any of the Rails configuration. We don't have any of the other surrounding stuff that that application is going to be providing, along with that test, to make sure that it's not some other gem, some other piece of tooling, something the user has done that is outside of RSpec. And we also don't have a Ruby version.
06:22
In the modern times, basically if you're on Ruby, I don't know, say 2.2 plus, variances between Ruby versions don't matter that much anymore. They're mostly stable and work all the time at this point but providing that information can be really, really helpful especially if you're on an alternative interpreter like JRuby or just if you think
06:42
something really weird is going on. And so, because we're missing a Ruby version here, we might have a slightly harder time debugging. Now, for this particular issue, it's not the case but I do think that is a really helpful thing to provide if you do have the time and ability to do so. In terms of things that could absolutely be improved with this issue, I don't have a backtrace.
07:02
I don't know what the exception the user is seeing is. I don't know where in RSpec the exception comes from. I really don't know what's going on. As a maintainer, I'm gonna have to go into this thing and do a bunch of an investigation before I'm able to actually work out where my bug is coming from. And whilst I applauded this issue
07:21
for providing Rails and RSpec versions, we don't have extended dependency information. I don't know all the other gems. I don't know what was in the lock file. I can't really begin to know whether or not this is an RSpec bug, a Rails bug or something in a gem which is monkey patching one of those things or monkey patching user objects.
07:42
These days, I think a really, really good standard to aim for is providing a full application, a full reproduction case that the maintainer can clone and immediately execute to be able to begin debugging this issue. So basically what that would mean is instead of filing an issue and finding a single spec, you file an issue
08:02
and provide a link to a git repository that I can clone, run bundle exec RSpec and see exactly as what is going on, because that will provide me a gem file, gemfile.lock, all of your tests, all of your code and everything I need to see in order to begin to understand what is going on. One of the things that's really great about RSpec
08:21
is it's a multi-maintainer project and so what happened immediately after this user filed this issue is another RSpec maintainer, Myron, comes in and is like, thanks for filing this issue. Please could you provide us a complete backtrace in order so that we can begin debugging this? And then the thing that happens next is kind of like one of those amazing, beautiful,
08:41
platonic, ideal, open source things where a different user to the one who initially files the issue comes in and provides all of that information and this is how open source is literally supposed to work. Multiple people are experiencing the same bug and they're literally collaborating on the issue and this is an amazing thing that we can only do
09:03
because our code is open and our tools are collaborative and so if you are using an open source framework and you're encountering an issue in it and you go to the issue tracker and you see that the issue is immediately at the top of the list because it's a new bug, usually they're new, you can collaborate.
09:21
If you can provide more information than the person who initially did and that information is helpful, that's really, really great and no maintainer is ever going to tell you off for providing extra data points. It's a really helpful thing and I super want to encourage you all to do that but we're still missing something from this issue.
09:41
We still don't quite have enough information to be able to begin debugging it and really, that's the ability to reproduce this issue in an automated, repeatable, easy to use fashion which is basically what you need to debug any real issue in open source these days.
10:00
So a third RSpec maintainer comes in and provides us with exactly that. Here we have complete steps to reproduce, Rails new, what to put in the gem file, which models to generate and how to write a test and so let's actually work out what this issue is. Let's actually debug this issue together live on stage and work out what's going on.
10:21
So I'm gonna go ahead and new upper Rails application at version 4.2.0. The reason for that is that this issue was filed against that exact version of Rails and we're now in the future and Rails 5.1 exists and so I need to be able to go back to that older version of Rails in order to do so.
10:40
Once our application is created, we can then begin getting ready to debug this. The first thing that we're going to need to do is put RSpec actually into our Rails application so that we can begin testing our bug and here what you'll notice I've done is actually specified path dependencies to the RSpec version instead of the actual official gem versions.
11:01
The reason that I've done this is that I have all of the RSpec repositories locally checked out on my computer so that I can do debugging, insert binding dot pry, insert print statements, do everything I'm gonna need to do to actually debug this. This also makes switching versions really, really easy. I can check out a specific git tag and get the exact version that I need
11:20
in order to do so. You might also notice I've specified rake 10 here. That is because the current version of rake is rake 12 and the older versions of RSpec aren't compatible with rake 12 but that's just sort of a side note. So now that we've updated our gem file, let's go ahead and bundle update and bundle install so that we have everything ready to go and then we'll actually begin debugging this issue
11:43
and making sure everything's good and to do that, we'll basically do this very standard Rails generate RSpec install and that will take a while because bundler is slow. Sorry, there's Andre in the room. I'm sorry, Andre, I love you
12:01
and then we'll go ahead and generate a user model which is beginning to get us towards the ability to reproduce our bug and once we've generated that user model, oh, we're having some video troubles. I had to rebuild all these videos like half an hour ago
12:22
because they were a bit blurry so please bear with me. So that's gonna go ahead and generate our user model and then we'll migrate our database because you have to do that, otherwise you won't be able to run your tests.
12:43
So because we did an RSpec install on our Rails application, it's automatically going to have generated a spec models user spec for us which we can then pop open and actually like replace the default generated test with the one that's going to reproduce our bug. Now because we didn't actually create a scope method
13:02
on this user object, I'm gonna go ahead and stub new instead of stubbing a specific scope method which should actually allow us to reproduce this bug exactly as we expected. If I do that and run my tests, you should be able to see that this test passes and the reason for that is that we've checked out
13:20
RSpec version 3.2.0 to confirm that this did actually ever work. The user has reported that this is failing and we want to make sure that it did work on the previous version as they are claiming. So what I'm gonna do here is go ahead and change into my RSpec development tooling repository which is a sort of meta repository used by RSpec maintainers to make working
13:41
with various versions of RSpec more easy. What I did there is run a command rake git checkout version 3.3.0, moving us from 3.2.0 to 3.3.0. Then I changed back into my Rails application and ran my tests and you can see that they are now failing. So we know for a fact somewhere between 3.2.0 and 3.3.0
14:01
our tests broke. Reproduction case check and we have a good commit range. So at this point we have a commit range, we have a passing and failing test and that is sufficient I think for most people to start doing debugging. Because this issue is small and probably is resulting
14:23
from a small change, it wouldn't be that difficult to just go ahead and stick a debugger in and work out what's going on. But we can do a little bit better. The year is 2017 and we have great tools for debugging systems now and for when a regression gets introduced like this, something used to work
14:42
and now it doesn't, there's one tool in particular that is better than all the others for working out what's going on and that's git bisect. For those of you who aren't familiar, git bisect is a tool where you tell git where the thing was working, where the thing was not working and git will move you in exactly halves of the commit range,
15:02
helping you find where the problem occurs. The important commands that you need to know, a git bisect start, which basically tells git you're going to do this, git bisect good, which labels a particular commit as functional and git bisect bad, which labels a particular commit as non-functional and by using these commands,
15:23
git will interactively walk you through a series of commits and help you debug and find the correct commit in order to identify where the issue came from. Unfortunately, we are slightly complicated in our ability to find the specific commit because there are potentially two repositories
15:40
at play here, rspec-rails and rspec-mocks. For those of you who aren't used to working with the internals of rspec, rspec is broken down into multiple different repositories to allow us to ship features independently from each other. rspec-rails manages all of rspec's integration with the Rails gems and rspec-mocks is the gem
16:03
that provides these mocking methods, allow, receive, and so on. And at the moment, we don't know where in either of these two gems it could be. Now, I sort of take the lead on maintaining rspec-rails, so I tend to assume that bugs exist in that repository first.
16:22
So, let's actually bisect across rspec-rails and take a look at what's going on here. So here I've moved into the rspec-rails repository, git bisect start, label version 3.3.0 as bad, check out 3.2.0, label it as good, and now we can begin debugging.
16:42
And you can see here, git's actually moved us to a specific revision and then we go back to our Rails application, we type bundle exec rspec and we explode. The reason for this explosion is that bundler can't actually resolve rspec-core equal to 3.3.0 with rspec-rails equal to 3.3.0 pre.
17:03
So far, so cryptic. Let me explain what's going on here. All of the rspec gems are released at the same version numbers so that we can maintain compatibility. While rspec provides a completely stable and semantically versioned public API, our internal APIs don't have to follow
17:22
that same compatibility guarantee. And rspec, individual rspec gems are allowed to call each other's private APIs where it makes sense because after all, these are sort of one combined project. So if I have rspec-rails checked out to 3.2.0, I must also have all the other rspec gems
17:40
checked out to a 3.2x version and in this case, they're all checked out to 3.2.0. The same is true for version 3.3.0, they must all also be checked out to that same version. But what's happened when we started the bisection is that the version tag for rspec-rails has changed to 3.3.0 pre.
18:00
This version tag doesn't represent a single commit of rspec but it's instead the version number that we use in the master branch of the repository while we're developing the next minor version of rspec. But earlier, we checked out all of the rspec gems to version 3.3.0 and 3.3.0 pre is not equal to 3.3.0
18:21
because trust me, and so this won't bundle. There is a very, very simple solution to this which is this command, git checkout head caret. What the caret character says is, git, please give me the revision that's immediately before the one I'm specifying
18:40
and head is the revision we're currently checked out to. So what I'm gonna go ahead and do is change to every single one of my rspec repositories and type git checkout head caret in what is a furious bunch of manual typing in order to get them all onto the 3.3.0 pre versions of the rspec gem so that we can bundle our application.
19:01
Once I've done all of this, I can go back into my Rails application and now if I run my tests, they will actually run because the bundle is now compatible on those rspec versions that are checked out. In this case, my tests are failing so I go back into the rspec Rails git repository, git bisect bad, go back to my Rails application,
19:21
run the tests, they fail, go back to Rails, git bisect bad and this happens basically in a loop marking revisions as bad every single time because the tests are going to fail for a while. Coffee.
19:41
This is what open source is like. This is my life. So anyway, I think we're nearly done. Zero revisions left to test after this one so we run our tests one more time, they fail, we go back to rspec Rails, we try to run the test but we can't because we moved and I'm bad at typing
20:02
and we mark as bad and we run our tests. Really, you too could be a maintainer who gets to do this. Okay, and git has given us the revision where we started failing and this says something really interesting. It says bunk to 3.3.0 pre. Well, that's kind of curious
20:20
because that would be a commit immediately after we released a version of rspec. So here we look at the git log and we see indeed the thing immediately before is releasing 3.2 and we can see here that our patch has absolutely nothing to do with active record scope objects and mocking and any of that stuff which means our bisection failed. Oh no, you know what this means.
20:44
Oh yeah, oh yeah, it's time to do open source people. So what we're gonna do is run our tests in our Rails app, make sure they're failing. Here I've reset everything to the 3.3.0 versions of rspec
21:01
and then go back into my rspec mocks repository this time and reset bisection because I was doing some earlier, check out the 3.2.0 release, mark it as bad. Sorry, 3.3.0 release, mark it as bad, check out 3.2.0, mark it as good and now we're going to do the same thing,
21:21
changing back and forth between our Rails application and running our tests. This will be accelerated because I like my time. So this time our test passed, we type git by site good, we run our tests and this time we get a blow up. We don't actually get output from the test suite. This is interesting, right? It says cannot load such file,
21:40
rspec support object inspector. Here's what's going on. Bisecting old code across multiple repositories is a dangerous and scary game. Because we've been actively developing in all of these repositories, classes have been appearing and disappearing as that development has happened.
22:01
And so if we go into the repository that has that file object inspector and take a look at what's going on, we can quickly identify the issue. Here we're looking at rspec support which is an internal rspec library to help us develop code. And you can see here there's a commit which just literally deletes this object which my tests depend on during bisection
22:22
because again, this is active development. So what I'm gonna go ahead and do here is grab the commit before the object was deleted, check it out and now go back into my Rails application and here we can begin bisection again. And here where the tests fail, I'm gonna type git bisect bad. Where the tests pass, I'm gonna type git bisect good and so on and so on through all of this process.
22:45
And basically what will happen is the same thing. Git will identify a specific revision where the project started breaking and once it's done that, we can actually go ahead and understand what's going on. So I'm gonna go ahead and skip like a bunch of bisection here
23:01
and sort of see good, bad, good, bad and eventually the commit gets popped out at the end. So what we go ahead and do is grab that commit number and head on over to GitHub and see what's going on and that can't be seen at all because the contrast on these projectors isn't the best. See, can anyone read that?
23:22
No. All right, basically what's happened here is a patch has been introduced that causes RSpec to do a callback every time you try and verify methods on an expectation on all of the subclasses of this object.
23:41
As we sort of debug through various lines of GitHub, we find these methods which do the things, we search for them, we discover they don't exist, we search for different ones, we discover they do and eventually we're led back to this piece of code. The important line of which is this one. If your object responds to a method called define methods
24:04
which is defined on active record base and everything that inherits from active record base implements, we call that before attempting to do any RSpec work on it. The reason we do that is that when you have an active record object for the first time before it's ever spoken to a database, all of the methods for all of the attributes don't exist.
24:23
If you have a database column called name and you try and call the method name, that method isn't defined yet and it's the method called define attribute methods which actually does that. I can show you this by going back into my terminal and firing up a Rails console and trying to call it.
24:44
If I call active record base define attribute methods, it's going to explode and the reason for that is that active record base is abstract. It doesn't represent any specific table and so there are no attribute methods to be defined. It doesn't have any attributes
25:00
and this is identical to the blow up that the user who provided us with a backtrace reported in their issue. We've done it. We've found the cause of the problem. It's that when we invoke this method, RSpec explodes and this method is being called because RSpec is calling this method on all subclasses
25:20
that respond to it. The problem here is that active record base is a subclass but it's not valid to call this method on active record base. We only want to call it for the subclasses and not active record base itself. We solve this by implementing a check that we're not going to invoke this on the thing that explodes and also because RSpec is a testing framework,
25:41
we add a bunch of tests that actually verify this behavior exists and works. There's a little bit of aftermath to this issue. Whilst this solution works for active record, the generic idea of being able to do this kind of crazy callback chain isn't the best and so a different patch gets applied to RSpec mocks
26:03
which is more generic and doesn't only work with Rails. While we like to fix the immediate issue, it's also useful to come up with generic solutions if you have the time. So let's do a quick retrospective on this issue. How did we do, how was this?
26:22
Well, it wasn't great that we shipped a version of RSpec Rails that was critically broken. I think a few people need RSpec to work with Rails applications, just sort of a hunch. But we had a really, really great debugging strategy. We had a backtrace, we had version information, we had steps to reproduce
26:41
and then we did very, very carefully calibrated bisection. Bisection is a really great method for finding specific bugs and exactly where they were introduced in the system. It's also worth noting that initially we bisected the wrong repository but that was just based on intuition.
27:01
If we'd spent a few more minutes looking at the backtrace that the user provided, it would have been obvious which repository had the issue in it but it's great because we actually eventually ended up finding the commit. Before I go too far, I think it's also worth noting that for this bug report, 99% of the time
27:20
that was spent is actually in trying to find out where the bug came in. The actual fix was trivial. It took about five minutes once it was obvious what the bug was and that's so common and so true. I think debugging is most of the time and then just implementing the fix, it's not hard. So let me give you a few pro tips.
27:41
Let me summarize all of the wisdom that I've been trying to distill into this talk. There are ways that you can make your bug reports infinitely better with just a few small tweaks. So for like one star, the most basic level of filing a bug report is just telling a maintainer that there is a bug. This thing broke, here is some code.
28:02
It's fine but it doesn't really provide us with enough information to be able to begin to assess and triage the issue and that's where we started with this issue and then our maintainers got to work actually working with the community to begin to have enough information to solve this problem. A second star I can sort of award to you
28:21
for providing a backtrace because backtraces are rich with information. They don't just tell us where the bug is but if you look carefully, you can also see that there's like version information baked in to these lines here. They actually tell us more or less exactly what stuff you had checked out and what it was doing when the problem occurred.
28:43
The next most useful thing is dependency information. If I know every piece of code that was running in the system at the time a bug occurred, I'm much more likely to be able to quickly and suddenly identify your issue. If there's a gem which is monkey patching what RSpec does or monkey patching what Rails does,
29:02
I'm able to tell you it's probably an issue with that gem and probably not an issue with RSpec or Rails. The big one, the thing that I wish everybody did is providing a reproduction case that I can just clone. Just type bundle install, bundle exec RSpec and see exactly what the bug is.
29:24
This kind of automated reproducible bug reporting is kind of like the golden standard these days. Many maintainers won't even look at your bug report unless you do that now because it takes so much time. The Ruby ecosystem is really, really complicated these days
29:41
and if we don't have this, it can be almost impossible to work out what's going on. And then for five stars, but really this should be more like infinity stars, is doing the bisection for us. If you can come, not just with an application that I can clone, run a thing and see it fail, but also then spend the time to see if this is a regression
30:02
to actually bisect across many, many revisions of Rails or RSpec to determine what's going on, that's gold because then you've done most of the work and that five minute fix that I talked about earlier, that can happen straight away. It saves me all the time in the world to not have to bisect your repo on your behalf.
30:23
And in fact, I have a sort of pithy quote about this from Sean Griffin, not Sean's baby, just to be clear. A few years ago, actually I think last year, I had a bug in Rails that was badly affecting RSpec and I had actually bisected down
30:41
to a single commit in Rails. And I was like, Sean, the bug came in here and it was just like, bam, six hours later, the issue's gone and he said to me, if everyone who filed an issue on Rails gave me an exact commit where the bug was introduced, there would be no more open bugs in Rails. Now there is a whole army of you
31:02
and there aren't that many people with a Rails commit bit so if you all want there to be no more bugs in Rails, this is a thing you can do, right? It's not that specialized. So that's approximately where I finish talking about bug reports.
31:20
It behooves me to just say one more thing, which is you've probably noticed DigitalOcean has been all over this conference. I work there. Who enjoyed our party? See, that's just easy pandering to the audience. Here's not pandering to the audience.
31:41
Very specifically, my team at DigitalOcean is hiring, not just DigitalOcean is hiring, my team at DigitalOcean is hiring. I have, I think, one open head count for this year and four open head count in 2018 and I want to build an amazing team filled with Rubyists who can ship great software. We work on internal tools
32:01
but also we write a lot of RSpec tests and if you want to see how a RSpec maintainer writes RSpec tests, this is an open opportunity. Please come grab me afterwards. We have socks. I am wearing them. I will give socks out until there are no socks left. First come, first serve.
32:21
Please don't tackle each other. So that's it. I'm done. I'm Sam Phippen everywhere on the internet. That's my email address.