Riding Rails for 10 Years
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Part Number | 14 | |
| Number of Parts | 94 | |
| Author | ||
| License | CC Attribution - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/30697 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
RailsConf 201514 / 94
1
4
7
8
9
10
11
13
14
16
17
19
21
24
25
29
30
33
34
35
36
37
39
40
42
47
48
49
50
51
53
54
55
58
59
61
62
64
65
66
67
68
70
71
77
79
81
82
85
86
88
92
94
00:00
FrequencyMathematicsMereologyRevision controlNumerical analysisContrast (vision)CASE <Informatik>Content (media)BitMultiplication signCodeElectronic mailing listLine (geometry)DivergenceEndliche ModelltheorieAssociative propertyStandard deviationGroup actionAdditionValidity (statistics)CountingTotal S.A.Server (computing)Computer-generated imageryInterface (computing)Web 2.0Computer fileRight angleEntire functionRow (database)Pattern languageLevel (video gaming)Pairwise comparisonGame controllerUniform resource locatorDifferent (Kate Ryan album)Cache (computing)State observerPoint (geometry)Data structureMobile appDirectory serviceModule (mathematics)View (database)Data managementPower (physics)Software testingText editorPrototypeReal numberComputer configurationRepository (publishing)TowerRoutingSoftware developerProduct (business)Computer animation
07:53
Goodness of fitMixed realityNumerical analysisCycle (graph theory)Online helpFigurate numberAliasingSource codeGroup actionMultiplication signCodeError messageStapeldateiStatement (computer science)Cartesian coordinate systemRight angleRevision controlView (database)Different (Kate Ryan album)Communications protocolElectric generatorPattern languageMathematicsState of matterDependent and independent variablesGenderType theoryIntegrated development environmentProcess (computing)NamespacePoint (geometry)Combinational logicSeries (mathematics)Order (biology)Server (computing)Contrast (vision)Web 2.0Library (computing)Web browserFile formatMappingException handlingConsistencyBitFundamental theorem of algebraLink (knot theory)Slide ruleMultilaterationReal numberRule of inferenceComputer configurationData managementPhysical systemComputer fileAreaWeb serviceCondition numberMobile WebMassFluid staticsIntegerCASE <Informatik>RoutingUniform resource locatorAttribute grammarIntrusion detection systemConfiguration spaceFunction (mathematics)Computer animation
15:46
Multiplication signTemplate (C++)MiddlewareMathematicsDifferent (Kate Ryan album)Cartesian coordinate systemMereologyTwitterDefault (computer science)Attribute grammarSimilarity (geometry)SubsetQuery languageAssociative propertySoftware developerWebsiteProduct (business)Type theorySoftware frameworkDynamical systemHash functionState observerSet theoryMobile appSpring (hydrology)CodeArrow of timeBitAdditionComputer fileLibrary (computing)Front and back endsObject (grammar)Proxy serverFlow separationIntegrated development environmentConfiguration space1 (number)Data managementPhysical systemChainRule of inferenceReal numberWhiteboardFrequencyStapeldateiSlide ruleLengthParameter (computer programming)Point (geometry)
23:39
CodeServer (computing)Scaling (geometry)Template (C++)Revision controlInformation securityComplex (psychology)Link (knot theory)Turbo-CodeLinear regressionStructural loadObject (grammar)Type theorySampling (statistics)Line codeDifferent (Kate Ryan album)Latent heatBitCASE <Informatik>Line (geometry)Set theoryComputer fileProduct (business)Multiplication signMathematicsTranslation (relic)Flash memoryRight angleData storage deviceParallel portVariable (mathematics)MomentumSubsetLibrary (computing)1 (number)Row (database)FrequencyCache (computing)Process (computing)File formatMaxima and minimaRule of inferencePattern languageVarianceMobile appService (economics)Level (video gaming)Computer animation
Transcript: English(auto-generated)
00:12
Yeah, so thanks everyone for coming. So my talk is Riding Rails for 10 Years. And so when I started thinking about this,
00:20
10 years didn't seem like that long of a period of time to talk about Rails and how it's changed. It's actually a long time. There's a lot of stuff that's happened. So I've tried to jam in as much as I could. So I'm going to try and go through it pretty quick. But there's lots of interesting things that have happened over the 10 years.
00:43
And so part of this talk is looking at Shopify and the Shopify code base and how it's changed over those 10 years of Rails. Why do we, why look at Shopify? And it's pretty interesting because Shopify started nearly around the same time as Rails 1 came out, just a bit before actually.
01:01
It's never been rewritten and we've used versions all the way from before 1.0 to 4.1, which we're running now. And the Git repository holds most of the content of all of these changes over time. So it's really interesting to go digging through it and seeing how Rails has changed and how Shopify has had to change
01:21
as Rails has moved along. And for this talk I put together a little timeline of kind of the releases over time. So these are the releases of Rails over time. Did it based on kind of the public releases and so they may be off a little bit.
01:41
But there's been a lot of releases over time, and a lot of changes to Rails over time. And then you can contrast this with Shopify and how we've been evolving the Shopify code base over time as well. So we've been following along with Rails and in some cases you can actually see the version numbers being a little bit before Rails releases because we were running on Edge Rails
02:01
for a long period of time in the early days. So this is kind of how Shopify's upgrades to Rails have mapped to the Rails releases. And as I was going through the history, I kind of pinpointed around 45 different versions of Rails that we had running in production
02:23
over the 10 years. So this is looking at all the major, minor, and tiny version updates that have been done. And this doesn't include anything that I might be missing from the history that we don't actually have in the Git repository because there's a few years of history that we don't actually have. So this is a lot of change over time
02:41
and a lot of change that both Rails has went through and the Shopify code base itself has had to go through to follow along with those changes. So I want to start just going back in time a little bit to that first commit that we have of Shopify and checking that out and kind of taking a look at what Rails looks like at that time.
03:03
And so the first commit we have is from August 2005. It's likely around Rails 0.13 and it's pretty much what Rails 1.0 was released as in December of 2005. So I checked out that commit because I could do that
03:21
and that was really easy and cracked it open in my editor and looked at the directory structure and it's pretty interesting because it's all very familiar. Nothing crazy about this stands out. It looks pretty familiar as a Rails app for most developers who've been using Rails.
03:40
A lot of the main pieces to what we know of a Rails app today existed back then in 2005. So we have the MVC patterns established, testings already baked in, there's even a deploy.rb file. At that time I think it was using something called Ship Tower.
04:01
Capistrano was released in early 2006 which Shopify now uses. So I went through some of the files as well and take a look at some of those to see how the pieces have changed over time. And so this was, I think I maybe cut out a line or two but this was basically the route file
04:21
for Shopify at that point, the entire routes file. And now our writes file is actually split into like five or six different files because there's so many routes, it's very large. But all this is very familiar. It doesn't look quite the same but you can see how it got to where it was
04:40
and it doesn't feel completely different from what you might be used to. You still have the fragments in the URLs, you still have controllers, you still have actions that you're mapping to. And then if we look at a controller, again it's still pretty similar. The naming of the actions is different because we didn't have as much standardization
05:00
around the naming of the actions at the time. But all in all it looks pretty similar to what you would know from a Rails app. And the models are the same thing. We have all the associations, there's validations. A lot of the same pieces that are there now
05:20
existed back in 2005. There's just a lot more to them. You can do a lot more with the validations, you can do a lot more with the associations that you couldn't do at this time. There wasn't much in addition to the associations here but now you have a lot of options that you can pass to associations and they're just generally a lot more powerful
05:41
than what they were at the time. It's also cool that JavaScript was also kind of baked into Rails at the time. We had the RJS responses, Scriptaculous and Prototype were stripped with Rails. So Ruby and Rails going along
06:00
with some bits of JavaScript has been the case since the earliest days. Then I looked into it a little bit more and there's a few other things that kind of exist and a lot of these things have actually changed over time. So at one point there was this idea of sweepers and you use those to expire your caches.
06:21
We had observers to encapsulate some of the callback actions. Views were RHTML instead of being HTML.ERB. And dependency management was handled by either including the gems in your vendor directory or having sub-modules or having SVN externals.
06:41
And then the interface to the web server was fast CGI at the time. So I also did a line of code count on that first commit which is kind of cool. So the total lines of code of Shopify at that time was 11,000 which is pretty tiny in comparison to what it is now.
07:01
Just to compare, we have close to half a million lines of code now, the majority of them being Ruby. So a lot has changed over time but it's interesting looking at kind of those early beginnings and where it came from. So yeah, a lot has changed but there's still a lot the same.
07:20
Like a lot of those early patterns and early practices were set up and established in those earliest days. So now if we follow through with kind of the evolution and start looking at some of the other versions of Rails, starting with Rails 1.2, this was released in January. In Shopify it actually showed up in November of 2006
07:42
because we were running on the edge and later on in 2007, we updated to 1.2.3. So in those early days, we were keeping pretty close with the versions, being on edge as much as possible and it was really easy to do that because Rails was actually checked directly into the code base in like vendor or externals.
08:03
So it would be changed even with like minor changes to Shopify and since the code base was pretty small, the impact was low. Rails 1.2 was a pretty interesting release. It added REST and the idea of resources. So DHH was talking about this earlier.
08:21
REST was one of the kind of fundamental things that came about in Rails and still exists now. But it wasn't perfect when it was initially released, kind of like Turbolinks wasn't perfect when it was initially released. I'll get to that a bit later. Rails 1.2 also added multibyte support, some routing was rewritten and you could do some nice things
08:44
with formats and response too. So I'll show you some code from the Shopify code base. And most of this code was like directly lifted from the code base in the history. I cut out like a few things to like make it fit on the slides but this is real code that we had running in Shopify
09:00
at that point in time. So routing once kind of the formats and respond to, so that's the format fragment in the URL. So this allowed you to do the HTML, CSV formats and Rails would just inspect the content type, the MIME type and just handle figuring out
09:21
which format to render for you. So this was really nice and cut out a bunch of like format handling that you would have to do on your own. REST was really cool. So this created a lot more consistency in your applications. So you could map your resources and then you would have a lot more consistent action names
09:41
and you wouldn't have to do all the mapping of the specific URLs. And a lot of this exists today and hasn't changed a whole lot over time except for the underlying routing. Then back to like that REST initial version.
10:00
So the initial version of REST actually shipped with the URLs being separated by semicolons. So after your orders ID, there'd be a semicolon and then the action name. And this was not handled very well by some browsers and libraries and web servers. Mongrel didn't handle it very well
10:20
which was kind of the defacto web server at the time. So they actually changed it in Rails 1.2.4 to be a slash. So this is kind of contrast with the Turbolink situations where the ideas were solid from the start but they didn't quite get it right and it kind of evolved to be the patterns
10:41
that we see today and to be really great patterns that we all use and love. So Rails 2, this was released in December of 2007 and Shopify updated to it pretty much right at the same time just a few days later after the release date
11:03
because we were running pretty close to the edge. And the Rails 2 series was probably one of the best series of Rails ever by a lot of accounts. And there was a lot of really cool stuff that was added in too in some of the point releases that a lot of people really love.
11:21
And it was a good combination of being really fast and then having a lot of features that got the job done and you didn't have to do a lot of stuff on your own. Like Rails was taken care of a lot for you too. So Rails 2, some of the notable things that were added were rescue from and fixture dependencies,
11:40
namespaces and routes. Multi-view responses was kind of cool so you could register your own mind type aliases. So this would let you do the registering like an iPhone alias for something if you want. This was pretty early on in the mobile web. Another interesting one is Action Web Service
12:03
which was removed in active resource which came in. So a lot of people probably don't know or remember Action Web Service but it was basically SOAP RPC protocol support and WSDL generation for APIs and all kinds of nice things like that. Luckily REST won out and Rails chose REST
12:22
over all of that stuff which is another kind of opinionated view and drove things forward. But they were exploring different ideas at the time. So looking at some Shopify code and what rescue from cleaned up, we used to have this override of rescue action
12:41
that was taking errors in and there was this massive case statement that I actually trimmed down. There was like 15 different conditions in there for all the different errors. And so rescue from, a lot of us know it still because it exists now. But it let us clean that up a lot so you could encapsulate each of those error responses
13:00
on their own instead of jamming them all into the same case statement. Fixture dependencies were a big win because at the time you had just static integer IDs that you were including. So you had to reference those directly which was really painful. So this let you reference the shot by its name
13:20
in the fixture instead of having to reference the ID. And then if you needed to, you could use the fixtures.identify if it's ambiguous. Namespacing also allowed us to clean up a lot of code where we'd have this mapping of resources with a mess of options there.
13:41
We can now nest them nicely and encapsulate the different namespaces and keep things from getting really cluttered and make it easier to understand and read the routes as they grow over time. Rails 2.1 didn't have a huge number of changes
14:01
but it's notable mainly because it added support for config.jems in the environment.ib file. So this was really cool because as I said, up until this point you were kind of left on your own to manage your gem dependencies and this was the first time you were given a pretty decent system to manage your gem dependency in your Rails app.
14:20
So what this looked like was in your initializer in environment.rb, you could just configure which gems you were using and you could even specify the source. At the time, a few months after this was released, GitHub actually became a gem source and you could have all your username dash gem name and there was a pretty big mess with that
14:41
which changed once Bundler and Ruby gems and everything kind of evolved over time. But it was interesting because this really helped to manage dependencies where we didn't really have any help at the time and you were kind of left on your own to figure out how to manage dependencies.
15:03
And Rails 2.3 is another big release. So this was released in March 2009 and Shopify actually hit it in March as well. And so I would say Rails 2.3 out of all the 2.0 releases is probably one of the biggest and one of the best
15:21
with the number of features and kind of what it gave you and it's kind of the pinnacle of the 2.0 release cycle. And again, like I said, it's like a good mix of strong performance and giving you a lot of features to kind of take care of what you need to do. So this introduced Rack which wasn't in Rails in the previous versions.
15:42
You have accepts nested attributes and there's a few other niceties that were added. Application templates were added which DHH showed with the dash dash API now. So a lot of the stuff building up to some of the things that we see now in our Rails apps as well. So Rack, a lot of us know how Rack works
16:02
and know what it is these days. But this is just kind of how Shopify started using Rack in those kind of early days. This was the first middleware we added to the application just as a simple blacklist for a application that was spamming our site. And so we would just 404 this particular request URI
16:22
because they were just kind of scraping our site. And it's a great use of Rack and these changes to Rails really made it easy to do these types of things. Accepts nested attributes is another one where we kind of take it for granted, I think. So we actually had a whole bunch of code in Shopify
16:43
where we were trying to do the accepts nested attributes kind of thing. And it was just like really terrible. I actually wasn't involved in any of this, but going through and looking back at it, it was just really interesting to see like the kind of hijinks you had to do to kind of make it work. And then if you switch over to using
17:02
accept nested attributes, it's really clean and really obvious what's going on. So it simplifies the code base a lot switching over to that. Well, Rails 3 is another pretty significant release. And it came out in August
17:21
and Shopify updated around October. So we were pretty close to that one as well. And this was a pretty large one. And it was also the first major release since the Rails team merged with the Merb team. So there was a lot of new ideas coming into Rails 3 being brought by the Merb team
17:41
by kind of breaking up different parts of Rails, making it agnostic to a whole bunch of different things and adding a whole bunch of niceties so that you could plug into Rails a lot easier and make it your own kind of framework. This came with a lot of performance degradations and this was a lot of the kind of flack
18:02
that Rails got at that time and a big reason why a lot of applications actually stuck with Rails 2, 3 for a long period of time. We moved on to 3 pretty quickly because the kind of developer productivity that we got was a lot higher than any performance impact that we saw
18:21
and it was worth the hit to be able to onboard developers and to just have a lot nicer code and easier to manage code base. So looking at a few of the features, Arel, like it's really popular now, like we see Arel everywhere and it was introduced in Rails 3.
18:42
Previously you had to do some kind of awkward syntax again to kind of do the same things and it just really tightened things up and made it really easy to do the things that you'd need to do with your associations and your queries. Bundler was also introduced at this time and so this took a lot of like kind of similar ideas
19:03
to what config gem in the environment RB was doing but pulled it out into a separate system and Bundler again is one of those things that early on a lot of people hated on because it was different and kind of changed the way you work but these days is probably one of the best things
19:20
about Rails and one of the things that actually makes it possible to work on a large Rails app with a lot of dependencies. I don't know, like it's hard to remember what it was like before Bundler and before this dependency management but it was pretty terrible time and Bundler makes a lot of this easier
19:40
and we actually converted Shopify to use Bundler before 3.0 so that we could take advantage of that while we were still running Rails 2.3.5. So the next set of releases are 3.1 and 3.2. So these ones are pretty big ones again and we actually didn't update to 3.1.
20:02
We attempted it but found a whole bunch of performance issues with it and then we never followed through with it. There was no dedicated team on it so it just kind of sat beside until we drove forward with 3.2. So within 3.1 and 3.2,
20:21
the noble changes are asset pipeline, jQuery becoming the default JS library and just lots of internal API changes. So when I was looking at what we eventually merged as the Rails 3.2 update, there was like 250 change files and like the additions deletions were pretty even
20:43
but there was just a lot of them because a lot of internal APIs changed particularly around associations and kind of the internal association proxy objects. So this was a pretty big change for us and asset pipeline is something that we had wanted to use
21:01
because we were working on the JavaScript front-end MVC app so asset pipeline was actually really interesting. So we ended up backporting asset pipeline to 3.0 so we could use it. So this was just our generally pretty messy set of updates but we eventually got onto 3.2 and got that pushed out.
21:28
Then Rails 4 came along and it was released in June 2013 and we got on it in February 2014. So at this point, Shopify's getting pretty large and complex.
21:41
There's a lot of moving pieces, a lot of code and a lot of people working on it and it's getting harder to keep up to date when we're not paying as much attention to it so the updates are being drawn out a little bit longer. But Rails 4 had a lot of really cool stuff that we wanted to take advantage of
22:01
so it supported Ruby 2 which we wanted to move towards, Turbo Links, the Russian doll caching, strong parameters, the killing off of observers and then just generally tightening up APIs like removing the hash and dynamic finders. So a few bits of code from Shopify here
22:22
and it just really getting more consistent APIs and tightening up on kind of the ideas that were put forward previously with Arrow and all of that and just making it consistent across the different APIs and different syntax that you might use.
22:46
So the latest version that we've updated to is Rails 4.1 and so this was released in 2014, April and we got on it in January 2015. And so we're on 4.1.8. And again, we didn't really dedicate a whole team to it,
23:01
it was kind of done on the side but we have learned a lot about keeping up to date with these things which I'll get to in a couple more slides. So kind of the notable things in Rails 4.1 are the Spring application preloader which was cool because we were actually using it previously and it was just really great
23:22
to see Rails like just baking in these same defaults and this kind of like the trend over time you see them take the really good parts about how people are building and using Rails and baking them in so that you don't need to do it yourself and it just comes packaged with all these great things to be able to build your applications.
23:41
And the other notable feature for us was variant templates because we were starting to use this for our mobile applications as well. So a little code sample from Shopify itself of the variant templates. It's pretty straightforward, basically just setting the variant based on the user type and then Rails automatically knows how to pick that up
24:02
and render that view for you. And so that brings us to today. And so looking back at kind of how we've evolved over time, it's been really interesting looking at how we've managed these updates from the early days when we were on Edge
24:20
and the code base was a lot smaller and more manageable to as we've moved to a much larger code base with a lot of people and the difficulties that that has brought. So it started a lot easier and it's been getting hard but we've been learning how to deal with that pain of just the large code base and what we're doing with it.
24:46
It's kind of what have been the hardest things over time. Marshaling changes have been a big deal and this isn't strictly related to Rails but it's come up with Rails updates. It comes up with more with Ruby updates. But just when Ruby or Rails are marshaling objects
25:03
or changing the format of our objects, that's been really difficult for us. So we have a caching layer that actually marshals full active record objects and so that was really difficult as Rails changed the internals of active record and some of the variables that it has. That gets difficult.
25:21
And then also changes to things like flash formats and sessions and just anywhere you're gonna be serializing an object and expecting one format and then maybe get it differently in another request. And so kind of the way we've solved those issues or have learned to solve those issues is to either write them in parallel in two different stores with two different versions.
25:42
So depending on the request, you can pull out the right one or having some translating code. So in the case of flash format, the translation between the version was pretty easy. So we just had a piece of back ported code that would figure out which version to deserialize and do that for you.
26:02
Another big thing is just maintaining the momentum with a large team. So we have over 150 people contributing to this code base and we don't wanna slow them down but we also don't want them undoing the changes that we've been making or reintroducing deprecations. So this has been really challenging and kind of the one, the biggest thing
26:21
that we've found to minimize that is just to release the changes as early as possible. It's even as simple as we add an environment variable so you can toggle between the different Rails versions really easily and then be able to work on the upgrade in parallel and ship as much of the code into production
26:42
as early as possible. So in the end, the Rails 4.1 and even the Rails 4 updates ended up being dozens of files changed, if that many. So it really small set of changes compared to that 3.2 update where we changed thousands of lines of code.
27:02
And then just the size of the code base and lots of edge cases. So this makes it really hard to actually know all the bits and pieces and everywhere that might be doing some specific Rails thing or might be monkey patching Rails in a particular way. And then running into performance regressions
27:22
just because of the scale of the code base and the amount of servers we're running on, the amount of load we're getting. We hit a lot of performance regressions that people might not see. So with all that in mind, why do we keep upgrading and why do we keep moving it forward?
27:43
And it's really important to us to move it forward. So the big reasons are the new features. So things like turbo links and variants and all those things are reducing the complexity of our code base and giving us a larger base to leverage. So as much that we can push into Rails itself
28:01
and take advantage and not have to maintain completely ourselves in that half a million line code base is a big benefit to us. Getting better security and updates and better practices is huge. Hiring is a big deal. So as we're growing and as we've been hiring a lot,
28:20
bringing people on is a lot easier when we're moving onto newer versions of Rails because they'll either know it or it's easier to learn because of the documentation or the existing resources that are out there. Then also the code base longevity. So the code base has been around for 10 years. It's likely gonna be around for another 10 or 20 years.
28:42
So really we actually need to do this. There's no sticking to a particular version. Otherwise it'll just become unmaintainable. If we are thinking about keeping this around for another 10 or 20 years, we need to continue moving forward with Rails.
29:01
So kind of recommendations for anyone working on keeping their Rails app up to date. And I kind of hinted at some of these. But a few big ones are like avoid monkey patching Rails itself. We've done this in a few cases and it's really hard to figure out and it ends up biting you really badly.
29:20
Keeping dependencies low. So every gem, every library that you depend on is gonna be another thing that you need to either investigate or update yourself or throw away and rewrite when you update Rails if the internal APIs are changing. So keeping those dependencies low and to a minimum is really important.
29:41
As I said, shipping changes early and often. So and having like a parallel CI running so you can do that is a big deal. So with Rails 3.2, we didn't really do that. We had like a big bang release of Rails 3.2. But with Rails 4 and Rails 4.1, we did a lot more progressive releases of the changes.
30:02
We had CI running so we could see what was breaking and like the final flip was like 30 lines of code for the 4.1 upgrade. And it was mainly one person part-time kind of working on pushing it through which was pretty significant. And then also having a dedicated team. So depending on the scope of the change,
30:22
making it someone's part-time job doesn't really work in a lot of cases. So dedicating people to it for a short period of time makes a big difference. And then being able to ship to isolated production servers has been a big deal for us. So we've been able to ship some of these changes, particularly the marshaling changes to a subset of servers so we can actually investigate
30:43
how it's gonna work in production and see how those changes are gonna take effect. And then we can roll them back really easily without impacting users very much. So that's it. If you wanna talk about anything, I don't know if we have any time for questions,
31:01
but I'll be around and just talk about how we've kept Shopify going on Rails and happy to show you guys what we've been doing or talk about what we've been doing. Thank you.