Okay, so I'm doing an experiment where

I'm trying to see how much I can get away with before they stop inviting me to these things. So, alright, I've asked the, if you follow me on Twitter, you know what's about to happen.

If you don't follow me on Twitter, you're in for a treat. I've asked the organizers to place security at the back door, so you can't leave now, and I will begin. I don't understand why people like static typing so much.

It's impossible for me to program without moving my hands. I heard a team of French computer scientists are working on a JIT for Ruby, they call it Legit. The second version will be called ToLegit. I recently went to a JavaScript conference, but all of the node developers put me on edge.

It's nine a.m., yeah! Apparently, Action Cable is all about streaming, so why didn't DHH call it Action Netflix? The problem with Action Cable is that I'm paying

for more channels than I watch. I'm really excited about the new text-based browser that DHH announced yesterday, Turbo Links. Where's my, how much time do we have left?

Okay, alright, I will stop punishing you for a while. Alright, hold on, hold on a sec, let's get this, let's get this started, I gotta get this started up here. Sorry, I should've prepared. One sec, sorry, hold on.

Okay, alright, I think we're up. So, you're probably wondering why I'm using Windows XP.

So, I'll explain why I'm using Windows XP, and the reason is because Kent Beck is here at this conference, he's gonna be giving a keynote, and so like, I'm a really huge fan of Kent Beck, and I went to his Wikipedia page, and it turns out that Kent Beck invented XP,

so I thought I'd use XP for this presentation. Unfortunately, Windows XP has no emoji, and I really like to use emojis, so instead I have to use Wingdings.

So, I'm sorry, this presentation will lack emoji. Anyway, so, welcome to RailsConf, I'm really happy all of you are here, and that you all came to watch me. I really enjoy going to RailsConf, it's like, if you attend RailsConf, you'll get to meet a lot of the core team developers, like me, and most of the core team comes to RailsConf,

and I mean, the reason that, you know, the reason that most of us core team members have to come to RailsConf is so that we know what the next version of Rails will have. Like an action cable. So, my name's Eric Patterson,

if you don't follow me on the internet, I am Tenderlove, if you do not recognize me, this is what I look like on the internet. So, just so that's clear, this is me, in fact. So, I gotta start out and say thanks to everybody. I work for Red Hat, so thank you Red Hat, yes, I'm using Windows, sorry.

I enjoy working at Red Hat, I'm on the Manage IQ team, we write software that manages clouds, so if you have a cloud, we will manage it. I also wanna say thanks to Ruby Central for having me here and having RailsConf, I'm really happy to be here. I gotta say thank you to a couple more people too.

Eileen Codes, she wrote a lot of the code that's in my presentation, you should've seen her yesterday, if you didn't, well, you should have. And also, Koichi, he is going to be giving a presentation right after mine today, I think he's one of the first talks, so you should go see his talk, he wrote Ruby's Virtual Machine, so you probably wanna see that.

And then I wanna say thank you to all of you for coming. And I need to have cat photos in this talk, because that's what I do, I've got cats, this is one of my cats, this is Gorbachev, Gorbachev Puff Puff Thunder Horse, this is my other cat, Choo Choo, and she helps me pair program a lot.

That is my keyboard, it's very convenient to have her there. I also brought stickers of my cat, so if you would like a sticker of my cat, come say hello to me, the reason I bring these to conferences is because I'm a very shy person, I don't know what to say to people, and I know people don't know what to say to me, so if we have this awkward moment,

you can say, Aaron, give me a sticker, and I will just give you one. So I've also been working a lot on doing extreme programming, like brushing up on extreme programming, and I'm thinking about putting out a new product that I've been testing out at home for extreme programming, I think we need

to do extreme programming a lot more safely, like I think we're doing it too dangerously these days, so this is what I typically wear is when I'm extreme programming, because I don't want to get hurt. So last year, last year I gave a keynote, last year I was a thought leader in training.

I just want to recap what's happened to me over the past year. Well, I actually graduated, so come on, it was very difficult. I got a degree in thought leader from an online university.

Unfortunately, after I earned my degree, I found out that online university is not an accredited university, and that thought leadering is not an actual degree, but I'm gonna use my thought leadering presentation skills anyway, so are you ready?

Yes, okay, so we're gonna do some thought leadering now. Okay, hold on, I gotta get my thought leadering face on. All right, Ruby. Rails. Technology.

Rails. Computers. Rails. Synergy. Rails. Innovation. Rails. Future.

Rails. These are all words that I am saying to you, because I don't know what to do with an hour on stage.

So, this is the worst presentation ever, I'm sorry. All right, anyway, so now I gotta do some trolling

on DHH, because this is what like, this is what I really enjoy doing, like I like to watch a presentation, and then like, ah ha ha ha ha ha. So anyway, like I was really really hoping that this year there would be like a TDD is dead thing like last year, because that was gold.

So I think this year, like it wasn't as extreme this year, so I think like I'm just gonna say it for him, SOA is dead. So, just that is it, that is it. It was really exciting to see that we announced Turbolink 3 RJS.

So also watching that presentation, learning about the new features that are supposedly coming in Rails 5, I thought to myself, I should announce something. Like, just something brand new, like okay,

we're gonna put this into Rails. So, you know, DHH talked a lot about his backpack. So he's talking a lot about a backpack. So I want to announce something today. I want to announce something that's a lot more lightweight than backpack. Much more lightweight, like I want to do, I'm working on a lightweight framework that's gonna ship inside of Rails 5,

and I'm calling it Action Fanny Pack. It only comes in one color, pink. Alright, let's talk about some tech stuff, finally. So, I'm gonna talk about performance,

because I love performance, that is what I love working on, I apologize for the early time, I guess it is only 9.15, and we are going to talk about some very technical stuff. Maybe it'll be in Rails 5, I don't know. I'm not bound by your timeline, saying,

versions. And do what I want. So, anyway, I'm gonna talk about some of the things that I'm thinking about and working on, and we're gonna start from the very, very nebulous things and get more specific as we go through the presentation.

So we'll start out with stuff that I'm just thinking about and working on proof of concepts, and there's, you know, not much, like it's definitely not gonna ship in Rails 5. Well, I can't say that for sure, but I'm gonna talk about more nebulous stuff and get more specific. But I have to admit, I had a very, very hard time

putting together this presentation this year, because last year I talked about adequate record and the work that I did on that, and the work that I presented last year, it took me three years to do all that work, so my presentation last year was the culmination

of three years worth of work. So as I'm working on this presentation, I'm like, what am I gonna talk about? This is going to pale in comparison to the speech that I gave last year. This sucks. But I'm gonna talk about the projects that we're working on, or that I'm working on, and I hope that we can present these,

or I can present these as adequate developments in the future. Adequate developments. Hopefully they will be fast enough. So the first thing I wanna talk about is automatic parallel testing, something that I really wanna have in future versions of Rails. Maybe Rails 5, I'm not sure. And what I mean by automatic parallel testing

is that, well, just exactly that. I want your tests to be able to run in parallel and do it automatically. So I want it so that when you upgrade to Rails 5 and you do Rails test, it'll just run all of your tests in parallel, and your tests will just automatically be faster, and you don't need to do anything.

So this is my dream, and the way that we're thinking about doing this is by having multiple test processes, splitting your tests up into multiple test processes. We introduced, I think Rails 5 will introduce randomized tests, which will help you run your tests in parallel eventually.

But the idea behind this is that we'll have maybe one database with multiple test processes hitting that database. So there's two ways that we can do this parallel testing. We can have one database and multiple processes hitting that database, or we can have multiple databases. And this is one of the experiments that I was working on,

and the idea behind this is that, well, you know, our website runs in parallel, we have multiple processes running, and all of those are hitting a single database. Clearly, we can do things in parallel. So why don't we have multiple test processes running against one single database, and then just each test opens a transaction

and rolls everything back at the end. That way we can get parallelization. So I tried this out. I actually put together a proof of concept with this and found that I could only get about, oh, 15% speed improvement on the active record tests with this particular technique, so I wasn't very happy with it. The other thing, so the other thing that I'm looking at

is doing multiple databases. Now, the idea behind this, and there's already a lot of gems out there that do this, is say, okay, we're gonna take our database and split it into multiple copies, and then just run a set of tests against each of those databases so we can do it in parallel. But unfortunately, the solutions that are out there today

are very manual. You have to say, like, well, I want this particular, you know, I need to set up end databases and do all this stuff. I want it to be completely automatic. So this is the next thing that I need to test out, and I said this is very nebulous. I have not tested this particular technique yet, but I think we can get much better parallelism

out of this technique. So the important things to me is that this has to be automatic. I don't want anybody setting up anything to get it to work. It has to be transparent, so it should work. I mean, you should be able to upgrade. All of your tests will run in parallel,

and nothing will break. So the next thing that I'm working on is caching compiled views. Now, this is a little less nebulous. We actually have some code that works with this. And what I'm talking about with caching compiled views is your actual views in your Rails application. So today, we do lazy compilation.

So if you have Haml or ERB templates, the way that it works in Rails today is when a request comes in, we get a lock, and then we say, hey, is the template compiled? If the template is compiled, we unlock it. If it isn't compiled, then we compile it, we save it into memory, and then we unlock.

And what I mean by compiling is taking that ERB or Haml code and turning that into Ruby code. So we take your template code, turn it into Ruby code, evaluate that Ruby code, and then cache it. So the problem with this particular setup that we have here is that this step, this compile and save step,

actually writes the memory. And the reason this sucks is because if you're running a threaded web server, like Puma or whatever, the threaded web servers have to lock, so you have this locking bottleneck there. Now, the other problem is this is also bad for forking web servers because we're wasting memory. Now, this template compilation doesn't happen

until after a request comes in, so when you start up your unicorn, unicorn forks off however many processes you have, and now every single one of those processes is compiling the same thing and storing it into memory there. What would be nice is if we could compile all of those on boot, do it in the master process, and then all of the children processes

will share that particular memory. So we can save a lot of memory in that case. Now, unfortunately, there are a lot of challenges with this project. I'm gonna talk about some of the challenges and how we're thinking about getting around them. Some of the challenges are like, well, I have too many templates. I mean, as I'm telling you this,

we're gonna precompile all the templates as soon as you boot up. Obviously, this is gonna impact your boot speed because we're compiling all the templates, and maybe you have a website that has millions of templates, but unfortunately, you only use two of them. So now we're compiling millions of templates, and in fact, we're actually wasting memory

because you normally would have just used two of those millions of templates. I would say you need to delete those templates, but we should probably have something that you can configure and say, I only want these particular templates to be precompiled. Now, that's not so much of a challenge. We can do that, not a big deal.

Configuration thing, default will compile everything. Otherwise, you can say I only want these particular ones compiled in advance. Now, this is not a very big challenge. The biggest challenge, in my opinion, is rendering with locals. When you do a render template and you say, colon, locals, and you give it that hash,

what actually happens under the hood, if you go look at the way the template rendering engine works, is you'll see it looks a little bit like this, where we say, okay, we're gonna put a prelude above your compiled templates. We take your template, convert it into Ruby code, and then we make this little prelude based on that hash of locals that you passed in,

where the prelude sets local variables equal to some particular value from the hash. Now, unfortunately, this is dynamic, and we can only know this at runtime. We only know this information as soon as that render call happens, right? Which means we have to do it lazily.

Now, what we're thinking of doing is looking at the source of the templates and trying to determine what locals you actually use. So maybe we can come up with heuristics. We can definitely come up with heuristics for templates that can't have locals, but maybe we can come up with heuristics for ones that do have locals and only precompile sets that we know.

So this is one of our challenges, and I'm not telling you, don't use locals. Please use locals, it's fine. Please, really. So this is one of the biggest, this is the biggest challenge that we have, and yes, it's still a little bit nebulous, but we do actually have code for this up on GitHub.

So the next thing I want to talk about is the integration test. Let's take a look at an integration test, or no, let's take a look at a controller test first. Let's look at a controller test. Okay, this is a controller test. All right, do a get, do an assert, and we're done.

Okay, everybody's familiar with this, great. Let's look at an integration test now. Okay, do a get, do an assert, and we're done. So controller test, integration test, controller test. All of our programmer brains are going,

hmm, these look very similar. So I thought to myself, I was like, why, you know, why do we write controller tests? Why, why? The only reason I could come up with is, well, integration tests are too slow. This is the only reason I could come up with.

What's interesting is if we look at these integration and controller tests, we're testing at different levels, but nothing about these tests says what assertions we can and can't make about these tests, right? Anyway, so we write controller tests because integration tests are slow. So something didn't make sense to me about this

because I just kept thinking about it. I'm like, well, you know, if integration tests are slow, does that mean that our website is slow too because it's an integration test? And, you know, something doesn't make sense. If the application is fast enough to be serving up

requests, why is it too slow to be writing integration tests? So this did not make sense to me. So for Rails 5, this is left of a question mark, I wanna have faster integration tests. So we've been working a lot on doing, implementing faster integration tests. So if we think about controller tests and integration tests, what's the difference between those two?

Controller tests have, I'm gonna tell you the difference between them, and this is at the framework level. If you go look at the difference between them, the main difference is that controller tests have no middleware, so when you're testing a controller, you're not going through any of the middleware in your application, where integration tests do have middleware, but these are really the only differences

between the two. Integration tests go through the middleware and hit the router, controller tests do not. You're testing directly against the controller. So what I wanna do is I want to delete controller tests. I just wanna delete them. Just go away. Okay, that's okay. You don't need to clap, it's fine, it's fine, it's fine.

And I don't mean I'm just gonna delete them and you're gonna upgrade and go, oh my god, my controller tests don't work anymore. What I wanna do is I actually just wanna delete the controller testing framework and implement it in terms of the integration testing framework, so that when you upgrade, everything will work the same, it's just that we'll only have an integration test

framework under the hood, right? So in order to do this, we need to figure out why these integration tests are so slow. So why so slow? First we need to know how slow they are. So the first thing we did, and by we, I mean Eileen.

She writes the code, I do the presentation. Anyway, so this is what a benchmark looks like for it. Now we're doing a benchmark, this is, the top one is a benchmark against an integration test,

the other one is a controller test, and we're testing exactly the same thing, we're testing the index. So the top one's integration, bottom one's controller test, and then we compare the two, and the thing is, I don't actually care how fast either one of them are, I only care how fast they are with comparison to each other, right? Because we wanna make sure that integration tests are just as fast as controller tests, as we can.

So when we ran this, we found, well, we did the comparison, and it was 2.4 times slower. Integration tests were 2.4 times slower than controller tests, so we needed to know what is slow, why is it so slow, and in order to figure this out, we used a gem called stack prof,

and this is built on profiling stuff that's inside of Ruby, maybe Koichi will be talking about it later, I'm not sure. But we used a gem called stack prof, and this is the test that we did, we said, okay, we're gonna run one method against the controller test, and I wanna know what the bottlenecks are in the controller tests, and then I'm gonna compare those to the bottlenecks

in the integration tests and see what the difference is. So if I can see what the bottleneck differences are, maybe I can understand what it is about integration tests that makes them so much slower than controller tests. So we have a controller test, we're doing a CPU profile on that, which means that we're timing

how long the code spends on the CPU. I'm gonna talk about that a little bit. And then we dump out the profile results to this particular file. So I wanna talk a little bit about CPU time versus wall time. We configured this test to use CPU time. What CPU time means is, well, I only wanna know about time that my code is spending on the CPU.

So to give you a concrete example, what this means is, well, the sleep command is slow, but it doesn't use CPU, right? So if you said sleep for 10 seconds, your program's gonna be slow, it's gonna take at least 10 seconds, but that's not gonna show up on a CPU profile. So you won't see things like sleep,

you won't see things like IO. So I said, okay, we're gonna view the stack, we're gonna dump out the stack. This is how you dump out the stack. The stack looks like this. We ran it, really excited, because if you look at the top 53% of our time was being spent in mini tests inside this method. And we're like, wow, okay, amazing.

So we can, if we fix this, our code is gonna be 53, or twice as fast, or whatever. Some number. It's early. Cut the time in half, I don't know. Oh boy. All right, so how do we make this method faster?

I'm gonna show you my number one favorite technique for making a method faster. This is my number one favorite technique, is to delete it. So we change it to this. We go, okay, let's just yield. We don't need to just delete all of that code.

So rerun the benchmarks, look at the results, and we see, okay, boom, that is no longer in our profile. 50% faster, I am extremely happy. Yes, wingdings. It's the worst, man. All right, so I timed the process,

and this is the before time. It took about 12 seconds. And then this is the after time. It also took about 12 seconds. You will note that this is not 50% faster. So I was sad about this.

So the thing is, we know that this test is CPU bound. We're not doing anything. We're not doing any sleeps. We're not doing any IO calls. What is going on here? So if we change the profile to say, if we change the profile to use wall time rather than CPU time, we should be seeing the same stack.

Wall should be similar to CPU in this particular case because we're CPU bound, we're not IO bound. So we rerun it. This is the CPU stack, and if you compare it to the wall stack, you'll see that they are completely different. Maybe not completely, let me look again. Are they completely different? Yes, they're totally different.

Which is not what we expected. So I contacted Eman who wrote the profiling tool, and I was like, hey, I've got a CPU bound test here. When I do a wall profile versus a CPU profile, it's totally different. And he said, well, there's probably a bug in your code. And I said, I don't think so.

And then I showed him, and I was like, it's really weird. And he was like, yeah, that is weird. And I said, can you fix it? And he's like, no, I have no idea. So then I contacted Koichi about it because the stack profiler uses Ruby's tools under the hood for doing this stuff. And I said, hey, Koichi, I've got this test.

It looks weird, the results are very weird to me. Can you take a look at it? He looks at it, too, and he's like, huh, yeah, those are weird. And I'm like, oh, okay. So he runs a test on Linux. He's like, I'm not getting the same numbers as you. Then runs it on OS 10, then gets the same numbers.

And it turns out that it seems like there is a bug in trap only on OS 10. So there is a bug, and we don't know what it is. So the point that I'm trying to make here is that even profilers have bugs.

So you need to verify the test results that you're getting. Now unfortunately, we've walked through all this stuff and we were talking about getting things faster and improving the, making speed improvements. And we made no speed improvements, so I have wasted the last 10 minutes of your time. I am very sorry.

So let's do something useful. Let's do something useful. Let's do some integration test profiles. And this time, we're only gonna be using wall time. So we looked at the profile of integration tests to see if we could get them as fast as controller tests. This is what our stack looked like for wall. And it turned out that we were spending 27% of our time calling delegate.

So calling delegate was slow, so we rewrote it a little bit. We rewrote our implementation to stop calling delegate. This is the patch that we applied. And this is actually, this is fairly old, but we made this change a few months ago. And there are other caveats about this which you should come talk to me about.

Astute readers will notice the issues. But this sped it up. So we'll check our progress. Before, we were 2.47 times slower. We run it again, check our comparison. We are now down to 1.45 times slower. And if we run time against our stuff, we'll see before it was 26 seconds,

and now we're down to 16 seconds. So we made some improvements. So even profilers have bugs. You need to check this. Don't use just one profiler. Check your numbers. Always measure the code that you're working on and verify the results that you have come up with

with something else. So this project that we did is about a few months old. Today, integration tests are approximately 10% slower than controller tests. And we've actually improved the speed of both controller tests and integration tests such that when we ship Rails 5, if we delete the controller testing framework,

the integration tests will be faster than the controller tests were in the previous version. So our goal is to make sure that if we did delete the controller tests, they had to be at least as fast as the previous version. So it should be faster, faster than controller tests in previous versions.

Don't thank me. Thank Eileen. Ha ha ha. So the next thing I want to talk a little bit about

is soup to nuts performance. I don't even know what this stupid phrase means. So what I mean by this is, I wanted to know, as a Rails core team member,

I wanted to know what is the performance of Rails, but I don't want to know just from inside the framework. I want to know as soon as I hit enter, what is it doing? How fast are we going? Can we make that faster? Now, in order to study this, in order to study this performance, I had to study Bundler.

And because very frequently my command is bundle exec. Bundle exec, whatever, right? And in order to study Bundler, I needed to study bin stubs. And what I mean by bin stubs, since this is extremely confusing, I am not talking about the stuff that you store in bin Rails. I'm not talking about Bundler's bin stubs.

I am talking about the bin stubs that RubyGems installs. So when you install a gem on your system, that bundle command is actually generated by RubyGems when you installed Bundler. So you're not actually running the Bundler bin file that ships with the Bundler gem.

You're running a bin stub, which in turn calls the Bundler bin file. So if we look at the bin stub source code, it looks like this. This is the bin stub source code. So I needed to study this. And you'll notice that we have this gem line, or it says gem in there, so I needed to study gem. And in order to study gem, I also had to study require.

Okay? So the question is, what does gem do? I gave a talk at a conference called CUSEC, C-U-S-E-C,

Canadian University of Software Engineers in Canada. It was a conference that was all for Canadian university students. So it was a conference put on by university students for university students. And of course, I had to make puns about John CUSEC, right? So I had,

so I've got like tons of references to his movies and stuff, but you know, they're all 80s movies. And I ask everybody, I'm like, okay, raise your hands if you're over 35. Two hands.

Anyway, so what does gem do? What does the gem method do? The gem method finds a gem spec. Finds a gem spec with a particular name, and then it puts that gem spec onto the load path. Puts it onto the load path. So it does load path manipulation.

And we can see that, you can pull up IRB and see this today. So this is an example. We're loading the bundler gem. So if you look at, this is duping the load path, loading the bundler gem, and then checking the load path afterwards, and you'll see that there's a new path on the load path.

So okay, we know about that. Now what happens when a file is required? This is another interesting thing we need to know about. When a file is required, it does require, and this is so confusing. RubyGems has its own require, which is different from Ruby's require, and I am talking about the require

that's built into RubyGems. When you write your Ruby code, you probably don't know which one you're using. And it depends on the situation. If you're using bundler, you're using Ruby's require, but if you're just writing a regular old Ruby script, you're using Ruby's require, which is super fun to figure out and fun for new people to understand.

And I am going to talk about RubyGems' require. This is the one that is inside of RubyGems. What does it do? It says, okay, if the file is in the load path, then we load it. If it is not in the load path, then we look for it in a gemspec, and then we add the gemspec to the load path, and then we load the file. Make sense? Okay, good.

I love these PowerPoint transitions. So awesome. All right, so I wanted to study RubyGems' performance. We'll talk a little bit about my test methodology here. Test methodology is, I wanted to require one file, and then increase the number of gems

that are on the system. So we require one file, and then see, increase, add more and more gems on our system, and see what the performance is of require as we increase the number of gems on the system. I also wanted to do the same thing with activating gems. So let's say like the gem command.

We add one gem, or we activate one gem, and then increase the number of gems that are on the system. So the same test. Now, I started testing this, and I'm gonna show you a graph from it. And as I was testing it, I found that the time was linear. So we expected, I saw linear growth while I was testing this.

So the x-axis is the number of gems that are on the file system that are installed, and the y-axis is the time in seconds. All of these gems are generated. I generated all of these gems. They all contain one file, and all of the gems have no dependencies. Now, what was interesting is, since we found that it was linear time, or since I found that it was linear time,

it meant that Ruby gems was probably looking through an array, right? It's probably trying to find something in an array, which would be linear time. So what I did was I found the array that it was searching through, and this graph that you're looking at, this graph represents the best and worst case scenarios.

So the pink line is if the file was at the very beginning of the list, and the green line is if the file was at the very end of the list. So probably when you're doing a require, you're gonna land somewhere in between these two lines. So this is how long it took to require one file. Now, I also looked at allocations to require one file.

So this, as you require a file, what do the object allocations look like? This is both linear growth, exactly the same thing, like same test, just testing memory allocations, so best and worst case scenarios. And I think it's pretty interesting. You can see the slope here is much different

than the slope on the time, and I think that says something about our garbage collector, that our garbage collector is good. So then I wanted to look at time to activate one gem. So this would be doing just gem bundler, or whatever gem, right? So I tried the same thing. I saw that it was linear time.

I'm like, show me the best, show me the worst, and it turned out they were exactly the same, exactly the same growth. So then I looked at allocations per gem, and I've only got one line here, also linear growth, and I only put one line because they're exactly the same. Okay, so we got require performance,

we got gem performance, so now let's take a look at bundler performance. Okay, so my test methodology for this, and I'm much more familiar with the Ruby gems code than I am with the bundler code, so I don't know why, I can't explain to you any of these numbers,

I can explain to you my test methodology, but I can't tell you why these numbers are the way they are. So we have, the test methodology here is I put one gem in the gem file, and then I increase the number of gems that are on the file system. Then I also did another test where I said, okay, I want to increase the number of gems in the gem file, but keep a constant number of gems on the file system.

Now remember, as I said earlier, all these gems have no dependencies, all of these gems are local to my machine, so I'm not hitting the network in any of these tests. So if we look at varying the number of system gems where we only have one gem in the gem file, we'll see we have linear growth doing bundle update. So again, x-axis is number of gems on the system,

y-axis is the time in seconds. Then if we look at bundle, exactly the same thing, just different times, both linear growth. I kind of expected this, since Ruby gems under the hood was linear growth, and we're linearly increasing the gems on the system, so this is not surprising to me.

The next thing I did was I varied the number of gems in the gem file, so I said, okay, let's keep a constant number of gems on the system, and let's increase the number of gems that are in the gem file, and look at the performance as we increase those gems. And I have extremely good news for you, the good news is that the time for bundle exec, if you run bundle exec, it is actually constant time.

So we're seeing a lot of noise in here, but it's actually about the same time between each run, all the way up to a thousand gems in the system. So that's very good. Unfortunately, that time is like one over a second, which is not good, we would like to get that smaller.

And what I think is interesting is if you time, what I found was that Bundler was loading all the gem specs on your system. So I found that loading those gem specs, if we did a breakdown and just timed loading all the gem specs, we'd see loading all those gem specs took about 710 milliseconds, and we're spending another 750 milliseconds

doing I don't know what, but the good news is that it's constant, it doesn't grow. So all we have to do is we find that other stuff, probably reduce that, see if we can find a way to reduce spec loads and we'll have constant growth that's smaller. Now, the next test I ran was okay,

I wanna do, I'm growing the number of gems on the system and I wanna run bundle update. This is the one that actually surprised me the most. This is what it looks like when you run bundle update and you increase the number of gems on your inside the gem file. So x-axis is number of gems in the gem file, y-axis is time in seconds.

Now, when I was first running this, I was doing, I wanted to test with 100 gems, 1,000 gems, 10,000 gems, and then 100,000 gems. That was my plan, so just do those numbers and see how it grew. Now, unfortunately, when I ran against 10,000 gems, I was sitting there for hours.

I don't know if you saw my tweets, but I was just waiting and waiting and waiting, and it didn't finish, so I decided okay, let's just go in 100 gem increments up to 1,000 gems. Now, I wanted to say okay, we've done a curve fit here, we see that it is polynomial time. How long would 10,000 gems take?

We can calculate this. So I was like okay, let's plug in the numbers to this equation, I plugged it in, the time came out, something I did not believe. So I said okay, I don't believe that number, let's take 200, plug it into the equation,

and make sure that that 200 matches the point on this graph. It did not. This equation is wrong. Thanks, Excel. So I was doing, I was trying,

okay, hold on, I'm really sorry. This is super embarrassing, I'm sorry, hold on. One sec, hold on, oh my God.

Hold on. Okay, okay, quickly, go to black screen, please. Okay, okay.

Why did Kent Beck invent this? This is the worst, I don't understand. Okay, okay, hold on, let me get to the slide. Hold on. All right, okay, sorry. Okay, back to our code here.

All right, so, this makes me even more nervous, sorry. So all right, so that equation was wrong. And the numbers were not correct, so what I did was I decided okay, it is time to pull out the big guns, let's use R. So I had all the, this is using R,

this is the R code used to fit that curve. I'm reading in all the code, or all the times from a CSV file that's on the first line. On the second line, we're doing a regression against a polynomial that's second order polynomial, that's what the number two means there in the poly function.

Then we plot those points, the points that I got from the CSV file, and then that last line is plotting the actual equation. So we're gonna get a bunch of points, and then we're gonna see a red line that is graph plotting that equation. And that's what this looks like,

so this is the graph that comes out of R. So you can see those circles, those circles are the actual data points, and the red line is the equation. Then I asked R for the coefficients, and we could get the coefficients for each part of the equation, and then check the R squared to see how close we fit that curve,

and the R squared is .998, so it's very good. And this is the, so this is the real equation. Thank you, PowerPoint. I could not change the font or color. So anyway, that is the equation. And this is the correct equation.

So what I did is I plugged this into R, and I said, okay, let's define a function where the code up there is just basically taking the coefficients from that prediction and then just doing the multiplication. So if we run this and check an empty gem file, our empty gem file will take about 11 seconds,

which I thought was interesting. You have 1,000 gems installed on your system, and your gem file is empty, and you run bundle update, it takes 11 seconds. Now if you have a 10,000 gem file that has 10,000 gems in it, remember I couldn't finish this. I wanted to know how long maybe would it have taken. It would have taken about 6,800 seconds or so.

All right, so I'm up here on stage talking about performance stuff, complaining about it, seeing that it takes forever, and I don't like doing that. I don't wanna get up here and be like, oh, boo hoo. Turbolinks is the worst.

I wanna talk about how to fix it, like what do we do going forward, right? Because who cares, it's nice that we can see this stuff. It's good that we can see how to model these things, how to get these numbers, but what really matters is like, okay, how do we fix it? What do we do moving forward, right? So if we take a look at all the stuff

that we've been doing, like activating gems, going through the gemfile.lock, if we look at it and think about the information that's there, like let's look at this code and think, well, what do we know? Let's look at these different things. We'll look at bin stubs, we'll look at gemfile.lock, and just think about what is the information that we have. So if we look in the bin stub source code,

we'll see the information we have is we have the gemfile name. Now, in gemfile.lock, if we look at the lock file, we'll see we have the name and the version of the gemfile that we care about. So if you go into RubyGems today,

you'll find some code that looks like this, and this is not actually the code. This is just a condensed down TLDR version. Essentially what it does is it says, okay, I'm gonna load all of the gem specs off of your file system, and then I'm gonna look through all of them for one that has this particular name, say, Bundler, whatever,

whatever you're looking for. So this is where our linear growth is coming from, right? We're loading all those gem specs and then trying to find the one that has that name. So if we say, like, okay, I wanna know, let's look at the impact of this if we had 100,000 gems on our file system. Well, 100,001.

We'll have 100,000 gems plus Bundler. If we do that, how long does bundle-v take? Okay? So my machine has 100,001 gems on it. I run bundle-v. It takes 50 seconds to do that, which I think is pretty ridiculous. It's too long.

So what I did is I said, you know, this doesn't make sense. We shouldn't load all the specs. We know we only need the Bundler spec. It says it in the code right there. It says, gem Bundler. Why am I loading all of them? So what I did was I changed the code to be like this. It's very stupid, very dumb change.

It's like, okay, instead of just getting all the gem specs, why don't I get the ones that just start with Bundler? Revolutionary, I know. So if I do that and run it again, we're down to 600 milliseconds.

So 50 seconds down to 600 milliseconds. Thank you, thank you. Now, unfortunately, that code that I just showed you is still O N, still linear time. It's just that the slope of this is much smaller than the other slope.

But I think we can actually make it O one time. And the way we can do that is we say, well, for the Bundler case, the Bundler case knows the gem spec name and it also knows exactly the version. It knows the gem spec name and the version. And from that, we can actually calculate the file name. So we can say, okay, let's just calculate

the file name for that. We load that specific one. Now, the Ruby gem's gem version, it doesn't know the exact version. It just wants the latest. So that one, we may still have to do a search for. However, I think we can make even that O one. And the way that we would do that is say, well, when you install the gem,

we can keep a manifest of all the gems that are on your system. So when you boot up, we'll load that manifest and just say, okay, you want gems with whatever name. Give me the first. For the bin stub case, we'll say, gems that, tell me all the specs with that name and just give me the first one.

Because we can pre-sort it on install. We don't need to do that for the Bundler case. Now, there's other future work that I want to talk about a little bit and that is making require constant time. I think that we can make require constant time too. If we're keeping caches when we install, this is very nebulous.

I am sorry we have gone full circle from nebulous to concrete and now just some ideas that I'm thinking about. I think that we can make require time constant time too. And the way I'm thinking about doing that is saying, okay, when you install a gem, we're gonna keep a manifest of all the files that are inside of that gem and then keep another manifest of all the gems with files.

So when you do a require on a file, we can just do a look up there and then require that file. Now, I guess I have a little time. It's fine. When you do a require and you actually send that to Ruby, so Ruby's require case, I don't have any slides for this but we'll just talk about it.

In the case of Ruby's require, so say you require foo and Ruby gems is not involved at all, it's still linear time growth. And the reason it's linear time growth is because it has to search through the load path. It has to say, okay, is it here? Nope, here, nope, here, nope. Keep going through all of those. But what's very, very interesting

is that if you give require an entire file path from the root of your file system, it won't do that search. It'll just go load that file. So we don't even do the search at all. So what I've been thinking about is when you do a gem install, if we have that manifest, when you do a require, we could look up the full path from that require

and just send that off to Ruby's require and not ever search the load path again. So I think we could get a lot faster, a lot better performance there but unfortunately I need to get through this, I need to finish up this spec caching work first before we can experiment with that. Anyway, so my throat is starting to hurt. We should finish this up.

What I want to leave you with today, what I want you to remember, hopefully remember from this talk, the main things that are annoying me about the performance of Ruby gems and the performance of Bundler is that all this stuff is static data. We know all this data in advance.

We know all of it. It's all there. Yet we're recalculating it at runtime and that really, really bothers me. Just down at my programmer core is like, I know this data. Why are we doing this twice? I sit there at my desk too. You can ask my wife. I go. Cats get scared.

So if you have data and you know it's static, don't recalculate it. You don't need to do that. Just use the static data. Use what's there. Take measurements. Take measurements of the things that you're using. Now I wouldn't have known, I would have totally expected that that bundle update case would be linear growth.

All the stuff that we were looking at previously would be linear growth and this one was not so I was very surprised. So you should take measurements of all those things so you can understand what's going on in your system. You should also be questioning your intuitions. Now I thought for sure, my intuition was that that would be linear time and it was not.

It turned out I was wrong. Also it turns out that maybe that particular algorithm after speaking to some of the Bundler team is that maybe that algorithm has to be polynomial time and that my intuition is just wrong. Maybe I'm wrong. So you need to talk to the other people who are involved with that particular code that you're testing. So the last thing is I have stickers.

You should come say hello to me. If you want to ask me any questions about any of this stuff, I am happy to talk about it. I don't actually have long hair. This is what my stickers look like. Thank you very much. Thanks.