Facepalm to Foolproof: Avoiding Common Production Pitfalls
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Part Number | 43 | |
| Number of Parts | 89 | |
| Author | ||
| License | CC Attribution - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/31510 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
RailsConf 201643 / 89
1
2
3
5
7
8
11
12
19
22
28
29
30
31
33
45
47
50
52
53
58
62
67
71
74
76
77
78
79
80
81
84
85
86
88
89
00:00
Speech synthesisProcess (computing)AreaFamilyRight angleData conversionRemote procedure callMoment (mathematics)Axiom of choiceLevel (video gaming)Office suiteService (economics)Link (knot theory)MathematicsSlide ruleSquare numberGroup actionTotal S.A.Real numberTouchscreenGoodness of fitMobile appMoving averageComputer animation
03:24
Mobile appCASE <Informatik>WebsiteRight angleBitMultiplication signWeb 2.0Point (geometry)HorizonDatabaseSpacetimeServer (computing)Computer fileProduct (business)QuicksortBenchmarkPower (physics)BootingOptical disc driveConcurrency (computer science)2 (number)Absolute valueResultantProcess (computing)WeightContext awarenessGame controllerMoment (mathematics)Real numberBound stateMultiplicationComputer animation
07:44
Semiconductor memoryMobile appDifferent (Kate Ryan album)Graph (mathematics)Operating systemMultiplication signMiniDiscGodRight angleComputer animation
08:35
Computer fileSound effectMedical imagingCartesian coordinate systemSemiconductor memoryComputer fontLeakACIDRight angleShift operatorMultiplicationMathematicsSoftware testingInheritance (object-oriented programming)Point (geometry)LoginBitAuthorizationDefault (computer science)Overhead (computing)VolumenvisualisierungElectronic mailing listWebsiteProduct (business)Mobile appLink (knot theory)Web pageForcing (mathematics)Parameter (computer programming)Automatic differentiationRevision controlData conversionNumberServer (computing)Operating systemMereologyError messageScripting languageIntegerProjective planeImage processingTotal S.A.Object (grammar)QuicksortTrailMoment (mathematics)Multiplication signCuboidRotationStandard deviationDivisorLimit (category theory)Core dumpReal numberBlogScaling (geometry)Service-oriented architectureStreaming mediaWave packetProcess (computing)PreprocessorUniform resource locatorContext awarenessLibrary (computing)Function (mathematics)AxiomPlotterGame controllerType theorySlide ruleEvent horizonWeb 2.0Internet service provider2 (number)Data storage deviceCache (computing)DebuggerSoftware developerComputer animation
18:17
Plug-in (computing)SimulationCloningFiber bundleConnected spaceRevision controlServer (computing)Computer architectureScripting languageArithmetic meanTable (information)Single-precision floating-point formatEntire function
19:23
Right angleIntegrated development environmentComputer-assisted translationGoodness of fitMobile appSoftware repositoryServer (computing)Multiplication signSoftware developerPoint (geometry)Revision controlKey (cryptography)Real numberGradientComputer virusMedical imagingConfiguration spaceLink (knot theory)RoutingComputer fileMereologyComputer animation
22:32
Mobile appDatabaseGoodness of fitGraph (mathematics)Hydraulic jumpBit rateSubject indexingAssociative propertyTable (information)Integrated development environmentMedical imagingStructural loadSystem callObject (grammar)Standard deviationLibrary (computing)CASE <Informatik>WebsiteBitGame controllerDigital photographyIntrusion detection systemVideo gameValuation (algebra)Performance appraisalHuman migrationView (database)Factory (trading post)Right angle1 (number)Functional (mathematics)Musical ensembleSoftware developerLoginComputer animation
26:38
Mobile appDigital photographyGame controllerMultiplication sign2 (number)Internet service providerServer (computing)BitQueue (abstract data type)Process (computing)Computer-assisted translationLine (geometry)Message passingState of matterMedical imagingService (economics)CodeMereologyCASE <Informatik>Dependent and independent variablesParameter (computer programming)AuthorizationLattice (order)Self-organizationWeightEmailSet (mathematics)Object (grammar)MassProduct (business)Real numberWordComputer animation
30:27
Computer animation
Transcript: English(auto-generated)
00:03
We're going to get started on the unenviable job of speaking last today, which means you're tired,
00:23
you want to go to sleep, you haven't had enough cake, or all your coffee is crashing. And I'm nervous, so let's do this. Turn to the person next to you and say, I promise you I will stay awake for this entire talk.
00:42
I'll wait. OK, that's a lot more conversation than I was expecting. Now, turn to your second choice, the guy that you didn't want to talk to, and say, did you know the guy on stage is homeless?
01:02
It's not true. I don't know why you'd say that about me, but we'll get to that in a second. Hi, my name is John McCarty. I work at Heroku. I help people fix things there. And I'm not homeless. Well, kind of I am. This is my house. All right, the link is bit.ly, B-I-T dot L-Y slash
01:24
railscomp16. There it is. It's on the screen. Go ahead. Thank you. All right, all right, so where was I? Homeless, joke, no, that's my house. All right, so, and there's a reason I'm telling you this.
01:40
So yeah, I actually, last year, about a year ago, a little year and change, I sold my house, and I moved into a 180 square foot trailer. And I've been living on the road for about a year now. That sounds kind of scary to a lot of people. And then you throw in three kids and a wife.
02:02
And 100, yeah, that's when everybody goes, oh. So yeah, that's my family. It's really cool. We get to travel around. I work remote for Heroku. We're about 50% remote. 60% of engineering is remote, 50% of the entire company. We get to go to some really cool places. It's up in Washington state last summer.
02:21
This is my home office. There's my desk chair there, right by my solar panels. It's my roof. And I tell you all this because I love traveling, which is going to set us up for what we're going to see here. I'm so tempted to hit play. Do I dare? OK, all right, I don't know what slide's coming next because I'm mirrored,
02:41
but we're going to work with it. I'll tell you all that because I love to travel. And in the light of my talk today is basically we're going to cover some really common things that we see at Heroku support. All of these issues that we're going to talk about, they're facepalm moments. Yeah?
03:03
No, it's fine. I'm good. I'm good. We'll roll with it. They're facepalm moments. They're these moments where you think, my app is so slow. What's going on? It's probably Heroku's fault. And you, hi. I'm totally, I'm good. Yeah, I'm good.
03:21
Oh, don't unplug it. It was OK. So they're facepalm moments. No, it's supposed to be that. That's cool. It's cool.
03:47
But I can't tell you who made these mistakes. Some of them are me that I've made early on riding rails. But a lot of these are really common customer issues that they ride in and things aren't going right. And then you just kind of start to poke around. You ask some questions. And a lot of times, they're like, oh, yeah.
04:01
It's this really simple thing. But I can't tell you who they are, so we're going to tell this entire story using a made-up company. And I like travel, so we're going to go with a travel theme. I think Airbnb is a fantastic site. I use it all the time. But they haven't really looked beyond the horizon, literally into space for space rentals.
04:20
So we're going to take advantage of that. And today, I'm launching Airless B&B. It's my new startup. If you are a VC, come talk to me afterwards. You're also not a very good VC. There's our logo. We are launching now. Thank you very much. And things are going OK.
04:42
In the last 10 seconds, we launched. And just poking around, seeing how things are going. And it looks like my Rails app is doing about 40 requests per second, or is capable of. If you want to play around with AB, that's fun. People send us AB results all the time.
05:01
40 isn't that great. And so I start to poke around. And I'm wondering what's going on. And well, let's go ahead and boot up. Let's see the server when it boots up. This is what's going on here. Anybody knows when I just do that in previous to Rails 5, what I'm doing? Yeah, I'm doing this right there.
05:24
Shocking, shocking amount of people running WebRic in production. Hopefully that's not you. If it is, if you don't know what Web server you're running, it's you. You're running WebRic. It's a face bomb. OK? Don't do this.
05:41
Let's talk about Puma real quick. Puma is the web server that we recommend at Heroku. We used to recommend Unicorn. Unicorn is great for a long time. It's susceptible to slow request attacks. So we kind of stay away from that. And there's a lot of other great things that Puma will do. You can look at the benchmarks on their website.
06:01
They're fantastic. And we'll talk a little bit about, too, kind of the ping that Puma gets is that it's threaded. And if you're using MRI, which doesn't really handle threads, what's the point? But in reality, most Rails apps are IO bound. And so while your app is outgoing to the database,
06:21
it can take advantage of threads, even in MRI. So Puma is fantastic, really easy to get started. Throw it in your gem file, boot it up, really simple. So this is just a home controller. I'm not doing any kind of context switching. And immediately, I get double the throughput. So if you're wondering why you shouldn't do this,
06:41
it's mainly because Webrick has zero concurrency. It technically does. But in Rails, it has no concurrency. What that means is it can only handle one request at a time, which is really terrible. Because while one request is coming in and getting processed, another one comes in, it has to wait, and it's blocking. With Unicorn or Puma, you're able to fire up
07:03
multiple workers and handle multiple requests at a time. So that is a fantastic thing. We're done. It's great. It's a great talk. Run, Puma. Yeah, thank you. Thank you so much. Or is it?
07:20
You're going to see a theme through all of these. And a lot of our questions that we get at Heroku actually start with, my app is running really slow. And that is one of the hardest places to start helping someone debug what's going on. Because almost every single thing I talk about for the rest of the talk starts with, this is really slow. And you have to start poking around
07:41
and figuring out what the reasoning is. In this case, I started poking around and I saw this. This is actually a Splunk graph. The red shows me how much memory I've used. And the blue shows me something that I don't want to see, which is my app using swap.
08:01
Now, swap, the first time I learned about swap, somebody gave me a Linux book about this thick. And I'm not going to cover that. So let me give you a really quick explanation if you don't know what swap is. I didn't when I started. So your operating system is going to have a certain amount of RAM. And then after that, once that fills up,
08:21
it's going to start saving stuff to disk. Your disk is way slower than in memory. So when your app starts going to swap, you're going to see a big performance hit. There's a couple different memory enemies. There's two main things that we usually see. The first one is, I'm just going to give you two links
08:41
and I'll give you the slides after. Memory leaks are kind of nasty and are going to take a much longer time to talk about. Go visit these two links. The first one is a library by Richard Shteeman, one of the guys I work with at Heroku. It's a fantastic little thing to run against your app. And you can actually see if you do have a memory leak.
09:03
So if you are concerned about that, look at that. He also wrote a fantastic blog post over at Codeship on debugging a memory leak. The second thing, hopefully, number one doesn't apply to you. The more common thing that we see, actually, is a really simple fix. People are running too many of those workers.
09:21
So they've got Unicorn, they've got Puma. And how many workers should I run? I don't know. Five is a cool number. Has a little hat on the top. But they don't think about what does that number actually mean, especially when it applies to memory. So if you look at a Rails app, a typical Rails app is going to run out of the box right around 180 to 200 megs.
09:44
So if you've got a gig of RAM, you go, oh, 1,024 divided by 2. I'll run five workers. The problem is now you're right up against that limit. You have no headroom. So if your app needs to actually allocate more objects, so you're doing any kind of image processing,
10:01
that's going to balloon up. And all of a sudden, you're going to start going to swap. So the better way to look at that is to just cut back on your web process, see how your memory looks after that, and then start to adjust from there. But when you start going into swap, you're going to have a bad day. Really easy to check this out actually is go onto your server,
10:22
type free-m. You're going to get that number, which is your total amount of memory. So we got eight gigs. Your operating system has allocated, so it's basically saving 511 megs for swap. And right now, I'm only using seven. So just ignore the other stuff. These are the numbers you care about.
10:41
When that number, that seven, starts getting really deep, you need to be concerned about it. Obviously, there's a lot of other tools that will help you visualize this stuff. But first place to look when your app is running slow is check your memory usage and adjust accordingly. All right, moving on. I've got people who want to pay me, which is awesome.
11:04
Found a great place on Mars, one bedroom. Problem is, we've got a bad guy, and he discovered a unprotected part of my website and has taken the money, and now shit's on fire.
11:26
This is why. Again, more common than you think. So we put up an SSL cert, and everybody goes to it. All the links that we post have the little S at the end. Why are people going here?
11:40
Because they did, and you didn't tell them otherwise. And so therefore, they're going to this checkout page, and it's completely unprotected. Really, really, really, really easy to fix this in Rails, actually. And we'll talk about how to do that in Rails and Nginx. The first one is this. This is it. This is all you have to do. I don't know why this isn't on by default.
12:01
Go into production.rb, go to force-ssl, and flip it to true. And you're done. Ship it. This will take any request that comes in on port 80 and immediately redirect the entire thing to HTTPS. Now, the problem with this, though, is that it's going all the way into your app. And it's kind of a lot of overhead
12:21
for just doing a redirect. So you can do this a little bit higher up in Nginx. List it on port 80. If somebody comes into 80, send them to 443. You can do this a lot higher up. Nginx is a much better place to do this. And we're moving on. We've got a few more to cover. I've got 25 minutes. All right, we are going to change our logo.
12:42
The rocket ship didn't test well. We're going to change it to a comet. Super easy to do. So we change it, and we push it, and our logo is gone. Nobody knows where it went. Is that an image? That should be IMG. I have. I'm a developer.
13:02
Designer probably did that. Burn. Why did this happen? Our friend, the asset pipeline. Yay. It gets a bad rap. And it's unfortunate, because I love the asset pipeline. If you ever had to deal with asset caching
13:20
before the asset pipeline with the question mark crap, so much better. So this image, my logo here that I put in, is actually going to turn into this MD5 digested file name, which is going to let us do cache busting a lot easier.
13:41
If you don't know what's going on here, basically what the asset pipeline will do is it looks inside of your image, and if it has changed, it will generate a new file name, which is how we do cache busting. The problem is that somebody did IMG, and they just direct linked to the logo. Instead, again, really easy fix, use the image tag helper.
14:01
This one isn't actually as common as the CSS version. This is way more common. I see this a lot more. So when you've got your CSS, stop using the URL and linking to assets, because this will not use the asset pipeline at all. Very few people actually seem to know this, but there is a bunch of CSS helpers inside of CSS,
14:23
STSS, that Rails provides. Image URL will actually do all of that handling for you. There's also, if you've got a fonts folder, it'll do font URL. There's actually just a base asset URL as well. But make sure that you're using this wherever you can. Teach the rest of your team to always use this.
14:41
Designers, too. On my old team, this is kind of what we would work with our designers whenever they're doing any kind of front end stuff. It's just to make sure that they use this, because it's really tempting just to plop in the old URL. Oh, yeah, one more thing, the pre-compile list. Thankfully, Rails, I think, forget which version it is, where it will actually warn you
15:00
if it's not in the pre-compile list. I think it was like 4.1. Rails will warn you if you try to render another JS or CSS file, and it's not in this list, but be aware of this list. So if you're trying to, by default, it'll render application.js and application.css. If you want another one, make sure to add it to this list so it gets pre-compiled.
15:20
That's another thing we see people run into sometimes, is it works locally, and then they'll push to production. It hasn't been pre-compiled, and it'll throw an error. All right, we get a bug. And somebody asked me, is it happening a lot? I don't know. It happened once or twice. Let's check the logs, right? Check the logs.
15:41
That's what we do. We've got a production.log. The problem is we have multiple application servers. This is a problem, because now I have multiple production.logs. And I've got really, really big production.logs. We've got a little problem here. If you've never read about 12-factor apps,
16:02
I highly encourage you to visit this URL. Let's take a look at what it says about logging in a 12-factor app. A 12-factor app never concerns itself with routing or storage of its output stream. It should not attempt to write to or manage log files. Instead, each running process writes its event stream unbuffered to standard out. What this means is your app should not be sending stuff
16:22
to production.log. It's really easy. I've seen a lot of apps where, usually not on Heroku, that will, just by default, it's like we're only going to be running one server forever, and who cares? We're never going to scale. And so we'll get to it later and just dump everything into production.log.
16:40
Lots of times, they don't even do log rotations, so this thing just starts growing out of control. In your Rails app, in your Ruby app, use standard out. If you log to standard out, now you're doing what a 12-factor app should do, and you're just sending everything to standard out and getting out of the way. This allows you to consume standard out in a much better way.
17:01
Hadoop, Hive, Splunk, you can use an add-on like log entries or paper trail to consume those logs and have a much better understanding of what's going on inside of your app. All right, we're moving. We're shipping. Ship, ship, ship, ship, ship. Everybody wants to ship. This is our deployment script.
17:23
Go ahead, find it. Sleep 4. So this is an actual restart script somebody gave me. I took over a friend's project,
17:41
and the former engineer said, this is our restart script. What do you think happens when a user comes and we are sleeping? What do you think they see? An nginx 503 error. It's terrible. The argument was we shouldn't do restarts because you could have two different versions of your app
18:01
running based on a Unicorn restart. That's a whole other conversation, but this is bad. What we need is an actual legitimate way to deploy our app that doesn't involve me SSHing into a server and running this. So you may have heard of Capistrano, and I hope you've heard of Mina.
18:21
If you have not, I'm going to tell you about Mina. It is my favorite, sorry, my second favorite way to deploy a Rails app. I'll tell you about my first one in a second here. Mina is fantastic. I started deploying with Capistrano a couple years ago, and then I found Mina. Mina is really similar to Capistrano if you're used to using Capistrano. Take a look at this.
18:42
There's a great plug-in architecture for restarting things like Unicorn and Puma and Sidekick, and there's great plug-ins for this. But what's great about Mina is it's really, really, really fast. It does the same kind of idea where it will do a shallow clone out of Git, make a new version folder, and then do the SIM linking and all that and bundling and DVMigrate, all that stuff for you.
19:03
But what it kind of does even cooler is versus Capistrano, which fires off every single command in its own SSH tunnel, Mina will look at the entire thing, bundle up the entire script, open one SSH connection to all of your servers, individual, and run the whole thing on the server. So it's way faster.
19:21
If you've never seen it, take a look at it. This is what it looks like when you deploy. Oddly similar to something else I'm going to show you. It will make the path, clone the Git repo, do all of the business, and then right at the end, it'll SIM link and restart the server. There's another way. This is what it looks like on Heroku.
19:42
Create your app, Git push Heroku master. Whether it's this or whether it's using Mina, you need to have a good way to deploy if you want to ship frequently. The teams that are usually unable to deploy quickly are teams that struggle with these deployment practices.
20:00
They think they'll get to them at some point, and they never do. And they get stuck because everybody's too busy doing something else. And then by the time they need it, it's almost too late and they can't deploy quickly. So have a way to deploy quickly so you can ship. Next problem. Somebody got into all of our S3 images and swapped them out.
20:24
Someone nefarious has taken all my beautiful Cosmos pictures and put in something less than appealing. How could this have happened? A long time ago, someone decided to put our S3 creds into the Git repo.
20:43
Then something happened. Maybe somebody forked a part of the app. Maybe somebody took the app and put it on GitHub and then deleted it. But it's still there in your Git history. There was a couple of years ago where you could actually,
21:00
I think you can still do this sadly. There's a search command. Be good. But there's a search command that will let you go through all of GitHub and look for S3 creds. And they're still out there. A lot of people have S3 creds all over the place. Shneems did it one time, right? Yeah, that guy right there. How big was your S3 bill in the morning?
21:20
3 grand, yeah. So don't do this. There are so much better ways of doing this. There is something called secrets.yaml inside of Rails now. This one works pretty good. I have a better way. But if you like this version, you can go this route. Throw your keys inside of secrets.yaml.
21:41
Please git ignore this file. And then you can use it inside of Rails by calling it thusly. Personally, I much prefer nvars. This is where all of your keys need to be in nvars. You don't commit them. They are different per environment that you're running on.
22:00
Really easy to do. In development, you can use this gem called .nrails. You make a .n file, git ignore it. You've got your key values. And then on Heroku, it's really easy. Heroku config add with your key value. And then in your app, you just call env and whatever the name is. Really easy to do. This is the right way of storing credentials.
22:22
It's really tempting, especially when you're going really fast at the beginning of an app. You're just like, oh, just give me the creds. I'll throw them in here, and I'll deal with it later. Start the correct way by using nvars way up front. All right, 15 minutes. So I've got to hurry up because I want to do questions. OK, I'm going to hurry up. This is my database graph.
22:41
This is my app graph in New Relic. The orange chunk right there is my database. This is no good. What on earth could be causing this? When we were yet a small little app and I was looking up users by their username, this call took 2.3 milliseconds. But now, the last 20 minutes of us being up and running,
23:00
we have jumped to 231,000 users. And suddenly, that call is 51 milliseconds. Who would like to tell me what's wrong? Indexing, thank you. So that little thing is now this big thing, and that's only going to get worse because we do not have an index on username.
23:21
This is one of my own. I bit this one really hard early on. And this is really, really easy to do. So database indexes basically tell your database, this column is something that's important to me. I want you to index it because I'm going to make calls against this column frequently. It's so easy to do in Rails. You basically just create a migration,
23:42
add the index on the table and the column that you want. And all of a sudden, this 51 milliseconds actually is faster than the original one. It's down to one millisecond with the same table load. One asterisk here is composite indexes. I didn't know about this one either. Suddenly, I want to look up by this one thing and then something else. That original index that you made no longer applies.
24:03
You need to create a composite index. Again, really easy to do in Rails. Pass an array of column names, and you now have a composite index. All right, two more. I'm going to blow through these. Awesome. We're so close to a billion dollar valuation. It's going to be great.
24:20
My first problem that I have here, start seeing, again, the app is a little slow. And I've got this controller here, iterating over photos. And then in my view, I am spitting out the photo and the username. Except this is what my log looks like. So I'm getting all the images.
24:42
And then I've got all of these user loads. What's going on? It is a n plus 1. Didn't know about this one either. These are really easy to ignore while you're writing an app. Because, again, your development environment isn't taking on the load that your production environment will. So all of a sudden, what works locally
25:02
is getting thrashed by the amount of users that you have. An n plus 1 basically is what happens is when you call one object that has a relationship, you start iterating over this and calling the related of the associated object. It's really easy to fix in Rails. All you have to do is instead of going just
25:22
photos.all or photo.paginate or whatever you want to do, includes. It kind of messes with you because it include is actually a standard library method. Includes will tell ActiveRecord that this is an association that I want you to load. And what happens is Rails does this really smart thing where it goes from this to this.
25:41
So now it will make only two calls, get all your photos, and then do an in and find all of the user IDs on those photos and grab them all in one shot, which is way better. In development, use a gem called bullet. This will show you when you've got n plus 1s. It looks like this in your logs. And it will actually tell you, add this on your finder.
26:03
So if you've got photo, go. It says right here, there's an association called users. Add this to your finder, include users, and life will be magical. Last one. My users are complaining that this thing is slow when they're doing certain functions, specifically adding photos.
26:21
And if you look around, you'll actually find plenty of cases where you can see where milliseconds will actually affect users' happiness. People will bail on websites because things are just a little bit too slow. So this is actually pretty important to us. Now, I take a look in New Relic,
26:40
and I see my app's pretty snappy, except this one thing called Photos Controller. It's taking almost five seconds. So I dig into it, and I see this. I'm spending almost 60%, 66.1% of my time dealing with S3. So what's going on here? This is what's going on.
27:01
So a request is coming in. It's hitting the controller. Now it's creating this photo. It's going off to S3 and waiting for S3, which means the user is waiting for us to do something that the user doesn't really care about yet. It wants to upload the photo and get on with what it was doing. So it's now going to wait for S3 before it returns a response.
27:20
So we're going to talk about background workers here in just a second. Let's talk about another use case. Let's just do SMS. SMS is a little simpler. We don't have to deal with image magic, if you followed that one. So let's talk about SMS. So let's do, we got this. We have a user object. We're going to send them a message with an SMS provider.
27:40
So the request comes in, hits the controller. We've got to wait for SMS provider before we return a response. This is a great example for when you want to use a background worker. Actually, an even better one, it's a little bit simpler to talk about, is mailers. If you go onto the Rails guys, you actually can see. I think they use, in the active job explanation, they use mailers. I'm going to talk about SMS here.
28:02
But mailers is a great example. Anything where you have to wait for an external service to return a response for something that's non-critical, sending an email, sending an SMS. If it can wait a second, send it off to a background worker. Don't make your users wait for it. So instead of user.sendSMS, in our controller,
28:20
it's actually going to do the job. We're going to call performLater, which is the active job method. You can use Sidekiq, whatever else you want. The active job uses performLater, and you pass it your arguments. And now instead of this, it's this. So we go to the controller. It goes and adds the job to the queue in Redis,
28:41
which is super fast, and gets the user back to what they were doing. Now the SMS job can do whatever it wants. This is what a basic job looks like. So it's inheriting from active job base. You just need a perform method and the arguments. And then you can do whatever was costly before. Uploading a photo, sending an SMS job, sending an email, whatever it is.
29:01
This is really, really small, bite-sized code and saves you a ton of time. And what's even better on the app provisioning, servers, Heroku side, you don't need as many app servers now because they're not all waiting around and consuming resources. They can actually get the user back and go let a worker take care of that,
29:21
which means you actually need less servers. So this is a great thing for the bottom line as well. That's it, we sold. We made a billion dollars off of a terrible idea, but mostly because we fixed all the problems that we talked about in this talk. We're actually pivoting. We're gonna be building a Tinder app for cats.
29:42
It's gonna be good. Don't steal that. So the bottom line here is if you are new or if you've been around for a long time, when you are running a Rails app in production, you will break things.
30:01
The important part is when you do fail on something, when something does break, learn quickly and continue to learn. There are so many things that are changing in Rails. There's so many things that are changing in the state of the art of the servers that you're running. Don't get discouraged. Keep learning and you're gonna be great. Thank you very, very, very, very much. Thank you very much.