Turbo Rails with Rust
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Part Number | 11 | |
| Number of Parts | 89 | |
| Author | ||
| License | CC Attribution - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/31570 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
RailsConf 201611 / 89
1
2
3
5
7
8
11
12
19
22
28
29
30
31
33
45
47
50
52
53
58
62
67
71
74
76
77
78
79
80
81
84
85
86
88
89
00:00
Line (geometry)Slide ruleGraph coloringState of matterStudent's t-testMereologyRight angleCodeConfidence intervalBookmark (World Wide Web)Installation artComputer fileOrientation (vector space)Block (periodic table)TrailMappingBitMultiplication signComputer programmingYouTubeCore dumpRow (database)Endliche ModelltheorieInternetworkingScripting languageReal numberJava appletProcess (computing)Functional (mathematics)Survival analysisFile formatRule of inferenceInsertion lossOffice suiteForestSystem callVideo gameTerm (mathematics)Social classDependent and independent variablesInheritance (object-oriented programming)Goodness of fitComputer animation
04:40
Speech synthesisSocial classRun time (program lifecycle phase)NewsletterCodeExtension (kinesiology)NumberImplementationParticle systemPresentation of a groupString (computer science)Default (computer science)Einbettung <Mathematik>Structural loadExpert systemCollisionObservational studyComputer programmingCASE <Informatik>Spectrum (functional analysis)Software frameworkMultiplication signCartesian coordinate systemSoftware developerField extensionRevision controlLocal ringCognitionSlide ruleInterface (computing)Right angleBarrelled spaceSampling (statistics)Graph (mathematics)Term (mathematics)PlanningException handlingWordCrash (computing)Different (Kate Ryan album)Skeleton (computer programming)Process (computing)Group actionOnline helpWritingEmailLevel (video gaming)Entire functionExtreme programmingMathematical optimizationComputing platformMereologyComputer engineeringMacro (computer science)Installation artCodierung <Programmierung>BitCommitment schemeMetaprogrammierungSpacetimeCoprocessorGoodness of fitMobile appUniform resource locatorBuildingProduct (business)Regular expressionSystem callComputer animation
13:16
Right angleMobile appCrash (computing)Exterior algebraData storage deviceRun time (program lifecycle phase)Multiplication signProduct (business)CodeProjective planeField extensionWritingVariable (mathematics)
14:47
System programmingWordDifferent (Kate Ryan album)Extension (kinesiology)Multiplication signFormal languageWebsiteProjective planeComputer animation
15:49
CompilerRun time (program lifecycle phase)Crash (computing)Error messageComputer programmingFlagComputer animation
16:29
Loop (music)Formal languageCompilerAbstractionCondition numberConcurrency (computer science)Multiplication signVariable (mathematics)SpeicherbereinigungCrash (computing)Computer programmingTrailDifferent (Kate Ryan album)BitCodeIterationRun time (program lifecycle phase)InformationCausalityUniform resource locatorSemiconductor memoryRandomizationConstructor (object-oriented programming)Level (video gaming)CASE <Informatik>WritingSensitivity analysisPole (complex analysis)ForcePosition operatorModule (mathematics)Graph coloringOnline helpLine (geometry)Event horizon
20:25
Formal languageRule of inferenceFunctional (mathematics)Compiler2 (number)Greatest elementUniqueness quantificationNumberoutputLine (geometry)Type theoryImplementationRevision controlCodeInformationPhysical systemStatement (computer science)Right angleSpacetimeMultiplication signLibrary (computing)BitBuffer solutionBlock (periodic table)Slide ruleComputer programmingProgrammschleifeCondition numberString (computer science)Combinational logicLevel (video gaming)Loop (music)MereologyKey (cryptography)Point (geometry)Equivalence relationSystem programmingPointer (computer programming)Variable (mathematics)CASE <Informatik>Semiconductor memoryIterationBoilerplate (text)Electronic signatureRing (mathematics)Bit rateProcess (computing)Connectivity (graph theory)Limit (category theory)State of matterFigurate numberSeries (mathematics)Computer-assisted translationSummierbarkeitComputer animationPanel painting
26:52
CodeExtension (kinesiology)Functional (mathematics)Forcing (mathematics)Core dumpProjective planeLibrary (computing)Right angleBoilerplate (text)Multiplication signRAIDThermal expansionInterior (topology)TrailType theoryTerm (mathematics)WritingMereologyField extensionSoftware testingManifoldHelixImplementationLogicSocial classBuffer solutionAbstractionComputing platformAlgorithmCodeRevision controlSet (mathematics)String (computer science)Line (geometry)Level (video gaming)Bit rateNumberSuite (music)Cellular automatonNumerical taxonomyMacro (computer science)Constraint (mathematics)WebsiteCausalityInstance (computer science)AreaMobile appExistenceModul <Datentyp>Array data structureDifferent (Kate Ryan album)FreewareStorage area networkStack (abstract data type)Computer animation
34:19
Line (geometry)Point (geometry)Web applicationImplementationSoftware bugCASE <Informatik>CausalityComputer programmingFormal languageSoftware testingDifferent (Kate Ryan album)Reverse engineeringWorkloadCodeBenchmarkReal numberMultiplication signDemosceneTask (computing)Run time (program lifecycle phase)Schmelze <Betrieb>Revision controlIterationMobile appDilution (equation)Entire functionCompilerBit rateMereologyLoop (music)Error messagePointer (computer programming)Physical systemCartesian coordinate systemBoilerplate (text)Scripting languageSign (mathematics)Subject indexingAlgorithmGastropod shellType theoryLogicRight angleGradientCore dumpNumberProcess (computing)SoftwareReading (process)Coordinate systemArray data structureArithmetic progressionGraph (mathematics)CountingProjective planeUtility softwareStapeldateiInternetworking1 (number)Green's functionHelixGreatest elementDatabaseVirtual machineSocial classMacro (computer science)Hard disk drivePressureSinc functionComputer animationDiagram
41:41
Computer animation
Transcript: English(auto-generated)
00:12
Should we get started are we ready for this? Okay? Let's do it Welcome to welcome to Kansas I
00:24
Have always been very excited about this rails comp because I always want to come to Kansas I I've always heard about this movie called Wizard of Oz and It's as you can tell it's a little bit ahead of my time But I watched it before I come here in preparation for a conference
00:42
But as we learned in Jeremy's keynote the other day it turns out these lines on the maps are state lines and Kansas is on the left and the other half is Missouri and turns out Kansas City is Actually not in Kansas, so I guess we're not in Kansas anymore
01:02
Thank you for that I Had to redo all my slides yet colors them, so if they don't look very good. That's probably why I am Godfrey you can find me on the internet says Chenkin code And I'm very excited to welcome you to your employee new employee orientation. I hope you're at the right place
01:27
Yeah, yeah, if you're looking for rails conf. I'm afraid that moved so no just kidding This is rails conf welcome to your else confident. Thank you for coming I Always say
01:41
There's a very personal thing about rails come to me, and I love coming back. This is actually my fifth rails conf five years ago and Austin that was my first rails convent actually I attended there attended rails conf Austin as With on the students student scholarship, so if you were there
02:00
Thank you for chipping in for my ticket and Thank you for giving me the opportunity to be part of this community and next year. I went back to Rails conf Portland and for some reason I had suddenly the courage to go But Aaron and say hey, I have a poor request and you merge it and for some reason
02:22
you end up merging it so I have my first commit in rails and The next year soon after the next rails confident Chicago I Joined the rails core team and the year after I spoke at rails conf for the first time which is last year and This year. I'm very honored to be part of the program community
02:44
program community program committee on rails cons and help created the behind the magic and you in rails five tracks, so if you went to those talks and Like those tracks Well, that's where they come from and
03:02
Like Jeremy said the other day rails is rails exists because of his community So thank you for being part of this community so following Aaron's lead I would like to now some new rails five features as you learned today rails five will come with PHP support and
03:25
Following his lead. I would like to announce JavaScript support for rails five and In fact you don't even have to wait for rails five you can just get it today by running gem install JavaScripts once you have installed gem all you need to do is require JavaScript on the top of your Ruby file and
03:44
There you go You can just wrap anything in a JavaScript block and write your JavaScript code in there Even supports things like functions, and that's very handy because what is everyone's favorite JavaScript feature of course? That's callbacks, and what's everyone's favorite rails features
04:01
Of course, that's also callbacks, so The JavaScript gem let you combine the best of both worlds like here is the active record model You can have your favorite before create callback, and you can write that in JavaScript, so actually Just Like the FUBY jam. This is a real thing you can use and an insane amount of engineering when it dead jump
04:25
And if you want to learn more about that I suggest you watch my talk at Garuko called dropping down to the metal You can find that on YouTube I Have another thing to plug If you went to the lightning talk from yesterday you would already notice
04:43
But I helped coordinate a newsletter called this weekend rails where you find the latest Commits and for requests and things like that that went to rails. That's that week And here is a sample from last weekend it includes sensational headlines like look
05:01
local scientists discover new method to manipulate time faster code found to perform better on the load JRuby builds too flaky to be useful and Regic experts to play debate left to right or right to left is one better than the other If you haven't already go to bit.ly slash rails weekly to subscribe to this newsletter The next issue will be coming out in a few hours, and you will
05:24
You'll probably learn something new from there speaking of newsletter I also I'm also part of in a newsletter which is a product newsletter, so if you are not already signed up to Scala you should Sign up at skylight.io and select the almost daily insider channel
05:44
email preference and will usually write about experience writing building skylight and a lot of times will Read about things like oh I had this problem today, and we don't know how to deal with it and some of the customers will say hey I had that problem last week, and here is the gem
06:02
I found that does that or sometimes the opposite would happen will be like we solve this problem today and How customers would write back and that's really great? I have the same problem at work So if you're interested you should go sign up and if you have pen paper you might want to write down that secret URL
06:21
That's actually my personal Referral URL, and I heard we have plans to Distribute bonuses this year based on referral credits in our accounts, so please help me out Anyway, let's talk about Ruby, so Ruby is great. We all love Ruby. That's why we're here
06:42
Ruby has a lot of nice features. It reads really nice meta programming is pretty awesome, but There's also a problem to Ruby, which is that it's pretty slow most of the time. It doesn't really matter, but occasionally You might hit a wall when you try to do something in Ruby And it's just too slow for use case on the other end of the spectrum
07:06
Or perhaps on the other extreme of the spectrum there C. Which is super low-level Just super fast basically. It's like as close to metal as you can get without writing assembly and That's great, but it's also Pretty dangerous you can very easily write code that crash your program at runtime
07:25
in an unexpected way and There are also a lot of concepts that are a little bit hard to grasp So in Ruby we have this feature called native extensions that give you the best of both worlds So for example when you run gem install JSON what you're getting is actually two different things for the price of one so
07:48
By default you would get a thing called JSON pure Which is a pure Ruby implementation of the JSON encoder, but if you're on a supported platform You'll also get another thing called JSON colon colon ext. Which is native extension, so it's the same
08:04
JSON encoder API written in C, and so that's like super fast But to you as a user you don't even notice the difference because you just call it like a regular Ruby class and you call its method normally and Under the hood it basically is
08:22
transparently called the Call the C methods that does the work for you and as a user you don't have to care about that and Chances are you're already using native version without knowing it So Native extensions they are great. Why don't we write more of them?
08:40
Well, there is a catch so it's indeed the best of both worlds But it's only the best of both worlds if you're the end user that's using the gem This is this is fine David who created rails Have said something like this to me. Here's a general value of rails development
09:01
We will jump through whatever hoops on the implementation side to make a user facing API nicer And I think I would personally extrapolate that user facing API to developer experience in general, and I think that's a good goal to have like it's good to Make the experience for your developer users as nice as possible
09:25
while keeping So keeping a beautiful interface on the outside and do whatever you need on the inside to make that work. So That's a great goal to have and Here is a good example of that. So Sam saffron who?
09:43
You might have heard of he did a lot of Ruby performance work. It was a Ruby hero from last year So he noticed something in rails so it turns out in rails in active support in particular There's a method called string dot blank So it turns out this method is called a lot both inside of the rails framework and also in
10:07
user code, so This is also the implementation of the flipside called present So you might have seen things like user or params colon user dot present Right. So this is actually the method it's talking about
10:22
This is the implementation in Active support as off a few weeks ago. There are some improvements since then we'll talk about that later So it's pretty short method, right? Like you basically reopen the string class and it basically check if the string is
10:41
All white space characters and that's all there's to it this one-liner It reads beautifully like all of your Ruby code, and that's great and apparently like evidently, this is a very useful method because We use it enough in our application and rails to make it a performance hotspot according to Sam
11:01
so what he did is He made a gem called fast blank that we implemented the same blank method in C and According to Sam this C implementation is up to 20 times faster than the Ruby version we saw on the other side slide and
11:21
He said in some application is again. You can get up to five percent performance improvement on the macro now as I said, there are some recent improvement to the rails version, but For the purpose of this presentation and this particular number doesn't really affect that much because the optimization
11:41
We did in rails is about the in edge case or common case that is when the string is empty so now if We can So as the user you don't really need to know the difference really you just get the fast black Blank gem in your app and all of your code works seamlessly because it provides the same
12:04
Stringed up blank question mark method with just with a different implementation under the hood So you can get the performance you want and user the developer experience you want then
12:20
That seems great. We're done here, right well There is a problem though the problem is me me being the developer that Knows just enough C to be dangerous if you give me a C program with a variable called PTR If the code doesn't work up the first thing I try is probably at a star to it
12:43
And if it doesn't work, maybe try more stars if more stars doesn't fix it. Maybe try amber sand and The problem with this is as I mentioned before If you if you're like me if you read your C code like this your program can
13:02
Sec fought at runtime And that's pretty bad because you're if you're embedding you see code inside a Ruby processor will crash an entire group process There's not like a regular exception that you can you can handle So as scally where we have a similar problem. Oh by the way. This is I don't know if I mentioned it
13:20
But this is where I work We have we are performance analyzer for rails apps and to do that we have to have our agent that we put inside your app to collect performance data on production and we We want to make sure that our agent is as lightweight as possible
13:41
We don't want the thing that's supposed to be measuring your performance to become the bottleneck itself so we could write that agent in C or C++ and Fortunately most of our engineers are aspiring me Then they don't randomly add stars and ampersands to variables in C, but even then
14:00
We don't feel very confident that we can get away with writing mean writing and maintaining our own C code that goes inside of all of our customers apps There are a lot of native extensions in The Ruby community like no Kogiri or JSON gem in their early days. They all had various issues of
14:27
sec fodding at runtime and those people way smarter than me and if Even after very careful Writing they still occasionally Crashes then we definitely won't want to recommend our customers put something like this and their app. So what is the alternative?
14:45
Well at the time when we started this project rust Just announced that they would They have made some very good very good improvements at the time, so we thought we will
15:01
Perhaps give that a try. So what makes rust different than writing your native extension in C or C++? Well, let's look at the rust website Rust is the system programming language that runs blazingly fast Prevent sec fods and guarantees threat safety. Well, that's a lot of words. That doesn't mean a lot to me yet
15:26
but they have another slogan that tries to put the same thing in different words which is hack without fear the goal of the rest project is allowed you to To basically make system programming more accessible to
15:43
More programmers and and it does that by having a a Compiler that knows about a lot of these things. So What makes rust special is that as a compiler that can find most of these? Well, actually all of these errors could that could cause your program to crash at runtime and flag them at compile time
16:05
And if you don't do everything correctly, and if you don't satisfy if you don't If you cannot convince the rust compiler that your program is sound it simply won't it was simply refused to compile your program and
16:21
Therefore you don't have a thing to run at runtime and so it cannot crash at runtime So there are some Features in rust that makes that possible. I can't get into a lot of details Of rust today and this talk is not really about teaching you rust But I'll try to describe them at a very high level so you can perhaps try to understand them without seeing the actual
16:48
code The first feature of rust that makes this possible is They take care of managing the safety of your memory without using garbage collector. So Ruby is also a
17:03
Language that offers memory safety guarantee and Ruby you cannot write you cannot write code that access random locations of memory and cost causes your program to crash at runtime that way so it's Maybe it perhaps sound a little bit
17:21
Crazy to say that oh like your program cannot crash at runtime if you write in rust But in fact, you're already used to that because your Ruby program also cannot crash at runtime The difference is that Ruby manages that by using a garbage collector and rust does that without a garbage collector and what rust does is basically tracks lifetime of all your
17:43
variables at compile time So it knows exactly when it when and where it needs to locate things and when it needs to clean it up so it doesn't need to pause your program and Clean up with a garbage collector periodically It also allow you to do
18:01
concurrency without data raises or raised conditions Which I don't have time to get into right now and the final feature that is particularly Relevant for us it it has this concept of zero-cost abstraction. So in Ruby or in frankly many at most other languages
18:22
You always have to make a trade-off between Abstracting your code versus performance So like maybe you notice that you are doing you're repeating a bunch of steps in a few places And you would want to extract that into a method in Ruby So that's fine, and that's what you should do most of the time
18:42
but unfortunately this incurs the cost of an extra method invocation and when When you're talking about really performance sensitive code you have to make the trade-off between how much you want to abstract your code between And how performant you want your code to be right so in Rails for example. We have a lot of
19:04
Modules, and we call super a lot, and that's how we decide we want to Abstract our code ideally, but occasionally there might be cases where we realize this is like really hard path And we really would prefer not to incur that cost and we have to inline some of the things right so in Rust
19:24
This is one of the biggest feature in Rust where You can actually you don't actually have to make the trade-off between Abstractions and performance because the Rust compiler is smart enough to notice that ah this method can be inlined into that other thing
19:40
So I would just do that or in fact a lot of time if you use higher level constructs in Rust like iterators It's actually giving the compiler more information about how you should how the compiler can optimize your code So it actually makes a program one run faster so for example in Ruby you might want to write things in each loop, and we often do and
20:03
That's fine, but that's incurring the extra cost of making Calling the each method and stuff on the other hand on Rust if you use an iterator actually makes it faster than writing a hand rolled loop because the compiler knows I'm this is going to be a safe iteration so it can remove some of the bound checks inside
20:22
For each iteration, so I can't get into a whole lot more details But yeah who they gave a talk on Rust at RailsConf here last year So if you are new to Rust and you're curious about Why you might want to look into this language you can look up his talk from last year, so
20:42
Let's get back to fast blank and I guess we'll look at the fast blank and implementation see quickly, so This is the fast blank body, so you probably cannot read it, but it's fine. We'll walk through it step-by-step together, so
21:02
basically At the top you basically have the method signature and some boilerplate to extract some pointers and So you have you then Have a line that quickly check Oh if the string is empty then return right away
21:21
so you know you can avoid doing a bunch of extra work and then the main part of the method is basically you loop through all of the characters inside the string and there's some more boiler plays about pointers and Finally you have a switch statement if you encounter a white space character you keep looping basically and if you
21:44
encounter a non white space character You know that this string is not blank so we can return false immediately Otherwise if you get to the end of the loop you know that all of your of the characters you have encountered In the string are all white space characters so again return true
22:01
So this probably looks a little bit more scary than it actually is perhaps only because it's on a slide But this is like 50 line of code right and it's not particularly Particularly Difficult to reason about so if we can get up to 20 times faster performance writing 50 lines of C code seems worth it
22:22
So Next let's look at the equivalent fast blank implement implementation in rust and here is so this is literally a one-liner function in rust that does exactly the same thing and Handles all the unique code edge cases correctly
22:42
Let's walk through it so basically define an Extern C function this basically tells rust please I know that this code is going to be called from a C program so please keep the Use the C function calling convention for when you actually compile a program
23:00
That part is not particularly important except To illustrate the point that rust is a pretty low-level system programming language that's designed to interrupt with Other C programs and the Ruby implementation that you're probably using the MRI Or C Ruby implementation is a C program so they work pretty nicely together
23:24
and Because like I said the rest compiler cares a lot about safety so you have to do a little bit of work to Annotate your code to the compiler and give it enough information to know that your code is doing Correct things so here. We're telling the compiler what the type of the input to this function will be
23:47
The specific type we use here is not particularly important except to point out that you need to tell the compiler Which type each variable is going to be it also helps rust to the rest compiler to figure out how to
24:01
Allocate memory for four of these things because as much as much as possible rust try to allocate things on the stack So it doesn't have to Which makes cleaning up a lot faster, which is another key to? rust performance so here we're annotating to the
24:21
We're telling the compiler that this method is going to return a boolean value because the blank question mark method is Expected to return either true or false depending on whether the string is blank or not and the actual body of the method is Perhaps gonna remind you about
24:42
The Ruby code that you're used to writing so here. We're getting other characters from The string as an iterator, and we're using these high levels combinators like dot all you're probably familiar with this in Ruby It's equivalent to the array dot all question mark method that loops through the array and check if each of those
25:05
items in the array matches a certain condition and you have things like block which is also a thing we like to use in Ruby and finally you are for each character in the in the buffer you're checking whether it is a
25:22
white space character or not and this is white space is a Method provided by the rust and library that knows how to Do the big switch statement we have in this C implementation, so? This is pretty terse and perhaps surprisingly so because we're now programming a system
25:43
system programming language and surprisingly it looks kind of similar to what you will write in Ruby right so It's just going to be Sacrificing a lot of performance compared to the C version because it's so Terson so high-level well we ran some benchmarks, and if you look closely
26:04
This is iterations per Second by the way so high the number the better so at the bottom you have the pure Ruby implementation and on top we have rust and Next to it is the C implementation if you look closely the rust version is actually a little bit faster than a C version
26:22
Even though we're using high-level things like iterators and dot all so Of course I'm not being very scientific here. I'm not trying to convince you that rust is faster in C I'm just trying to convince you that you can get away with writing the high-level code that you used to without sacrificing Performance at least you would get code that is in the same ballpark as the C equivalent
26:46
so Let's look at this code again Turns out there's a catch it didn't tell you about The catch is there is a lot of glue code that I didn't show you here if you look at the full rust versus C Extension you notice that on the left the red part is the one-liner
27:05
I showed you and on the right That's the very big C function body that I showed you and indeed the C function body is Much larger than the one-liner on the left however if you count all the boilerplate code around it They're roughly the same size in terms of line of codes
27:26
Not all is lost though. I should point out that on the rest side a lot of those are Common shared abstractions like the buffer implementation and things like that that could be extracted out Whereas on the C side that's literally your business logic
27:41
So you have to write that amount of code every time you want to write is the extension like this whereas on the left? That's actually the only the the one-liner is The only thing that's specific to your extension still this is not very nice, so Yeah, we didn't I have been working on
28:00
this project called helix and Our goal is to make most of those world up late go away and take advantage of Rust features like zero cost abstraction to make it To help you focus on writing a thing you want to write without having to roll all the boilerplate all the time
28:20
So here is the entire fast blank extension written in helix, so We are using Rust macro feature which lets you write things. That's pretty similar to Ruby DSLs and So here we are At the top we're importing a library and here we're declaring a bunch of Ruby types
28:43
And the first thing we do is we try to reopen the Ruby string class and we add a method called is blank and you might find this syntax somewhat familiar and finally in the in the code we have the one-liner that we saw earlier and
29:02
This is all the code you need to write to write a extension like fast fast blank and rust and helix so my personal Goal for this going forward is I would like to be able to experiment
29:22
implementing some of Rails features in terms of in in helix so we already have an extensive test suite and in rails and there are a lot of modular parts that That is like stringed up blank that can be swapped out into
29:42
Into your rust extension without affecting any of the user facing API so We can continue to iterate quickly on the Ruby implementation on the sorry on the pure Ruby implementation on the rail side while taking advantage of the existing rails test suite and
30:01
write these experiment with these lower level implementation and As long as we keep running against rails test we'd look we can be fairly confident that things are in parity and if So it would of course be an optional thing that you can install just like fast blank but if you're if your platform supports it and if you
30:25
Trusted enough it might give you some substantial performance gain and there are a lot of low-hanging fruit for a smaller piece like For example the active support duration class like when you do one dot day, that's the active support duration class that's created and
30:42
So there are probably a lot of pieces small pieces like that that we can experiment with using helix and Eventually we can probably work on Writing bigger pieces like perhaps the the core routing library in rust someday
31:01
So that's all great, but What about my app well so while we're working on this project? We have a friend who works at sesti, and they have a pretty interesting problem for us so they they for the most part there the stack is a Ruby stack and
31:24
They are catering company in San Francisco that tries to Deliver organized meals for for companies based in the Bay Area And one of the problems that they have is they have a meal matching algorithm so for example
31:43
There will be some constraints like I Some of my employees have these allergies so they cannot have meals with these items in it or sometimes There will be the opposite some of my employees preferred these Features and the meals and you should make sure the meals have
32:01
Have the like vegan or gluten free and etc so at the end of the day There the problem boils down to something like this you have some tags for for each meal, and you have some tags for different preferences your users might have and
32:22
What you want to find out is whether the meal the tags in the meal fully satisfy all the requirements in the in the second array Right so this is the tax on the meal and what you want to see is if this first array fully contains all the numbers in the other array and
32:41
that basically boils down to What we call set containment problem, and there is a known trick for that Oh, I should mention the reason they care is They have written this Algorithm in Ruby, and it's taking a long time to run like sometimes like and measured in minutes
33:01
Maybe up to 30 minutes right and they measured it and a lot of time is being spent in this Set containment algorithm, so It would be interesting to see what you can do if you implement that algorithm in rust so There's a known trick for a setting
33:21
Testing set containment if you go to stack overflow and look for it. You'll probably find a solution similar to this basically To check whether an array fully contains another array You can do an intersection between the two arrays basically make a new array that only contains
33:40
Items that is in both of the original and the the other array right and then you can check if that array is equal to The preferences array so this would tell you whether your meal fully satisfied your preferences so this is really nice to read like off the Ruby code that you're used to writing and
34:03
Usually it's fine right like I think it's good to be able to start with something like this and So they have this in their code base, and they have an existing test suite To test that this is working so we Replicated that test suite and replicated this method, and we confirmed that it indeed works as advertised
34:26
But we can do better than that Because we know we happen to know that all of the numbers in the array is Sorted and all of the numbers in the array are unique, so there are no duplicates in the array so After thinking really really hard we realize we can do better
34:42
So here is a different algorithm in pure Ruby to do the same thing I Will walk through it quickly so first of all you check some edge cases if either of the array is empty Then you can very quickly decide one way or the other without having to do a bunch of extra work then there's some boilerplate to checking tracking the
35:05
position into like the index of the other array and basically you loop through all of the items on the longer array and keep pointing to keep a pointer into The other array and try to advance a pointer as you find
35:21
Similar item if and finally if you get to end of the loop you know that you didn't You there are some items that is in the other array. That's not in the original array, so It took us a while to figure this out, so if you don't immediately understand it don't worry about it But the point is we wrote this and we run it against the our spec
35:42
test we have and it passes and according to a benchmark is up to seven times faster and The usual case is more so depends on the particular test case right the usual case is more like 2x And but it goes all the way up to you seven times
36:00
So that's pretty great. You basically figure out the Google interview question You should give yourself pat on the back at this point, but what if we write that in rust so that's what we tried next Again, we're using helix. So we are using the declare types macro. We're reopening the rate class and
36:20
we are defining the same fully contained method with the type annotations and here is the same code that checks for the edge cases and The same boilerplate for tracking the array index Interestingly here in rust as I mentioned earlier if you use iterators the compiler is actually able to do smarter things
36:44
So here we are actually using an iterator instead of writing a hand rolled loop and because we're using an iterator The loop body end up looking slightly better than the Ruby version in my opinion But otherwise the same exactly the same algorithm and
37:02
Finally if you get to the bottom you return false And we benchmark this and it runs up to one hundred seventy three times faster than the Ruby version Depends on depending on the workload, and I plotted this on a chart. You probably can't really see all the details, but each
37:20
basically we rent a bunch of different test cases and The blue line is the pure Ruby implementation, that's the one-liner the green lines are The fast algorithm written Ruby and the yellow lines are the rest implementation of the same algorithms so you probably can't see a lot of blue and greens because they're mostly at the bottom and I
37:45
Think the numbers are roughly The rust implementation is usually tend to up to like hundred and seventy four three times faster than the Ruby implementation, so that's pretty great and
38:02
Here is we are almost out of time so rubber quickly so So here is where you can find the code for the helix project is major work in progress but we are on a pretty good track and if you would like to help us you can come talk to us afterwards and
38:22
finally I would like to close with Was this historically Scripting languages were slow, and they're still relatively slow today and because they're slow They were mainly historically used as a coordination layer that delegates to
38:42
Heavier tools written in system languages like C So for example you might write your shell script and bash that delegates to Unix utilities like sword grab or work count to do to have to do the heavy lifting However it turns out that a lot of time
39:01
We are actually doing IO bound operations like reading and writing to the hard drive or waiting for network or talking to your database Because I operations are usually so much slower the performance difference between a scripting language and System language doesn't actually matter so much a lot of the time
39:22
But because scripting language is so much easier to write and so much more pleasant to use We end up moving a lot more of the code into the scripting layer and instead of writing everything in C So since web applications are so IO heavy That worked out wonderfully for things like rails and the fact that you can have
39:42
60 in a 32 unicorn Process on like an eight core machine is probably a good It's probably a good empirical evidence to say that that's true However at the end of the day we still have some some occasional computationally heavy operations and now application turns out
40:04
Business logic is what defines your application what set it apart from other apps So I think we're entering a new era where we have taken a lot of the advancement from scripting languages Especially on the ergonomic side and move that into system languages like rust so with helix you can readily move any
40:25
competitionally heavy part of your app back into rust without having to switch your entire stack to something like goal and In my opinion rust is particularly suited for this task and rails teams Thanks to the safety guarantee offered by the rust compiler
40:42
More of your team members will be able to tanker and experiment with the rust code Without worrying that it would cause problems at runtime the worst you can do is not compile make the program not compile So like you probably try get help at that point in our team Everyone end up picking up enough rust to be able to fix bugs or make minor tweaks to our rust agent in
41:03
Roughly half a year or so and we think that will continue to be true. No pressure Rocky So our goal with the helix project is to make this even more accessible to more Ruby teams So we can keep writing most of your code in the language you love without fearing that it will be too slow eventually
41:21
So because you can always drop down to rust if and when you need to do that So once again, that's all I have today and you can find me on the internet's as champion code. Thank you for your time Let's make Ruby great again. Thank you