Building CLI Apps for Everyone
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 66 | |
| Author | ||
| Contributors | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/37515 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language | ||
| Producer | ||
| Production Place | San Antonio |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
Ruby Conference 201560 / 66
8
10
11
14
18
19
21
22
23
25
26
27
30
31
35
36
38
40
41
42
46
47
48
51
52
54
55
57
60
61
62
64
66
00:00
Common Language InfrastructureBuildingMobile appTwitterWhiteboardBuilding40 (number)Mobile app
00:38
TwitterStorage area networkLecture/ConferenceComputer animation
01:30
Information securityMultiplication signMetropolitan area networkLecture/Conference
02:12
Online helpView (database)Line (geometry)Single-precision floating-point format
02:45
Order (biology)Computing platformMultiplication signSoftware developerDialectCartesian coordinate systemRight angleMetropolitan area networkInstallation artDecision theoryData managementComputer animationLecture/Conference
04:00
Installation artReal numberMultiplication signProjective planePhysical systemBridging (networking)Computing platformVapor barrierProcess (computing)Software developerSoftware as a serviceFront and back endsLecture/Conference
04:35
Revision controlInformation securityDefault (computer science)Revision controlMultiplication signSoftware bug1 (number)Inheritance (object-oriented programming)Computer animationLecture/Conference
05:14
Computing platformRun time (program lifecycle phase)Installation artPhysical lawRight angleProcess (computing)WindowHydraulic jumpCuboidBit rateOffice suiteRevision controlSign (mathematics)Public key certificateLecture/Conference
06:18
Common Language InfrastructureClient (computing)Revision controlFormal languageSystem programmingDifferent (Kate Ryan album)2 (number)WindowMultiplication signOrder of magnitudeRevision controlComputer filePhysical systemIntegrated development environmentVariable (mathematics)Speech synthesisProcess (computing)Metropolitan area networkCartesian coordinate systemWordLine (geometry)Projective planeHacker (term)BenchmarkEntire functionLevel (video gaming)Single-precision floating-point formatDivisorOperating systemCodeEquivalence relationDefault (computer science)Run time (program lifecycle phase)Lecture/Conference
09:04
Directory serviceRun time (program lifecycle phase)Extension (kinesiology)MereologyWindowRevision controlInstance (computer science)Projective planeMultiplication sign1 (number)Cartesian coordinate systemException handlingFigurate numberComputer animation
09:52
Different (Kate Ryan album)Formal languageProjective planeRight angleDependent and independent variablesLie groupBuildingComputer configurationScripting languageCloningVideo gameVulnerability (computing)ResultantLimit (category theory)Software repositoryResource allocationCartesian coordinate systemWritingMereologyTwitterQuicksortRevision controlMultiplication signHuman migrationProcess (computing)ParsingComputer animationLecture/Conference
11:59
Computer configurationRevision controlDirectory serviceData structureCodeLibrary (computing)Point (geometry)Computer fileResultantComputer animation
12:30
Binary codeComputing platformFunction (mathematics)Directory serviceBinary fileFeedbackBinary codeMultiplication signOrder (biology)IterationProjective planeSingle-precision floating-point formatCartesian coordinate systemProduct (business)Run time (program lifecycle phase)Loop (music)Variable (mathematics)BootingPhysical systemSoftware developerInheritance (object-oriented programming)Integrated development environmentBuildingCommon Language InfrastructureDifferent (Kate Ryan album)Computing platformOrder of magnitudeComputer fileTouchscreenDifferential (mechanical device)Stress (mechanics)Letterpress printing2 (number)Video gameDataflowMetropolitan area networkGoodness of fitFamilyWindowRevision controlWordVotingBenchmarkLie groupLecture/Conference
15:55
Computer virusGastropod shellRevision controlComputer fileOrder (biology)Binary codeComputing platformMatrix (mathematics)Gastropod shellWindowDataflowLie groupTemplate (C++)Task (computing)Pattern languageMobile appPlotterVideo gameComputer animationLecture/Conference
17:42
ImplementationFormal languagePhysical systemTemplate (C++)Order (biology)Computer architectureSemiconductor memoryBootingMultiplication signType theoryMultiplicationDifferent (Kate Ryan album)Projective planeMoving averageLecture/ConferenceComputer animation
18:26
FirmwareCodeRouter (computing)Multiplication signComputer animationLecture/Conference
19:00
Network socketComputer fileThread (computing)Different (Kate Ryan album)Category of beingMultiplication signThread (computing)File systemType theoryComputer architectureLevel (video gaming)outputSubsetAndroid (robot)Operating systemCopyright infringementPhysical systemFunction (mathematics)Block (periodic table)String (computer science)FreezingComputer animationLecture/Conference
20:06
Block (periodic table)Computer programmingCodeComputer virusInteractive televisionGastropod shellBinary codeFormal languagePatch (Unix)Block (periodic table)Reading (process)Order (biology)TouchscreenComputer programmingOpen sourceCartesian coordinate systemCodeProcess (computing)Computer fileBytecodeFlagMessage passingRepresentation (politics)Interpreter (computing)Binary codeCompilerGoodness of fitFunctional (mathematics)Parameter (computer programming)WritingScripting languageLetterpress printingCompilation albumPRINCE2AreaWordMultiplication signComputer animationLecture/Conference
22:14
Drill commandsCurve fittingCodeComputer configurationRevision controlSoftware testingCompilerBinary fileDisintegrationExecution unitComputer iconChainFile archiverLinker (computing)Core dumpParameter (computer programming)BitCodeFormal languageScripting languageComputer fileWrapper (data mining)Binary codeBuildingDifferent (Kate Ryan album)Order (biology)Configuration spaceCentralizer and normalizerPhysical systemTask (computing)Repository (publishing)Representation (politics)Design by contractLatent heatMetadataOperator (mathematics)Equivalence relationWritingLibrary (computing)Directory serviceCuboidService (economics)Uniform resource locatorDefault (computer science)Power (physics)Process (computing)System callControl flowEntire functionCASE <Informatik>Touch typingComputer animationLecture/Conference
25:19
Core dumpDebuggerCompilerEnumerated typeDirected graphExtension (kinesiology)Software developerTerm (mathematics)Event horizonComputer animationLecture/Conference
25:57
Core dumpComputer iconChainSoftware testingError messageLattice (order)Computer fileFunction (mathematics)Process (computing)Total S.A.INTEGRALExecution unitSoftware testingElectronic mailing listEntire functionSoftware bugPhysical systemComputer fileSoftware repositorySoftware frameworkoutputCloningBinary codeRepository (publishing)CodeEquivalence relationBinary fileLibrary (computing)Point (geometry)Line (geometry)Inheritance (object-oriented programming)Unit testingDialectOperating systemMereologyRevision controlRight angleOrder (biology)Process (computing)Projective planeMultiplicationVideo gameLie groupSpacetimeBuildingComputer animationLecture/Conference
29:49
Extension (kinesiology)CompilerWindowOrder (biology)MultiplicationProjective planeComputing platformRegulärer Ausdruck <Textverarbeitung>Compilation albumCASE <Informatik>ParsingAuthorizationOperating systemExtension (kinesiology)Power (physics)Computer animationLecture/Conference
30:38
Revision controlContent (media)Term (mathematics)Java appletMenu (computing)Execution unitGlass floatCodeCore dumpSource codeUser interfaceData miningOperating systemLie groupProjective planeSummierbarkeitRevision controlMixed realityLine (geometry)Video gameComputer configurationMoving averageString (computer science)Unit testingComputer fileParsingInteractive televisionTask (computing)Web pageFamilyBootingWindowKeyboard shortcutPhysical systemReal numberWave packetCodeBuildingSpecial unitary groupTouch typingSemiconductor memoryExecution unitINTEGRALSoftware testingEntire functionCommon Language InfrastructureBinary codeLecture/Conference
34:58
BuildingCommon Language InfrastructureBinary codeFunction (mathematics)Installation artCompilerAdditionSocial classProjective planeString (computer science)Cycle (graph theory)CalculationCartesian coordinate systemCuboidMultiplication signBuildingAdditionBinary codeFlow separationComputer animation
36:25
Limit (category theory)BuildingTask (computing)Factory (trading post)Hydraulic jumpProjective planeDialectWritingSoftware bugMultiplication signLatent heatLibrary (computing)Computer configurationFormal languageSoftware testingClosed setSinc functionBytecodeBinary codeComputer fileComputer animationMeeting/InterviewLecture/Conference
Transcript: English(auto-generated)
00:24
I'm here to talk about building CLI apps for everyone. So if you were looking at the board, and you saw something about managing a community for Ruby Association, this is not that talk. Fortunately, he couldn't make it. So yeah, so I'm Terrence Lee.
00:41
I go by Hon02 on Twitter. I come from Austin, Texas, which is only an hour and a half north of here. We have some awesome tacos. If you're ever in town, I'm more than happy to take you out for some tacos. I know some people have collected on that. There's also a Torchy's Tacos, I think in San Antonio,
01:02
halfway between here and the airport, is what I've heard. They have awesome tacos. With a few fellows here, and Davey, who is not here at RubyConf, we run this weird conference called Keep Ruby Weird, which is in Austin, Texas, just happened last month.
01:21
Stay tuned for year three next year. You probably hassle Ernie of how great or terrible the conference was. He was there. I'm also a big proponent of Ruby karaoke. If you did not go to Ruby karaoke at RubyConf, you missed out on a really awesome,
01:40
magical experience. This was us singing Bohemian Rhapsody, which is a classic, of course. It really started, I was inspired by Charlie Nutter back in, last year, during RubyConf Taiwan. He karaoke'd for a marathon like seven hours.
02:01
I only managed to do like five, but it was a really great experience. This is him singing a duet with Plexus, which is always a good time. I have to give a shout out to PJ Haggerty, who cannot make it here. But he has been a huge proponent of Ruby karaoke
02:21
and has helped organize a ton of them, so much that I've actually organized none of them this year. But there's been one in every single Ruby conference I've been to, which has been pretty great. I work at Heroku, I do Ruby things there with Richard Schneeman, who's sitting over there. He was the, he had that picture on the first slide,
02:41
drinking that medium-sized margarita. Everything's larger in Texas, apparently. So on to the actual talk. So I'm here to talk about packaging in Ruby and why we should be building things in Ruby. And in order to get a sense of how this works,
03:01
I'm gonna talk about the Heroku tool belt and the story behind it, and where we started and where it's going today. Okay, and so when we first started, we were just a Ruby gem, so in order to get started, you just ran gem install Heroku, and I think even some people today are still using the Heroku gem,
03:21
even though you're not supposed to. It's been deprecated. And at the time, it was a great decision because Heroku was just targeting Ruby customers. That was the only thing we supported on the platform. And so plugging into the Ruby gem's ecosystem made it really easy and convenient to get up and running. We didn't have to deal with package management.
03:42
We didn't have to figure out how to deal with all this stuff because Ruby gem already had a specification, and if you're a Ruby developer, you already had Ruby gems installed to already build your Rails and other applications. So that was a great win for us because it allowed us to move quickly and not have to deal with many other things.
04:01
But the downside over time that we learned was that it required people to actually have Ruby installed on their system, which as we moved to be a more polyglot platform, that became an issue. I've talked to some other people who have, like there's the SaaS project, which requires you, when it first got started, to have Ruby installed to do SaaS stuff,
04:21
which was a huge barrier to entry for front-end developers who may not be familiar with getting Ruby up and running. If you ever helped out at a Rails Girls or Rails Bridge, I'm sure you're very familiar with how painful this process can be. And the other thing, even if you were a Ruby developer, we couldn't even guarantee the version of Ruby
04:40
you're running was the same one. So we had to essentially support like many versions of Ruby. On OSX, it was famous that they had Ruby 1.8.7 as the default version of Ruby for a long time. And even after Ruby 1.8.7 was deprecated and like no security updates or anything were being added to it,
05:00
we still had to make sure our gem was backwards compatible, so we couldn't take advantage of any of the new features or syntaxes just because we had to make sure it would work. Which made debugging and kind of maintaining it a pain in the butt. So we moved on to essentially a package up installer, where we would package up the Ruby runtime
05:21
into this thing called the tool belt, and we would take the gem and basically have the gem and the runtime with it. And so we would have to build that for every single platform that we wanted to run. I don't know if any of you have tried to package up Ruby for Windows, but it is not very fun.
05:42
We had like a special box that would do that. And even for OSX, we had like a Mac Mini when we first started this, like under someone's desk at the office that was just there for package up like OSX Rubies. And we've automated a lot of that stuff since then, but it is still a lot more work
06:01
than just pushing up a new gem, right? Like if you're just updating a gem, you can just run rake release or some other task, and it's like very simple to get versions and stuff out. Now it's a much more involved process, and with the new OSX signing stuff, like we have to have certificates and other things in place.
06:20
So then there was this HK project that was started internally by an employee, and it was basically rewriting the entire CLI inside of Go. And one of the big motivating factors for doing this was that speed is a feature, like fast is a feature.
06:41
And so at the time when this was benchmarked, running Heroku version on the Ruby gem or the tool belt took almost two seconds. And the Go equivalent was 16 milliseconds. And so you're talking about like orders of magnitude of difference here. Admittedly, the Go version did a lot less
07:01
than the tool belt version, but I think it does show some baseline on performance difference there. And I think a lot of this can be attributed to the fact that having to load a bunch of these things inside of Ruby at runtime can be a slow process. If you haven't heard of this before, but require is pretty slow.
07:22
There is a performance impact for doing that. And so especially for a command line application, you want to split up your application as it gets bigger, because you don't want to have the monolithic Perl, like 23,000 lines of Perl code that you're trying to parse through for a single command. So you end up splitting it,
07:41
but then you have to do all these hacks, which we've done in the tool belt to only load the files that we need to at the time that they're needed all throughout the code base as the default version of Ruby for a modular and actually maintainable. The second thing that was really appealing about Go for us when we were doing this was that
08:00
you could statically link each binary, which meant that we could build a single file that we could distribute to every single operating system that we cared to support. Which meant for Windows and OS X and Linux, it was one file, we could just package that up and then give it to someone and they could install it on their system. And it wasn't this folder of files and things
08:22
plus some runtime and path environment variables to make sure that all that stuff is set up. And if you had an existing Ruby install, we had to make sure we weren't like munging and overwriting stuff while you're running the tool belt, that it would pick up other things that you already had set. So I don't think the Heroku tool belt story is unique.
08:42
I know Hashicorp has had a similar story. Their Vagrant project, a lot of their CLIs were originally written in Ruby. Vagrant is still written in Ruby, but they went from like being a Ruby gem to not being a Ruby gem anymore last year to just packaging up itself and then all their newer CLIs stuff is written in Go
09:02
for a lot of these same reasons. If you do want to package stuff up in Ruby though, there's a project called Traveling in Ruby by the Fusion guys, probably most well known for ModRuby and Fusion Passenger. And the nice thing here is that you don't actually have to rewrite your entire application,
09:20
you can still use Ruby and it kind of handles the tool belt part of packing up a run time and having extensions that you're gonna use if you have native extensions like Nokogiri for instance. But the downside is that you're actually restricted to all the run times and extensions that they have precompiled and the exact versions that they have compiled.
09:41
So if you want to use something else, you're kind of out there on your own to try to make all that work if you want to support Windows and Linux and OSX all at the same time. And when I was in Portugal, I actually ended up talking to SF Eric and he was telling me about this Crystal project.
10:01
So besides Go, there's also Crystal which is a statically typed, compiled, Ruby-like language that allows you to build things. And after kind of seeing this over the last few years and we have a ton of Ruby application or Ruby CLI is like an ecosystem for it
10:22
with like option parsers and GLI and Thor and things like that. And there was a lot of CLIs that originated in the Ruby ecosystem and it's a little sad to see them all go away and I think a big part of that is because our packaging story isn't great for packing job MRI. Like to deploy a Rails application, we have these Chef scripts that go
10:42
and like clone your repo, run Bundler and then like do all this setup stuff to hopefully make it all work. And instead of using other languages, I wanted to continue to build things in Ruby which is kind of the inspiration behind all this stuff. And I gave a version of this talk
11:02
at Rocky Mountain Ruby and Steve gave a kind of really poignant tweet in response that I think applies heavily here. There's nothing wrong with using other languages like Go and Crystal are both great languages in their own right and if you are happy using that stuff, like you should continue to go ahead and do so.
11:20
This isn't a language bashing talk. But I want to continue to build things in Ruby and I wanted to find a way we can shore up the weaknesses for continuing to do stuff in Ruby where people want to choose to do so. And I think picking Ruby shouldn't be a technical limitation of why you're working on your project, that you have to migrate everything off of it.
11:42
So how do we make packaging possible if you want to continue using Ruby? So Zachary Scott and I started a project called mruby-cli back this early summer. We spent a bunch of time trying to solve this problem. And we had a few design goals in mind.
12:01
So if we're gonna write a thing for Rubyists, you should be writing most of your code in Ruby. I think that's a pretty important point. And so inside of mruby-cli, you have an mruby lib directory and then you can essentially just have a bunch of RB files which are Ruby files. And you can add sub-folders in them
12:20
like you would in a Ruby gem. You can put anything, you can put all these RB files in any kind of organizational structure that you want. As long as it resides in this folder. The next thing is that performance is a feature and I learned that through the HK story as I saw that develop. And what I really mean by that is actually start up performance.
12:42
So generally like when people do these benchmarks, you take it with a grin of salt, but for command line applications it's actually really pertinent, right? Like how fast can you boot up the Ruby runtime and get something out on the screen? Because most command line applications are hopefully gonna be like this quick thing where you're interacting with it relatively quickly and you want to be able to get that fast feedback loop.
13:04
So inside of MRI I'm sure most of you have written this application before where you literally just print hello world to the screen. So on Ruby 2.2.2, which is what I tested this on, it took 40 milliseconds to boot up, which is not super slow, and that's totally tolerable
13:21
and it's pretty respectable to deal with that I wouldn't mind writing command line applications in this, but as you start to add require and other things, this will definitely slow down. And so when I did this in mruby-cli as we started to build out this project,
13:40
the simple hello world application was three milliseconds. So it's an order of magnitude faster. And to me that gives you a lot of headroom to do a lot more interesting things inside of the project and still get that really fast iterative kind of workflow that you would want out of a CLI. Like if you're building a CLI as a product to your company and people are running a bunch of commands, you definitely want to have
14:02
that kind of loop and feedback system. So like I was saying earlier, there's no require inside of mruby, which means that as you continue to add files into it, you're not going to get that kind of runtime slowdown that you would in MRI itself. So as the CLI apps get bigger,
14:20
it's only the performance differential for the startup time is only going to get bigger. And the third design goal that we had in mind was that like the Go thing that made that so attractive is that the product and artifact that we have to produce is this single binary that people can ship because that's a very attractive system
14:40
and not having to figure out some way to package all this stuff up and set up all these environment variables and things like, if you just have a single binary that has all the runtime and things included, you can just ship that to the customer and they can get that up and running relatively quickly. So inside of mruby-cli, we have a bunch of different platforms and you essentially, when you run the build,
15:01
you see the build name and then it produces a single binary. So for the hello world thing, this is the build summary that you get. And we have one for OSX as well as Windows, so you can cross-compile for all Linux OSX and Windows, which is a really great experience. And to give you a sense of how big these files are
15:22
that include this mruby runtime, the OSX binary is only 421 kilobytes. So not the smallest thing, but not terribly large either. And the final thing was, in order to actually get any traction on this, the setup has to be really simple to get up and running.
15:41
In order to do all this cross-building stuff like Go does, like you have to set up all these tools, and we wanted to make that as simple as possible. Because we knew that mruby-cli would be a real underdog compared to all the other solutions out there. So we're leveraging Docker, we built a container. If you're not familiar with Docker,
16:02
it's basically a Linux containerization thing that allows us to provide a simple setup for people who are interested in trying this out. So we have a tag pushed up onto Docker Hub that has basically all the tools and things you need. So in order to get started, all you need is Docker and the mruby-cli binary.
16:21
You don't have to have Ruby install, you don't have to have GCC or any C compiler, none of the cross-compiling tools for OSX, and nothing for Windows either. So this allowed us, as well, to only target one platform to then cross-build for everything and not have to handle that three-by-three matrix of how you compile from one thing to the next thing.
16:43
So the simplest hello world example of what this flow looks like is when you have mruby-cli on the path, you pass this dash dash setup, and you name the thing like in Rails new of what you want to call it. So hello, and once you CD into that directory, you have a bunch of files,
17:00
and you run this compile task through Docker Compose, and it will, you go and grab some coffee, wait for it to compile, and then you can run a shell or just execute the actual target build that is specific to the platform you're actually building this on, and when you run hello, you see hello world.
17:20
So that's kind of the workflow of like how you would get something up and running from nothing. And so when we actually run the setup, like in Rails, we produce a bunch of files so you don't have to write them yourself, and this allows you to get up and running relatively quickly. And basically for the rest of the talk, we're gonna kind of go through like what is generated and kind of how all this stuff works and fits together.
17:44
And mruby-cli itself is actually just this idea and really just that Rails like template generator, and that's all it is, and it pieces all this stuff together. And most of the leg work is actually done by mruby itself and the entire build system and everything involved with it.
18:01
So in order to really understand how any of this works, we have to take a step back and just talk about mruby. So what is mruby? mruby is this embeddable Ruby that is meant to be lightweight because it is being embedded in multiple different architectures and types and whatnot. So it's meant to have quick boot up times
18:23
and lightweight in memory. And people ask me all the time like, okay, this is a really cool project, like where is it actually being used? And I was talking to Matz yesterday at our BoF and asking about it, and they use mruby in Japan.
18:43
There's a company that produces routers and they use it to basically extend like the firmware. So you can write mruby code to extend the firmware stuff that you're using and not have to say drop down to C or assembly or whatever the firmware is being written in, which is really cool and it makes it really powerful and flexible.
19:01
And here's some of the contributors. I've been working on mruby. So what are the big differences between mruby and MRI? Most of you are probably not super familiar with mruby and it's maybe your first time really looking into it seriously. So since mruby is being built and targeted for different architectures besides just x86,
19:25
it can compile to Android and iOS as well as it's often used for IoT type of things as well. And Raspberry Pi, Arduino, stuff like that. That means that we can't guarantee that there's actually a file system there or any like the OS level things that we're used to having inside of MRI itself.
19:44
So that means no file, no socket, no input-output kind of thing. It's not thread safe because threads are an operating system level like feature so that also means there's no threading or forking as well. And the syntax itself is a subset of Ruby 1.9
20:01
with maybe some stuff from Ruby 2 like StringFreeze which got added recently. But on the flip side, it is an actual Ruby language. So that means stuff like procs and blocks that we're used to having, DHH freedom patching that he talked about a few years ago and metaprogramming and literals, like things that we're used to having inside of Ruby
20:22
are there and they work like you would expect inside of Ruby. So in order to understand how like all this kind of comes together, we have to look at how you actually run mruby code. Like I write a Ruby file, like how do I get it to actually print something to the screen or do what our command line application does.
20:45
So the simplest way, like inside of MRI, you can actually, when you build mruby, you can get an mruby binary which is just the interpreter. So you can pass dash e, print stuff out, you can also take your Ruby file and then pass it in as an argument to the mruby binary
21:01
and it just goes through and interprets the code. And we also provide an IRB binary as well so you can get all that REPL goodness that was really exciting when I first got started in Ruby. You can do that as well in mruby.
21:20
But where it starts to get really exciting is that there's an mruby bytecode compiler. So you can take any mruby script that you're writing and if you use the mruby bytecode compiler, it actually takes the code that you've written and puts it into mruby bytecode and you get this .mruby file that you have. And once you have that, you can use the mruby interpreter,
21:42
pass the dash b flag to specify that this is a bytecode representation and run the code. So it means at runtime, you do not actually have to go and do that compilation process. And if you take this a step further, you can actually take that and embed it inside of a C file
22:00
and get the mruby bytecode compiler to actually produce C code with an array of all this stuff that you need. And then you can go and write a main function that invokes all this stuff and then you can go ahead and basically compile down a static file. So if we take this to its natural conclusion,
22:21
you can actually produce a single binary with all this stuff. And so with mruby CLI, since we wanted to focus on people producing Ruby code and not writing a bunch of C, I assume many Rubyists are not familiar with writing a bunch of C. C is not my best language. We produce basically a C wrapper script for you that takes all the arguments that you need
22:42
from the C code and passes it down to mruby. So inside of your Ruby code, you have access to all the arguments and you can just write all of your stuff inside of Ruby and not have to write any of the C bits. So the only contract we actually have is that you have to define, we call out to this __main__ method and the argv is a Ruby representation
23:03
of the arg array that is being passed inside of the C code. So like I was saying before, we have an mruby lib folder that has all the Ruby code and again, it's really important that we're letting people write stuff in Ruby. And one of the awesome things about mruby is that all the stuff in it,
23:22
the whole entire build system is built on top of Ruby itself. So the entire build system uses Rake. There's a bunch of Rake tasks and stuff, so in order to compile and test and do things, you can just run a Rake task and that's the same for mruby CLI. We also just leverage the Rake system.
23:43
And so since this stuff is being built, one of the unique things about it is there's this buildconfig.rb file that basically helps you configure what you're building. So you can essentially create a new build for the targets that you want to build for. And in our case, we generate one for OSX
24:01
and this is an example of what it would look like to do a new cross build. You don't have to actually write any of this stuff out, like we generate all this stuff by default. We generate that build config and we populate with all these things. So in general, you don't really have to be touching this file unless you want to be overriding stuff. Like you have specific requirements and whatnot, or you don't care about compiling to some operating system,
24:20
you can just delete this or comment it out. And so when we run compile, we see that we get all that stuff built. And there's also this system called mruby gems, which is probably the closest equivalent to what MRI has for Ruby gems. And inside of this mruby gem or Rake, it's similar to the Ruby gem specification.
24:41
If you've ever written a Ruby gem, probably a lot of this stuff looks familiar to you. You specify all the metadata. And where it starts to get really interesting is this add dependency. Like this looks different than actually inside of MRI itself. So it takes two arguments. There's the name and basically where this gem comes from.
25:00
Since there's not exactly a central repository like RubyGems.org that's really standardized as a service where you can push up new releases and stuff, you actually have to tell mruby where to get this stuff from. So there's three different kinds of locations that it supports out of the box. There's core, so inside of the mruby directory itself,
25:22
there's an mruby gems folder. Inside of it, there's a bunch of mruby gems that you can use. And the reason that it's split out is because you're trying to, again, have a small footprint, which is great for CLI development. So we only want to be pulling in the things that we want to be using. There's many extensions that we're probably familiar
25:40
to having inside of Ruby for Ray and enumerators and things like that. And if we want to actually leverage that, you would essentially pull that in as a dependency. So we can really limit what we're actually pulling in and not have the entire kitchen sink. So that's all bundled in and maintained by the mruby team. There's mgem, which is the closest equivalent
26:01
to RubyGems.org, but it's not quite the same thing. Essentially it's a repository on GitHub and when you want to add something to it, you send a pull request that adds your mruby gem into this system and it basically clones it and then there are essentially just other GitHub repos for the most part, but it's a system that allows you
26:21
not having to know where all this stuff lives and you can just specify the mgem list. Of course you can also just use GitHub itself, which means if you want to start testing a thing that you didn't feel should be on mgem yet because it's not solid, but you need to test and build it, or if you want to fork a gem and you want to do that, there's an easy way to add that there.
26:43
And so one of the things inside of build config, like I was saying, overriding stuff, one of the things you can do is actually override any dependencies that you have conflicts for. So if a mruby gem has another dependency but you need to fix some bug there, inside of, in MRI, like in Bundler,
27:00
you probably would fork that thing on GitHub. You would have to fork the parent thing to have the gem spec point to your other thing and then make sure all that stuff's included. Inside of mruby, we can actually just specify this bottom line here, the conf.gem, and it will use that over anything else that is specified inside of mruby gem rake
27:20
and all the other mruby gem rakes that you're pulling in for the other mruby gems. Since we're rubies, we need to test stuff. So there is unit testing through mtest and that is a gem that can get pulled in and then you can essentially have a folder of test files that are all unit tested.
27:41
One thing to note is that since these are unit tests, they're running inside of mruby, so you're limited to the mruby syntax and code and the dependencies that you're actually pulling in, so you can't pull in extra stuff if you don't add it in. But it does allow you to write really fast unit tests. So this is an example of what that looks like.
28:00
It looks probably pretty familiar to most people who just run any other unit test. The next thing is that we actually also support bin tests and this is built into mruby and this actually uses MRIs, so you see those require files which should give you that hint and it just leverages Open3 and it's a framework for kind of wrapping all that stuff around it
28:20
and basically you just want to invoke the binary that you're producing and just test input outputs there. But since you have Ruby to the entire MRI system to your disposal, you can do a lot more fancier things inside your integration test for setup and whatnot. So this is an example of what an integration test
28:40
would look like inside of mruby. So this all sounds great, but what are the gotchas that you're going to be facing when you want to build something like this? So the first thing is that MRI has a rich standard library out there. I mean, in Aja's keynote, you saw like Rinda and tuplespace and stuff like that.
29:03
None of that stuff is really available inside of mruby and even the stuff that is is fairly trimmed down and ordered by design to keep it lightweight and small. So if you wanted to port something over to mruby, it's not as easy because there's many, even if it had no dependencies inside of MRI,
29:21
it probably leverages a ton of the standard library which may or may not be working 100% inside of mruby. And that also means you can't leverage RubyGems, right? Like that entire ecosystem of RubyGems that is out there, I think in, I was told it's like 7,000 gems.
29:41
Like none of that can be directly just like pull this thing in for RubyGems and expect it to work. So that's a huge bummer. And in order to support all this cross compiling stuff, the mruby gems that have native extensions in C and whatnot need to support cross compiling in order to get it to work across multiple operating systems.
30:02
So as Zzack and I have gone through and gotten stuff to work for mruby CLI and the mruby CLI projects that we've been building, we've gone and started adding support for cross compilation. So like libyaml and the XML parsing stuff with, or no, the Regex stuff with onigurama.
30:21
We've gotten all that stuff to cross compile and work on Linux OS X and Windows fine. So this is definitely like in a huge uphill battle because I think a lot of the gems are essentially built for the use case of the author. And so they don't need to support all these other platforms. But there is, I did want to talk about one success story
30:44
that we've had with mruby CLI. So there's this project called JRuby Launcher inside the JRuby ecosystem. And normally when you download JRuby, you get like either a bash file that boots it up on a Unix system or you get a bat file on Windows.
31:01
And they essentially had to maintain both of these things, which can be kind of a huge pain in the butt. And so they have this gem that you can gem install called JRuby Launcher that is this C++ code that can be used to basically launch your thing and they can have a unified code base for all the operating systems they wanted to support.
31:22
So inside of this JRuby Launcher code, you have a bunch of C++ and header files. And just to pull one of these files, there's this copyright from Sun Microsystems at the top of the main JRuby CPP file. And its last copyright was 2010, it's kind of old. And Sun Microsystems doesn't even exist anymore
31:41
as a company, so there's that. And if we look at some of these commits, some of these files, like that CPP file hasn't been touched in six years. So you can imagine how fun it must be to maintain this thing. So a coworker of mine, Joe Kuttner, started this project called MJRuby.
32:02
And essentially it's this entire JRuby Launcher rewritten on top of mruby-cli. So inside of it, he essentially rebuilt all this stuff using Ruby code, a mix of Ruby and C code. And he built some libjvm bindings inside of mruby to actually interact with the JVM itself.
32:22
But all the option parsing and stuff is written in Ruby, which is definitely much nicer for a Rubyist to do than to do all that string manipulation in C++. And one of the also great things about it is that he has unit tests, so this is a test folder of all unit tests to test all the option parsing. I don't believe the JRuby Launcher code
32:40
has any unit tests, they're all integration tests. But in mruby-cli, we can have both. So it's really great to see projects like that come up, and especially for a project like JRuby, I'm sure a lot more people will be willing to get involved in MJRuby than JRuby Launcher, like to actually contribute back
33:00
and be able to understand and see what's going on. So what can you do with mruby-cli, like how do you fit into this ecosystem of stuff that we're building? So one of the things is, like I said, we started it early this summer, which means the community's really young. mruby itself is only like five or so years old, I think, since its inception publicly.
33:24
But that also means that, like it's, although it doesn't have a ton of commits that are flowing through all the time, like the mruby GitHub, mruby's on GitHub, and we support pull requests and things unlike the MRI project, which is still on its version.
33:40
So for a lot of Rubyists, this is a lot more familiar to work with. And as you can see, like, Matz does read the pull requests and comments on stuff, which is great to be able to interact with Matz and other people on this project. So it's a fairly active project, but it's not so traffited that you can't follow through. Like, there's only a few commits
34:00
that usually get into master a day, if that, and the people who are active on the project are fairly active. mruby 1.2.0 just got released this morning by Matz, and I prepped a release this morning as well for mruby CLI.
34:20
So before we used to depend on master on mruby, which was not the greatest thing, because if something in master came out, it could potentially break your build, which was not ideal for me. So now that we depended on a lot of new features, but now that 1.2.0 is out, we can now lock it down to the release of 1.2.0, which means future versions will be a lot more stable.
34:40
So you can go to the mruby CLI project, and if you go to the releases page, there's a binary for every single, or for at least OSX Linux and Windows, 32 and 64-bit binaries that you can download and use, and you just need to put it on your path, and it will be a self-contained thing that works. So now that you've seen all this stuff,
35:01
how do we actually go and build a binary? So like I was talking about downloading, go download the mruby CLI binary. If you don't have any of the Docker stuff set up, you need to install the Docker toolbox to get Docker and Docker Compose, and then mruby CLI setup will go through and generate a new application, and then go ahead and modify it.
35:22
So one of the things that I got, I think one of the first C++ intro CS classes I took, the assignments was make a calculator, make addition and all that stuff work, or maybe build the HP calculator that Aaron was talking about in his talk, do all that string manipulation in C++, and then you just run this Docker Compose compile command,
35:40
and it should go and fetch the containers for you from Docker Hub if you don't have it, so you don't need a separate thing for that, and then you basically just rinse and repeat this cycle of modification and recompilation, and you'll see that once you have it actually compiled the first time, subsequent compiles are actually relatively quick and fast. And I love to actually hear about anything that people build.
36:02
We have a few projects that people have been working on, but no matter how small or if it's large that you're building, I would love to engage and talk and see what works or what doesn't work for you on this project. This is a huge passion project of mine, and I really enjoy working on it,
36:22
so definitely love to engage there. And I appreciate the fact that our community as a whole is really awesome, so we have stuff like Friday Hugs, things like Ruby Karaoke, the fact that our community enjoys doing all this stuff. I want to continue to build things in Ruby,
36:41
and I hope you do, and I hope we can help remove technical limitations of not choosing Ruby as a technology for doing stuff. And so let's go out there and build things in Ruby. Thank you. Is it, how much time do I have?
37:02
So the question was, is the bytecode spec published anywhere? I don't believe it is, but Matz would definitely know better. I don't know if he's here. I don't think he is. You should ask him during the Q&A in the, I guess at the keynote, closing keynote, since he's not here. Sorry.
37:20
So the question was the stuff you append on in the test, if you're able to separate that from the final binary. So there's actually, the test build is a specific separate build that's specialized inside of MRB, so the tests that you write are not actually in the final build that you produce. And then there's a add test dependency that got added recently in 1.2,
37:42
and so that allows you to add dependencies inside of your MRB project in general that won't get produced in the final build. There were, that feature's relatively new, so I ran into some issues with it, but supposedly it's fixed now. And yeah, so you should be able to have
38:02
a test specific thing that runs that isn't included. And oftentimes, so the first build you have is usually a host build, that's the thing that you run on your local machine, and in that build I have debug and other options set, so you can see the stack trace and back trace of stuff,
38:22
and normally all those things, so then even if the target build was the same as my host, I would have a separate target build that didn't include all that stuff. So the question was, is the Roku Toolbelt gonna use this? The answer is no. I'm not involved with the Roku Toolbelt heavily anymore, and that ship kinda has sailed already with the Go stuff
38:42
like if you're already happy using some other language, there probably isn't a huge reason to then port your entire project back over. Like a reason might be maybe you are in a company that has a bunch of people that are Rubyists and it would be easier for other people to get involved, but we have someone who's in charge of the CLI
39:00
and they do both Ruby and Go, so they're comfortable just continuing to work on the Go stuff. So the question was, when you package up an mRuby gem, it looks very similar to a Ruby gem, and is there any compatibility between the two? So I mean, I think even the file names
39:22
aren't the same between the gem spec, like it's blah blah dot gem spec, and then in Ruby gems, it's mruby gem dot rake, and it's the same name for all the mRuby gems. I mean, I imagine you could probably produce a package that had both of those things, but I would imagine you would run into a lot of issues for most people,
39:41
that there's many more libraries that you need inside of an mRuby project, but I'm sure that you could conceive something that mruby gem dot rake pulled stuff in that was missing, that wasn't in your gem spec, but I haven't seen anyone actually use that,
40:00
but I don't think it's out of the realm of possibility. Cool, thanks everyone.