Rsense Knows Your Code
This is a modal window.
Das Video konnte nicht geladen werden, da entweder ein Server- oder Netzwerkfehler auftrat oder das Format nicht unterstützt wird.
Formale Metadaten
| Titel |
| |
| Serientitel | ||
| Anzahl der Teile | 65 | |
| Autor | ||
| Lizenz | CC-Namensnennung - Weitergabe unter gleichen Bedingungen 3.0 Unported: Sie dürfen das Werk bzw. den Inhalt zu jedem legalen und nicht-kommerziellen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen und das Werk bzw. diesen Inhalt auch in veränderter Form nur unter den Bedingungen dieser Lizenz weitergeben. | |
| Identifikatoren | 10.5446/37631 (DOI) | |
| Herausgeber | ||
| Erscheinungsjahr | ||
| Sprache | ||
| Produzent |
Inhaltliche Metadaten
| Fachgebiet | ||
| Genre | ||
| Abstract |
|
Ruby Conference 201420 / 65
2
3
4
6
9
11
14
17
18
19
20
25
27
29
30
32
34
35
39
40
46
47
50
53
55
56
58
61
63
00:00
Maschinencode
00:24
InformationsspeicherungHilfesystemComputeranimation
00:44
SoftwareentwicklerSystem-on-ChipSoftwarewartungCoxeter-GruppeGoogolMaschinencodeSoftwareentwicklerSoftwarewartungTypinferenzProgrammiergerätComputeranimationVorlesung/Konferenz
01:14
InformationMaschinencodeLogischer SchlussQuellcodeRefactoringVollständigkeitInformationQuellcodeTypinferenzMaschinencodeGebäude <Mathematik>FehlermeldungRefactoringSpieltheorieAppletFormale SpracheHydrostatikProzess <Informatik>TypentheorieArithmetisches MittelVorlesung/KonferenzComputeranimation
01:51
VollständigkeitMaschinencodeTexteditorAppletVollständigkeitXMLComputeranimation
02:21
KreisflächePlug in
02:38
Logischer SchlussTypinferenzProgrammierungVollständigkeitLesen <Datenverarbeitung>TypentheorieMaschinencodeComputeranimationVorlesung/Konferenz
03:01
Logischer SchlussProgrammierungMeta-TagHalteproblemTypinferenzComputeranimationVorlesung/Konferenz
03:20
Computeranimation
03:38
Walsh-FunktionensystemSyntaktische AnalyseVersionsverwaltungPunktVersionsverwaltungQuellcodeSyntaktische AnalyseElektronische PublikationComputeranimation
04:08
GoogolMaschinencodeQuellcodeSelbst organisierendes SystemElektronische PublikationProgrammierungt-TestOpen SourceGoogolDatenverwaltungStellenringSoftwaretestSelbst organisierendes SystemProjektive EbeneMaschinencodeComputeranimation
04:54
SoftwaretestFehlermeldungZeitrichtungFehlermeldungNP-hartes ProblemProgrammfehlerMultiplikationsoperatorComputeranimationVorlesung/Konferenz
05:21
QuellcodeMultiplikationsoperatorSoftwaretestProgrammfehlerQuellcodeComputeranimation
05:58
DebuggingMIDI <Musikelektronik>SchnelltasteMaschinencodeDebuggingGeradeSchreib-Lese-KopfComputeranimationVorlesung/Konferenz
06:22
Wort <Informatik>MathematikComputeranimation
06:40
Kartesisches ProduktAlgorithmusProdukt <Mathematik>Modifikation <Mathematik>TypinferenzParametrische ErregungMultiplikationsoperatorComputeranimation
07:01
HilfesystemSoftwaretestProgrammfehlerComputeranimation
07:17
sinc-FunktionMultiplikationsoperatorAppletComputeranimation
07:42
MenütechnikAuswahlaxiomInstallation <Informatik>TexteditorTexteditorInstallation <Informatik>StrömungsrichtungPlug inComputeranimation
08:08
Prozess <Informatik>AppletMaschinencodeComputeranimationVorlesung/Konferenz
08:27
ServerWrapper <Programmierung>TexteditorSpeicherabzugInstallation <Informatik>AppletSystemaufrufPlug insinc-FunktionTexteditorServerWrapper <Programmierung>DiagrammSystemaufrufInstallation <Informatik>Computeranimation
09:02
TVD-VerfahrenProdukt <Mathematik>Kartesisches ProduktAlgorithmusFormale SpracheLogischer SchlussKartesisches ProduktTVD-VerfahrenFormale SpracheAlgorithmusVerschlingungProjektive EbeneHydrostatikGemeinsamer SpeicherVorlesung/KonferenzComputeranimation
09:43
MaschinencodeAbstrakter SyntaxbaumDatenstrukturQuellcodeTopologieSelbstrepräsentationTopologieSystemaufrufSelbstrepräsentationMaschinencodeDatenstrukturQuellcodeAbstrakter SyntaxbaumSyntaktische AnalyseParserAbstrakte SyntaxComputeranimation
10:06
Syntaktische AnalyseWurzel <Mathematik>ZahlenbereichBitSyntaktische AnalyseDatenstrukturMaschinencodeVerzweigendes Programm
10:28
Übersetzer <Informatik>Abstrakter SyntaxbaumQuellcodeDatenstrukturWurzel <Mathematik>Syntaktische AnalyseProzess <Informatik>Datenstruktur
10:52
ParserQuellcodeLaufwerk <Datentechnik>ProgrammbibliothekEindringerkennungGraphAbstraktionsebeneAbstrakter SyntaxbaumGebäude <Mathematik>InterpretiererGraphOrtsoperatorProgrammbibliothekAbstrakte InterpretationQuellcodeSyntaktische AnalyseElektronischer ProgrammführerProgrammierumgebungParserComputeranimation
11:21
Objekt <Kategorie>UmwandlungsenthalpieMusterspracheComputeranimation
11:44
KnotenmengeObjekt <Kategorie>KnotenmengeGraphMaschinencodeComputeranimation
12:03
Knotenmengep-BlockLaufzeitfehlerSystemaufrufKnotenmenge
12:26
Knotenmengep-BlockLaufzeitfehlerMaschinencodeGoogolSyntaktische AnalyseMetropolitan area networkTopologieDifferenz <Mathematik>ParserAbstrakter SyntaxbaumProjektive EbeneComputeranimation
13:03
VersionsverwaltungMaschinencodeMathematikRechter WinkelElektronische Publikation
13:47
InterpretiererMaschinencodeMultiplikationssatzAbstrakte Interpretationsinc-FunktionMaschinencodeInformationGeradeComputeranimationVorlesung/Konferenz
14:11
Constraint-PropagierungTypentheorieGraphKnotenmengeComputervirusMaschinencodeGraphPropagatorSystem FZahlenbereichNebenbedingungTypentheorieObjekt <Kategorie>ZeichenketteComputeranimation
14:53
SpeicherabzugInformationInformationSpeicherabzugProgrammbibliothekComputeranimation
15:13
GasströmungMeta-TagRegulärer Ausdruck <Textverarbeitung>TeilmengeBitTypentheorieProgrammierungMeterDynamisches SystemValiditätMaschinencodeTypinferenzComputeranimationVorlesung/Konferenz
15:40
TypentheorieLogischer SchlussNotepad-ComputerCASE <Informatik>TypentheorieResolventeResultanteMereologieSinusfunktionComputeranimation
16:22
TypinferenzMultiplikationsoperatorTypentheorieComputeranimation
16:40
StichprobeKlasse <Mathematik>VariableElement <Gruppentheorie>BimodulZeichenketteMessage-Passingp-BlockTexteditorDemo <Programm>URLKlasse <Mathematik>ZahlenbereichComputeranimation
17:41
RechenschieberDemo <Programm>URLTexteditorPlug inComputeranimationVorlesung/KonferenzXML
17:59
MultitaskingTexteditorLokales MinimumMetropolitan area networkPlug inXML
18:22
GraphROM <Informatik>HalbleiterspeicherProgrammbibliothekApp <Programm>CachingInverser LimesGraphComputeranimation
18:55
GraphRefactoringGebäude <Mathematik>TexteditorMailing-ListeAppletProjektive EbeneFehlererkennungImplementierungTypentheorieGebäude <Mathematik>AuswahlaxiomPlug inTypinferenzServerTexteditorVollständigkeitGraphMaschinencodeComputeranimation
20:30
Twitter <Softwareplattform>SoftwareEreignishorizontVideokonferenzComputeranimation
Transkript: Englisch(automatisch erzeugt)
00:18
I'm gonna go ahead and get started.
00:22
Hi, my name's Eric West. Before I start, I just have to say thanks. I had a tragedy this morning. Discovered it looked like the insulation had been burned through where the wires had gotten frayed. Pretty horrible, but thankfully, I got a little help from some friends
00:42
and just wanted to say thank you. My name's Eric West. I'm a Ruby developer at Lonely Planet, a former JRuby Google Summer of Code participant, and currently the maintainer of RSense. I've got a few goals for this talk today.
01:01
I wanna explain what RSense is, encourage you to get involved, and kind of a bigger goal of motivating you to stretch yourselves as programmers. So what is RSense? RSense is a type inference tool for Ruby. It reads in the source code, and then it provides information about that source code,
01:23
and that means we can take that information and build tools to do really cool things like code auto-completion, find by definition, error detection, especially type-based error detection, and automatic refactoring.
01:41
These are all tools that are typically available in statically-typed languages, and RSense makes it possible to bring that kind of tooling to Ruby. So where did this start? I started programming about three and a half years ago now, and I dabbled a little in Java,
02:01
did a little node, and then came to Ruby where I fell in love, found home, and the Java showed me, I guess, what was possible. So when I came back to Ruby, and I started using an editor called Sublime Text, I wanted auto-completion for Ruby there.
02:21
There was a plugin that claimed to provide it. I don't know if you can see very well, but in that little circle that says Ruby, it claimed to provide this. I tried everything to set that thing up, and it just didn't really ever quite work,
02:40
and it didn't work with JRuby, which was something I had started getting really interested in. So I dug a little deeper. What would it take to make this kind of tool on my own? Well, everything I read said that to do code completion, you have to do type inference, and doing type inference on Ruby was basically impossible.
03:01
A lot of quotes like this one, but I'll read it because I really like this quote. Metaprogramming makes the type inference problem equivalent to the halting problem. For anyone that doesn't know, the halting problem has been proven mathematically to be basically, or to be impossible, but I'm stubborn.
03:26
So I kept digging around, and I found this tool called RSense that did everything I wanted to. It was created by Tomohiro Matsuyama. Unfortunately, it was abandoned.
03:42
The last commit was in 2010, and when I got interested in it, it was hard to install. It only worked with Ruby 1.8 syntax, and it was using a really old version, like 1.3, 1.4 of the JRuby jar to do the parsing of the Ruby source.
04:04
So being naive and really new to programming, I thought I could just swap out the jar file and pop in the new JRuby, and everything would work. You may not be able to read there, but Bad Poker Face Guy is saying, of course I knew that wouldn't compile,
04:21
but I had no idea. So I found out about a program called Google Summer of Code. It's a program Google does every year where they allow students to work on open source projects, and they pay them a really nice stipend over the summer to do that. And I found out that JRuby
04:41
was gonna be a participating organization, so I went out to my local community college, signed up for a class so that I would qualify. Then I applied for the program. That summer, Tom helped me fix over 800 errors and 100 failing tests.
05:02
I learned a lot. We got everything working, and it was really awesome. So how does a noob like me manage to work on something that is pretty hard stuff? Well, there were a few things that I kinda had going for me.
05:22
I was comfortable with moving really slowly, one bug at a time, one failing test at a time. You can't hear it, I'm sorry.
05:43
But reading the source code, just diving in there, not understanding it, and being okay, being comfortable with not understanding what you're looking at, and gradually a picture will start to emerge.
06:00
Now, when I say debugger, a lot of you probably think of these things, but I found that a debugger that let me step through the code line by line and learn how it was executed, how it flowed, really allowed me to get my head
06:20
around what was going on. Reading papers, of course it's gonna be full of lots of words you don't understand, lots of math you don't understand, maybe. I didn't. But gradually doing all these things, a picture will start to emerge.
06:41
These are a couple of the papers that I read many, many times before I really understood how arsonists worked. The most last thing, though, was being really stubborn. Just being determined that this thing was gonna work, and I didn't care how much time I had to put into it.
07:02
So, at the end of the summer, had squashed a lot of bugs, fixed a lot of broken tests, things were working, but had you looked on RubyGems.org, you wouldn't have found a way to install Arsonist. I had some questions before I released anything.
07:20
Why, since Arsonist had been around for so long, had it not been more widely adopted? This is cool stuff. It's a cool tool. People had a really hard time installing it, and it was written in Java, so people, Rubyist, the people the tool was made for, probably maybe didn't feel comfortable contributing to it.
07:42
So it needed a better install story. Now, over a year later, if you look, you will be able to gem install RSense, and then install the editor plug-in.
08:01
Currently there's three, and we'll talk about those later, but there should be more. And it'll work for either MRI or JRuby. RSense, the gem, uses Spoon, which is based on POSIX spawn, to launch a JRuby process. This makes it possible for you to install
08:21
essentially Java code base with MRI and still run it. This then runs the RSense server, which communicates between whatever editor plug-in and RSense core, which is just a very thin wrapper around the RSense jar.
08:43
You can see in this diagram how that works. You gem install RSense, run RSense start, Spoon then spawns JRuby, which starts the server. That talks over HTTP to whatever editor plug-in. Then it makes Ruby method calls into RSense core, which calls into RSense jar.
09:03
RSense is implemented with a variation on the Cartesian product algorithm, named after Descartes. CPA was first implemented by Ole Agassen for the language self, and I referenced that paper earlier.
09:22
There is also a paper describing a project called Ecstatic and the algorithms described in the Ecstatic paper are almost identical to what is actually implemented in RSense. If you search for it, you'll be able to find it if you're interested in reading it, or I'm happy to share links to those papers.
09:44
How does RSense implement CPA? First, code has to be read in and parsed into an abstract syntax tree, which I'll call an AST hereafter, by JRuby parser.
10:01
An AST is just the representation of source code in a tree-like structure. We've got some very simple Ruby here. It's just an assignment of the number one to a variable A. If we parse that with JRuby parser, we get back something kind of like that.
10:23
Here's a little bit slightly more complicated, bit of Ruby, and if we parse that into an AST, then you have a structure that's just more suitable for processing. So here we've got the if node, like in the code,
10:41
and that's got some branches. It's got the conditional, and then it's got what happens if it's true and what happens if it's not true. JRuby parser is a library that originated as, it was extracted from JRuby,
11:02
and it has special methods useful to IDEs and tools like RSense. One of the really good things about it is that it tracks source code position. So RSense walks the AST after it's been parsed out by JRuby, and builds up a graph of nodes using an abstract interpreter.
11:25
Walking the AST, it uses something called the visitor pattern to do that. Every node in the AST will implement an accept method, which then will call the specific visit method
11:41
on the visitor that's passed in. So here is some code from RSense called create vertex. This is creating a vertex in the graph that RSense builds up, and you can see it calls accept on the node object,
12:01
and then passes itself in. Then here we have one of those visit methods, this one for a call node, and you can see that then it does things specific to a call node, like a call node will have a receiver.
12:21
So it creates a receiver vertex by passing back to the create vertex method we just saw. And this is how it walks through the AST. ASTs are pretty awesome. One of the things that I did while working in the Google Summer Code project was took some things out of RSense
12:43
and ported them back into JRuby parser. One of them was the ability to do a semantic diff. So a usual diff like what GIF does is text-based, but a semantic diff is actually looking at what the code is, what it represents.
13:01
And so you can do some smarter things with it. I made a tool from that called Smart Diff, and we'll look at a little GIF here. On the left is the older version of the code, and on the right is the newer version of the code. The stuff in red is stuff that's been removed, stuff in green is stuff that's been added,
13:23
and the stuff in blue is stuff that didn't change but might have moved around. Now, in a text-based diff, it usually can't tell if you move methods around inside your file. It will think that that's completely new code. But with a semantic diff, it can, and it can track it so that when I click on one side
13:42
or the other, it will move the other side to align those methods back up. Abstract interpreter, so after, well, walking the AST uses what's called an abstract interpreter, which I like to think of as a virtual Ruby.
14:02
It goes through each line of code as it would be executed, but it doesn't actually execute it. It just understands it. It gets information about it. Then it uses what's called constraint propagation to take the types that it knows and move those along the graph
14:21
to the objects in the graph where the type is unknown. Here we've got some simple code, x equals one, then we set y equal to x, then we set x equal to hello. Because we know x is a number, or one is a number, we assign that type to one. That propagates through the graph down to x and onto y.
14:45
Then because we also set x to hello, a string, we propagate the string type into x. RSense uses some built-in methods to stub out the core library and the standard library.
15:03
This gives it information that would be missing otherwise, and so it gives it a base to build itself up from. This is what those stubbed out methods look like. You can see above them, there's some type annotation,
15:21
and we'll talk about that a little bit more in a minute. All this would be almost enough, but Ruby's dynamic metaprogramming feature, it means that there's a small segment of valid Ruby code that RSense can't do type inference on. It's very small, but it does exist.
15:42
This quote comes back. But RSense does have a solution for this too. It uses an annotation resolver, taking a special type annotation syntax in the comments, and it uses those to get around those rare edge cases
16:01
where it otherwise wouldn't know. Because it does that, it prevents bad results caused by those from bleeding into the rest of the results. For the most part, it's only necessary to have those in the built-in stubs, but what I wanna do is move it to a syntax
16:24
like yard uses, the documentation tool. It has a great type annotation syntax that many of you already know, and so you could improve your documentation and get better type inference at the same time.
16:41
So that's a goal. So here we have a module with a class inside it. There's a method, it takes a variable, and it yields a block to that variable. So we'll pass in an array with a string inside
17:06
and a number, and then we'll pass it a block. So we'll get the first element of that array,
17:22
and we'll see that it knows it's a string and provides us a string method. We'll get the second element of the array. So I start to type, it'll clear out the... And we'll get abs, which is a fixnum method.
17:42
And if you wanna take a look for yourself at that demo, you can at that web address. So where does RSense stand today? There are currently three editor plugins working, known to be working. One for the Atom editor that I wrote, another for the Sublime Text editor that I also wrote,
18:03
and a third for TextMake 2. I have very recently switched to Emacs, and so now I have motivation to make an Emacs plugin. I'm really looking for somebody that wants to take on making a Vim plugin.
18:22
It works great currently for Sinatra apps, and when you're working on a gem or a library. But it does currently fall down on Rails apps. It runs out of memory. Rails, plus all of its dependencies, is just too big for the graph.
18:42
But I've got a solution for this. We could do some kind of caching of the graph, and that would be able to get around, work around that limitation. It just needs to be implemented. So I've got a big to-do list here of Vim and Emacs plugins.
19:04
Caching the type graph, switching to yard-style type annotations. Currently, RSense, the Java code, actually has implementations for find definition, where, and type inference,
19:22
beyond just the auto-completion that you saw demoed a minute ago. So those just need to be exposed to Ruby and to the command line or the RSense server so that editors could take advantage of that.
19:41
Then on top of it, we could build error detection tools. There's a project called Laser that also does type inference and has a lot of error detection capabilities built into it that could be ported over to RSense.
20:00
I would just say just use that project, but it doesn't look like it's being actively maintained anymore, although I could be wrong about that. So we could port those over to RSense, probably pretty easily. Automatic refactoring tools, I think these would be really exciting to have in Ruby.
20:21
So RSense needs you to use it, to help finish it, and to build on top of it, and to make a plugin for the editor of your choice. You can find out more about RSense at rsense.github.io. And thank you very much.