Perl 6 as a new tool for language compilers
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Title of Series | ||
| Number of Parts | 561 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/44480 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSDEM 2019398 / 561
1
9
10
15
18
19
23
24
27
29
31
33
34
35
38
39
40
43
47
49
52
53
54
55
58
59
60
63
65
67
69
70
78
80
82
87
93
95
97
102
103
104
107
110
111
114
116
118
120
122
123
126
127
131
133
136
137
139
141
142
148
153
155
157
159
163
164
168
169
170
171
172
173
174
181
183
185
187
188
193
196
197
198
199
200
201
205
207
208
209
211
213
214
218
221
223
224
226
230
232
234
235
236
244
248
250
251
252
253
255
256
257
262
263
264
268
269
271
274
275
276
278
280
281
283
284
288
289
290
293
294
296
297
300
301
304
309
311
312
313
314
315
317
318
321
322
327
332
333
334
335
336
337
338
339
340
343
345
346
352
353
355
356
357
359
360
362
369
370
373
374
375
376
377
378
383
384
387
388
389
390
391
393
394
395
396
406
408
409
412
413
414
415
419
420
425
426
431
432
433
434
435
436
438
439
440
441
445
446
447
448
453
455
457
459
466
467
471
473
474
475
476
479
480
484
485
486
489
491
492
496
499
500
502
505
507
508
512
515
517
518
529
531
533
534
535
536
539
540
546
550
551
552
553
554
555
557
558
559
560
561
00:00
CompilerProgramming languageComputer programmingProgramming languageLatent heatMereologyWordInterpreter (computing)CompilerParsingComputer animationLecture/Conference
01:00
CompilerPoint (geometry)CompilerLatent heatMoment (mathematics)Programming languageDifferent (Kate Ryan album)Revision controlTable (information)Lecture/Conference
01:59
CompilerInterface (computing)MultiplicationObject-oriented programmingSet (mathematics)Process (computing)Bootstrap aggregatingSubsetProgramming languageSource codeNoise (electronics)Medical imagingRepository (publishing)CodeLevel (video gaming)RandomizationCompilerComputer animationXML
02:40
Computer filePower (physics)Computer programmingMereologySlide ruleBinary codeRegular graphRevision controlRegulärer Ausdruck <Textverarbeitung>Programming languageCartesian coordinate systemCompilerFormal grammarLecture/Conference
03:48
WindowView (database)Computer fileOnline helpSpring (hydrology)Formal grammarStatement (computer science)Execution unitBlock (periodic table)Function (mathematics)Variable (mathematics)Token ringGastropod shellStatisticsEmpennageHash functionDeclarative programmingRule of inferenceLine (geometry)System callVariable (mathematics)Software testing2 (number)Functional programmingLetterpress printingToken ringSocial classFunction (mathematics)IdentifiabilityNumberSpacetimeMultiplication signWordExpressionMereologyVarianceStatement (computer science)Type theoryData storage deviceHash functionBlock (periodic table)Formal grammarGroup actionParsingFlow separationData structureFree variables and bound variablesComputer programmingVideo game consoleCodeProgramming languageString (computer science)QuantificationEquals signMaxima and minimaMessage passingCASE <Informatik>Regulärer Ausdruck <Textverarbeitung>Electronic mailing listCompilerImplementationDot productOperator (mathematics)Extension (kinesiology)Machine codeSingle-precision floating-point formatMoment (mathematics)DigitizingVertex (graph theory)Computer animation
12:22
Statement (computer science)Variable (mathematics)Function (mathematics)System callGroup actionData storage deviceContent (media)Line (geometry)Functional programmingImpulse responseComputer animation
12:57
Token ringVariable (mathematics)WindowStatement (computer science)Function (mathematics)Hash functionFormal grammarView (database)Spring (hydrology)Gastropod shellLine (geometry)Function (mathematics)Computer programmingData structureCASE <Informatik>Letterpress printingComputer animation
13:46
Statement (computer science)NumberOnline helpWindowView (database)Token ringVariable (mathematics)Spring (hydrology)Formal grammarFunction (mathematics)VarianceFormal grammarFunctional programmingPrototypeDeclarative programmingType theoryGroup actionRule of inferenceSocial classAttribute grammarMereologyMultiplication signTouchscreenSystem callFlow separationInformationVariable (mathematics)CodeRegulärer Ausdruck <Textverarbeitung>Programming languageComputer animation
15:26
Hash functionStatement (computer science)Function (mathematics)Variable (mathematics)Token ringWindowView (database)InformationFormal grammarType theorySocial classFlow separationPerspective (visual)Software developerProgramming languageComputer animation
16:01
Declarative programmingHash functionMoment (mathematics)Element (mathematics)Key (cryptography)Variable (mathematics)Functional programmingSyntaxbaumType theorySystem callLecture/Conference
16:20
Computer iconWindowToken ringSimulated annealingView (database)NumberSocial classString (computer science)Formal grammarObject-oriented programmingMatching (graph theory)Source codeComputer animation
16:44
Function (mathematics)WindowToken ringVariable (mathematics)CompilerBookmark (World Wide Web)Computer fileMoment of inertiaConvex hullType theoryHash functionProgramming languageComputer animation
17:10
WindowInternetworkingView (database)Hash functionStatement (computer science)Function (mathematics)Variable (mathematics)InternetworkingSoftware testingFormal grammarGodComputer fileDirectory serviceLecture/ConferenceXML
17:39
Gastropod shellCompilation albumHome pageHash functionDisk read-and-write headSimulationFormal grammarStatement (computer science)Social classSoftware testingProgramming languageComputer animation
17:59
Functional programmingSyntaxbaumLecture/Conference
18:16
Floating-point unitHome pageGastropod shellSimulationEmpennageMachine codeWindowView (database)Linear partial informationNumberIntegration by partsLink (knot theory)Statement (computer science)Formal grammarState of matterRule of inferenceType theoryComputer animation
18:33
Computer fileWindowBookmark (World Wide Web)NumberStatement (computer science)Rule of inferenceLoop (music)Formal grammarStatement (computer science)Type theoryDifferent (Kate Ryan album)TouchscreenXMLLecture/Conference
18:52
WindowBookmark (World Wide Web)NumberProgramming languageStatement (computer science)Rule of inferenceVariable (mathematics)Computer fileToken ringFunction (mathematics)Exterior algebraFormal grammarSingle-precision floating-point formatRule of inferenceStatement (computer science)Computer animation
19:14
Programming languageFormal grammarType theoryLecture/Conference
19:56
Software developerType theoryFormal grammarLecture/Conference
20:33
SpacetimeRule of inferenceDefault (computer science)Lecture/Conference
21:03
Lecture/ConferenceComputer animation
Transcript: English(auto-generated)
00:07
Let me start. Yeah. Okay. Hello, everyone. Thank you for coming. It's really nice to see so many people here. My name is Andrew and what I'm doing, today I realized that I'm doing Perl for 20 years since 1999.
00:22
I'm also organizing the Perl Conference. So, it's PerlCon this year in August. So, you can take a look. But I'm not going to promote Perl here. What I'm going to do, I'm going to talk about one specific part of Perl 6 which is really interesting especially in this audience for creating
00:40
some compilers, parsers, translators, whatever interpreters for other languages. So, I will demonstrate how Perl 6 can do this stuff today. So, basically just a couple of words about what Perl 6 is. So, a few years ago, well three years ago, Larry Wold, the creator of Perl itself,
01:02
announced that this Perl 6 is 0.00 release, and it was done here at FOSDEM. And since then, basically you have this table version of the language specification, and you also have a really nicely working compiler. You have to realize that Perl 5 and Perl 6 are basically different languages.
01:22
Of course, they have many things in common, the same spirit of the language, but still it's a separate language, and all that I'm going to talk about is only about Perl 6. So, it's different language. There's only one compiler at the moment which you can use, it's called Rakudo, and you can
01:41
download it from rakudo.org, and you can use this compiler to run all the examples that I will demonstrate later. The interesting thing about Perl 6 is that this compiler is also written in Perl 6. So, it's self-hosting language, and that's one of the reasons why it took so long to bootstrap the language.
02:02
Also, there's the MQP, Not Quite Perl. It's a subset of Perl 6, which was developed to help to build this bootstrapping process. This is just a random, really random. I just took some random code from a repository. This is the source code of Perl 6, and it's written in Perl 6 plus some,
02:20
you see, MQP somewhere which uses some MQP calls to do some low-level stuff. But this image is not aimed to show you how difficult Perl 6 is. It's not a noise language. It's internals of the compiler, so it has to be something difficult.
02:40
Basically, in Perl 6, you can do things like this. So, you can use Unicode characters, you can use, for example, superscripts to create the power of a variable instead of using two stars, for example. But that's a side story. So, what's interesting in Perl 6 for creating other languages is that they have
03:02
the updated regular expression engine, and the regular expressions are not called regular expressions. Well, they're not regular, that's regular. They're now called reg-exes or reg-exes, whatever you like to read it. The second part is that Perl 6 has extended version of regular expressions,
03:22
which are called grammars. That's the end of the slides. What I'm going to do next, I'm going to show you how quickly you can create a compiler. Well, not necessarily it should be a compiler. By compiler, I mean that it's
03:40
some program that translate that reads that understand language, not necessarily compiles it to an executable file in binary code. So, here's the empty file, and what I'm going to do, I'm going to create a program that will be able to understand this program.
04:00
For example, it's just a random language, it's not language, it's big enough, and I'm going to print it. So, imagine that this line is the line in language, which is the minimalistic language, and I'm going to parse and execute it using Perl 6 grammars.
04:22
So, what I'm going to do, I have to create the grammar, so I have to describe the language. It's somehow similar to what the BNF syntax is using. So, I create the grammar, I can call it whatever I want, and that's basically the class. So, the syntax really resembles how
04:42
you define classes in other programming languages, but instead of methods, you have rules and tokens. So, my first rule, and it should be called top, so the beginning, my first rule is to parse the whole sentence, the whole program. So, my program is basically a list of statements.
05:05
A list of statements, and what I'm typing now, this line five is actually the regular expression, which is embedded inside the Perl 6 code. So, grammar, line three, rule, line four are Perl 6 code, inside line five is basically already a regular expression,
05:23
not irregular, a regular. So, I'm going to have more than one statement, or actually it's zero. Syntax is really like a regular expression, so the star quantifier means there can be zero or more repetitions of something on the left.
05:43
And then I tell that these statements are separated by the semicolon, so that's simple. What I have to do now, so I described this grammar to some extent, and now I can use this grammar to parse
06:01
the string containing this program in this programming language. That's it, after that, I for example can say, say is operated to print in Perl 6. So, I can use this test.pl, no, no, yeah, of course, because there are no statements,
06:23
so I have to define them. So, a statement actually is the next rule which I have to implement, statement, and what is a statement? A statement is, in my example, it's either the vertical bar for or,
06:43
either a variable declaration, so I can type variable declaration, or the printing instruction. Well, actually we can call it just a function call, right? And from this moment, I have to define what is a variable declaration,
07:01
what is a function call, and again, it's really simple. So, I can, variable declaration, what is it? It's a string var followed by variable name, variable name followed by equal sign,
07:24
followed by some number, and so on. So, I have to also explain to Perl 6, what is a variable name? So, I create another rule, but this time it will not be a rule, it will be a token.
07:42
Why a token? Because variable name is something that does not contain spaces inside. So, Perl 6 understands that for example, in the line one, I will omit spaces around the equal sign, it will still work, so I don't have to do any extra work to explain that it is allowed there.
08:01
So, Perl 6 will understand as soon as I'm using the rule keyword, it will understand the spaces around parts of the expressions are valid things. So, what is a variable name? Basically, it's just some letters or whatever, some word characters more than one.
08:21
The same for the number. Let's do a simple thing. For example, numbers are only digits, so it's integers, positive, but for our example is more than enough. What else? We have the function call thing. So, let's create another rule,
08:41
function call, which will expect a function name. In our case, it's only print. So, we are only having a single function in this language, followed by a variable name. So, more or less, that's it. So, what I also want you to take a look at.
09:20
So, for example, line 13,
09:22
rule variable dash declaration. In Perl 6, dash is a valid character for, it can be used inside the identifier of the, for example, variable or function or method name. Okay, so it looks like okay. Let's try running it. No, still no method statements, because it has to be statement,
09:42
and S is like, yeah, it's the star. Yo-ho. Okay, so let's take a look at the output of this program. So, what it says, it says that it managed to parse the top line contained in the program, and basically, it then demonstrates, it contains a statement, bar X is 20.
10:03
The second statement is print X, and each statement has some details about what it is. In the first time, in the first case, it's a variable declaration with the variable name and the number that is assigned to this variable. In the second case, it's again the function call. The function name, well, the function name is missing here because we didn't create the special rule
10:21
or token for the function name. But still, we have the variable name X extracted. So, basically, here, we're already parsing the program. So, Perl 6 understands, well, my program, my test program understands what's the structure of this program. And the next step is just to execute this.
10:41
So, yeah, and also notice that, unlike traditional approaches to creating compilers, there's no separation for lexer and parser. Everything goes in a single grammar. So, my variable declaration, my numbers, and like equal sign or var keyword are leaving here.
11:02
There's no special lexer which extracts the parts of the language first. So, now the second part. The second part is to execute this. So, what I want, I want to print 20 in the console. And what I will do, I will add actions,
11:20
so-called actions to my grammar. So, this is the placeholder for the action. So, again, line 15 starts with some Perl code inside which we have some regular expression, inside which we have, again, a block which contains Perl 6 code. So, here, where the line 15 is,
11:41
I will type some Perl code, and I will just save this variable in some variable storage. So, this is how I can do it. Variable name is number. I will explain this plus. And, of course, I have to create some storage for this.
12:01
So, I can create the hash var which will contain my variables, and after the program is passed, I can just type it and see what's there inside. So, what's there inside? You see this line, the last line? Yeah, sorry. This line.
12:22
So, for the last, like, Perl. So, this line demonstrates the content of this variable storage after I passed and executed the action. So, you see, so, x contains 20, and now, actually, I can create another action
12:42
which will use this value to print it. So, I have to go to the function call rule, and here, I will just print. So, in Perl 6, there are two functions that print. Well, actually, there's more. So, there's print, but also, there's three letters, say,
13:04
which also adds the new line at the end of the output which is really handy in many cases. So, what I'm going to do, I'm just going to print this exact value. That's it. So, you see, just above,
13:21
there's 20 in the output, so it works. And, of course, it is not limited to this program only. I can extend it. I can add, for example, another variable, which is 42, and I can print it. And, you see, so 42 is here,
13:40
and I also see what the structure of this program is. So, basically, that's a really short demonstration of what you can do with grammars, and you can do really quickly. So, it took me five minutes to type this, and it's already a working prototype of what you can expect.
14:00
Of course, the bigger the language becomes, the more is your desire to split Perl 6 and regular expression parts of this grammar. So, what can be done, you can extract these parts of the grammar to separate class, yeah, class A, for example,
14:21
and what you have to do, so we are going to move this to the methods of this class, and the name of the method should be the same. So, its method this time should be the same as the method at the rule name that is activating this action. So, it's variable declaration,
14:41
method variable declaration, and it's just the Perl code, which, okay, I have these, which does what it did before, but you also, well, expect something in this variable which is passed to this method, and also, I can move the function call,
15:05
function call action to a separate method. So, I will just remove it from the grammar, and I will move it to the function call, and then when I'm passing the string, I can pass in the named attribute actions,
15:23
these class containing actions, and if everything is okay, you will get exactly the same behavior, so nothing changed from the user perspective, but from the perspective of the developer, you have a separate class with Perl 6 code, you have a separate grammar described in the language.
15:40
And also, so I promised to talk about this, so basically, this dollar slash variable, you can name it differently, you can just have dollar something, I mean, you can type letters there, it will contain the information which contains some information
16:00
similar to what we have here, this parse tree, the fragment which is passed at this moment, like function call or variable declaration, is passed further in this method, and you can use it, so basically, when you type this, you are referring to this variable, you are taking it like an element from the hash, and the key for this element is variable name,
16:21
and on the right side, it's the number, this, what is selected is a object of a grammar class, the so-called match object, if you type plus before, you will just ask Perl to make a number out of it somehow, so Perl 6 knows how to convert this match object to a number, this is exactly what we need,
16:41
and actually here, it converts the match object to a string, you can extend it, like you can type the tilde before this, but you don't have to do it, Perl 6 will do that, so that's the short example, and if you, I can also show you,
17:03
so there's github.com slash hash slash lingua, as is an example of much bigger language, it's using the same thing that I just demonstrated, what it contains, it contains a few files,
17:20
so there's the grammar, oh my God, yeah, you can go to itself, if you have internet here, but nevertheless, it's just the bigger thing, for example, we go to the test directory of this,
17:43
I created some tests, for example, there's if statement implemented in this language, and again, you can find it in the actions and in the grammar class, how we do it, it's really easy, just step by step to increase your language, starting with the example I demonstrated earlier, and finally, you will have to deal somehow with,
18:03
for example, functions, so you have to put the function, so you have to understand that this is a definitional function, you have to put the parse tree somewhere, and then you have to evaluate it, so if you will, on that github link, so you see,
18:22
there's the grammar, and my top rule states that it's not only statements, it also can be a comment, for example, and also I'm clearly separating different type of statements, just to make the rest of the grammar simpler, yeah, so,
18:43
so it means that instead of, so I can have the vertical bar means or, so it's either this or this or this, but inside, remember on the previous screen I showed you the, this thing, dollar something, so when you are inside an action
19:00
or inside the grammar rule, you can refer to one of those alternatives using the single name, so it's much, much easier than, you don't have to create a, like, if this is a statement, if it's assignment, if it's something, so yeah, so okay, the final thing, I would recommend you to read this book,
19:20
it contains a lot of examples of how you can use grammars, and also there's something in this book, written by me, you can buy these books downstairs in the Pearl Booth, so thanks a lot for listening, I hope it was interesting, and yeah, success with Pearl Six, and languages.
19:47
I will take picture, so, yeah. Okay, what type of grammar is it? This is so, thank you. Yeah, I would recommend you to,
20:01
what type of grammar is it, LL1, this one you mean, right? That's the question to developers more than to me. Actually, I think that in the documentation of Pearl Six, you don't have the explicit answer to this question, if I'm correct. So basically, yeah.
20:23
If it satisfies, yes? Yes, it's when you're using the rule keyword,
20:40
if you're using token, then it's not allowed, but you also can redefine, so there's the built-in rule called WS, white space, which is explaining what white space is, so it's not just a space, it's like new line, whatever, and you can redefine it, for example, to implement comments.
21:00
Yeah? Okay, then thanks a lot.