We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

How (not) to create a language specification for Perl 6

00:00

Formal Metadata

Title
How (not) to create a language specification for Perl 6
Subtitle
Lessons learned
Alternative Title
Perl - Perl 6 Lang Spec Lessons Learned
Title of Series
Number of Parts
150
Author
License
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language
Production Year2015

Content Metadata

Subject Area
Genre
18
20
Thumbnail
55:22
24
Thumbnail
49:05
26
Thumbnail
45:24
30
Thumbnail
25:44
37
Thumbnail
26:33
87
89
90
104
Thumbnail
22:20
126
Thumbnail
16:49
127
Process (computing)Formal languageTerm (mathematics)Latent heatSoftware developerMultiplication signResultantVideo gameCategory of beingFlagSoftware bugTask (computing)ProzesssimulationGroup actionPoint (geometry)Presentation of a groupMathematicsDirection (geometry)Reflection (mathematics)Variety (linguistics)BitProcess (computing)CodeData conversion6 (number)Goodness of fitWordXMLComputer animation
Pascal's triangleNatural numberFormal languageData modelImplementationService (economics)Product (business)Revision controlUniform resource locatorInterpreter (computing)Suite (music)Software testingRadio-frequency identificationStandard deviationFormal languageSuite (music)Rule of inferenceSoftware testingInterpreter (computing)Uniform resource locatorNumberProcess (computing)Form (programming)Latent heatRight angleComputer programmingStandard deviationEntire functionInternetworkingWorld Wide Web ConsortiumRevision controlBit1 (number)Electronic mailing listVariety (linguistics)Programming languageImplementationHypertextMarkup languageSet (mathematics)MathematicsSoftware developerOrder (biology)Term (mathematics)ArmHeat transferContext awarenessComa BerenicesPlanningInternet service providerWordProfil (magazine)CASE <Informatik>CodeComputer-aided designCommunications protocolComputer animation
Formal languageImplementationLevel (video gaming)Radio-frequency identificationSystem programmingPhysical systemLatent heatFormal languageImplementationData conversionSoftware developerNumberOnline helpMultiplication signPermanentDirection (geometry)Slide ruleMereologyLevel (video gaming)Sinc functionAdditionWordSet (mathematics)Process (computing)Order (biology)Dependent and independent variablesExecution unitProjective planeArithmetic meanRight angleDifferent (Kate Ryan album)Perspective (visual)SequenceSource codeBit rateComputer animation
ImplementationSystem programmingHost Identity ProtocolLimit (category theory)Web browserCodeInternetworkingStandard deviationTrailIndependence (probability theory)Operations researchKolmogorov complexityInternetworkingStandard deviationProcess (computing)Latent heatImplementationMereologyDescriptive statisticsAreaFormal languageMultiplication signArithmetic meanExistenceElectric generatorSoftware developerTrailWeb browserParallel portSimilarity (geometry)BitCodeOperator (mathematics)Independence (probability theory)FreezingExtension (kinesiology)Computer animation
Modul <Datentyp>Radio-frequency identificationTime evolutionLogical constantComputer programmingOperator (mathematics)ImplementationTerm (mathematics)MathematicsRevision controlSuite (music)Software testingFormal languageStandard deviationOffice suiteSocial classIntegrated development environmentLimit (category theory)CASE <Informatik>Duality (mathematics)Data conversionEvoluteImplementationWordMereologyCompilerLatent heatFormal grammarType theoryCartesian coordinate systemSingle-precision floating-point formatLevel (video gaming)Multitier architectureCodeNumberSource codeFilter <Stochastik>BuildingSoftware testingDifferent (Kate Ryan album)Right angleInterpreter (computing)Core dumpMultiplication signSuite (music)AdditionRevision controlSoftware developerTerm (mathematics)Logical constantSet (mathematics)Domain-specific languageProgramming languageOperator (mathematics)Mechanism designModule (mathematics)Reflection (mathematics)Ocean currentDescriptive statistics1 (number)Inheritance (object-oriented programming)Adaptive behaviorEndliche ModelltheorieMacro (computer science)Programmer (hardware)Computer animation
Similarity (geometry)Repository (publishing)MathematicsIndependent set (graph theory)Revision controlSuite (music)Software testingImplementationRevision controlLatent heatSoftware testingSuite (music)ImplementationFormal languagePhase transitionLevel (video gaming)Right angleBitSet (mathematics)Point (geometry)MereologyForm (programming)QuicksortChainRepository (publishing)Arithmetic progressionPatch (Unix)SubsetCompilerMechanism designHome pageComputer fileLine (geometry)Web pageCheat <Computerspiel>Optical disc driveWordCASE <Informatik>Multiplication signRow (database)PreprocessorFiber (mathematics)FlagDrag (physics)TesselationPattern languageVector spaceArtificial neural networkTraffic reportingComputer animation
Revision controlSoftware developerInformationSheaf (mathematics)Level (video gaming)Fluid staticsSoftware testingFlow separationImplementationTime evolutionStandard deviationCore dumpFormal languageRevision controlNumberLatent heatMereologyMultiplication signPrototypeQuicksortInformationRight angleConcurrency (computer science)Cycle (graph theory)SequenceBeta functionSuite (music)Standard deviationFlow separationImplementationModule (mathematics)Numbering schemePlanningComputing platformPhysical systemReading (process)Proof theorySubsetCorrespondence (mathematics)EvoluteMechanism designInternetworkingArithmetic progressionProjective planeLevel (video gaming)Series (mathematics)Point (geometry)Software testingCodeRow (database)Video gameAuthorizationSoftware maintenanceKey (cryptography)Cellular automatonDistanceTheoryComputer animation
GoogolComputer animation
Transcript: English(auto-generated)
I got gonged already? OK, well, thank you for coming. So good morning, everybody. Can everybody hear me OK and back? All right. So my talk today is how to create a language specification for Perl 6.
And you'll notice the very key word not there. So this is lessons learned in my 10 years now of working with Perl 6. And when I say that, when I say I've been working with Perl 6 for 10 years, it just makes me go, ah, what happened in this decade of my life? And why has it taken so long and so forth?
As I go through my talk today, please feel free to ask questions. I don't expect that I will take the full 50 minutes that's been allotted. So if any of you are thinking, how will I get lunch? Now you know. You can have some time to get some lunch. And we will go from there as soon as I get my thing there.
So overview of the talk. First of all, I'm going to talk a bit about what language specifications really are. Because when I was putting this talk together, and this is the distillation of a lot of thinking about it over the last two or three years, all of a sudden
it became apparent to me that when we say language specification, it's not always obvious what that means. And so I'll talk about what are language specifications. I'll talk about Perl 6's design process and what I see in retrospect as being some mistakes that were made in the design process. Maybe one or two, but that's all.
Some important features that I've learned for language specifications and where I hope that the Perl 6 development team will be able to take things from here on this specific topic of language specifications. So how did I come up with this talk? When I was invited to give a presentation here at FOSDEM,
I didn't know what I would talk about. If you have been following Perl 6 development, then you know that for about the last 18 months, maybe a little bit longer, I have not been actively involved in it as I had been prior to that. There are a wide variety of reasons for this. And so I couldn't really talk about anything
I've done recently in terms of the code base. And so I said, well, maybe I'll do some reflections on the language design process. And these are a couple of topics that are really near and dear to me at this point because I think it's important that we make some new directions, that we make some new changes. You'll see those here. In particular, when talking about the specification,
it bugs me how much imprecision there is among the development team about the language spec and what that means. And so my crusade, my task going forward from here is to try and get some precision back into that. In fact, as we're talking about getting to a Perl 6.0.0 release
sometime in the near future, we're going to have to get a handle on what we mean by the language specification. And so some of these topics need to be discussed now. The opinions that I give here are, at this point, just my opinions. I've spouted them in little small groups from time to time over the past couple of years. Now I'm actually going public with it
and saying I'm planting a flag here and these are things we need to do. So let's, first of all, look at what is meant by the whole term of language specification. How many people know what a language spec is? Right, you're wrong. Because I had to think about it.
And so much of what comes up in this talk is the result of conversations with people that I've had, usually an answer to the question, when will Perl 6.0 be released? And that has led to a lot of thinking about how languages get developed. So what is a language specification? Many languages have them.
Ada, Pascal, C, they have language specifications. Many languages don't have them. So Perl 5.0 doesn't have a language specification, not in any formal sense of the word, certainly. PHP doesn't have, well, it does now. But prior to 2014, PHP had no sense of a language specification. And you can see that in the language.
I mean, if you look at PHP, you know it wasn't specified anywhere to be able to come up with this. But if you take anybody in a language that they work with, whether it's any of these languages, and you say, what is the language specification that you use for your language, what you'll discover is across languages, the form of the specification
can be very, very different. So when we say, what is the language specification for, say, Ada, there's a document. There's a set of documents that very precisely and rigorously define the Ada language. Whereas if you go and say, what is the language specification for C, what is the language
specification for C? Anybody know? Well, there's C89 is one. There's 99. There's C11. There's K and R. There's a whole bunch of them, right? There's a whole bunch of them. And if you go and look at the language specification that we now know as K and R, it wasn't very formal.
It was very informal. It was basically a book that it comes through. So the form in which a specification can take place will be different from one place to another. And yet, in the Perl 6 community, we often talk about the language spec. And what one person thinks of as being a language
specification could be very different from what another person does. And yet, we talk about it as if it's this fixed, well-known entity, when in reality, it's not. So let's just back up instead of language specification. What is a specification? So I went and looked on Wikipedia, because Wikipedia knows everything.
And Wikipedia offered this, which I thought was pretty good. So a specification may refer to an explicit set of requirements to be satisfied. And that was the key thing that I took away from this, that if you talk about a language spec, then it's an explicit set of requirements to be satisfied. So a specification should not be a plan,
should not be conjectural. It should not be, oh, we think it's going to be like this. If you're talking about a specification, then in many contexts, what you're looking at is requirements, something that this absolutely meets in order to do things.
In which case, we should call them speculations instead of specifications. And we have a lot of editing to do. So let's get to work. But if we're talking about specification, in some contexts, if you talk to somebody about, or in most contexts, if you say, we have a language specification, what you're really saying
is, these are the requirements of the language. So again, here's some example ones. I mentioned C already, the C programming language originally from Kernighan and Ritchie in 1978. And then you go to NCC, which became C89,
which became an ISO standard and then became C99 and C11 and so on down the list. Python is another one that doesn't really have a formal language specification. But they do have the Python language reference that's kept up to date by a variety of things. And the Python language reference, if I looked at it,
I'd say, oh, that's pretty close to a specification. It describes the language, but even it admits that there are quite a few details that are left ambiguous to give some room to the various implementations of Python to do things slightly differently. Even though it's not what we would normally think of as a programming language,
certainly HTML, the hypertext markup language, has a specification to it. And so it is, in fact, a language. And it also has a formal specification that is maintained by W3C. The early versions of HTML were, in fact, RFCs from internet RFCs.
And just to go along that theme a little bit further, when we're talking about specifications or language specifications, the entire RFC process is itself a way of making specifications. And some examples for that would be like HTTP, the Hypertext Transfer Protocol, or SMTP,
or URLs. All of these things have documents that formally define how they look. And so we don't have to limit ourselves to programming languages when we are discussing language specifications. So what is the Perl 5 language specification?
How is Perl 5 language specified? Is it by the Campbell book? Or is it by the interpreter? Or is it by the test suite? Or is it by a brick wall, I mean, by a layer wall? Anybody know? I mean, I know some people know. But where is the Perl 5 language specification? Larry?
These days, it's widely considered that it's the interpreter and the test suite form the Perl 6 language specification. And anybody who wants to disagree with me on that, you're probably right. But Perl 5 doesn't have a separate written language specification.
The interpreter and its tests, that defines Perl 5. So whatever the Perl 5 interpreter does, that's the standard behavior. And I noticed, if you've done much with Perl and Perl 5 development or Perl 6 development, you know the two rules. Everybody know the two rules of Perl 5 development?
First rule, rule number one is? No, no. Larry's always right. Rule number two is? If Larry changes his mind, he's still right. It occurred to me that Perl 5 language specification is the two rules.
Whatever Perl 5 says it does is correct. And if it changes its mind, that's correct. So there is no real standard specification there in terms of something that exists. It's that Perl 5 changes its language definition with every release.
So that's kind of an interesting way to go about doing things. So with Perl 5, Perl 5 was what brought us to wanting to create Perl 6. And so let's look at the language specification history for Perl 6.
So this story has been told many, many times. Perl 6 was announced in July 2000. July 2000, that was 14 and a half years ago. Ouch, right? There were a number of RFCs, a request for comments commission. Larry says that he expected to get about 20 things
that we might fix in Perl 5 in order to make it Perl 6. And he got 361 that were all over the map. Larry refined these into the apocalypses and synopses and very early on it was decided unlike the way Perl 5 had been developed where there was no language spec, Perl 6 would start with the language spec
and then we would implement it. Got that? So very, very different process. Perl 6 will start by creating what the language should look like and then we will implement it. And my response to that is, oops, I think that was a mistake. And I only think that now in retrospect.
I only think that looking back on it and things that I've learned about language specification since then. In retrospect, targeting the creation of a language specification before you do a lot of implementation is a mistake. And it really confuses things when you try to do that.
And in preparing this talk, in addition to all of the conversations that I've had with people about why it's taken so long for Perl 6 to be developed and all of the details about the language spec, I found out some interesting things. First of all, if you say we're going to create the language specification at the outset,
that word specification is too loaded with meaning. Because going back to the Wikipedia definition I gave earlier, specification implies a set of requirements that have to be met. And it implies some level of rigidity and permanence that you don't want at the beginning of a project.
You don't want there to be all that permanence. You want there to be room for experimentation. So just the name, just by going through and saying we would start by creating a specification, trying to throw things into the wrong direction. So can we have a design plan? Sure. Can we call them synopses?
Yes. That would be fine. But by saying that we would start with a specification, that has led to a lot of confusion about how Perl 6 would develop and how it should be developed. So this next slide is a summary of quotes that I've had to deal with over 10 years of working
with Perl 6 that really illustrate the confusion, part of the confusion. Now there are other sources. But the number one question I always hear when talking to somebody outside the Perl 6 community about Perl 6 is, when will it be ready? You still ask this.
When will Perl 6 be ready? And you kind of go and you say, well, we're working on it. And we're busy putting together a specification. And we're still figuring out the design. And we're still working on the implementation. And the next question that comes up by somebody who's trying to be very, very helpful is, well, have you at least finished the specification yet? Right? How many Perl 6 developers have heard this sequence?
Right? Right? Have you at least finished the specification yet? And so then we get into a discussion of, well, we're still working on figuring out what the specification should have in it. So that becomes, well, no wonder it's taking so long if you can't even figure out what the spec is first. Right?
If you can't even decide on the specification, they say there's no way you'll ever get to an implementation. And then it comes to my least favorite question in talking about Perl 6, or comment is that I will hear from somebody. They'll say, Perl 6 needs to freeze the specification right now, implement that, and then release it.
To which my answer is, no. That would be the wrong approach. But I cannot tell you how many times I've heard this in conversations about Perl 6, where very well-meaning people will say, the answer is to freeze the spec, implement it, release that, and then all of our problems will be solved.
All of this, I think, comes down to the notion that we have supported, that we have encouraged in our own discussions of the idea that specification leads development. But that's not really what happens. Right? Many people really assume that specification
precedes implementation. And as I said, it's common. We reinforced it in our own discussions of Perl 6, our own ways of saying how things would happen. But the reality that I didn't realize until probably about five years ago, successful languages, if you pick any successful language or system
that we have today, they almost always started with an implementation that was later converted into a specification. So Perl 5, or may not even have a specification, so Perl 5, clearly successful. And it doesn't have a language specification.
You can create a successful language without having a specification to begin with. PHP didn't have a specification until 2014. Ruby, HTML, HTTP, all of these different systems that we use every day that it would be held up as the primary systems that we use that have been very, very
successful all started out with an implementation first and then was later converted into a specification. So that's one of the big mistakes that I think was there. There are some striking examples of specifications going wrong. How many people have ever heard of HTML Plus?
Right? HTML Plus. How many people have seen a browser that implements HTML Plus? Never happened. So in the early days of the internet, HTML came out. HTML came out in 1990. The first specification for HTML came out in about 1992.
Again, another example of an implementation preceding a specification. And right after that, they said, HTML needs some extensions. And so there was widely published this document called HTML Plus that would define the next generation of HTML and what it would have it. And it was never implemented.
And as far as I'm concerned, the existence of this document basically delayed development on HTML because nobody could implement this specification. Nobody had the time, resources, or whatever to implement that. But since there was a specification,
nobody wanted to do anything that was outside of it. They wanted to stay within that specification before going on. Doing research for this talk, C99 came out in 1999. And after that, when they were coming up with C11, which is the 2011 standard for C, but after that, the C standards committee
adopted new guidelines. And the new guidelines were specifically to overcome the problems that showed up in C99 where things went into the specification, but there were no implementations of those. And so nobody was bothering to implement them, and it was actually making it very hard to implement.
So they reversed their standards to say that they would not adopt new features into the specification until they had been tested by implementation. That was kind of neat to discover. And then Wikipedia again. If you go and look at the article on language specification in Wikipedia, it has this very nice little quote here,
which I will just read out loud. Writing a specification before an implementation has largely been avoided since Algol 1968. What? I didn't know this. And why do you do it? Because of unexpected difficulties in implementation when implementation is deferred.
This is exactly the experience we had with many parts of Perl 6, that we would create a specification, and we would go and implement it sometime later, and all of a sudden we discovered, oh no, that was wrong. Start over, come up with something quite a bit better. But apparently, this has been known for some time.
So the lesson I learned about language specification, I have more lessons coming up. But first of all, specification freezes are not anything like code freezes. When you freeze a specification, when you declare something is a specification, that needs to be something that is retrospective.
Needs to say, we're codifying what already happened, as opposed to we're codifying what should happen next. And specification releases aren't really like code releases either. There may be a lot of similarities, but my lesson here, at least going forward,
is that if you're going to release a specification for a language, it's not exactly the same as a code release. Specifications should be very retrospective. An interesting parallel, when I discuss about the specification process for languages, and people say, oh, you should have the spec and then implement the language,
is to look at how things happen in the internet world. So internet standards, first of all, the RFC process has to be considered highly successful. It has worked really, really well for many, many decades. In the internet standards process, a proposal starts out as something called an internet draft, and this is a working document
that people working on that particular area of the internet kind of agree on. And eventually, it graduates to become what's known as an RFC, a request for comments. And that is kind of a formal description of whatever part of the internet is being proposed to be changed or to be adopted. And those RFCs can go onto something
called the standard track, as a proposed standard. And that's where many RFCs end up living for, I mean, for a very long time, is as just a proposed standard. They don't become internet standards until four things have to happen. And the interesting part about that is that the first thing that has to happen
is there have to be two independent operating implementations of the standard. And this, again, is different from this notion of start with a specification and then implement. In the internet world, you start with implementations, and once you have that, then you can declare a specification, the implementations must come first.
And there's some other things that have to go into that standard. But the big one that has always struck me is that internet standards are always developed after the implementation is largely considered to be complete. So that's one big mistake,
I'm off my soapbox about that. Let's go look at some language design standards. So another lesson that I've learned in working on Perl 6 is that a programming language is never frozen unless the language is dead, right? But languages always evolve. One of the things that Perl 6 gets very right
is to explicitly recognize that evolution will happen in your language, and to prepare for being able to deal with it. And so a lot of features have gone into Perl 6 that basically future-proof the language for the days when the language evolves. So that includes the things like,
if you modify the Perl 6 language, which you can do as a user, as a programmer, then your modifications are lexically scoped. The specification for modules and for the Perl 6 language you happen to be using are all versioned. The version numbers refer to the language version. We of course have features to replace
and get rid of the evil that is source filters. So now we have macros. We have the ability to custom build operators. You can define your own operators in Perl 6, which means you are modifying the grammar as you go. You're actually modifying the Perl 6 grammar. We have domain-specific languages. We have slangs, which again,
allow you to modify the grammar of the language that you're there. We have the ability to augment classes. You can take the base level classes that are in Perl 6 and you can add new methods into them. But if you do that, you have to declare that you're doing the evil by saying you're doing monkey typing. But you can go and modify the base level classes.
Classes in Perl 6 are never finalized. Well, actually the application is allowed to, but a class cannot say I'm final and no inheritance, no things like that. There's not a way to do that in Perl 6. Again, explicitly recognizing that what we might think of as being the final word today
may not be that way in the future. So evolution is a constant part of languages and your specification needs to be able to accept that. Another part of language design that Perl 6 gets correct is that we are making, unlike a lot of other places,
a sharp distinction between our specification and our implementations. So the term Perl 6 always refers to the language. It never refers to the compiler, right? And that again is different from Perl 5. And this question came up at dinner last night
where we were having dinner and one of the people there said, whenever I was talking about the compiler, I would always say recudo Perl, which is the compiler that I lead development for. And finally, one of the people there at dinner said, why do you always say recudo Perl 6 instead of just Perl 6? And my answer was, Perl 6 is the language.
It's this specification we're working on and the compiler has to be something different. Another thing that we get right is that there's no official implementation of Perl 6. So there is a language spec, but there's not a compiler that will be considered to be the official Perl 6. And that's important.
And that has always been something that's important to me, even though I'm working on one of the implementations. I mean, there could be a desire to make recudo Perl 6 into the Perl 6 standard and that everybody else then becomes in a second tier. But I think that's bad for the ecosystem.
I think that's bad for the language as a whole because we have to be able to adapt. And the way that you have adapt is you have multiple individuals. You have evolution. You have the ability to evolve. And so declaring or creating or anointing a single implementation as being the official one is probably not a good idea, yes.
I'm fine if people think that working with recudo is okay.
Yeah, I'm not sure I can solve that problem. So the question was, how do you prevent people from thinking of recudo Perl as being the official Perl? And the way I always get around it is to say, well, which compiler is the official C, right? And as you correctly point out, some people think GCC is the official C.
But those of us who've been around know that's not the case. And so I'll generally say, just as there's not really just a single C compiler, there should not be a single Perl 6 compiler. And then people kind of get it to be able to work on it. So multiple implementations are key to the long-term
evolution of this language. In fact, the question came up last night. Recudo currently runs on top of Parrot and the JVM and on top of more VM. And the question came up, well, why not just do it on JVM? Why have all of these other ones? And the answer is multiple environments gives us capabilities
and the ability to try things, to test things, to overcome limitations that would be insurmountable if we picked only one. And so this is a case of there's more than one way to do it. I mean, it really is the Perl 6 motto in many, many ways. Another lesson, and this is another one of my rants,
is that what we know as the synopses, which are design documents, those should not be the specification and we need to stop referring to them as if they ever were. And we need to get that notion, that duality of synopses as spec out of the conversations.
I think it's caused a lot of confusion, both within the developer community, within the Perl 6 bubble, but more importantly, outside of the bubble. When people say, where's the Perl 6 specification? We point to the synopses as the specification and it's confusing. It's difficult to work with on the synopses
and worse, they change a lot. There's a reason for the synopses changing and that is because they're not a prescriptive specification. They are a reflection of our current understanding of Perl 6, of what Perl 6 should be. And there are many times where the implementation leads those documents,
where the implementation, Recudo Perl 6 or Nieca or PUGS, will be doing things that were trying out features but it's not reflected in the synopses. And there are places where the synopses kind of articulate where we think Perl 6 will go, but we don't have any implementations
that implement that. And so we need to get rid of this notion of synopses as being a specification document. I'm fine with calling them design documents. I'm fine with calling them synopses. But I think we need to eliminate this specification thing. Because they change so frequently, whereas we think of specifications as being more rigid,
that's another reason why the synopses should not be equated with being a specification. And it's difficult to version them, just the mechanics of making versions of the synopses is hard. This was recognized sometime around 2008 in Synopses 1. Prior to that, it was kind of well understood
within the community that the synopses were our working set of specification documents. In 2008, Synopses 1 was changed to explicitly say that Perl 6 is anything that passes the official test suite. That is, the test suite becomes our specification. Whatever set of tests are there, if you can create a compiler or an interpreter or something
that passes the official test suite, then you have something that you can call a Perl 6 implementation. So where should we go from here? All right, first of all, as I've been saying throughout the talk, we need more precision in our descriptions. We are working to establish and release Perl 6.0.0.
It's been a goal for the last three or four years, at least, and the core contributors for Mercuto Perl 6 and Perl 6 in general are all pretty much in agreement that we're getting really, really close to the time when we can say, this is Perl 6.0.0,
and to be able to make that release. But before we can do that, in addition to all of the code level things that have been identified, I think we need to have a much better understanding of exactly what we mean and say when we talk about the Perl 6 specification.
And there needs to be a better understanding in our community that specifications follow implementations. They don't precede them. And for Perl 6, we've already declared in synopsis one, but we have not made this widely known as much as we should, that our specification is a set of tests. It's not a set of design documents, right?
Now this means you really need a good test suite. I mean, if you really want to test all of the features of the language, you've got to make sure that your test suite tests all of the different features of the language. I suspect that the first couple of releases of Perl 6, the language, there may be some holes in the test suite,
but those will be rapidly closed. We need to get rid of all of the false references to synopses as official Perl 6 specification, either by making it clear in the documents we produce going forward, or in some cases, going back and annotating what already exists
to indicate that the synopses really are no longer our specification documents. And just doing a couple of Google searches where I would do Perl 6 language specification brought up pages where it says the synopses are the official Perl 6 language specification.
In fact, last year at the Austrian Perl workshop, I finally got fed up with the Perl 6 homepage for the synopses on perl6.org, started out by saying the synopses are the official specification for Perl 6. And so I just finally committed a patch and said, I'm done with that. We need to stop doing that. I have another nit to pick with that.
Our GitHub repository that holds the synopses is currently called specs. I'm fine with that if that were generally understood. But most people are going to look at specs and say, oh, these are the specification documents. So to all of my Perl 6 compadres, fair warning,
I will change this soon. I don't know when. I'm going to go rename the repository to something else. If you have a preferred name, design is fine with me. How about just synopses, right? I mean, I can do synopses.
But specs has got to go. It's wrong, and it's confusing for people. So I will do it. I'm not going to give any more advanced warning when I'm doing it. And if you don't like it, well, forgiveness is better than permission. So it's going to happen. You can revert my commit if you like, but it's just going to happen. All right?
Because this has actually been bugging me for about four years. And I keep saying, we need to change the name of that repository, hoping to get a groundswell of support for people saying we should do it. And I'm tired of waiting for that, so I'm just going to do it. I'll note, right? I'm the Rakuto Perl pump king.
I'm the leader for the compiler. That doesn't make me the leader for the Perl 6 specification, which is why I've been a little reluctant to do that before now. But I'm really going to adopt forgiveness better than permission. Where we're going, I think we need to consider and look at the details of what our toolchain is going to look
like for the Perl 6 specification. Because if the Perl 6 language spec is a set of tests, and we already have a pretty robust test speed, it's called ROAST. Probably should be called specs, but it's called ROAST.
I'm sorry. Spec test is the? Oh, really? What's the ROA? Oh, backronym ahead, backronym ahead. Be careful. Then I need to go put that in the README.
That way nobody will find it. Let's see. So the current test suite, the specification will be defined by the test suite.
But I don't think we should just snapshot the test suite as it exists and say, that's Perl 6.0.0. Because the language is evolving. And inside the test suite, there are things that we would consider to be a solid part of the specification. And there are things that we would consider to be very conjectural, that are being implemented,
thus they need tests. But they're not really part of the specification. So there's always going to be some extra tests in there. So what I'd like to see us develop over the next few months to a year is some sort of mechanisms to extract the part of the test suite that is our consideration of being the Perl 6 specification.
And we can do a couple of dry runs of this beforehand. A lot of people will often say, well, it's already in GitHub, so just use a git tag to tag the version of the specification. I think that's useful, but not sufficient. We need something a little bit more than that. One of the nice things about Roast, or the official test
suite, is that it already has a subsetting capability built in. So if you have a common set of tests that are being used by multiple implementations, what are the odds that every implementation passed every single test in exactly the same set of tests?
And the answer is, it doesn't happen at all. Some implementations need to mark tests as to-do, and other implementations need to mark some tests as skip. But those don't always match. So many years ago, there was a tool created called Fudge. What does that stand for, Larry? Fudge. It doesn't stand for anything.
It stands for Fudge, right? Fudge. Cheating. Well, that's what we do. Is it still cheating if it becomes an official part of the way you do things?
Right. Correct. Correct. Right. So inside of the test file, there
are lines that look like this. These are our to-do markers. And there's things like to-do and skip and the like. And Fudge will take the official test file and pre-process it into producing only those tests needed for your particular implementation.
So for example, if this line appears before a test line inside of a test file, then Fudge, if it knows that the test is about to be run under the more version of Recudo, then it will mark it as to-do just for that particular implementation.
But the rest of the time, it's considered a comment line. So it's still all valid Perl 6. But there's some pre-processing that takes place. So the idea I have is we may be able to do that for Perl 6 versions. That the roast suite, the repository of all tests, would also be able to have markers in it that either declare these are the tests part of Perl 6.0.0
or declare these tests are not part of 6.0.0. I don't know which way it will be yet. But using Fudge, we could then run it on this large superset of tests to produce exactly the set of tests needed for a given version of Perl 6.0.
So we need to work on that tool chain quite a bit. I think we also need to explicitly recognize that features have lifetimes in a language. And that a feature may start out as being conjectural. 10 minutes, got it? Thank you. A feature may start out as being conjectural. Or it may move to then being a work in progress
where people are trying things. Eventually, it may come to a point where it's adopted. It's officially part of the language. And then, of course, we have the declining side of a feature. There may be a point at which, well, it's still there, but we don't recommend you use it, where it's discouraged.
All the way down to the form of deprecated, which is where we say, well, it's still there, but it's going to go away. And then retired, which is when it's officially removed. And I don't know. I've never seen this sort of notion of language features
having lifetimes, where you kind of identify the phases that they go to, not for the language as a whole, but for individual features. And it might be nice if our specification could do something like this as well, go down to individual features or individual tests and identify what stage they happen to be in.
Right. So perhaps we should explicitly recognize this somehow. We need some guidelines on versioning. I mean, the buzz within the community that we expect to have is all about 6.0.0.
But what defines 6.0.1? What defines 6.1.0? How do we decide, after we get to 6.0.0, where do we go from there? And I know from dealing with version numbers in Parrot and in Recudo and in various other projects,
getting the version numbers right is a pain. So the version numbers in Recudo, does anybody know what the version numbers are in Recudo? Our first release was release 0. Our second release was release 1. Our next release was release 2. And that's it. Right. Now, we do releases on a monthly cycle.
So there are also month numbers that go along with it. And they are a four-digit year followed by a two-digit month. Right. But release sequencing is just a sequential number. We didn't put any major, minor specifics in there. And that was a direct reaction on my part to the bizarre numbering scheme that Parrot adopted.
It was also a reaction to the very nice versioning scheme that I like from the Ubuntu series of versions. Ubuntu always does theirs as being a year-month release naming scheme. And that makes it very, very easy to work with. I'm certain that Perl 6 version numbers will be 6.
Minor number dot, you know, some other, or major.minor, something like that. But at some point, we're going to need to decide, how do we decide what constitutes a new major version and a new minor version? And maybe Larry already knows. But if so, I don't know what it is.
And the other thing that occurred to me is that in Recudo and in Parrot and in all of my projects that I do now, we do time-based releases. That is a very, very useful mechanism for making sure that progress is made. And I don't even think about doing projects now without considering time-based releases. Should we have something equivalent
to time-based releases for the spec? I'm not sure. I wouldn't do it monthly, but maybe every six months, like Ubuntu does, or once a year, or that's probably not often enough. But a time-based release system might be something worth considering. And then what kind of started me down this path
was several years ago, people were asking when the synopses were largely considered to be spec, they said, can we get just a version of the synopses that has only the parts that work now? As opposed to what might work in the future? Because if you're reading those documents and you go say, hey, here's a cool feature,
and you go try it in Recudo, Recudo comes back and says it's not yet implemented. Well, that's kind of annoying. So I would like to find a way to tag the synopses documents with language version information. And the idea I had was doing it at the paragraph level a couple of years ago. And I wanted to write a tool, a proof of concept tool, I just never got around to it.
The nice thing about doing this is that we could then extract the synopses and say this subset of the synopses corresponds to this version of Perl six. And you can kind of see the correspondence there without all of the extra about where we're planning to go and hopefully without the extra of where we've been,
but just the parts that you would need to know. The nice thing about this is that unlike a specification, the synopses could actually go back and rewrite history to make more sense to people. Because we could go back and say, oh, a better way of describing history would have been this. And so that would be something nice to do.
So to recap, again, in thinking about language specifications, all of a sudden I discovered that within many communities, there's this widespread misconception that specifications precede, got it, five minutes, precede implementation. I think specifications work best when they are historical markers.
What has been done, let's snapshot that, let's make that official. Certainly that's the way the internet has done it. Next, language design needs to recognize that languages evolve. That's very important, it's key, and I think it's something that sets Perl six apart from most other languages that have been developed,
is how much effort has gone into handling the multi-decade evolution that we hope Perl six will begin. So Perl six has robust features for that. I think it's very key that we separate specification and implementation and maintain that separation. I like very much that we're doing
test-based specification, and I really think we need to come up with language specification versioning standards. And so with that, I will take questions. If there are any. Yes.
I can see several paths, and I can't speak for the language so much, but I can see several paths.
Certainly there are going to be features that we would consider to be good candidates for the language, that we would like to see first implemented as modules. So as opposed to, and part of Perl six design is that you can create modules to customize the language however you want. And so you could start by creating a module
that would add that particular feature. And then if everybody says, oh yeah, that really should be part of the core language, then it would be relatively easy to put it in. If it's a feature that can't be done that way, or if it's something that so obviously needs to be part of the core, then yeah, you could prototype an implementation. And I wouldn't necessarily have a problem
with it going into the code base. If there's some sort of general agreement that this is something we should start. There are places where we do that now, concurrency being a big one. That there was no specification for concurrency. And the reason there wasn't is because we felt like it was definitely premature to do a specification for concurrency. Let's do some implementations first.
See how we like it. And then, I'm sorry, what? Yeah, exactly. So if that's something that really needs to be in there, certainly see how much trouble it is to put in. And then we can start to consider those kinds of things. So, but yes, I would, again, the idea is, is that let's build a platform
that makes it easy for people to experiment. And then we can adopt the things we like. Yeah, the challenge is, let's be careful about adopting things before we're sure that's what we want. Because Perl 6 history is replete with examples of us adopting approaches
and going a good ways down that path and then discovering, oops, that was a mistake. We really need something radical the other way. And so that's another reason why standards need to follow implementation and at some distance. Any other questions? Yes?
Yes. I think our intent is that we will have, again, specifications should be along with implementations. So we will probably declare Perl 6.0.0 as a test suite.
I would expect there would be some betas or release candidates leading up to it. Corresponding at the same time, there will probably a version of Recudo that we will declare to be meets the 6.0.0 spec. Does that sound right, Larry? Actually, for more details, see Larry's talk tomorrow.
I could if I knew it.
And I would agree with that timeline.
Right. And I'm sure my time is up. Yep, all right. If you have any more questions, I'm here today and tomorrow. I'll be glad to do it. Thank you all very much. Thank you. Thank you.