Celebrating Guile 2020
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Title of Series | ||
| Number of Parts | 490 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/46973 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
00:00
Normed vector spaceFibonacci numberNumbering schemeBitCompiler2 (number)Arithmetic meanCore dumpScaling (geometry)MereologyVirtual machineFunctional programmingSound effectBenchmarkProgramming languageLine (geometry)Cartesian coordinate systemNumberMultiplication signData managementMaxima and minimaTask (computing)ResultantInterpreter (computing)Lecture/Conference
02:36
LeakNumbering schemeMereologySoftware configuration managementFormal languageData managementTask (computing)Real numberFormal languageImplementationPower (physics)WordPhysical systemLambda calculusDialectMultiplication signNumbering schemeElectronic mailing listDreizehnRight angleCalculusLecture/ConferenceComputer animation
04:35
ExistenceTransmitterEmbedded systemExtension (kinesiology)Local ringInertialsystemDivisorInterface (computing)Source codeBinary codeCommon Language InfrastructureNP-hardCompilerUnicodeComputer fileNumbering schemeMetrePattern languageParallel portEmpennageInterface (computing)Multiplication signDialectNumberNatural numberAuthorizationMaxima and minimaBitMathematicsPattern languageDirection (geometry)Event horizonControl flowHill differential equationParallel portComputer programmingMathematical optimizationFormal languageWeb 2.0Numbering schemeEvoluteStability theorySound effectState of matterContext awarenessSoftware developerSoftware bugInclusion mapExpressionQuicksortFunctional programmingCodeObject-oriented programmingCompilerThermal expansionState observerStrategy gameDefault (computer science)Software maintenanceRight angleRevision controlVariable (mathematics)SpeicherbereinigungLibrary (computing)Run time (program lifecycle phase)String (computer science)Process (computing)Real numberExtension (kinesiology)Projective planeFigurate numberMacro (computer science)Food energySubject indexingSelf-organizationVideo gameGraph (mathematics)Physical systemWordMoment of inertiaLink (knot theory)Line (geometry)Point (geometry)Different (Kate Ryan album)GodArithmetic meanPerspective (visual)Axiom of choiceFlow separationEntire functionComputer animation
13:16
MetrePattern languageParallel portEmpennageTexture mappingHuman migrationTable (information)Variable (mathematics)Atomic numberFrequencyInterface (computing)Series (mathematics)Uniform convergenceObject-oriented programmingVector graphicsStability theoryNormal (geometry)Formal languageCompilerPressureModul <Datentyp>Function (mathematics)Macro (computer science)Numbering schemeCodeSoftware maintenanceNetwork topologyMemory managementMaxima and minimaLocal ringElectronic mailing listMessage passingMathematicsEmailSpeicherbereinigungIntegrated development environmentComputer programmingPoint (geometry)Configuration spaceCodeFormal languageArithmetic meanFingerprintNumberIdeal (ethics)Functional programmingRevision controlLibrary (computing)Flow separationAuthorizationQuicksortMultiplication signObject-oriented programmingCausalityInterface (computing)Commitment schemeParameter (computer programming)Exception handlingStability theoryConcurrency (computer science)MereologyPattern languageSeries (mathematics)Computer clusterEvoluteExtension (kinesiology)Level (video gaming)PressurePointer (computer programming)Memory managementElectric generatorMacro (computer science)View (database)Module (mathematics)Online helpOnlinecommunityKeyboard shortcutImplementationGoodness of fitType theoryDependent and independent variablesResource allocationCompilerNumbering schemeSoftwareRun time (program lifecycle phase)Software maintenanceAttribute grammarAdditionSocial classDifferent (Kate Ryan album)Peer-to-peerMeasurementWebsiteEndliche ModelltheorieLetterpress printingBenchmarkRight angleFundamental theorem of algebraPiRun-time systemSoftware bugCASE <Informatik>AverageRoutingPerspective (visual)Subject indexingFiber (mathematics)InternetworkingJava appletScripting languageVotingKey (cryptography)GodWordStandard deviationComputer animation
21:56
Stability theorySource codeProgramming languageConstructor (object-oriented programming)QuicksortParallel portNumbering schemeFormal languageSource codeLevel (video gaming)Library (computing)Default (computer science)Installation artDimensional analysisException handlingComputer fileDialectSoftware developerFreewareIntegrated development environmentDifferent (Kate Ryan album)Cycle (graph theory)Computer clusterTable (information)Hash functionStability theoryPerspective (visual)Computer animation
23:31
Hacker (term)GoogolFormal languageImage resolutionMultiplication signCurveRight anglePhysical systemPoint (geometry)Perspective (visual)Order (biology)Functional programmingTraffic reportingSoftware bugPlanningLecture/Conference
26:20
Point cloudOpen sourceFacebook
Transcript: English(auto-generated)
00:06
You can start. Oh, yeah? OK. Thank you. Hi. Thanks for coming to my talk. I appreciate that. This is two small talks in one talk. We just made a new major release of Guile, Guile 3.
00:21
So I'm very happy and relieved and tired. And so we wanted to celebrate that a little bit. And then also, I was thinking about some minimal languages. Obviously, the theme of the room. Guile is minimal, but it's old. And old things have a tendency to grow in size. Or how do you maintain a minimal aspect through time?
00:41
And in fact, how do you maintain a thing through time? Like, what have you observed? So I have a second mini talk, which is things I've learned as being part of Guile over the last, I don't know, 15 years or so. OK. So Guile, I describe it as something which is old, which is ancient, but which is also spry.
01:01
And spry meaning that it's active and lively. And we just had a new release. And to show that, I can start with some microbenchmarks, which you obviously can't see. But this is comparing Guile 3 versus Guile 2. And the more paint is on there, the better Guile is than Guile 2. If you see some very thin white lines there,
01:22
each one of those light lines is two times more. So more than half of the benchmarks are more than two times as fast. And one of them, for some reason, is 32 times as fast. So it's a microbenchmark result. And if I take a larger, more real result, perhaps, this is benchmarking a function we
01:41
have in Guile, which is the core of the interpreter, called eval. This is eval that you learn in CICP or things like this. It used to be in Guile 1.8, when I started with Guile about 15 years ago or so, this function was implemented in C. And we changed to have it implemented in Scheme when we added the compiler.
02:01
When we did that at first, things slowed down quite a bit because our compiler wasn't that great, and our virtual machine wasn't that great. It had a number of advantages. But as you can see, we went from computing the 30th Fibonacci number using the interpreter from .32 seconds up to 4 and 1.5 seconds. Again, a log scale on the y-axis here.
02:22
But as we've improved from Guile 2.0 to 2.2, and now finally to 3.0, we're back with the performance of primitive eval, except now it's implemented in Scheme, whereas before it was implemented in C. So I'm pretty pleased with that effect. And if we take a larger benchmark, like something like Geeks, Geeks the package manager written in Guile, on some tasks it can improve
02:43
by, I don't know, 5%, 10%, 15% perhaps. Depends on how much other things you're doing, IO or things like this. So I think pretty good. But when I actually looked up the definition of spry, it didn't say just active and lively.
03:01
It said, especially of an old person, right? Which is funny because it's the word I thought about to describe Guile, and then I was offended by the definition, and then I realized, you know, this is reality and I should just accept it. I turned 40 last week also, so this is related.
03:22
So Guile is old, Guile's real old, you know? It's not on the upswing like a new language like Rust, for example, it's not Elixir, it's old, right? It's older than probably some people at Fazdem, right? I guess older than JavaScript,
03:40
which I realized today was three to the power of three years before Guile 3. And it's built from old things, right? Guile itself was not built out of nothing. I wasn't around then, obviously, I was 13. Guile was built on top of an implementation called SCM, which itself was built on top of an implementation called Scheme-in-One-Def-Un, S-I-O-D, in 1988,
04:02
before Bash, before TCL. And these things themselves are built on old things, right? Like they're implementations of Scheme, which is old, old, old, right? Which is like a dialect of Lisp, which is old, which is an implementation of the lambda calculus, which was three to the four power years ago.
04:23
So what you have after evolving with time, the world is not the same as it was 81 years ago, nor 27 years ago, nor nine, nor three, like we have to keep adapting. So how do you maintain a minimal system as we go forward in time? And I thought about this, and I understand history as a dialectic process, right?
04:43
You have tensions between things, and when you resolve these tensions, you're not ping-pong back and forth between stability and change, or between minimalism and complete, you know, everything's included. When you resolve these tensions, you get to another state and you have new tensions, right, and these tensions drive forward
05:02
the motor of history, and as Marx says, men make their own history, men, you know, language of the day, but they do not make it as they please, they do not make it under self-selected circumstances, but under circumstances existing already. We're always building on things before, we're never creating new minimal things out of nothing, we're always advancing from where we find ourselves,
05:22
and then once we've advanced, we find ourselves somewhere else, and so we have to move and keep on moving on. And so how do we remain minimal over time? And in the context of Guile, like what are the tensions that operate, and how do these tensions kind of turn the crank of history, like moving Guile forward to a newer and newer state in time, right?
05:42
And is it going forward or backwards? But it's certainly like developing. And so I see the tensions as like the problems in the world, like the web was not a thing in 1936 or whatever, between like having something stable that you want to build on versus having something that can change and adapt to new circumstances
06:01
or can improve itself. And there are a number of other tensions in here. And so in this talk I want to give some observations that I have on how these tensions operate and can pull language forward. So one of the lessons I've taken from Guile
06:22
is that as far as an optimization pattern, often you think about where you are and how can I get immediately better, right? You take a step in that direction. This is hill climbing, and it's widely known that you're not going to find necessarily a global maximum if hill climbing is your approach
06:41
for language development or for program maintenance or anything because you could climb and you could reach one hill but there could be other hills that you don't know of. And if you knew of them, obviously you'd be on them already, but you just can't know because you only see what's directly around you. And so like as a language evolution strategy, I find hill climbing is a recipe for death in the end
07:03
because if you ever reach that maximum, then you have nowhere to go, right? You're just, you're stopped. Like why would you step down in anywhere that's worse? And so I found that that's where Guile was 15 years ago or so. We had optimized and optimized and optimized to be a scheme that you would link to from C.
07:23
And we got to what we thought was a good position, but it had a lot of downsides that we couldn't reach through incremental steps to getting somewhere else. So while hill climbing has its places, it's not enough, right?
07:42
Another thing I learned in Guile is that users stay with you unless you push them away, right? Just out of inertia. They'll bleed off through time, right? You'll get new users through other processes possibly. But the real exodus events are when you break things for users, right?
08:00
And the way to break things for users is to change your interface. And there are many kinds of interfaces. Like it's the I and API and CLI and ABI, right? Whenever you change any of these things, you're going to lose some of your users, right? So one lesson here can be don't change, right? Never change. Which is not, this is a way to move forward
08:22
with your users. Obviously not the whole thing, but if we take some examples, Python 3 obviously has some problems in this nature that I'm not expert on, but I certainly know they exist. Guile had a thing in 1.8 called local eval, where you could evaluate an expression
08:41
within a lexical context at runtime. We took it away because we thought that when we added a compiler that we couldn't support this feature. But it turned out to be so important that we had to figure out how to add it back. Otherwise we were just going to lose some important users. There is some scheme itself is evolving in different ways.
09:02
One of the evolutions was a standard called R6RS, which includes some syntax which conflicts with old Guile syntax. And we don't really have a good way to deprecate syntax. So we don't have the R6RS syntax on by default right now. And finally, you would not design Guile today
09:21
with mutable pairs. You wouldn't design it with set, right? You wouldn't design it with mutable variables. That would not be what you would design. It's not necessary, there are other ways to do it. They're much more clear. But we have them around because if we took them away, our users would go away. At the same time, you can't keep all your users, right?
09:42
You're going to change, right? So if you can't take the previous strategy of never changing anything, then you have to say, well, okay, sometimes you have to lose some users. And so you need to know when that's okay. And it's hard to know, right? We've made mistakes in the past. But if you don't change at all,
10:00
you will reach one of these local maxima, and your project will be dead, right? So for example, we switched in Guile from, in 1.8, we had this thing where macros, which are user extensions to program syntax, would be expanded the first time a bit of code was evaluated. And this, it had a number of downsides,
10:21
and we switched to an approach where the whole program was expanded before anything was evaluated. Well, you could evaluate during expansion, but that was a separate concern. And that's when we switched to P syntax. And so that, if you have side effects when you are expanding in your macros, then they run at a different time
10:41
when you do expansion eagerly versus lazily. And we switched to this eager expansion, and it broke some things for people. But I think the benefits were enough that we got a more expressive system that we were able to recoup the users on the other side. But it was a pain. Similarly, we switched to garbage collector. We currently use the bone collector, which is pretty weird for a modern language,
11:01
but it works really well for some uses. One issue is that it runs, generally you don't need to tell it how to trace your object graph, but when you do need to tell it how to do it, it's very tricky. And so users which had used this API that allowed them to augment the garbage collector's notion
11:20
of what is live had a really hard problem, and still have a really hard problem, migrating off of Gal 1.8, which had its last release 12 years ago or so. The introduction of a notion of compile time is, in a way, runs against minimalism in some ways. And it complicates things for users,
11:41
but we did that. And also, we did the Python 3 thing more or less, more or less, of using Unicode for strings. And I think it was the right choice, but it was painful for some people. So every interface you have is cost, as maintainer.
12:01
And I hope that these lessons, I speak from a Gal perspective, but I think that every library author, every program author creates a language of some sort for their users. So the things I say, I think should apply for other programs as well. That in Gal, we have a binary API, and so the easiest way to do this is just to minimize your entire interface.
12:22
So for example, Go does not have a dynamic linking interface because it statically compiles. It's a great radical solution to this, I think. Geeks has a monorepo, right? Monorepos are also a kind of great solution because you don't have too much of an interface between what you work on and separate things that develop at different paces. Recently, we released Gal 3.0,
12:41
and I forgot to make some functions exported in the C API, and now I get bug reports, and I have to add them back. There's unintentional change, but this thing that was just out there that was providing no value, I accidentally broke it, and now I have to do some work. I don't like doing work. So anyway, the pattern we have in Gal for allowing us to change,
13:01
allowing us to evolve and change, is parallel installation, and this applies everywhere that we change. This is the pattern. So for example, we have libgal-2.0.so, which you can install in parallel on your system with libgal-3.0. And this philosophy was outlined almost 20 years ago by Havik Pennington in this great article,
13:21
if you haven't read it yet, I definitely suggest you take a look at it. And the corollary of this is that if you, you cannot change the meaning of any existing interface, right? You can extend the meaning, you can add new meaning, that doesn't conflict with old meaning, but if you ever change the meaning of something, you need to add a new name, like you need to give it a new name, whether it's Gal 3.0 versus Gal 2,
13:42
but in the case, we had a function called make struct that took this extra argument to support something that was deprecated, this was a very low-level facility, we weren't able to actually adapt this function, so I had to make a replacement called make struct no-tail, at some point I'll deprecate make struct and maybe swap them back, but it takes a long time to do this.
14:02
Additionally, we have like M4 macros that can allow you to say, you know, actually don't move me to Gal 3, right? Keep me on Gal 2.2, right? Cause I can't, I don't have time to deal with this right now. And as a counter example, libtool does not help us at all. I don't know what it gives you, the libtool versioning, it's never solved any problem for me. I mean, maybe it's good for distros, but it doesn't help me evolve Gal.
14:22
We can talk about that later. The thing that facilitates change though, is that you add a new name, right? You want people to migrate to the new name. Our pattern that we use here is deprecation, and it applies everywhere. On the C API, you can add attribute deprecate, functions, types, variables, and you can give a message when you do this
14:41
to say, please migrate to this. So when the users who are capable of making this change compile this, they see this message like, oh man, it's giving me all these warnings, you know, I gotta get on this. If they decide to switch to the new API, right? You have to give them time. We have this also in Scheme, which if you run your program at the end, it might print out, you use some deprecated features,
15:01
please set an environment variable to let you tell us, we'll tell you more about these features when they come. And we have facilities to kind of if def them out, if you're running in a configuration where you disable deprecation, because you want to know, is my program compatible with the future? Can I migrate at my own pace? So that gets to our arch pattern of maintaining minimality,
15:23
which is, okay, you need to change something, you replace it with something new, you deprecate the old thing and ask people to migrate over, and then eventually remove, and remove is important, because otherwise you're maximal, you're not minimal, right? So, and this allows us to change anything, and the only question here is,
15:41
how long do we have to keep something deprecated? And for some things, they might stay deprecated for 10 years. It depends on how active your user community is, like how, what their change philosophy is. Guile, coming from a kind of old GNU perspective, we're definitely starting from a like, oh, don't change things point of view.
16:01
But generally, for Guile interfaces, we'll go, we'll deprecate something, and let's say we deprecate in 3.0, when 3.2 comes out, we'll remove the deprecated interface. So you can only migrate onto 3.2, which presumably provides you with better things once you remove the old thing. So make struct is still around with this crappy interface.
16:21
We just migrated to C99, I'm so ashamed to admit it, off of C89, yeah. So we had like, defines for UN8 that are now obviously replaced by standard int, now from 20 years ago. This thing is still in Guile, but now it gives you a deprecation warning and we'll remove it next time.
16:41
And sometimes we don't succeed, maybe the replacement didn't take into account the use cases of users. So I added something called foreign objects to replace a facility in Guile where you can extend Guile with new types that are backed by some C implementation. And it hasn't been a real success, so I don't know if we'll be able to actually migrate off or if I'm gonna have to remove foreign objects, I don't know exactly.
17:01
Anyway, so once you change, you reach a new stable point. So for me, unstable series and stable series are also part of this pattern. Like, you get to this stable point in 3.0, right? I haven't looked at the mailing list since 3.0 came out because, you know, I can't change some things.
17:20
You know, I'll have to fix the bugs, but users are going to be able to rely on 3.0 being the same for some number of years at this point. I also didn't look at mailing lists because I went on holiday, but unrelated. That in a stable series, you yourself have dependencies and they must share your same ideals
17:40
with regards to stability and change. So we can't depend on anything that isn't as stable as Guile's goals. Whereas if you're like a new minimal language that comes out, you know, in 2020, you might be able to depend on stuff that's very, you know, fast changing because you haven't built up the users yet to be change-averse. They're still active in your programs.
18:01
And if you change something in six months or 12 months, then maybe you can take your users with you. But as you get older, we definitely need more stability. So we only depend on very, very stable things, with the exception of Gnulib, which is one of these include-only libraries, which has helped us to migrate forward without much risk of change there.
18:21
Right, so the other thing is that as a author of a language, you know, be that language a library or a Guile or whatever, you're going to get pressures for users for you to add things, right? To make your language bigger, to add features or what have you. And in Guile, being based on Scheme,
18:40
I found that it's been already really good that we empower our users that the language constructs they create sit on a first-class level with the language constructs that are defined in Guile. So users obviously can extend Guile with modules and such, but I think that the syntax question is really important as well. We all know the problems that come in extending
19:01
the syntax of JavaScript or Python or something like this, because syntax keywords are a first-class part of the language that users don't really have access to. Whereas in Scheme, we're able to evolve this syntax without my permission, my permission as Guile co-maintainer. The other thing is that often you have maybe a performance difference,
19:21
especially in language implementations. A user defines some facility, it doesn't perform as well as something implemented in Guile, because Guile has access to all the speed, you know, shortcuts and things like this, and this was a real problem we had in 1.8. It pushed users towards a overuse of C extensions.
19:40
It added to our interface there. We still have a lot of that interface that we still have to maintain. And having made this leap to have a better compiler and runtime system, users can now define things in Scheme that are perf-acceptable, generally. So my lesson here is that if you empower your users
20:01
to create things that sit on, as peers, to the facilities defined in the language, then they need less from you, which is good, because I don't want to do things. Speaking of not wanting to do things, users come with code sometimes, and they want you to take it, right, in an extension to your software, the software you're responsible for.
20:21
And this is almost always a burden. And so for me, I really prefer this empower the users, separate modules sort of things, except when the feature is just quite stable and it's not going to demand anything of me. But that involves a commitment from the users as well. Like they have to realize that this thing that they're adding is stable,
20:41
or participate actively in Guile maintenance. So you just need some agreement there. And so some things we have in Guile are in because they're super stable, and some things are out because like Fibers, for example, is a lightweight concurrency facility. I haven't added it to Guile because I don't know if I, I don't know if it's stable enough, I don't know.
21:01
So, right, just a couple more things. The greatest problem we have in Guile right now with regards to evolution is memory management on the C interface. It would be really nice for us to have a precise generational garbage collector. Instead we use a sometimes precise, sometimes conservative, details, the BOEM collector,
21:23
which doesn't offer bump pointer generational allocation. If you look at the microbenchmarks, the way to get the next bump to reach a scheme is to have a good garbage collector. But it's very difficult on the C level to have all of your users change this API
21:41
in a very fundamental way. I'm not really sure how we're gonna get there. Guile's not the only language with these problems, even PyPies has issues with regards to when the delete finalizer gets called, or destructor, or whatever it is in Python, relative to CPython, which, where it happens a different time. And if we look in the future though,
22:00
so we're around in a circle, like we've gone from stable to change, and now we're back to stable again. And in Guile's perspective, I wanna add another dimension to parallel installability. Users can do most anything they want with Guile defining new constructs that are on a par with Guile's constructs, with the exception of whatever language it is
22:21
at the top of your file, right? At the top of your file, it's still kind of this default Guile environment, this default Guile syntax. And Racket has this really wonderful, elegant solution to this, which is called hashlang. You just sort of note what language your file is in at the top, and you can have pretty significantly different syntaxes. You can have a more friendly,
22:42
perhaps, approach to using libraries. So I wanna have that. I find that to be parallel installability for source languages, even source languages that are radically different than Scheme. And additionally, I've been working in the weeds, right? I haven't been working so much on the higher linguistic levels. And I think Racket has shown a lot of idioms
23:00
that are really useful, that we would like to sediment down into Guile's riverbed. And so the thing about the dialectic is it doesn't advance Guile itself. It's still people doing things. And until now, it's been me, sort of cranking around from stable release to development cycle, and then back to stable release. And I want to see if I can step back and let other people hold that some more.
23:23
Whether that's like, I put a lot of effort into community, or if I just step back for a year and see what happens, I'm not sure what's gonna do it. But one of those things will work. So yeah, that's my thoughts on evolving language. Questions, welcome.
23:50
Yes, in the back. I think, whew, that's a very good question.
24:03
The question is, when is it a good idea to lose users? From the selfish perspective of Guile, it's better to lose users if you recoup more users than you lost. But you don't recoup new users tomorrow, right? It's a function that you expect
24:20
that it's going to come in at some point in time. So is it better if you lose 10% of your users, but you get 20% more in three years, or does it need to be six months? It really depends on your goals as far as your growth. If you expect to grow a lot, then you can lose all your users. Right? I think Guile's more on a smaller curve.
24:43
Upwards, I hope. Yes? Is Guile still using auto tools? The question is, is Guile still using auto tools? Yes, it does.
25:01
It's horrible. It's horrible in ways I understand. I don't have any particular plans to switch off, but if somebody wants to give a crack at that, that could be interesting. But I would note also that I got bug reports after I released Guile from Itanium people and HPPA systems.
25:21
And so is it okay to lose those users? I don't know. I don't know. I can't let other people contribute,
25:42
but I remember in some of your previous, either a talk or a blog post, that you worked best alone. So how do you see that? So the question is, I would like evidently to allow other people to facilitate other people to contribute,
26:02
but I've said previously that I worked best alone, which is true. I don't know what the resolution there is. I don't know. Right now it's a desire without a plan. Okay. Thank you very much.
Recommendations
Series of 24 media
Series of 10 media