Nim Metaprogramming in the real world
This is a modal window.
Das Video konnte nicht geladen werden, da entweder ein Server- oder Netzwerkfehler auftrat oder das Format nicht unterstützt wird.
Formale Metadaten
| Titel |
| |
| Serientitel | ||
| Anzahl der Teile | 287 | |
| Autor | ||
| Mitwirkende | ||
| Lizenz | CC-Namensnennung 2.0 Belgien: Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen. | |
| Identifikatoren | 10.5446/56965 (DOI) | |
| Herausgeber | ||
| Erscheinungsjahr | ||
| Sprache |
Inhaltliche Metadaten
| Fachgebiet | ||
| Genre | ||
| Abstract |
|
FOSDEM 202249 / 287
2
4
6
8
12
17
21
23
31
35
37
41
44
45
46
47
50
62
65
66
67
68
71
73
81
84
85
86
90
92
94
100
102
105
111
114
115
116
117
118
121
122
124
127
131
133
135
137
139
140
141
142
145
149
150
156
164
165
167
169
170
171
172
174
176
178
180
183
184
189
190
192
194
198
205
206
207
208
210
218
220
224
225
229
230
232
235
236
238
239
240
242
243
244
245
246
249
250
253
260
262
264
267
273
274
277
282
283
287
00:00
Diskrete-Elemente-MethodeStichprobeMAPTemplateMakrobefehlLokales MinimumTypentheorieTopologieReelle ZahlProgrammierungTemplateMAPFolge <Mathematik>OvalDateiformatMathematikMakrobefehlNatürliche SpracheGenerizitätWechselsprungTypentheorieLeistung <Physik>FunktionalZeichenketteFormale SpracheMeterÜbersetzerbauCodeBildschirmmaskeMatrizenrechnungGemeinsamer SpeicherDämpfungShader <Informatik>VektorraumMetaprogrammierungLokales MinimumVektor <Datentyp>DiagrammComputeranimation
03:05
DickeSoftwaretestGenerizitätTypentheorieMessage-PassingFunktionalVektorraumLeistung <Physik>NormalvektorVersionsverwaltungDämpfungGanze ZahlInstantiierungTeilbarkeitBenchmarkMatrizenrechnungQuick-SortSoftwaretestDickeMathematikEinfache GenauigkeitZusammenhängender GraphBinärcodeXML
04:45
Objekt <Kategorie>BildschirmmaskeTopologieDateiformatVarietät <Mathematik>Objekt <Kategorie>ZählenZeichenketteCASE <Informatik>Hook <Programmierung>Computeranimation
05:26
TermMakrobefehlTemplateZeichenketteCodeCompilerObjekt <Kategorie>ROM <Informatik>BetriebsmittelverwaltungMAPSpeicherabzugMultiplikationsoperatorObjekt <Kategorie>SubstitutionCodeMAPTemplateZeichenketteReelle ZahlHook <Programmierung>Umsetzung <Informatik>DifferenteDefaultModifikation <Mathematik>TypentheorieLeistung <Physik>Dynamisches SystemFormale SpracheSpeicherverwaltungGenerizitätSyntaktische AnalyseNEC IC Microcomputer Systems Ltd.SystemaufrufCASE <Informatik>HalbleiterspeicherSnake <Bildverarbeitung>InstantiierungRechteckURLZählenMereologieJSONXML
08:42
TemplateRechteckTypentheorieQuick-SortFormale SpracheAlgebraisch abgeschlossener KörperProgrammierungBenutzeroberflächeCodeWinkelBefehl <Informatik>BaumechanikNichtlinearer OperatorSondierungSubstitutionCASE <Informatik>MinimumUmwandlungsenthalpieJSON
10:31
CodeTemplateVektor <Datentyp>MathematikMultiplikationAlgorithmische ProgrammierspracheFehlermeldungNichtlinearer OperatorTeilbarkeitDifferenteTemplateCodeARM <Computerarchitektur>TypentheorieDivisionGeradeKomplex <Algebra>VektorraumSystemaufrufComputeranimationJSONXML
11:49
SchnelltasteEin-AusgabeÄhnlichkeitsgeometrieEin-AusgabeKontextbezogenes SystemProgrammfehlerNetzbetriebssystemProjektive EbenePhysikalisches SystemFormale SpracheSchnelltasteInstantiierungPRINCE2TopologieEreignishorizontBildschirmfensterComputeranimation
12:49
TemplateKlasse <Mathematik>SystemaufrufLaufzeitfehlerInterface <Schaltung>VererbungshierarchieSchnelltasteCodeTemplateObjekt <Kategorie>ZeichenketteFunktionalCASE <Informatik>Endliche ModelltheorieFormale SpracheMAPMultiplikationsoperatorPlotterTypentheorieGesetz <Physik>BeobachtungsstudieKommunikationsprotokollSchlüsselverwaltungJSON
14:47
DatenmodellMakrobefehlCompilerMAPTemplateTypentheorieLeistung <Physik>CodeEndliche ModelltheorieMAPCompilerMakrobefehlParallele SchnittstelleGenerizitätÜbersetzerbauComputeranimation
15:27
CompilerCompilerCodeDatenflussNeuroinformatikEgo-ShooterFunktionalUniformer RaumMultiplikationsoperatorShader <Informatik>VektorraumMatrizenrechnungSchreiben <Datenverarbeitung>Folge <Mathematik>Endliche ModelltheorieObjekt <Kategorie>MathematikAusnahmebehandlungSoftware Development KitSoftwaretestMessage-PassingHalbleiterspeicherComputeranimationJSONXML
16:24
CodeBefehlsprozessorMakrobefehlPhysikalisches SystemBitShader <Informatik>Komplex <Algebra>MakrobefehlCodeNim-SpielMathematikDeklarative ProgrammierspracheZeichenketteSystemaufrufSelbstrepräsentationPoisson-KlammerFunktionalÜbersetzer <Informatik>Gemeinsamer SpeicherKoroutineDifferenteExistenzsatzSoftwaretestKontextbezogenes SystemGraphfärbungATMKomponententestBefehlsprozessorRechenwerkSchreiben <Datenverarbeitung>MultiplikationsoperatorAssoziativgesetzFormale SpracheBitrateTwitter <Softwareplattform>Funktion <Mathematik>Ego-ShooterArithmetischer AusdruckProgrammfehlerBildschirmmaskeRechter WinkeleCosBefehl <Informatik>Codierung <Programmierung>Grundsätze ordnungsmäßiger Datenverarbeitung
18:33
ComputerBefehlsprozessorATMFunktionalVersionsverwaltungCodeMathematikFunktion <Mathematik>Ego-ShooterMultiplikationsoperatorVektorraumSystemaufrufImmersion <Topologie>Shader <Informatik>NeuroinformatikHardwareVererbungshierarchieWasserdampftafelBeanspruchungTopologieGüte der AnpassungFolge <Mathematik>DateiformatATMBetriebsmittelverwaltungBildschirmmaskeRechter WinkelPlastikkarteFormale SpracheJSONXMLComputeranimation
19:45
SchnelltasteQuick-SortVideo GenieFormale SpracheCodeStützpunkt <Mathematik>Güte der AnpassungMultiplikationsoperatorGraphikbibliothekUltraviolett-PhotoelektronenspektroskopieComputeranimation
20:30
ÄhnlichkeitsgeometrieFormale SpracheObjekt <Kategorie>LoopKonstanteNichtlinearer OperatorOverhead <Kommunikationstechnik>KreisbogenLokales MinimumSyntaktische AnalyseVektorraumVektor <Datentyp>BildschirmmaskeThumbnailCASE <Informatik>Snake <Bildverarbeitung>Übersetzer <Informatik>Rechter WinkelKonstruktor <Informatik>StereometrieSchnelltasteFormale SpracheSpeicherbereinigungBildgebendes VerfahrenCall CenterKonstanteMakrobefehlObjekt <Kategorie>GraphfärbungInformationFunktionalAlgorithmische ProgrammierspracheElektronische PublikationZeichenketteNichtlinearer OperatorMereologieTypentheorieOrdnung <Mathematik>VersionsverwaltungSyntaktische AnalyseEndliche ModelltheorieParametersystemDefaultMatrizenrechnungDatensatzAggregatzustandEinsNamensraumNormalvektorTermInstantiierungProdukt <Mathematik>Proxy ServerFigurierte ZahlGarbentheorieSchwach besetzte MatrixZahlenbereichFolge <Mathematik>Prozess <Informatik>TexteditorHill-DifferentialgleichungMultiplikationSuite <Programmpaket>Keller <Informatik>Bridge <Kommunikationstechnik>UmwandlungsenthalpieUniformer RaumGenerator <Informatik>HilfesystemInformationsüberlastungDivisionMultipliziererMailing-ListeAbzählenDifferenzkernObjektorientierte ProgrammierspracheSystemaufrufDifferenteOverloading <Informatik>MultiplikationsoperatorInverser LimesGeradeE-MailGanze ZahlNim-SpielDatenfeldNEC IC Microcomputer Systems Ltd.MathematikGrenzschichtablösungBimodulSkalarproduktComputeranimation
28:45
MakrobefehlATMTypentheorieKonstanteCodeFunktion <Mathematik>SpezialrechnerSynchronisierungTaskMAPTemplateLokales MinimumTypentheorieVersionsverwaltungKonstantePhysikalischer EffektBitBildschirmfensterElektronische PublikationSchnelltasteFunktionalInterface <Schaltung>Funktion <Mathematik>GenerizitätMakrobefehlSynchronisierungTaskMAPFormale SpracheNim-SpielVideo GenieOrdnung <Mathematik>CodeTemplateLeistung <Physik>MetaprogrammierungProjektive EbeneMinkowski-MetrikKomplex <Algebra>TermSoftwaretestFehlermeldungMeterProgrammierungZahlenbereichFamilie <Mathematik>Rechter WinkelComputeranimation
31:21
System FBildschirmmaskeSchreib-Lese-KopfComputeranimationBesprechung/Interview
32:06
TabelleFolge <Mathematik>Data MiningCodecDifferenteBildgebendes VerfahrenProgrammierungSuite <Programmpaket>Mehrschichten-PerzeptronBesprechung/Interview
33:23
MAPWort <Informatik>Formale SpracheObjekt <Kategorie>FunktionalSchnelltasteVideo GenieBesprechung/Interview
33:57
Objekt <Kategorie>Produkt <Mathematik>BitHypermediaTypentheoriePhysikalischer EffektMailing-ListeBesprechung/Interview
34:39
GamecontrollerGenerator <Informatik>SchnelltasteFormale SpracheBesprechung/Interview
35:16
ProgrammierspracheÜbersetzer <Informatik>Formale SpracheRechter WinkelSchnelltasteGruppenoperationVollständigkeitSuite <Programmpaket>DickePunktBildgebendes VerfahrenBesprechung/Interview
36:27
Formale SpracheSchnelltasteSuite <Programmpaket>VerschlingungNim-SpielGüte der AnpassungCoxeter-GruppeEinsGenerizitätBitVektorraumBesprechung/Interview
37:08
FehlermeldungMultiplikationsoperatorDelisches ProblemVektorraumRechenbuchTypentheorieGenerizitätMathematikComputerspielTeilbarkeitBesprechung/Interview
38:24
Rechter WinkelChatten <Kommunikation>Multiplikationsoperator
39:30
Computeranimation
Transkript: Englisch(automatisch erzeugt)
00:06
Hello, welcome to my talk, NIM Metaprogramming in the real world. I'm Andre van Hock, you can find me on GitHub as treeform. Let's get into it. Why use metaprogramming? This is the question I want to answer with this talk and I want to give you real
00:24
world examples from some of the libraries that I have written. Hopefully by the end of this talk you'll know why and how to use metaprogramming in NIM. First, I want to get across this idea of metaprogramming power levels.
00:41
And the lowest end we have generics, which sometimes isn't even considered metaprogramming, but it is a way to get interesting functionality. Then you have templates, which is basically like code substitution, like copy pasting code. And then finally we have macros, which basically can do almost anything.
01:01
But they're kind of hard to write and they put you into the chair of a compiler writer. It is always a good idea to shoot for the most minimum metaprogramming power level you have. So if the problem at hand can be solved in generic, you should just use generics. If it requires templates, use templates.
01:21
And if you need to use macros, use macros. But don't jump on macros right away when a simple problem can be solved in a simpler way. Okay, let's look at some examples. Let's look at level one, generics. Generics allow you to compose different types.
01:41
In a type language, you almost can't do anything without generics. For example, we have a sequence. If sequence didn't have generics on it, it would be very cumbersome to use. For example, we have to create sequence of integers, sequence of strings, sequence of some other format, and that's like just a lot of sequences and a bunch of repetitive
02:05
code. That's why generics exist to get us out of this problem, right? Just create a sequence with a generic T, and then you can create sequences of a bunch of types, including sequence of integers, sequence of string, sequence of float. It's really easy to use then.
02:21
In NIMM, generics, I think, are really powerful, and you can use them for some very interesting results. Let's look at V math. V math is one of my libraries. It's a vector graphics math library for NIMM that aims to follow GLSL shader naming conventions and be very fast.
02:42
Why GLSL? There's a lot of documentation and examples on how to use GLSL shader languages, the accompanying vectors and accompanying matrices. V math tries to just follow those conventions so that there's less surprises and less things to learn.
03:00
You can find V math on my GitHub at treeform slash V math. Here's how I use generics in V math. I have a generic vector of size three, and you can see it's an arbitrary type T, and then I have X, Y, and Z component be of that type. Using this generic vector, I can create more concrete vectors that you would expect from
03:22
GLSL, for instance, the binary vector, BVAC3, as well as an integer vector, IVAC, as well as an unsigned integer vector, as well as a normal vector three that a lot of people use with GLSL, and then the D vector, the float64 version of the normal float vector.
03:41
Using that generic gives me a lot of power. Here's an example, length. Now, I can just write a single length function, and that will work with any type of vector, with integer vectors and with float vectors, float32, float64, it doesn't matter. I write the function once.
04:01
If the vectors weren't generics, I'll have to rewrite this function over and over and over again for each type, and while doing that, I'll probably make some mistakes, or it's just hard to look at and hard to reason about. So, this is why I use generics, and all of the functions in VMATH are some sort of generic version of the general vector or general matrix.
04:25
Generics don't cost any performance, they get compiled in and are very fast. This makes VMATH fast as well. Some other things we do is we have a bunch of benchmarking tests and we profile the VMATH library pretty often. If you find a faster vector math library for NIMH, send me a message.
04:46
Next library I want to talk about is JSONy. JSONy is a fast JSON library with hooks. The idea when writing this library is that you want to turn the JSON you have into the objects you want.
05:02
The idea here is that in a while JSON can be in a variety of formats, and you also have objects that you like, that is structured the way you think. JSONy allows you to map those while JSON into the objects that you want without an intermediate step by using hooks.
05:22
You can find JSONy at treeform.jsony on GitHub. JSONy has a very simple API. You define your object, say an account object with name, score, and account ID, and then you have some JSON, which basically has the same thing, account, name, and score, and then you just call from JSON on that string, and then you get the account back.
05:44
If you want JSON back, you can just, you know, on the account, call toJSON. It does some nice things, for instance, it does the name conversions for you. In JSON it is very common to use snakecase with the underscores, while in NIMH it's very
06:01
common to use camelcase with the capital letters. JSONy just translates those different naming conventions automatically. The other thing that JSONy does is if there's any values that are missing, they just get default values in NIMH. Again, use the JSON you have, not some theoretically clean JSON.
06:22
Interesting thing about JSONy is it's all written using generics with polymorphism. There's no macros, there is no templates. It's so generic, and it's all built using parse hooks. This makes it very, very extendable. Here's an example.
06:40
It's very composable. Very often in JSON, you encode dates as strings. And here's an example of date being the string. With a parse hook, you can actually turn it into a real date time object. You just do parse hook, you first parse it as a string, and then you just convert it automatically to date time.
07:00
Now, wherever a date like this appears, and wherever you need a date time, JSONy will use this hook to auto-convert. It's very nice for this reason. Likewise, you can create a dump hook, which takes a date time that you have in your object and converts it into JSON by first formatting it and then dumping it as a string.
07:23
Again, if you have any date times in your objects, converting them to JSON that's just too JSONy is very easy. And the cool thing is these dump hooks are very composable for any type you have, for almost any situations that you might run into. So this is what makes JSONy powerful.
07:41
And this is also what makes NIMS generics pretty powerful as well. Again, generics are compile time code. They don't hurt performance like you would have in a dynamic languages or with dynamic dispatch. This keeps JSONy very fast. Other reasons why JSONy is fast is it does not need to create an intermediate JSON nodes.
08:04
There is no step which just parses everything, creates a bunch of nodes and then converts it to objects. No, JSONy converts the JSON as it's parsing it directly into the objects that you've supplied it. This saves a ton of memory allocations. And as we know, memory allocations are pretty slow.
08:22
So that's why JSONy is fast. Let's look at another level, level two templates. Templates enable code substitution. They're pretty simple to understand. It's almost like copy paste, but they can be used for very cool things.
08:40
Let's look at them. Here's a great example for why I love using JSONy for. So you have kind of like a little DSL languages to create either UIs or HTML. Here I'm just giving a very toy example. So at first we have the rectangle body, and then we create a rectangle inside it called button,
09:01
and then on button we create an on click handler called click. And that just echoes some text, hey, you clicked. Pretty simple. With the templates on the bottom, you can see I create a rectangle and it takes a name, well, ID in this case, and then it takes a body, which is untyped. Untyped is a special type reserved for templates. What it does, it basically takes whatever you pass that
09:22
and will substitute the body inside the actual template. As the template starts, we have a push rectangle, which means we do some sort of initialization, and then once we're done, we'd pop a rectangle and do some decentralization. Who knows what else? The interesting thing is these templates, they nest really nicely.
09:42
And then you have another template called the click handler. Again, it just takes a body and that body goes inside the if statement. Just if mouse overlaps the current rectangle, you do something, and this is pretty cool. And this actually just turns into simple NIM code. We have a push rectangle,
10:00
then we have another push rectangle, then we have the if statement, and they have pop and another pop. Doing this in other languages might involve closures or callback handlers. None of that is needed in NIM. With templates, you can write pretty clean, pretty clear code with just a few lines. And I think templates are great for these kind of small DSL languages.
10:21
DSL stands for domain-specific languages, languages that are only basically live inside your program and don't have to go outside. Let's look at VMATH. Again, remember the VMATH is one of my vector graphics math libraries that is very fast. It's very error prone to write repetitive code.
10:42
And this is where templates win. Well, they do copy paste, you know, after all. So here we have a bunch of procedures that we need to define for all the vector types I have. A bunch of additions, a bunch of simplification, multiplications, divisions. Like this will just go on and on, and I'll probably make a bunch of copy paste errors. And when something needs to be changed,
11:01
again, copy paste errors, repetitive code, very hard to look at. You can fix that really easily with a very simple template. For example, here I have a very simple template that generates code, it's called gen operator. What do you do is you pass it an operator and then it generates an operator for all the different vector types. And then all I have to do is just call gen operator
11:22
on all the operators I have. It's a lot less typing, you'll make a lot less errors, but it is more complex. I don't recommend using templates to save a few lines. If you have a lot of lines to save, it's great, but they do introduce extra complexity, extra something you have to understand and debug.
11:41
So I only recommend templates when you can actually save those lines of code, when it actually makes sense. Let's look at another cool library I made called Windy. Windy is a windowing library for NIM that uses OS native IPIs to manage windows, as well as set up OpenGL context
12:01
and receive input events, mouse and keyboard. It's very similar to GLF, W or SDL. You can find it on treeform slash windy. The idea here is I wanted to fix a couple of bugs in GLF, W as I found, but it's very big project and it's very hard to understand why,
12:21
because it's not written in NIM. GLF, W and SDL both kind of have not exactly NIM APIs to them. I ended up writing almost like a layer on top of GLW to be more NIM-like. And then I said, why do I have this layer where I can just go to the operating system directly and understand directly how operating system work.
12:41
And I can get some other features which GLF, W didn't want to support. For instance, IME for Chinese, Japanese and Korean language input. Here's an example of using templates in Windy. You can see I'm using here to generate some Objective-C classes that are needed to talk to the Mac API. We use a low-level Objective-C interface from C.
13:03
It takes a lot of code to generate these objects normally, but we have created a template that is very simple. Just add class, the class you want, the superclass and then whatever NIM class you want this to bind to. It's the same thing for methods. You do add methods, the string method in Objective-C.
13:21
As you know, Objective-C method names are kind of strange. And then the NIM functions that you want to bind this to. We just create this object in NIM that looks almost exactly like an Objective-C class would look like. We just create a bunch of them and it saves a lot of code in Windy. This is how that template looks like.
13:41
You can see it starts out with a template, add class. You pass it the two class names and then other things and the body that you're going to use. We call low-level Objective-C things like Objective-C allocate class pair and other methods that are needed to make this work. Then we actually have templates inside the template. The cool thing here is that add protocol and add method,
14:02
they don't exist outside of add class. If you try to use them outside, it'll just not work. But inside the body, you can use them. And that's another use case for domain-specific language, kind of cool. Again, we have a methods to add the protocol, which Objective-C classes have, as well as add method,
14:21
which is another class add method low-level Objective-C call that's kind of strange looking. But after that, we have the body. That's where we get all that other code comes in here. And then also those add methods just get turned into these calls. And then in the very end, we register the class pair with the Objective-C runtime.
14:40
Again, templates saved us a lot of repetitive code and it is just so much easier to use. Templates are awesome because another model they have is very simple. It's basically, if you can do copy paste, you can do templates. Yes, there's some weirdness and some typings you have to do, but otherwise templates, I think, are great.
15:01
I highly recommend when you have a bunch of code to use templates. Again, remember the power levels. If you can solve your problem and judge generics, do it. Don't go for templates. But if you can use templates to solve it, without going to macros, use templates. Finally, we are at level three, macros. Macros makes you basically a compiler writer.
15:22
They can do almost anything in NIM as I'm going to show here. I want to introduce my library, Shady. Shady is a NIM to GLSL compiler. Again, you can find it on my GitHub, treeform slash Shady. Shady allows you to write cool shaders like this one here.
15:42
This is computed by Shader. It creates a very cool looking flare. Here's an example of the flare function. You can see it is just pure NIM code, maybe exception of that uniform over there for the time, but that just turns into normal float32. No problem there. You can see almost all the code is pretty math heavy inside it.
16:01
It was all using the V math vectors and matrices. It's pretty cool and it allows you to basically write shaders in NIM. There's something that you cannot do with this thing is to allocate memory. So using sequences or ref objects is basically not allowed in this model.
16:21
So you have to find workarounds. This is how easy it is to use. We created that flare thing and we can easily test it by passing it a test color, a vector, and then a time, and then we get a test color back. We actually run that flare on the CPU. There were no shaders involved. It just runs, but you can also compile to shader by calling to GLSL macro.
16:43
And it just returns you the string representation of the function that you can pass to the rest of your system to initialize the shaders. GPU shaders are really cool and bringing NIM to GPU shaders is also pretty awesome. But shaders are very hard to debug
17:01
in their GLSL form, right? There's like no echo statements. And yeah, it's just hard. Writing my shaders in Shady allowed me to put all kinds of echoes and see everything that was going on. It's actually was really nice. Shaders are really hard to test as well. I've never seen a GLSL unit test, but with NIM you can just write unit tests
17:22
for your shaders. It's pretty neat because you just run them in the CPU mode at the time of the test. Another problem with GLSL shaders is you can't share them with the rest of the code. You've written a cool math routine and you want it to be available for both shader and the CPU. And now you can't because they're in different languages, right?
17:42
But now my problem doesn't exist. Another cool thing that I found is there's no need to context switch. When I was switching from writing shaders to write a NIM code, I would constantly put on semicolons, I'll put the variable declaration in the wrong place, but that's gone. I can just use normal NIM now. It's like much easier to write the shader code.
18:03
Shady is just a very large macro. To GLSL just takes a NIM node. That's what macros take. And then it goes and it's like, what kind of NIM node you are? There's all kinds. There's like an assignment. There's an influx, which is a plus or minus. And there's a call. Is it a dot expression? Is it a bracket expression?
18:21
Like I just look at the NIM node and then I generate the appropriate GLSL translation. You know, it takes a lot of work, but each little bit isn't that complex. And this is what the output of that flare function that we saw. You see, it outputs it as main because that's like the main, that is what was a requirement by a GLSL.
18:42
And then you can see we have a vector, sequels, you know, a vector, and then a bunch of math code. It's the same, exact same code as in the NIM version. It just slightly translated and actually doesn't look that bad. I highly recommend this method of writing shaders. It has saved me so much time. Shader allows you to write compute shaders as well.
19:03
Compute shaders are really fast and super cool, right? For some workloads, the compute shaders just blow the CPU out of the waters. Of course, not all workloads are good for the GPU. For instance, if you're doing lots of allocations or like tree walking, it's not as good.
19:20
But if you just have a bunch of data in sequential format and you want to crunch through it, it's amazing, right? Another cool thing about writing your compute shaders in Shady is that you can just, if, for instance, you're running it on Mac hardware where the OpenGL compute shaders are not supported, you can just choose to run them in the CPU mode. Yep, it is slower, but it does run.
19:42
So I don't know, it's pretty cool. Another one of my libraries I want to talk about is Geni. Geni generates a shared library and bindings for many languages, sort of like SWIG, but for NIM. Please visit it at treeform.geni on GitHub.
20:00
The general idea behind Geni is that I believe NIM can get a wider adoption if it's easy to use with other languages. I want these NIM libraries to infiltrate themselves into code bases of larger teams that use many languages and then slowly grow there over time. If a team uses NIM for a small thing, they can go, hey, we already used NIM.
20:21
Let's use NIM for something else. And I think that's a good way for NIM to infiltrate big companies, startups, and other teams. Before looking at Geni some more, I also want to introduce Pixi. Pixi is our to-do graphics library. It's written in NIM. It is similar to Cairo or Skia.
20:42
It can do all kinds of vector graphics, which is really cool. You can find it at GitHub, treeform.pixi. I originally started Pixi as a way to learn how Cairo or Skia work, and I've ended up writing a full-fledged library. I think Pixi library is pretty cool. I want other people to use it.
21:01
I want people to use it from C or use it from Python or use it from Node.js or use it from Ruby. I want my Pixi to be available in a bunch of places. How to do that? What do I need to do to make this possible? I think a high quality bindings as a shared library is the way to do this.
21:22
Jenny's goal is to generate the best possible binding for the target language that we support. What do we need to do to achieve that goal? Here's a bunch of Jenny ideas. First off, I think the naming convention that the Jenny language should be familiar in the target language. If the language uses camel case like NIM does,
21:42
the name should be relatively unchanged. If the language uses snake case, for instance, like Python or C, then yeah, we should rename everything to snake case as we export that. If the language just uses something weird, like kebab case, mostly Lisp languages, yeah, it should convert to that. It should feel natural in the language where it is used.
22:03
Some more of Jenny's ideas is that make the exportable objects behave like object-oriented objects with members, method and constructors. Many of the popular languages are object-oriented while NIM isn't particularly object-oriented. It does the uniform call syntax, but it should behave if we export it to another language,
22:24
just like another object. It should just feel natural in that language. In other things that would make it feel natural if we pass optional arguments as well as enums and other constants, like if that just works normally, how it usually works in the language, I think that would be great.
22:41
Some other ideas is to generate helper methods like equality or as note checks, you know, overload math operators, plus, minus, multiplication divisions, all that should be handled if the language support it. For example, C doesn't support these operations while Python does, JavaScript doesn't while Ruby does, those kind of things need to be tailored for a language.
23:02
Another thing is I want to export sequences just like normal sequences, like you would use in a language with the bracket syntax. It needs to feel natural to a native array. It is possible, for instance, to do that in Python and Ruby and less so in C and JavaScript, but yeah, you have to meet where the language is, kind of make it nice to use,
23:22
basically make it so you don't even feel like you are not using the native language. Some other more complicated thing is that we need to synchronize the bindings between the language garbage collector and NIMS-ARC garbage collector. So for example, if you have an object in JavaScript that holds a reference to one of NIMS objects,
23:43
if that object gets delegated by the GC, it needs to also de-locate NIMS object as well, transparently, that needs to work. And we even do some nice things like we copy the comments to the bindings so that other tools can use them, like the editors or the IDs,
24:01
they can just show the comments as well as doc generators, et cetera. Again, the library must feel natural in that language. So what does Jenny look like? How would you use Jenny to expose pixie? Here's how. You would import both Jenny and pixie in your bindings and you would go and you would use a macro,
24:20
export constants, right? And it would be take a bunch of constants from pixie, like default miter limit or auto line height, those constants just exist in pixie, but now we'll export them through Jenny, right? Those constants don't exist in like in the shared library, they are exist in kind of like a header file for each language. Then we have other things to export like enums.
24:40
Enums don't behave the same in different languages like C prefers enums as constants. All the languages do different things. So when we export the enums, they just feel natural in that language. Then we have another macro that exports procs. Again, you just export that proc, you do read images, one of our pixie procs, read mask, another pixie proc, read typeface, right?
25:01
You just like stick them in there and then Jenny will figure out how to export them correctly for the target language. Next, we have another section called export object, which is export simple non-ref, allocated on the stack object. So here we have a matrix three, has a constructor, a bunch of languages have a special place
25:22
for the constructor. Nim doesn't really, but a bunch of languages do. So we have to honor that, we mark a constructor. Then we have a bunch of procedures. Here we have a multiply matrix by matrix. This will actually get turned to a multiplication symbol. If the language support that, it will be operator overloaded. For instance, in Python,
25:41
it will be multiplication, in Ruby as well, but in languages like C or JavaScript, now it will be a mole function that you call on the matrix, just like it is natural in that language to use. And then finally, for the most complicated macro, we have export ref object that exports an image. It exports it by our reference, so not on the stack.
26:00
So a bunch of access like field access requires special handling. But again, we tried to make it feel very natural. If you create an image in Python, you can just do dot width, dot height on it. But in Nim, it gets translated to calling the thing and then going through like this bridge and then returning the width back. It's really cool.
26:20
Again, we mark a constructor. Here's something interesting. We're actually marking a specific constructor that takes an integer and an integer, part of a macro scan to scan that type information and find the correct function. Then we export all the procedures that will get attached onto the image. So we have the right file. As you know, Nim has the right file where you'll write the string data to the string file name.
26:42
We also have our own write file that writes images, but you wanna attach that to the image objects in whatever language we are working with. The same thing with copy. Pixie has many copy methods, but here's the one specifically for the image. And then we get couple regular method to get color, set color. Here's an example how would you draw a heart
27:01
in Nim using Pixie. You have an image fill and then you have the SVG path. Then you parse the correct color and then you send it in. And that's how you would fill that using Nim. Here's how you would draw a heart in Pixie Python. This is our Python bindings to Pixie. There's several different things here. First off, Python prefers module names to be present.
27:22
So we have pixie, the module name, dot. Then we have the parse path, which is the, you know, it is different from Nim because Nim was in camel case, but this is a snake case. You've kind of translated the path name to something that looks like, that looks normal in Python, right? Then we, again, we have pixie.paint,
27:40
which uses a constructor. Then it uses a constant pixie pk solid color. Then we have, then we set the color again using the normal Python stuff. And then we fill image using the fill image path with our path and paint, simple. Here's how you draw a heart in pixie C.
28:00
So here you can see that we have, C requires a lot more typing. So we create a path, in C doesn't really have namespaces. So we're using function prefixes like it's very normal to do in C. So we have pixie parse path again in snake case. Then we have, we create a paint. Again, because C doesn't have constructors, but we have a function pixie new paint.
28:22
It's very normal to do that in C. Then we have pixie paint set color, again with a parse color. Then we have pixie image fill path, very similar to the other ones we saw, except in this one takes a bunch of extra parameters. Why? Because C doesn't have optional parameters. So we have to specify them. So we have to specify a default matrix
28:41
and the winding order. Again, Geni is just set of large macros. What do those macros do? They walk the Nim AST node once, actually walks it once as untyped in order to produce that nice language. It creates a type version and then it walks them again
29:01
after typing with another macro. So we get the types back. You know, it gathers functions, it types and constants and enums, like it just gathers all that. And then it generates the Nim glue code as well. Cause remember you have to have a little bit glue code to access a Nim. And then it generates the kind of glue code on the other side in the language that it's for.
29:21
Here's an example of what Geni output. What does it give you? It gives you a pixie DLL for windows. Then it gives you a pixie SL for Linux and then pixie DLL for Mac, right? And then you have the individual binding files. So we give you the pixie Pi, which actually uses C types to access the DLL. And then we give you the pixie H, which is needed for C to bind to those DLL.
29:43
And then we also give you pixie JS, which uses the foreign function interface to connect to that. And then we give you pixie RB, which is the Ruby version of foreign function interface again. And so you can use pixie in all these languages. Imagine trying to maintain all these bindings manually without Geni. This task is very repetitive and error prone
30:02
and extremely boring. The bindings would need to be kept in sync and they need to stay up to date. It's like a lot of like busy work that no one wants to do. This exactly where the power of the macro comes in. You can create amazing things with macros. And you know, Geni is just one of those examples.
30:23
Again, I want to remind you about the meta programming power levels. You have generics on like kind of least powerful and then you have templates and macros. Remember, they're really cool. They have each of their uses and you know, use the most minimal power that you need for your project.
30:40
And don't go overboard. Make sure the macros and the templates actually save you space and complexity rather than just introduce just another thing for someone to understand. Hopefully during this talk, I've answered the question, why should you use meta programming? You know, through real world examples. Give you example, how I use meta programming in VMath,
31:02
in Jsony, in Shady and in Geni, how the power levels work. Thank you for listening to my talk. If you have any questions, please feel free to post them in chat or contact me directly through email or on my GitHub. Thank you very much.
31:39
Okay.
31:41
So that was it. The final talk for today. So I'm joined here by Treeform. Hey guys. Speaker of this fantastic talk. Sorry, I have my, I had it running in the background.
32:01
I started hearing myself. Yeah, it's disorienting. Yeah. So yeah, unfortunately we had some technical issues for the last talk, but this one seems to be working fine. So we have gotten a couple of talk or a couple of questions here.
32:23
First, do you want to say something about the why post fix? A lot of people seem to have picked up on that. Oh, the why post fix. So the, like at first I was just naming my libraries, you know, something, some name, and then some of them just happened to end on Y.
32:42
And then it's like, oh, you know, it kind of is a brand now. I should add more libraries on Y. And so, yeah, that's how it started. I think a bunch of the NIMM libraries ends on S. So if you do the standard NIMM libraries, they all like S of things like tables and, you know, there's the sequences and there's like a bunch of them in an S
33:01
and it's like, oh, you know, I started naming mine on S as well. And then there were like conflicting, but they have a libraries because like a standard thing is just a plural. And then I said, oh, I should have like a brand. And that's when the why thing got born. So, yeah. So now I think when you see librarians on Y, you kind of think it's one of mine, which I think is cool.
33:21
Yeah. And yeah, I mean, the talk, this talk was more or less a talk about the different ways of doing metaprogramming, but a lot of the questions we have gotten in here seem to be more about your specific libraries. So for example, one here is about when you're genying,
33:44
is that an official word? I don't know. I think I don't say that. I just said, you know, using geny or generating this geny is short for generating, right? But yeah, when genying bindings for an object oriented language, how do you tell whether a product should be turned
34:03
into methods or standalone function? I guess you touched a bit on that in the talk, Paul. Yeah, a little bit. So it is true. I don't think it's possible to automatically figure out how to like create an object out of NIM functions, right? That's why we made the DSL, right? Like you kind of described the objects
34:21
that you want to export. It's not automatic, but I think it's pretty, pretty, pretty good. Cause sometimes you don't want some things exported and you want some other things exported. So I think it's important to like list exactly where you want stuff to go and what types you want them to be. I mean, it definitely makes, it definitely makes it easier for the library
34:43
to do it right. And I think it's, I think it's better to have to have a good correct bindings than doing it automatically and getting it wrong. Yeah, you want control over the binding generations. Yeah, exactly.
35:01
Another question here. I see now Node.js is supported in Jenny. This is new with respect to the latest release, correct? And he asks, what is the next language going to be? All right, yes. So our primary Jenny target is Python because, well, I feel like a lot of Python
35:21
could use things like this. Yeah, the second one will be JavaScript because those are like arguably two of the biggest programming languages. So with Node.js binding is very kind of a crossroads. So before that we were using the JSON FFI library to connect similar to C types. So it required no extra compilation with respect to JavaScript,
35:41
but we actually think that is not the correct way to buy into JavaScript. And we want to generate like the C bindings and those will have to be, you know, generated per release. Right now, the FFI works for many releases, so it almost doesn't matter which release we support. But we're probably going to change that model soon. So that's what's happening with Node.js.
36:02
The same thing might happen to other languages as well, but our top languages so far are Python, C, JavaScript, and Ruby. So I guess what's the next language going to be? It's probably going to be Ruby because it's almost done. After that, it's less clear what's going to be next. It's probably something along like Java or C sharp
36:22
or something like that to be next. Yeah, I guess you really, you want to either bind to a language that doesn't have the same performance or which doesn't have the same ergonomics. By need to- Yeah, the whole idea of the JN is to get NIM more out there.
36:42
So I think it's like binding to C or C++, I think is a good thing because many people use those languages. I just want to get NIM out there, NIM used more in the real world. Yeah, that's definitely something we should do more of. Yeah, I guess most of the other ones
37:01
you pretty much answered during the presentation. Yeah, I want to touch a little bit maybe on the vector one, because I think it is about criticism. Why aren't vector sizes also generic? I thought when I did that, so originally the math was much simpler. It was just, it had to use no generics.
37:21
And the reason why I did that is because when you use generics, it is more complicated. You do get more error messages. So for like half its lifetime, the math actually used no generics. It was just floats and it was just vector one, two, three, and four, right? But as time went on, I actually needed the other types.
37:40
So I kind of said, yeah, all right, we have to switch to generics because it's weird. Because I had like another almost clone of e-math called D-e-math for double vectors to do astronomical calculations. And it's like, you know, it'll just be much easier if it was just a, if the type was generic. And so, but I didn't still want to give up the simplicity of, you know,
38:02
just having the one, two, and then three vectors. Because I think you do get weird errors, even with generics currently, you do get weird errors if you get some types wrong. What's going on here? I would just hate to get even more weird errors saying like, you know, the length aren't correct and like, you know, like the type isn't correct. The error is probably going to be horrendous.
38:23
Yeah, yeah, for sure. All right, so someone, right. Yeah, that was really, really related to this one. I guess we should answer him in the other chat. But I think, I guess that's all we had time for,
38:42
for the official Q and A. So if people want to get more, you can join this room that we're in right now after the talk ends.