Evolving API design in Rust
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 15 | |
| Author | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/52169 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
2
3
4
6
10
13
14
15
00:00
Hydraulic jumpComputer-generated imageryLine (geometry)CodeLibrary (computing)Function (mathematics)Keyboard shortcutEvent horizonStructural loadTexture mappingQuadrilateralStrutContext awarenessPoint (geometry)Scale (map)Vector spaceRotationPoint (geometry)BuildingVideo gameFormal languageCompilation albumQuadrilateralMereologyData structureParameter (computer programming)Default (computer science)Functional (mathematics)Fiber bundle1 (number)QuicksortObject (grammar)State of matterContext awarenessOpen sourceGame theoryInterpreter (computing)Revision controlStructural load2 (number)Information overloadExpressionTexture mappingGoodness of fitBitProjective planePlanningPerspective (visual)Inheritance (object-oriented programming)Fundamental theorem of algebraTable (information)MathematicsSoftware frameworkWritingSlide ruleSoftware maintenanceSoftware bugPower (physics)Metropolitan area networkDrop (liquid)CASE <Informatik>Electronic mailing listRotationBit rateProper mapArtificial neural networkSet (mathematics)Traffic reportingTransformation (genetics)JSONXMLComputer animation
06:44
StrutContext awarenessPoint (geometry)Scale (map)Vector spaceRotationDefault (computer science)Data conversionLink (knot theory)Type theoryLibrary (computing)Game theoryArtistic renderingVideoconferencingWeb browserStructural loadComputer-generated imageryEvent horizonInclusion mapKolmogorov complexityMereologySynchronizationCloningHash functionSicNormed vector spaceScalar fieldDigital photographyData modelSet (mathematics)Collision detectionPredictionStatisticsDisintegrationParameter (computer programming)Matrix (mathematics)Different (Kate Ryan album)Disk read-and-write headMathematicsSoftware frameworkPhysicalismMultiplication signCollisionHacker (term)Operator (mathematics)Programmer (hardware)PlanningParameter (computer programming)Loop (music)State of matterEvent horizonQuicksortFunctional (mathematics)WindowSystem callGoodness of fitOperating systemComplex (psychology)Game theoryMereologyTerm (mathematics)Computer configurationType theoryKeyboard shortcutContext awarenessSimilarity (geometry)Point (geometry)RotationPower (physics)Default (computer science)CASE <Informatik>Data structureLibrary (computing)Structural loadBit rateCodeGeneric programmingPerimeterEngineering physicsCuboidLevel (video gaming)SineIdeal (ethics)AlgebraNeuroinformatikArithmetic meanComputer animation
13:20
SicScalar fieldCloningMIDILocal GroupData structureSemigroupLoop (music)Source codeAbelian categoryAbelsche GruppeAlgebraic closureEquivalence relationAxiomInverse elementAssociative propertyIdentity managementCommutative propertyMatching (graph theory)Term (mathematics)CompilerMathematicsType theoryPhysical systemAlgebraSoftware bugFormal grammarGraphics processing unitError messageVector spaceScaling (geometry)State of matterCASE <Informatik>Level (video gaming)Multiplication signSource codeXMLComputer animation
14:24
Local GroupData structureSemigroupLoop (music)Source codeAbelian categoryAbelsche GruppeAlgebraic closureEquivalence relationAxiomInverse elementAssociative propertyIdentity managementCommutative propertyComputational complexity theoryLie groupInheritance (object-oriented programming)Fluid staticsString (computer science)CuboidFluid staticsWritingPoint (geometry)QuicksortFlow separationFile systemString (computer science)VirtualizationComputer fileType theoryNormal (geometry)CASE <Informatik>Structural loadGame theoryData storage deviceMedical imagingComputer fontConfiguration spaceOperating systemDirectory serviceComputer programmingAuthorizationMobile appMereologyProgram slicingObject (grammar)Java appletCompilerRevision controlUniform resource locatorInformationFormal grammarCase moddingMetadataParameter (computer programming)NP-hardMiniDiscPhysical systemCodeEndliche ModelltheorieGoodness of fitProcess (computing)Instance (computer science)Message passingComputing platformService (economics)Formal languageData structureMathematicsFunctional (mathematics)Table (information)1 (number)Decision theoryWindowECosSystem callMathematical singularityComputer animationXML
19:49
Dean numberFlagDew pointDecision theoryCASE <Informatik>Figurate numberAuthorizationTrailIteration
20:48
IterationMIDIComputational complexity theoryBit rateCurve fittingoutputCoefficient of determinationRevision controlResultantMultiplication signFunctional (mathematics)Disk read-and-write headType theoryCodeCASE <Informatik>Dressing (medical)WritingControl flowQuicksortEncapsulation (object-oriented programming)Computer animation
23:02
JSONXML
Transcript: English(auto-generated)
00:07
As she said, I am Simon Heath. I am talking about evolving API design in Rust. I have been using Rust pretty much since 1.0 and I'm interested in programming languages and I'm interested in compilers and I'm building infrastructure and making video games.
00:25
So I took these things and put them together and made a game engine because it's more fun than making a game called GGEZ and This is the first major Rust project that I had been working on and it's also the first major open-source project that I've worked on
00:46
and the goal is to make it easy to make 2D games because this was something I wanted to do. It's a good way to learn Rust and whenever someone says I need an idea for a project to learn Rust with, I can say you should make a game.
01:04
It is based on a Lua game framework called Love2D, which also has a similar goal of making 2D games easily, which is to say that I went through the Love2D API docs function by function and wrote everything in Rust.
01:21
Because I didn't really have a great plan for how to make a good 2D game engine, I just wanted something that was simple and would work and was easy to use. So I knew Love was simple and worked and I'd used it before, so there we go. GGEZ is actually used in a few games made by real people who are not me, which is awesome.
01:48
It's nothing, so far there's anything like super huge or complicated besides Xemiroth, which is this one which has been worked on since before I started working on Rust.
02:02
But hopefully someday, I'll actually get to write games in GGEZ as well. That's my goal for 2019. So to do all this stuff, GGEZ brings together a lot of other crates from the Rust ecosystem and has to take all of these libraries and make them play nice with each other and convince the ones that don't have cool logos to
02:23
make cool logos, so I can put them on the next slide. It has to take these crates and have them interoperate. It has to take whatever API they expose and be able to kind of wrap it up in a consistent way and make it easy to use for Rust newbies.
02:43
It has to actually be able to use all of these crates successfully, and so I have gotten to be very good friends with some of the maintainers of these crates because I would keep submitting bug reports, or I would keep saying I need to be able to do X, how do I do X? And usually they tell me and life is good.
03:02
But I wanted to do this talk because I also hang out on Reddit and IRC a lot and probably too much, and I keep seeing things like this. So everyone who first learns Rust and writes something big has to ask for advice on it and so it was it was weird because I don't see this a lot in like Python or
03:24
C-Sharp or whatever. Maybe the world would be a better place if people did do this in those languages, but either way people who learn Rust seem to have trouble figuring out how to write Rust APIs, or at least they have anxiety about writing Rust APIs. They keep asking how do I write idiomatic Rust? And so that's what I want to talk about.
03:44
So how do we design a good API in Rust? In my case, I didn't have to design an API. I just copied an API and made it rusty. So that was Let's take a look at the API. I copied it a little bit but I'm going to start with some GG-EZ examples
04:02
and then I'm going to look at some of the other crates that GG-EZ uses and how those sort of look from an end-user perspective and what's good about them and what's bad about them. So here is a very simple Love2D game. It's all in Lua. I don't know how many people out there know Lua, but we have some functions. We have
04:23
load, update, and draw, which are the sort of fundamental parts of your game. We have a global player, which is just a table dict essentially. That's my Python showing through. And we have Update detects if you are pressing buttons and changes the world state if you are and
04:42
Draw draws stuff. Great, and these are basically callbacks that are loaded by the Lua interpreter and Love basically has a version of the interpreter that is built with a bunch of libraries and looks for these callbacks and loads them and runs your game.
05:02
So this is pretty different from how Rust works. I mean, there's no curly braces at all. But we have these sort of magic callbacks that the interpreter looks for. It's dynamically typed. There's mutable state everywhere.
05:21
So it's kind of... I didn't even know if it was going to be possible to make this in Rust. I was like, well maybe. I mean look at just the Love2D draw function. We have four different overloads for the draw function. I like your expression responding to this. We have
05:43
some drawable objects, or you can replace the drawable object with a texture and a quad saying which part of it to draw. You can have a transform, which is a structure that basically bundles up all of the drawing parameters, or you can just list all the drawing parameters that are possible individually. And it also turns out that you can actually omit the ones at the end and they'll just default to
06:04
zero or one or whatever is appropriate. And so you can just leave all of those off and just have X and Y and R and it works fine. So I was kind of like looking at this and saying well, I just want to make something work. So worst case, I'll just make a separate function for each of these variants.
06:23
So I started with that and I sort of squished it together and got something like this. So we have a struct that has all the draw parameters in it that you can have. And we have a function that takes a drawable object and draw params and a context which like just holds on to the graphics context state and appears everywhere.
06:44
And it draws it based on whatever parameters you give it. And then well, okay we have a simplified function that just takes a destination point and a rotation and you can use that if that's all you need. And if you need the full power function, then you can use that instead. It's like, okay. Yeah, fine.
07:01
Eventually, I discovered that the default trait and the struct update syntax exist and you can do something like this. Which actually is halfway decent. It's not great. It's not terrible, but it works. It's not too pretty. I never really liked it. But something that I realized as time went on is that ggEZ is an opinionated framework.
07:23
And so lots of people have opinions about it whenever they try to use it. It's actually fairly low-level. It doesn't provide animations. It doesn't provide a physics engine. And so everyone says, oh, why don't you do it this way or why don't you add this or why don't you not add that?
07:40
But nobody in the last like two and a half years or whatever has actually complained about this horrible hack. It's not a problem. Like it actually has quite a few advantages. It's simple. Like even the most basic Rust programmer can understand it. It's completely obvious what's going on and like where the data is coming from, where it's going, and where it's used.
08:05
And with this nice syntax, it's even not too terrible. So it works. So the harder case was dealing with Love2D's sort of callback structure. And I ended up starting with something like this, where we had a trait called game state.
08:25
And I should have cut out all the inconvenient code, but it provides load, update, and draw methods that are just like Lua's or Love2D's load, update, and draw methods. And then down here you have this game struct that is generic on your
08:43
the type that you implemented the game state trait for. And it just creates your game state by calling the load method and then it has an event loop inside it that calls update and draw and takes keyboard events and all that stuff.
09:00
So this is the closest I could get to something that looked like Love2D, where it just had these magic callbacks that did everything for you. And it sucked. Nobody hated it, or nobody liked it. I didn't like it. I got tons of questions like, oh, how does the game state actually get created? Like where do I put the new method? How do I, like who owns it? It's owned by the game
09:21
type. How does it know what type to load? Well, it uses, you have to annotate it with that generic parameter. Like how do you, where does the context get created? Well, it gets created in the game, and like it was just complicated and nasty. Eventually I wanted to be able to even like take apart the event loop and let
09:43
the user sort of write their own with their own update functionality if they really wanted to, because Love2D does allow you to do that. So eventually I ended up with this, which is kind of similar. We have a event handler trait. It defines update and draw methods.
10:02
But the main, the game state is just a struct you create, and there's nothing special about it. You create a context which is sort of the handle to all the GTEZ library functions. Like it handles the sound state, the window state, it talks to the operating system, etc.
10:22
You have this event dot run or colon run function, which is literally what you would just write. It's a while loop that pulls the operating system for events, calls event handler update, and then calls event handler draw, and it just does that. Now we, by making, by trying to do less magic, everything becomes way better.
10:46
And so, and I didn't like, it was like this, this doesn't really look rusty. This is kind of dumb as a pile of bricks, but how do we design a good API in Rust? Everything, a good API is still a good API in Rust.
11:00
That was the sort of what I figured out. Love2D started off with a pretty good API, I turned it into Rust, and it's still a good API. Rust doesn't add or remove anything too magical from it. And when it down, keep it simple. I like the term complexity budget that the last talk used, because having the,
11:20
putting the, putting my complex budget into trying to make it look exactly like a Lua API wasn't worth the extra complexity. So, another problem that I deal, have often when people are trying to get into Rust game dev, is that some popular crates are very hairy.
11:41
So they say, well, what are my options for drawing graphics? And someone mentions GFXRS, which is awesome, and which Love2D uses, and they go to the docs and they see this. Now, I don't know about you, like, usually I call it quits after only like seven or eight associated types. Here we have 12.
12:01
Or they say, okay, well, how do I do matrix math? Well, look at an algebra, and they see this, like, this is, this is the matrix type. This is, what's even going on here? We, it's almost impossible to read. When I see stuff like this, I call it trait salad, because it's just a pile of different stuff all mixed up, and you can't make heads or tails of it.
12:20
But if you're, like, trying to learn Rust, and you come at this, and this is what you see, then you, obviously the creator is this guy, who either is just a mentat who is one with the computer, and it knows everything there is to know about everything, or he's just a sadist, and he likes torturing the users of his crates.
12:43
I mean, let's look at, I mean, we saw his math framework. Let's look at his physics engine. This actually looks kind of reasonable. I mean, it's just a bunch of methods. This is part of the collider type, I think, and it's pretty obvious, like, what's supposed to connect. So it's not that he's a sadist, and it's not that he is, like,
13:03
operating on some plane beyond human comprehension, so what's going, what's actually going on here? Like, we have a bunch of traits. They all have really complicated bounds, like, we have one here called abstract magma. I usually think of this when I see that, or maybe this.
13:24
However, it turns out that what I should have been thinking of is this. It's a math term, and so what an algebra is doing is teaching the Rust compiler how to do fundamental math by encoding it in the type system. That's not something that would have ever occurred to me to do,
13:43
but it also means that it completely rules out a lot of math errors at compile time. You can't, like, if you have a transform vector and a scale vector, you can't add them together. The type checker catches it. GFX is similar. It's doing something kind of like that, trying to encode the state of a graphics card in the type system.
14:04
Which is really complicated and low-level, but you end up, it ends up catching a lot of bugs. So, these aren't bad APIs. They're just very sophisticated and specific and low-level, and are geared towards a certain type of use case.
14:20
I've talked to people who have a formal math background and use an algebra, and they love it. Like, they don't see this magma, but they, all of this pops out at them, and they're like, oh, I know how everything fits together. So, the next lesson is, know your audience. Who do you want to be making these crates for?
14:43
What do they want to be doing, and how do you make what they want to do easy? If you're making something for yourself, and you have a background in math, then you end up with an algebra, and if you don't, then you end up with something else. So, also, as an audience, know your tools. It's a lot easier to understand why an API is the way it is, and
15:05
where to look when you need some functionality, when you know who the writer is making it for. Maybe it's not for you. Maybe it's something you can learn, but either way. So, we've discovered that API design in Rust is hard. It's hard in any language.
15:23
Rust seems to make it trickier, though. Why do people get such anxiety about it, and what can we do to make it better? Well, in Rust, API mis-features are actually really nasty. This is true in any language, like Java or Python or whatever, but Rust is good at making things subtly terrible in ways that aren't obvious to people who don't know Rust very well.
15:48
For instance, GGEZ has a sub-module for loading resources from file paths, and keeping the files in some platform-specific location that's different on Windows, on Mac OS X, on whatever.
16:03
So, it uses a crate to just ask the operating system what paths it should use, and the crate is called Aptors, and it works basically like this. You create an AppInfo struct, which has the name of the app and author, and the operating system has some
16:22
specific location based off of this information that it uses to store images or fonts or whatever. You just say, okay, get me the user config directory for whatever operating system I'm running on, and it gives you a path. That's it. So,
16:41
the API is this. There's more than this, but this is the part that GGEZ uses the most. So, we have a struct, and we have a method, and the struct has some static strings, and everything is good. So, I wanted at one point to write a program that loaded these paths from a config file or something, and so I had something like this. We have author and app name, which are our own strings, and we feed them into AppInfo,
17:07
which has static string slices, except these are our own strings, but these are static string slices. You can't get there from here. How do you do it? Well, you can probably figure, like, you go on IRC and you ask, how do you
17:23
turn a own string into a static string slice, and someone will say, well, you can probably do it with unsafe, but we don't like doing that because the point of unsafe is that you never need to use it. It's there in case, it's like the shotgun on the wall in your dad's room. It never leaves the wall. It's there in case you need it. You just never need it.
17:42
Okay, so we'll fix that somehow. Let's just get on with the file system code. I want to be able to write something like this, where I have sort of several virtual file systems, like one of which will, one type will be able to load things from the disk, just from the normal file system, and
18:01
one will load things from a zip file and pretend it's a file system that's sort of been overlaid on top of it. This is handy for games because, especially if you want a game that's moddable or something, you can have all the sort of formal game resources in one zip file, and then you can have a mod that just sort of replaces a few of them, either in a zip file or in the directory, and it's nice.
18:21
And we just have, I wanted to just have a trait, the file system be a trait object that exposed a few methods, and we just have a vec and we just look at which ones, we just look through each of them in turn and use the first one that has a file. Great, so I found a crate that looks like it does this, and it's called, and it has a trait like this, and it's called VFS.
18:46
And so we have the path, which this crate is, the path uses, it uses the path as the entry point, kind of, like you use path and then you open a file and it's very object oriented, but it works okay, and then we have an associated file and metadata type, and then you create a path through
19:02
this method on the trait, and it takes something that can be turned into a string and gives you the path type for that trait. Well, hang on, I wanted trait objects, and this has a generic parameter with a trait bound, but you can't turn that into trait objects
19:21
because the trait object needs a Vtable, and this will make one, the compiler will make one version of the path method for each type that you specialize it with, and so you can't, you can't make a trait object from that. The compiler would have to be able to look into the future and see what types of T
19:44
you would use it for and compile those as well. And it can't, like, so you end up with these situations where perfectly reasonable explanations or perfectly reasonable design decisions just get into weird places where you, and it's impossible to do anything about it.
20:03
And so this isn't, and these are the easy cases if you try to use GFXRS or Tokyo and you end up with, end up in one of these weird corner cases, then even Rusty can't figure out what's going on and tell you what you're doing incorrectly. And so you just end up with these weird,
20:24
these weird situations that aren't obvious, or at least aren't obvious if you're not looking for them, where you want to do something that the crate author didn't want, didn't think of you trying to do and Rust isn't allowing you to do it.
20:41
So they made some design decisions that had unexpected consequences. Unfortunately, I don't really have a great answer to this one besides iterate, because people will always come up with interesting use cases that you never thought of. I've spent the last two weeks with some trying to get GG easy to work on iOS
21:04
because someone really wanted it to, and I never really considered doing that before. So your users will always come up with something that you didn't expect and the nice thing is it's not a terrible thing to redesign an API in Rust,
21:22
at least for small things, definitely, because it's Rust, like if it compiles it will probably work, and it's very hard to break things by accident. So, have people actually use your crates, use your own dog food, but also have make sure you share your dog food around and have other people taste it and see if it is to their
21:41
palate. Rust is great because you have cargo, everyone tends to use semantic versioning, and it has good encapsulation, and so if you make an update, an update to your crate, like futures or something, and people don't like it, that's fine.
22:02
They just use the old version and it's no problem. So, everything you know about API design still works in Rust. If you can make a function that takes borrowed types and returns a result when necessary, then you're writing idiomatic Rust code. It doesn't matter how non-fancy it is, you don't need the fanciness. The fanciness is there when you need it,
22:24
but most of the time you don't. Always make sure that you have some idea of who you're writing this crate for, just so when the next interesting feature pops into your head and you say, oh, I wonder if I could do it, if it could do X, you stop the second and say, well, does X really serve my purpose or not?
22:42
More importantly, if I write X, how do I explain it to someone who's trying to use it? Iterate and keep working and keep writing code. So, thank you.