Zero-Cost Async IO
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 8 | |
| Author | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/52175 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
RustLatam 20192 / 8
2
4
6
00:00
Open sourceFacebookParity (mathematics)QuicksortProcess (computing)Computer animationLecture/Conference
00:39
Self-organizationLecture/Conference
01:20
SequelQuery languageTable (information)DatabaseAbstractionDependent and independent variablesMereologyCartesian coordinate systemOrder (biology)Different (Kate Ryan album)Functional (mathematics)SynchronizationSound effect2 (number)Moment (mathematics)ForceDirected graphQuicksortWeightFormal languageMultiplication signOperator (mathematics)CodeSampling (statistics)Row (database)outputType theoryString (computer science)Business objectPlanningComputer programmingImplementationDomain nameLecture/ConferenceSource code
05:40
AbstractionGreen's functionThread (computing)Image resolutionSoftware testingBlock (periodic table)Scaling (geometry)Formal languageSoftwarePrimitive (album)Service (economics)Form (programming)Computer programmingoutputConnected spaceMereologyGreen's functionThread (computing)ScalabilityDependent and independent variablesScheduling (computing)Task (computing)NeuroinformatikSpacetimeQuicksortTraffic reportingPhysical systemSystem callLibrary (computing)Lecture/Conference
07:54
Green's functionThread (computing)Execution unitMereologyFormal language10 (number)Green's functionMultiplication signRun time (program lifecycle phase)CASE <Informatik>Thread (computing)SoftwareService (economics)Right angleContext awarenessOverhead (computing)Endliche ModelltheorieAbstractionJava appletComputer programmingCoroutineScheduling (computing)Cartesian coordinate systemBitOrder (biology)Sheaf (mathematics)CodeFunctional (mathematics)Semiconductor memoryPrimitive (album)QuicksortLibrary (computing)SpacetimeKernel (computing)Operating systemGoodness of fitLecture/Conference
11:26
BefehlsprozessorEndliche ModelltheorieBitScheduling (computing)MereologyComputer programmingMemory managementResource allocationDynamical systemAbstractionBoundary value problemDifferent (Kate Ryan album)Combinational logicBefehlsprozessorFormal languageResolvent formalismRepresentation (politics)Loop (music)Multiplication signThread (computing)SoftwareQuicksortLibrary (computing)BuildingConcurrency (computer science)Primitive (album)Object (grammar)Event horizonSynchronizationGame controllerConnectivity (graph theory)Musical ensembleoutputLecture/Conference
15:10
State of matterEvent horizonTablet computerCycle (graph theory)AbstractionCombinatory logicBoundary value problemScheduling (computing)outputFormal languageGame controllerPhysical systemMemory managementResource allocationOverhead (computing)Data structureState of matterSpacetimeSingle-precision floating-point formatEndliche ModelltheorieBitQuicksortCombinational logicCASE <Informatik>Parameter (computer programming)Resolvent formalismDifferent (Kate Ryan album)ImplementationBenchmarkAdaptive behaviorEntire functionCartesian coordinate systemIterationString (computer science)Functional (mathematics)MereologyFinite-state machineLevel (video gaming)Dependent and independent variablesRevision controlAbstractionCodeEvent horizonCycle (graph theory)Library (computing)Machine codeUniform resource locatorError messageMetropolitan area networkPlug-in (computing)Loop (music)ResultantArithmetic progressionRight angleSign (mathematics)Point (geometry)Telephone number mappingLecture/ConferenceProgram flowchart
20:42
Fluid staticsFeedbackFunctional (mathematics)WeightLoop (music)Multiplication signPointer (computer programming)CloningOverhead (computing)CodeQuicksortError messageCompilerCombinational logicType theoryFeedbackOnline helpRadical (chemistry)SummierbarkeitCellular automatonLevel (video gaming)Streaming mediaExpressionAlgebraic closurePositional notationSpeicherbereinigungChainValue-added networkFluid staticsLecture/Conference
23:08
FeedbackField (computer science)FeedbackQuicksortOrder (biology)String (computer science)Self-referenceCodeQuery languageNormal (geometry)Natural numberService (economics)Field (computer science)Error messagePattern languageWriting2 (number)Limit (category theory)TheoryDatabaseHookingGoodness of fitSequelSurfaceFrame problemState of matterVector potentialDirected graphWeightPoint (geometry)Multiplication signFormal languageSign (mathematics)Lecture/Conference
25:38
QuicksortMereologyPersonal identification numberCuboidMemory managementCodeFunctional (mathematics)Uniform resource locatorSelf-referencePointer (computer programming)Semiconductor memoryType theoryCASE <Informatik>Run time (program lifecycle phase)Endliche ModelltheorieOverhead (computing)MathematicsAbstractionAdditionFinite-state machineWrapper (data mining)WritingLecture/Conference
29:38
Electric generatorStreaming mediaCASE <Informatik>Personal identification numberProcess (computing)Loop (music)WritingSoftwareService (economics)QuicksortState of matterIterationSingle-precision floating-point formatSynchronizationStability theoryoutputLecture/ConferenceComputer animation
30:48
Electric generatorEndliche ModelltheorieSoftwareDependent and independent variablesStreaming mediaWeb 2.0Socket-SchnittstelleCASE <Informatik>Electric generatorFunctional (mathematics)Limit (category theory)CompilerLecture/Conference
31:30
Functional (mathematics)IterationStreaming mediaQuicksortState of matterRevision controlPersonal identification numberWeightPoint (geometry)Finite-state machineEndliche ModelltheoriePole (complex analysis)Lecture/Conference
32:05
Open sourceFacebookParity (mathematics)Musical ensembleLecture/ConferenceComputer animation
Transcript: English(auto-generated)
00:08
Hey everyone. Thanks for having me. My name is Without Boats, as in, yeah, Cimbotes. You can, most people call me Boats, which is actually even more
00:22
confusing. I am a researcher at Mozilla. I work on Rust. It's my full-time job. It's pretty cool. I think, so this talk is going to be about, sort of, it's a feature
00:40
that I've been working on for about the last year and a half, and before me, other people were working on for, I mean, honestly since before, the very beginning of Rust, before 1.0. But before that, I just wanted to thank the organizers for having me and for all of their work in putting this conference on. I think I, maybe someone's already said, pointed this out, but I
01:06
believe this is the first Rust conference outside of either United States or Europe, and so as someone who works in Rust, I'm really, like, excited and glad to see our global community, like, thriving and growing, and it's really cool to see all the conferences that are happening this year.
01:21
Come on to the technical stuff. So the feature that I've been working on is this thing called async await. It's sort of going to be probably the biggest thing that we do in the language this year. We're planning to ship it sometime in the next few months, and it's the solution to this problem that we've
01:41
been struggling with for a really long time, which is how can we have a zero cost abstraction for asynchronous IO in Rust. So I'm going to explain what zero cost abstraction means in a moment, but first just to kind of give an overview of the feature. So async await, it's just these two new keywords that we're
02:05
adding to the language, async and await, and so async is this modifier that can be applied to, like, functions, where now the function, instead of when you call it, it runs all the way through and returns. Instead it returns immediately and
02:20
it returns this future that will eventually result in whatever the function would return. And inside of an async function, you can take the await operator and apply it to other features, which will pause the function until those features are ready. And so it's this way of handling asynchronous concurrent operations using these annotations that makes them much easier to write.
02:43
So here's the little code sample just to sort of highlight and explain the feature. This is in, like, basically just an adapter on, like, a kind of ORM type of thing. It's handwritten, where you have this get user method, which
03:01
takes a string for a user name and then returns this, like, user domain object by querying the database for the record for that user. And it does that using async IO, which means that it's an async function instead of just a normal function. And so when you call it, you can await it. And then just to walk through the body of this method, you just, the first thing is it creates
03:23
this SQL query, interpolating the user name into, you know, select from users table. And then we query the database. And this is where we're actually performing some IO. So query also returns a future because it's doing this async IO. And so when you query the database, you just add this await in
03:43
order to wait for the response. And then once you get the response, you can parse a user out of it. This user domain object, which is, you know, part of your application. And so this method is just sort of like a toy example for the talk. But what I wanted to highlight in it is that the only
04:01
difference really between this and using blocking IO are these little annotations where you just mark the functions as being async. And when you call them, you add this await. And so it's, you know, relatively little overhead for getting, using non-blocking IO instead of blocking IO. And in particular in Rust, the really great thing about our implementation that makes me really excited about it is that our
04:24
async await and futures are this zero cost abstraction. And so zero cost attractions are sort of our redefining feature of Rust. It's one of the things that differentiates us from a lot of other languages is that we really care
04:43
about when we add new features that they are zero cost. We didn't actually come up with the idea. It's a big thing in C++ also. And so I think the best explanation is this quote from Bjarne Stroustrup, which is that a zero cost attraction means that when you don't use it, you don't pay for it.
05:01
And further, when you do use it, you couldn't hand code it any better. And so there's these two aspects to a zero cost abstraction, which is that the first is that the feature can't affect, can't add cost to people who aren't using the feature. So we can't add this global cost that will slow down every program in order to enable this feature. And the second is
05:25
that when you do use the feature, it can't be slower than if you didn't use it. And then you feel like, oh, well, I would like to use this really nice feature that makes it easier, but it will make my program slow. And so I'll just write this thing by hand. It will be a much bigger pain. And so I'm going to kind of walk through the history of
05:44
how we've tried to solve async IO in Rust and some of the steps along the way we had features that failed the zero cost test on both principles. So the specific problem that we're trying to solve is
06:02
async IO. And so normally IO is blocking. So when you use IO, you'll block thread and that will stop your program and have to be rescheduled by the OS. And the problem with blocking IO is it just
06:20
doesn't really scale when you have been trying to serve a lot of connections from the same program. And so for these kinds of really high scale network services, you really need some form of non-blocking or asynchronous IO. And Rust in particular is supposed to be designed for languages that have these really high performance requirements. You know, it's a systems programming language for people who really care about the
06:43
computing resources they're using. And so for Rust to really be successful in the network space, we really need some sort of solution to this asynchronous IO problem. But the big problem with async IO is that the way it works is that when you call the IO, you know, system call, it just returns immediately
07:03
and then eventually you can continue doing other work. But it's your program's responsibility for figuring out how to like schedule coding back to the task that you had to pause on when you're doing the asynchronous IO. And so this makes, you know, writing an async IO program much
07:20
more complex than writing one that uses blocking IO. And so a lot of languages that are trying to target these like scalable network services have been trying to come up with solutions for this problem that like take it, make it not the end user's problem, but a part of the language or a part of the libraries. And so the first solution that Rust started with was the idea of green threads, which have
07:41
been successful in a lot of languages. And so green threads basically look like blocking IO. They look like spawning threads and then just blocking on IO. And everything looks exactly the same as if you were using the native OS primitives, but they've been designed as part of the language run time to be optimized for this use case of having these network services, which are spawning, you know,
08:03
thousands, tens of thousands, maybe millions of green threads at the same time. A language I think that right now that is like having a lot of success with this model is Go where they're called Go routines. And so it's very normal for a Go program to have tens of thousands of these running at the same time because they're very
08:20
cheap to spawn unlike OS threads. And so just the advantage of green threads is that the memory overhead when you spawn an OS thread is much higher because you create this huge stack for each OS thread. Whereas green threads, the way they normally work is that you will spawn a thread that starts with a very small stack that can grow over time. And so spawning a bunch of new
08:42
threads that aren't using a lot of memory yet is, you know, much cheaper. And also a problem with using like the operating system primitives is that you depend on the operating system for scheduling, which means that you have to switch from your program's memory space into the kernel space and the context switching adds a
09:02
lot of overhead if you, once you start having, you know, tens of thousands of threads that are all being like switched between really quickly. And so by keeping that scheduling in the same program, you avoid these context switches and that really reduces the overhead. And so green threading is a pretty good model that works for a lot of languages, both Go and
09:20
Java, I believe, use this model. And Rust had it for a long time, but removed it shortly before 1.0. And the reason that we removed it is that it ultimately was not a zero-cost abstraction, specifically because of the first issue that I talked about where it was imposing costs on people who
09:40
didn't need it. So if you just wanted to write a Rust program that didn't need to use green threads, it wasn't a network service, you still had to have this language runtime that was responsible for scheduling all of your green threads. And so this is especially a problem for people who are trying to embed Rust inside of like a larger C application. It's one of the ways that we've seen a lot of success in people
10:02
adopting Rust is that they have some big C program and they want to start using Rust, and so they start integrating a bit of Rust into their program, it's just writing one section of the code in Rust. But the problem is that if you have to set up this runtime in order to call the Rust, then it's too
10:21
expensive to just have a small part of your program in Rust because you have to set up the runtime before you can call the Rust functions. And so shortly before 1.0, we removed green threads from the language, we removed this language runtime, and we now have a runtime that is essentially the same as C. And so it makes it very easy to call between Rust and C, and it's very cheap, which is one of the key things that makes Rust really successful. And
10:45
having removed green threads, we still needed some sort of solution to AsyncIO, but what we realized was that it needed to be a library based solution. We needed some sort of way of providing a good abstraction for AsyncIO that was not a part of the language, was not a
11:00
part of this runtime that came with every program, but was just this library that you could opt into and use when you needed it. The most successful library based solution in general is this concept called futures. It's also called promises in JavaScript. So the idea of a future is that it's this, it represents a
11:23
value that may not have evaluated yet. And so you can manipulate it before you actually, before the future is actually resolved. Eventually it will resolve to something, but you can start running things with it before it's actually resolved. And there's been a lot of work done on futures in a lot of different languages, and they are a great way
11:42
for supporting a lot of, like, combinators and especially this async await syntax that make it much more ergonomic to, like, build on top of this concurrency primitive. And so futures can represent a lot of different things. So AsyncIO is kind of the biggest and most prominent one where, you know, you
12:01
maybe make a network request and you immediately get a future back, which once the network request is finished, will resolve into whatever that network request was returning. But you can also represent things like timeouts, where a timeout is just a future that will resolve once that amount of time has passed. And even things that aren't doing any IO or anything like that, where just CPU intensive
12:20
work, you can run that on, like, a thread pool and then just get a future that you hold on to that once the thread pool is finished doing that work, the future will resolve. The problem with futures was that the way that they've been represented in most languages is this callback-based approach, where you have this future and you can schedule a
12:42
callback to run once the future resolves. And so the future is responsible for figuring out when it resolves, and then when it resolves, it runs whatever your callback was. And all these distractions are built on top of this model. And this just really didn't work for Rust because there's a lot of
13:00
people experimenting with it a lot, and found that it just was forcing way too many allocations, essentially every callback that you tried to schedule had to get its own separate, like, create object heap allocation. And so there were these allocations everywhere, these dynamic dispatches, and this, like, approach failed the zero-cost abstraction
13:20
on the second principle, where now it was not affecting people who weren't using it, but if you did use it, it would just be way slower than if you just had it handwritten something yourself. And so why would you use it? Because if you wrote the thing yourself, it would be much faster. But all hope is not lost.
13:45
Sorry, I also have a bit of a cold. This really great alternative abstraction that was poll-based.
14:00
Yeah, so we write this model, I really want to give credit to Alex, who's here, and Aaron Turon, who came up with this idea where instead of futures scheduling a callback, there instead, you poll them. And so there's this other part of the program called an executor, which is responsible for actually running the futures,
14:21
and so what the executor does is it polls the future, and the future either returns pending, that it's not ready yet, or once it is ready, it returns ready. And this model has a lot of advantages. One advantage is that you can just cancel futures very simply, because all you do to cancel them is you stop polling them.
14:42
Whereas with this callback-based approach, it was really difficult, once you've scheduled some work, to cancel that work and have it stop. It also really enabled us to have this really clean abstraction boundary between different parts of the program. So most sort of futures libraries come with an event loop,
15:01
and that's just your futures are scheduled across this event loop with this way of doing IO, and you really don't have any control over it. But in Rust, we have this really clean boundary between the executor, which schedules your futures, the reactor, which handles all the IO, and then your actual code, and so the end user can decide which executor they wanna use, which reactor they wanna use, giving them
15:21
the kind of control that is really important in a systems language. But the real advantage, the most important thing about this model is that it enabled us to have this really perfect, zero-cost way of implementing futures, where they're each represented as this kind of state machine. And so the way this works is that when the futures, the code that you write gets compiled down to,
15:42
it gets actually compiled into native code, where it is this kind of state machine where it has one variant for each pause point, for each IO event, essentially. And that variant then has the state that it needs to resume from that IO point, and that is represented as this essentially like an enum,
16:01
where it's this one structure where it's the variant, the discriminant, and then a union of all the states that it could possibly need. And so this is an attempt to visually represent that abstractly, where this state machine has, you perform two IO events, and so it has these different states, and at each state
16:20
it has the amount of space it needs to store everything you would need to restore to that state, and the entire future is just a single heap allocation that's that size, where you just allocate that state machine into one place in the heap, and it's just no additional overhead.
16:42
So you don't have all these boxed callbacks and things like that, you just have this perfect, really truly zero cost model. So I feel like that is usually a bit confusing to people, so I try to, this is my best keynote, visually represent what's going on,
17:02
which is that, so you spawn a future, and that puts the future in the heap in this one location, and then there's a handle to it that you start in the executor, the executor pulls the future until eventually the future needs to perform some sort of IO, in which case the future gets handed off to the reactor,
17:22
and the reactor, which is handling the IO, registers that the future is waiting on this particular IO event, and then eventually when that IO event happens, the reactor will wake up the future using the waker argument that you passed in when you pulled it. And so waking the future up passes it back to the executor,
17:40
and then the executor will pull it again, and it will just go back and forth like this until eventually the future resolves. And so then when the future finally resolves and evaluates to its final result, the executor knows that it's done, and then it drops the handle, it drops the future, and the whole thing is finished. And so it forms this sort of cycle where you pull the future, wait for IO,
18:03
wake it up again, pull it again, on and on in a loop, until eventually the whole thing is finished. And this model ended up being quite fast. This is a sort of the benchmark that was posted in the first post about futures, where benchmarked futures against
18:22
a lot of different implementations from other languages. Higher is better, and futures is the one on the far left. So we had this really great zero-cost abstraction that was competitive with the fastest kinds of implementations of async IO
18:40
in a lot of other languages. But of course the problem is that you don't wanna write these state machines by hand, right? You have your whole entire application state as a state machine is not very pleasant to write. But that's where the future abstraction is really helpful, is that we can build these other APIs on top of it. And so the first solution that we had
19:02
was this idea of futures combinators where you can build up these state machines by applying all of these methods to the future, sort of similar to the way that iterator adapters like filter and map work. And so this function, it just, what it does is it requests rustling.org and then converts that, the response to a string.
19:25
And so instead of returning just a string, it returns a future of a string because it's going to be an async function. And it has these futures in the body that it's going to be calling, and those are gonna actually be the parts that do some IO. And then they're all sort of combined together
19:41
using these combinators like and, then, and, map. And we build all these combinators like and, then, map, filter, map error, like all kinds of different things. And this works, it has some downsides, especially these like nested callbacks which can be really difficult to read sometimes.
20:02
And so because it has these downsides, we also tried to implement an async await implementation. And so the first version of async await was not part of the language. Instead, it was this library that provided this through like a syntax plugin. And this is doing the same thing that the previous function did.
20:22
It just fetches rustling and turns into a string. But it does so using async await, and so it's much more like straight line, looks much more like the way normal blocking IO works, where just like in the example that I showed originally, the only real difference is these annotations. And so the async annotation, you know, turns this function into a future instead of just returning immediately.
20:42
And then the await annotations await on the futures that you actually construct inside of the function. And await, under this pull model, desugars to this sort of loop
21:02
where what you do is you just pull in a loop, and every time you get pending back, you yield all the way back up to the executor that you're pending, and so then it waits until it gets woken up again. And then when finally the future that you're awaiting finishes, it, you know, finishes with a value, and you break out of the loop with the value, and that's what these await expressions evaluate to.
21:28
So this seemed like a really good solution. You know, you had this async await notation, which is compiling down to these really awesome zero-cost futures,
21:41
and so sort of released futures into the wild and got feedback, and that's where we ran into problems. That essentially anyone who tried to use futures quickly ran into very confusing error messages, where it would just kind of complain about how your future isn't static
22:01
or doesn't implement this trait, and it would be sort of this baffling thing you didn't really understand, and the compiler would make helpful suggestions which you would just follow until eventually you compiled. And so you would be annotating closures with move, and you would put things into reference-counted pointers and clone things and this and that, and it all felt like you were adding all this overhead to the thing that you didn't seem necessary,
22:21
you didn't understand why you had to do it, and also when you were done, your code ended up looking like soup. And so tons of people were bouncing off of futures, and it didn't help that the combinators produced these really huge types where your entire terminal would be filled up with the type of one of your combinator chains, where it'd just be like an and then of an and then of a map error of a TCP stream and so on.
22:42
And you have to dig through this to try to figure out what the actual error that you encountered was. And I found this quote on Reddit, which I think really beautifully sums up all of the complaints about futures, which was that when using futures, the error messages are inscrutable. Having to use ref cell or clone everything
23:01
for each future leads to over-competing code, and it makes me wish that Russ just had garbage collection, which is, yeah, not great feedback. So looking at the situation maybe a year, a year and a half ago, it was clear that there were sort of two problems that needed to be done to make,
23:22
like needed to be solved in order to make futures more usable for people, and the first was we needed better error messages, and so the easiest way to do that is to build the syntax into the language, and then they can hook into all of our diagnostics and error message support so that you can have really good error messages for async await. But the second was that most of these errors
23:42
people were running into were actually them bouncing off a sort of obscure problem, which is called the borrowing problem, where there was this fundamental limitations in the way that futures was designed that made it so something, really common pattern was not possible to express, and the problem was that you can't, in the original design of futures,
24:01
you could not borrow across an await point, so if you were to await something, you couldn't have any references that were alive at that time, and so when people were having these problems that they were bouncing off of what they were actually, ultimately what they were doing was they were trying to borrow while they were awaiting, and they couldn't do that, and so if we could just make it so that was allowed, then most of these errors would go away,
24:22
and everything would be much easier to use, and you could just kind of write normal Rust code with async and await, and it would all work, and these kinds of borrows during awaits are extremely common, because just the natural API service of Rust is to have references in the API, and so,
24:41
but the problem with futures is that when you actually compile the future, which has to restore all that state, when you have some references to something else that's in the same stack frame, what you end up getting is this sort of self-referential future, and so here's some code from the original like get user method, where we have this SQL string,
25:00
and then when we call query with it, we use pass a reference to the SQL string, and so the problem here is that this reference to the SQL string is a reference to something else that's being stored in the same future state, and so it becomes this sort of self-referential struct where you have these are the fields of the future,
25:21
in theory if it were a real struct, it would have the database handle that you were in the self aspect of it, then it would also have the SQL string, and the reference to the SQL string, which is ultimately a reference pointing back to a field of the same struct. And self-referential structs are sort of
25:40
a really hard problem that we don't have a general solution for, and the reason that we can't allow you to have reference to the same structs is that when you move that struct, then what happens is that we make a new copy of the struct in the location that you're moving it to, and when the old copy becomes invalidated, but when you make that copy,
26:01
the reference that was self-referential is still pointing to the old copy, and that becomes a dangling pointer, and it's the kind of memory issues that Rust has to prevent. So we can't have self-referential structs because if you move them around, then they become invalidated. But what made this really frustrating in the futures case
26:21
was that we actually don't really need to move these futures around. So if you remember the model where the future's in the heap and a handle to it is getting passed back and forth between the reactor and the executor, the future itself never actually moves. And so it's totally fine for the future to contain self-references as long as you never move it,
26:43
and you don't need to move it. So we really needed to solve this problem with some way to express in the API of futures that while you're pulling it, you're not allowed to move it around. And then if we just could express that somehow, then we could allow these kinds of self-references
27:00
in the body of the future, and then we could just have these references in your async functions and everything would work. And so we were working on this problem, and ultimately, we came up with this new API called Pin. And so Pin is a sort of adapter around other pointer types
27:20
where they become this pinned reference instead of just a normal reference. And a pinned reference, in addition to whatever other guarantees it has, it guarantees that the reference will never, the value that reference is pointing to will never be moved again. And so it guarantees that it's going to stay in the same place until eventually it gets deallocated. And so if you have something in your API
27:41
which says that it has to be taken by Pin, then you know that it will never be moved again, and you can have these kinds of self-referential structs. And so we changed the way that futures work so that now, instead of just being a boxed future, it's a boxed future behind a Pin. So we know that wherever we boxed it up, put it in the heap, it's guaranteed now by part of the Pin API
28:03
that it will never move again. And then when you pull the future, instead of just passing a normal reference to it, we pass a pinned reference to it. And so the future knows that it can't be moved. And the trick here that makes this all work is that you can only get an unpinned reference
28:20
out of a pinned reference in unsafe code. It's an unsafe function to be able to do that. And so the API looks roughly like this where you have Pin which is just a wrapper around a pointer type. It doesn't have any runtime overhead or anything, it just demarcates it as being pinned. And then
28:42
a pinned box can be converted into a pinned reference, but the only way to convert a pinned reference into an unpinned reference is to use an unsafe function. And so what we did was then we just changed the futures API so that instead of taking a normal reference,
29:00
it takes a pinned reference. otherwise the API is the same as it was before and this is essentially the API that we're going to be stabilizing. And so with that change, this code from the first example just works the way that it's written. And so you can just write code exactly the way
29:21
you would write it with blocking IO, add these async and await annotations. And then what you get is this async IO with this really awesome zero cost abstraction where it's basically as cheap as if you hand wrote the state machine yourself by hand.
29:41
So the situation today, pinning was stabilized in the last release about a month ago. We're in the process of stabilizing the future API, so probably in 135, maybe it will slip and be 136, which is in about two or three months basically. And then we're hoping sometime this year
30:01
we'll have async await stabilized, hopefully by the end of summer even, we're gonna have this stabilized and so that people will be able to write non-blocking IO network services using this syntax that makes it very similar to writing with blocking IO. Looking beyond that stabilization,
30:21
we're also already starting to work on these sort of more long-term features, like streams I think is probably the next big one, where it's an async future, so a future is just one value, but a stream is many values being yielded asynchronously, it's essentially like an asynchronous iterator,
30:43
and so you'll be able to loop asynchronously over a stream and things like that. And it's very important for a lot of use cases where you have streaming HTTP, web sockets, HTTP2 push requests, that kind of thing, where instead of having a networking like RPC model, where you make a network request and get a single response,
31:02
you have streams of responses and requests going back and forth. There's also today a limitation that async-fn can't be used in traits. There's a lot of work being done in the compiler to make it so that it would be able to support that, and looking out even beyond these features, someday we wanna have generators
31:21
where sort of similar to generators in like Python or JavaScript, where instead of just having functions that return, you can also have functions that yield, and so then you can resume them again if they yield, and you can use these to write functions as a way of writing iterators and streams, similar to how an async function lets you write a function that's actually a future.
31:41
But I guess just sort of recapping the real critical insights that led to this sort of zero-cost async IOM model, where the first was this poll-based version of futures, where we were able to compile these futures into these really tight state machines, and then secondly, this way of doing async await syntax,
32:02
where we're able to have references across the waypoints because of pinning. I'm supposed to say thank you, I don't know how it became an X. I'm fun. Yeah.