Gate project
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Title of Series | ||
| Number of Parts | 490 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/46991 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSDEM 202065 / 490
4
7
9
10
14
15
16
25
26
29
31
33
34
35
37
40
41
42
43
45
46
47
50
51
52
53
54
58
60
64
65
66
67
70
71
72
74
75
76
77
78
82
83
84
86
89
90
93
94
95
96
98
100
101
105
106
109
110
116
118
123
124
130
135
137
141
142
144
146
151
154
157
159
164
166
167
169
172
174
178
182
184
185
186
187
189
190
191
192
193
194
195
200
202
203
204
205
206
207
208
211
212
214
218
222
225
228
230
232
233
235
236
240
242
244
249
250
251
253
254
258
261
262
266
267
268
271
273
274
275
278
280
281
282
283
284
285
286
288
289
290
291
293
295
296
297
298
301
302
303
305
306
307
310
311
315
317
318
319
328
333
350
353
354
356
359
360
361
370
372
373
374
375
379
380
381
383
385
386
387
388
391
393
394
395
397
398
399
401
409
410
411
414
420
421
422
423
424
425
427
429
430
434
438
439
444
449
450
454
457
458
459
460
461
464
465
466
468
469
470
471
472
480
484
486
487
489
490
00:00
Logic gateProjective planeLogic gateMultiplication signComputing platformLine (geometry)1 (number)Computer animation
00:38
Functional (mathematics)Computer programmingState of matterServer (computing)System callException handlingFile formatNormal (geometry)Type theoryPoint (geometry)Computer fileBinary codeLogic gateService (economics)Run time (program lifecycle phase)Graphical user interfaceSheaf (mathematics)Focus (optics)Line (geometry)Level (video gaming)ResultantDifferent (Kate Ryan album)Similarity (geometry)Multiplication signWeb browserBinary fileRun-time systemCommunications protocolVirtual machineAreaLatent heatStandard deviationAdditionClient (computing)Dependent and independent variablesClassical physicsIntegrated development environmentCASE <Informatik>Cartesian coordinate systemInteractive televisionInterface (computing)Complex (psychology)CodeExtension (kinesiology)SoftwareInternet service providerPortable communications devicePhysical systemWeb 2.0PrototypeEndliche ModelltheorieGraph (mathematics)BitRoundness (object)Human migrationIterationPattern languageDynamical systemMikroarchitekturLocal ringMusical ensembleMereologyProgramming paradigmResidual (numerical analysis)Insertion lossHypermediaComputer architectureNeuroinformatikContent (media)Direction (geometry)Programmer (hardware)Right angleMathematicsBit rateOrder (biology)Representation (politics)Form (programming)CompilerMixed realityProjective planeGodLecture/Conference
08:17
CompilerRun time (program lifecycle phase)Core dumpArmLatent heatImplementationAndroid (robot)Process (computing)Service (economics)Message passingInterprozesskommunikationCompilation albumDigital filterPhysical systemState of matterTelecommunicationInstance (computer science)Library catalogComputerLogic gateComputer programmingData modelComputer programIdeal (ethics)Android (robot)Run time (program lifecycle phase)ArmCodeMereologyLogic gateFunctional (mathematics)NamespaceCompilerMultiplication signMessage passingMaxima and minimaPoint (geometry)Software developerDependent and independent variablesLocal ringWrapper (data mining)Extension (kinesiology)Closed setAsynchronous Transfer ModeNetwork socketDemo (music)Focus (optics)Set (mathematics)1 (number)Endliche ModelltheorieComputer programmingParallel computingService (economics)ImplementationLibrary catalogLogicMikroarchitekturNormal (geometry)Proof theoryProcess (computing)Computing platformState of matterComputer fileBinary codePopulation densityProjective planeNetwork topologySystem callComputer architectureLatent heatMoment (mathematics)SpacetimeCASE <Informatik>PlanningComplex (psychology)Total S.A.2 (number)Scripting languageRight angleComputer animation
12:49
Demo (music)Core dumpGeneric programmingDeterminismComputing platformDemo (music)LaptopLogic gateComputer animation
13:06
FreewareSoftwareElectronic data interchangeComputer fileDeterminismComputer animation
13:24
InformationResource allocationElectronic visual displayComputer networkRead-only memoryLoop (music)AdditionExecution unitInstance (computer science)State of matterSoftwareContent (media)Metropolitan area networkComputer animationSource code
13:47
Electronic visual displayWindowVirtual machineComputer networkHill differential equationVertex (graph theory)SoftwareArmPoint cloudVirtual machineSource code
14:03
Information securityInformationPhysical systemStructural loadMaizeRead-only memoryAddress spaceWater vaporCellular automatonArmTouchscreenGame theoryRight angleLogic gatePosition operatorComputer fileSource code
14:28
InformationRead-only memoryUbuntu <Programm>Address spaceState transition systemGame theoryFocus (optics)DeterminismPoint (geometry)Logic gateSource code
14:41
Library (computing)Convex hullFunction (mathematics)CodeInformationStack (abstract data type)File formatWebsiteInformationService (economics)Logic gateStack (abstract data type)Functional (mathematics)Sheaf (mathematics)Standard deviationOpen sourceVariable (mathematics)Directory serviceDeterminismState of matterPoint (geometry)Buffer solutionContent (media)Staff (military)Source code
15:31
ComputerCompilerPoint cloudFacebookOpen sourceComputer animation
Transcript: English(auto-generated)
00:05
OK, our next speaker is Timo Savola, who's going to talk to us about the Gate project. So please give a warm welcome to Timo. Thanks. So my name is Timo Savola. And today I'm not here to solve any of your problems.
00:23
My purpose is to give you a bunch of new ones, because I'm presenting a new platform. So that usually isn't the end of problems. This is a project I've been working on for quite a long time on my spare time. And I think it's high time that I tell a lot of people about
00:42
it and see what they think. So the concept of my project, I've titled Portable Execution State. And that's a pretty opaque title. And to put it short, it means that I'm supporting migration of live programs with various kinds of systems.
01:04
So basically, you can think of virtual machine migration, but between the similar systems with different CPU architectures and so on. And what you can do with that, the result of that is
01:22
that you get a new kind of programming model. And the disclaimer I have to mention is that this sounds like a blockchain, nothing to do with blockchains this time. And this is a high level illustration of what I'm
01:41
talking about when I say that it changes the way you write or design software. So up there, the upper dotted line is the new focus of program supported by Gate. So traditionally, when you have software installed on a
02:02
computer, the code is immutable until you upgrade the pre-installed program. But you can freely exchange data with other systems. And that layer used to be the focus of indirection. And for example, in this graphic, it's between
02:24
application and data files where the complexity of external interfaces happens. And I'm trying to remove that completely with this approach. And the remaining interface remains with human interaction with the application.
02:43
So what we can actually accomplish is a single file that contains both application code and data. And both are equally dynamic and exchangeable and so on. Another way to look at it, or another use case for this is in a network environment. So in a classic request response networking model, you
03:03
have a client which implements, for example, HTTP client. Then you have a server that implements the STPD. And then maybe in the server backend, you have actual functionality in addition to the network layer. So in my model, you are removing the network protocol
03:23
in compatibility or compatibility programs, problems, and performance problems from the shared standardization specification area. Instead, what you are focused on is local APIs inside the server.
03:40
So when the client moves its own code to the server to act as the controller, we not only have to bother about network protocols. Instead, we are just doing IPC or local function calls inside the server. And that changes the dynamics of how you can optimize.
04:02
For example, if you have a large latency between the hosts, the messaging pattern has to be designed in the specifications and so on. But now it's an internal client detail. They have to take an iterate and also do a few round trips
04:21
for the use case and so on. So it's a little bit like a graph QL, but with actual programming. And that's just querying. And concrete stuff. So like I said, I've been working on this for my free time so far.
04:41
To my knowledge, this has no users outside of myself. And I've been playing with these ideas for a long time. And the latest prototype called Gate is pretty far along, technically quite advanced. But it needs, let's say, a community or something on top
05:02
of it to actually be useful. So I'm hoping to get some interest from you people today here. These are the foundations of Gate. WebAssembly is all the rage nowadays. And it's deservedly so. It's everything I dreamed of when I was coming up with
05:22
these ideas for this portable stuff. And the next thing you need is the runtime environment for the WebAssembly. WebAssembly itself is pretty secure, but only if the runtime implements it correctly. And the biggest difference between, let's say, other
05:44
server-side WebAssembly runtimes on the Gate is that I'm not actually implementing any of the web APIs or even VASI, which is a server-side interface because of
06:01
this portability requirement. So I have to provide custom APIs, unfortunately, for that. And those are accomplished by extension services. And I'll talk about those things later. So the headline feature is this portable snapshot and restore. And it doesn't require any support from the user
06:22
programs running in Gate. And programs can be suspended at any time. They're suspended immediately. Or if someone is trying to work around that, it might not be immediate. But it will be pretty fast as well so that it's built
06:40
into the design. The program format is WebAssembly, and I'm also building on that for the snapshot format. So WebAssembly allows custom sections in the binary file. So I'm adding Gate-specific sections there.
07:01
And if you are wondering what that would look like if I load it to, let's say, Chrome, Chrome would just see a no-operation binary file without any entry points. So it could actually inspect it, but not actually run on anything in it, because that would require
07:23
Gate-specific features. One special type of snapshot program is one that has finished, its entry function has returned. And those are almost like normal WebAssembly programs that come out of a compiler, with the exception that it's
07:42
actually representing the final state of the function call that returned. So you could repeatedly execute the program by making repeated entry function calls in the same state. And those actually could probably pretty easily be
08:02
loaded into, let's say, a web browser by adding a lot of Gate-specific APIs there. And the internals, because I'm primarily intending to use this on the server side, and I go on the server side, so I implemented this Gate with that in mind.
08:24
And part of this effort was a custom WebAssembly compiler implemented in Go. And to my knowledge, it's the only WebAssembly compiler implemented in Go. So if you are interested in that, it's a standalone project, although its features are geared toward Gate.
08:43
And this secure, safe, untrusted code execution requires all kinds of tricks and language-specific namespacing and whatnot. So there's a C in there, and the runtime maybe functions in platform assembly. And the two most important CPU architectures are currently
09:01
supported explicitly by the WebAssembly compiler and then the Gate runtime. So x86 and ARM 64-bit variants. And I also ported this runtime to Android. It required some tweaks, but I have a working proof of concept. So basically, the state features are quite similar to
09:23
normal WebAssembly runtimes. They are defined by the logical sandbox of WebAssembly. And then the, let's say, hardening feature outside are like container runtimes. But there are some distinct stuff. So many, let's say, fast WebAssembly solutions out there try to get high density by doing isolation in process
09:45
for many, many customers. But Gate is conservative and using one process for its invocation for safety. And also, unlike containers, Gate is not containing anything. It's an empty container. Just does all the Linux namespace stuff,
10:02
cleans out all the modes, all files, everything, and smuggles the binary in a file descriptor in there and just runs that process in the isolation, not the container. And because of that, every exchange between
10:21
the service APIs has to go through pipe socket. And that has a really nice feature. It has overhead, but the nice feature is that I can actually employ a second filter that is not much longer than the original strict second mode. I couldn't make that work. I had to add problem and protect
10:41
and close in there as well. But I think that's quite much shorter than, for example, something you would need with Docker in a typical scenario. So I'm pretty happy with that. Okay, services are like extensions.
11:01
You can implement any Go or with a Go wrapper and some other language. Internally, Gate implements only two services, catalog and origin. I have this proof-of-concept local host thing that you can use to migrate
11:22
the legacy request response thing to use Gate. And a few others, experimental ones that aren't listed. They're probably not gonna be there for long. And the services, unlike user programs, have to deal with serialization. So it's a complex thing compared to like,
11:42
let's say, a trivial implementation of that feature and other settings. Oh, my next focus in development is try to enable a service that enables user programs to communicate between each other, which is one of the points
12:03
of interest for me. And on the other side, user program APIs, unfortunately, because of the portable scenario, we cannot just expose the Linux calls and whatever. Gate does support VASE, unlike I said before, but only through stubs and with the minimum functionality
12:21
required to do the Gate-specific stuff through the pipes. And it's an asynchronous programming model because of that IPC messaging. And currently, the best way is to use CLang and C++. But I would hope that soon I can say
12:42
that the ideal thing is to use Go or Rust. So I have a small amount of time to do a demo. Let's try the demo codes. What I'm gonna do, because every platform needs a Doom port, so I'm first gonna run Doom for the port of the Gate
13:01
on my x86 laptop. Like this, there's a WebAssembly Doom and I'm piping it in the data file and there it is. And we can play some Doom.
13:24
Next what we are gonna do is send a secret to suspend it. So there's the, well, whatever, it's an instance ID. And with that, we can create a snapshot of the state.
13:42
Now it's in the snapshot of the content. And there's the shot of the content. Let's see if the network works. Let's copy it with SSH to my ARM64 machine in the clouds and the network worked.
14:02
Let's go there. Let's see that it's actually ARM, AR64, yeah, it's ARM. And now that the file is there, we can run it with Gate and it's using x-forwarding to run it.
14:21
So it starts from the same position of the game that we used. Now, because of x-forwarding, it's really slow to update the screen. But here you can see that it's actually running just fine. And I have one minute to, okay, let's quit the game. One minute to show the details of the snapshot.
14:43
I'm now using Gate's debugging features to show the backtrace of the point of suspension. And you can see that the name of the Linux Doom, it's the open source release of Doom. It's just a directory name. So it's suspended in the get package function.
15:01
And there are some stack variables there. And finally, we can look at the snapshot file. You can see that there's a bunch of standard WebAssembly sections and then this Gate dot whatever custom sections that contain the stack state, IO buffers,
15:21
service state buffers, and DWARF debug info that comes from CLang and so on. So thank you. That was it.