Yjs: A CRDT framework for shared editing
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Title of Series | ||
| Number of Parts | 490 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/47045 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSDEM 202011 / 490
4
7
9
10
14
15
16
25
26
29
31
33
34
35
37
40
41
42
43
45
46
47
50
51
52
53
54
58
60
64
65
66
67
70
71
72
74
75
76
77
78
82
83
84
86
89
90
93
94
95
96
98
100
101
105
106
109
110
116
118
123
124
130
135
137
141
142
144
146
151
154
157
159
164
166
167
169
172
174
178
182
184
185
186
187
189
190
191
192
193
194
195
200
202
203
204
205
206
207
208
211
212
214
218
222
225
228
230
232
233
235
236
240
242
244
249
250
251
253
254
258
261
262
266
267
268
271
273
274
275
278
280
281
282
283
284
285
286
288
289
290
291
293
295
296
297
298
301
302
303
305
306
307
310
311
315
317
318
319
328
333
350
353
354
356
359
360
361
370
372
373
374
375
379
380
381
383
385
386
387
388
391
393
394
395
397
398
399
401
409
410
411
414
420
421
422
423
424
425
427
429
430
434
438
439
444
449
450
454
457
458
459
460
461
464
465
466
468
469
470
471
472
480
484
486
487
489
490
00:00
DiagramCollaborationismRevision controlImage resolutionConflict (process)Text editorFront and back endsContent (media)Mathematics1 (number)System callSoftware frameworkSocket-SchnittstelleIntegrated development environmentCartesian coordinate systemPeer-to-peerRevision controlPosition operatorComplex (psychology)Web 2.0Lattice (order)Server (computing)SynchronizationCollaborationismFlow separationDatabaseImage resolutionMultiplication signMathematicsCombinational logicFront and back endsCodeWhiteboardText editorImplementationGraphics softwareCommunications protocolInformation and communications technologyState of matterPattern languageLimit (category theory)Task (computing)MultiplicationSoftwareINTEGRALRepetitionDigital electronicsMusical ensembleBit rateChainUniform resource locatorBranch (computer science)Core dumpCASE <Informatik>PlanningRight angleGoodness of fitReal-time operating systemComputer animation
05:37
CollaborationismContent (media)MathematicsType theoryImplementationRepresentation (politics)Network socketSubject indexingGraphics softwareKeyboard shortcutDemo (music)WebsiteConnected spaceDatabaseCASE <Informatik>Context awarenessData storage deviceRevision controlMathematicsImage resolutionDiallyl disulfideTrailDemo (music)State of matterCartesian coordinate systemCommunications protocolProduct (business)Peer-to-peerCodeType theoryService (economics)Flow separationData miningDifferent (Kate Ryan album)Web browserLink (knot theory)SoftwareText editorMobile WebDatabase transactionSoftware frameworkImplementationVermaschtes NetzServer (computing)Module (mathematics)Multiplication signContent (media)Keyboard shortcutNeuroinformatikEstimatorAdaptive behaviorLevel (video gaming)ArmTelecommunicationGroup actionParticle systemCategory of beingEndliche ModelltheorieProjective planeFrame problemSubject indexingPlanningExtreme programmingPhysical systemScripting languageShared memoryTraffic reportingInformationCursor (computers)Row (database)MetreCoprocessorECosComputer animation
11:08
IntelSoftwareWordWebsiteWeb browserBackupMultiplication signComplete metric spaceInheritance (object-oriented programming)Repository (publishing)Doubling the cubeRight angleWindowGroup actionEvent horizonProgram flowchartComputer animation
12:16
Convex hullMathematicsVideo gameWordPeer-to-peerGodProcess (computing)Text editorInsertion lossWebsiteWave packetRevision controlDifferent (Kate Ryan album)Content (media)SynchronizationCursor (computers)Asynchronous Transfer ModeTrail
13:46
SynchronizationSampling (statistics)Graphics softwareFlow separationText editorComputer animation
14:14
Monad (category theory)MaizeCollaborationismWave packet2 (number)Coma BerenicesDemo (music)Physical systemBitInformation technology consultingECosProcess (computing)Service (economics)Software frameworkInternet forumLink (knot theory)Internet service providerComputer animation
15:18
Point cloudFacebookOpen sourceComputer animation
Transcript: English(auto-generated)
00:05
Hello everybody, I want to introduce you Kevin Jansz with a talk about CRDT framework for shared editing. Give him a warm welcome. Thank you.
00:22
So yeah, I'm Kevin Jansz, hello, first in. I'm a contractor, web developer, mainly focusing on creating collaborative applications. And that's what Yjs is for. It is a framework designed to specifically help you to make collaborative applications or make your existing application collaborative.
00:43
But first, what is collaboration? You're probably familiar with collaborative text editing, collaborative drawing. Maybe you have seen that a couple of times. I'm personally really invested into collaborative whiteboarding. Think about meeting rooms, multiple people coming together, working on the same problem.
01:02
It really makes sense to provide tools that enable you to do all these things at the same time and so it really enables you to create stuff together. All these creative tasks like drawing, like text editing, meeting rooms, these are best
01:22
served doing as a collaborative application. You would be really hindered if you would use a Git application to do something like that because it's an asynchronous tool. You really want to have realtime shared editing in many applications. And this is what Yjs does. So what's the problem that Yjs tries to solve or does solve?
01:44
It's automatic conflict resolution. It's kind of magic. So assume that two users are working on the same document, they have a text document in this example, and both are inserting a character at the same time at the same position. They first apply their change on their own document and then they propagate the change
02:00
to the other peers. So what happens is this conflict that occurs in this scenario here is automatically resolved and the documents are merged automatically. It's kind of magic. So both users will end up with the same document and that's really important in these
02:20
kinds of applications. This is very much contrary to a Git application. So this is manual conflict resolution. This is what you have in Git. Two users are doing something concurrently, working on different branches, and then they merge their stuff and they end up with a Git conflict. And this is perfect in my opinion because a Git application is an asynchronous application
02:45
and therefore a lot of conflicts might occur that you want to resolve manually because well, if you would just automatically merge those changes, you would end up with weird stuff basically. And this is also a problem that Yjs solves.
03:03
Think about offline editing. So you have a collaborative application that is supposed to also work offline. You want to be on a plane, write a document, and then later sync those changes to the other peers. Then you want to have manual conflict resolution and Yjs is also a tool for you to do that.
03:21
So it's very much like Git in a way. There are tons of existing solutions and there's a pattern there. All these existing solutions, they solve exactly one problem. They have their own editor implementation that they want to make collaborative or their own diagramming solution, drawing application that they want to make collaborative.
03:44
And there are so many existing tools that do exactly that and they all have a specific backend that you have to use and integrate into your application. They don't work peer-to-peer and where's the fun if you don't have, can implement a distributed application with this framework, right? You want to integrate this shared editing solution into your own environment.
04:05
Maybe you work at a company with an existing communication protocol already in place. Maybe you have something like web sockets already set up, you have a specific database set up, and then you want to use your own tools instead of relying on a separate
04:21
server implementation. This is something that really bothers me and something that I want to solve with Yjs. I've been working on Yjs for six years now. I've been here at FOSDEM six years ago talking about the same stuff, but now I'm really ready to do all these things that I'm talking about. Yjs' mission is really to provide a solution that is network agnostic.
04:44
It works with all the existing communication protocols that are out there. It works peer-to-peer, it works on the DAT, on IPFS, it integrates into your own existing environment. It's really awesome, I named this network agnostic because it doesn't care what the
05:01
underlying communication technology is. Furthermore, I want to support all the editors. At this time, I support five of the most prominent text editors, rich text editors, code editors, and they all work in combination with Yjs, but that's not the limit. I want to make everything collaborative, drawing, and maybe even interfaces.
05:25
You can make React state collaborative, that's all possible with Yjs, and Yjs gives you the benefit that it already solves all these complex things that happen in collaborative applications. For example, shared undo redo, it's a really complex problem, think about that.
05:42
I'm working on a document and I hit undo. I should not be able to undo the changes that you created, I should only undo the changes that I did, and that's a huge problem. There's a lot of state tracking involved, and that's a problem that Yjs solves, and it gives you undo redo, collaborative shared undo redo in all your applications just
06:04
by using this framework. It solves the problems of propagating the awareness information, that is information, for example, who is currently online, what are they currently selecting in the text, where is their cursor, what is their name, these are all problems that are solved in Yjs.
06:23
Yjs has many more features, for example, it does versioning, it does change tracking, you can track who created which content. This is very much like get blame if you want, so you can select a portion of text and find out who created that specific text. It does offline editing really well with all the problems that are associated with
06:42
offline editing and merging offline content, so yeah, pretty cool. But Yjs itself, here, it's just a really small CIDT implementation. It is based on the idea that there are shared types that you can use to manipulate data.
07:01
A shared type is very much like a JavaScript type, like the JavaScript array or JavaScript map, with the only difference being that all the changes that you do to these types, these shared types, are automatically propagated to all the peers that are out there, and they're automatically merged, and you can observe these types and see, okay, when
07:25
did they change, what happened, and who did those changes, you can do transactions on these changes. This is how Yjs tries to solve the problem of designing a collaborative application. Then there's a huge ecosystem of modules that work together with Yjs.
07:42
I put all these modules into three categories, and these are only the modules that are officially supported by Yjs, and so they are the connectors. Basically they are all about how you connect and share your data using different communication protocols.
08:00
There are the persistence adapters that store the data in a database. This can be server-side, for example, in LevelDB, or browser-side, or in IndexDB. Then there are the data bindings, and in this case here, only editor bindings. Yjs supports five editors, including ProseMirror, Quill, and several code editors, and we
08:22
are currently working on integrating Yjs in Gutenberg, which is the official editor of WordPress. You probably know about that. It really integrates well into all these products. I could talk hours for each of these modules individually.
08:40
They do a lot of stuff. They allow you to scale your application. I do, actually, you can check out the podcasts we created around Yjs. Later the slides, there's a link to that. I want to pick out a few of these modules individually and give an example. There's YindexDB, the persistence category, which allows you to store all your document
09:02
updates efficiently in the browser database. IndexDB is a database that is available in all your browsers, in all modern browsers, and YindexDB stores document updates incrementally efficiently in the database, so the next time you reload your page, you will already have the content there.
09:21
You don't need to request it from a server. It's already there on your local computer, and it will immediately load up. It's really fast. It also allows you to create offline-ready applications, because if your website is available offline, for example, using service workers, you can create an offline-ready application.
09:41
Then there is, and this is the coolest connector of them all, in my opinion, then there's YwebRTC. It's a really ambitious project of mine. WebRTC is a protocol, basically, that allows you to create a peer-to-peer connection from one browser to the other, and YwebRTC is a method, a framework, to create a mesh network
10:11
of WebRTC connections, of many peer-to-peer connections from browser to browser. There's no central server involved that does conflict resolution. All of this works peer-to-peer.
10:21
Of course, Yjs also integrates into existing projects, for example, the DAT protocol. This is something I'm currently working on, also with them, to make this happen. Yeah, I'm pretty excited about that. Now it's demo time, and I invite you to not only trust what I say, just go on this
10:42
website, yjs.dev, it's a website. I would be really happy if a lot of people would show up, and we could try out this peer-to-peer application with a lot of browsers and mobile devices here. I'm gonna say beforehand, the FOSDEM network is not the most reliable, but even if a lot of connections break, the WebRTC network is kind of fault tolerant, so I think it
11:07
might work. Actually, I'm gonna use this. This is my website, the Yjs website. There are a couple of examples here, many more examples available in yjs-demos.
11:23
It's a repository full of ready-to-use examples. This website is completely available offline, so the next time you reload this website, all of the data is already there. Did nobody join, or does it just not work? Well, that's what happens in demos, right?
11:43
You just gotta have to trust my word on it, but I have a backup solution for this. So if you can see other user cursors, it will work. Apparently the network here is not strong enough. So the idea here is, and now I open two browsers that you can work collaboratively, text,
12:03
you see me typing on the left window, it's having stuff on the right window, and all of this, of course, works offline. Actually, I'm gonna go in this. Now that I have no network anymore. Airplane mode. I can reload this website, it will still work.
12:21
Even I can see my cursors here, which is really cool, I think. And it will sync even while offline, and the cool thing here is you can make versions. So I can create a version here and see all the differences that will happen that was created by different users. I can highlight them by the user who created them.
12:41
And now imagine me writing a lot of content here while I'm offline, and I go back online and sync all my changes back to you. Just imagine it would work. What would happen is that there would be a strange mishatch of weird words appearing on your side
13:02
and on my side. Oh my god, there are a lot of users. Holy shit. Ooh, it does work. Cool, so it does work. So now all my changes are just synced back to you, but I wanna know what happened. What did you create in the meantime since I wasn't here?
13:21
Apparently, some people just copy-paste the text and inserted some content, so I can really distinguish the users who did what, what happened here. And this is really important in offline editing. And by the way, this live tracks all the changes that you make, so when you do an insertion here, I'm gonna see who did that.
13:42
So yeah, this is really important in offline editing because you wanna be able to proofread the changes that you did to your document before you sync them to the other peers. And this is also something that YGist allows you to do. It's all peer-to-peer, offline. This uses YWebRTC, this is the ProseMirror editor,
14:01
but as I said, it's not limited to one editor. There are several editors that YG supports. And there's a drawing sample. You can coloratively draw. Let's have you guys draw something here. Oh man. Okay, it might be a bit overwhelming.
14:22
I think there, oh, there are a lot of users joining. This is basically my demo. I would really much like to have at least a few questions. So this is me. This is YGist. Find us in the GitHub community on the discussion board. Work together with TechOne to provide professional services
14:41
around YGist, shared editing, collaborative applications, distributed systems. Write us at YGist at techoneconsulting.com if you're interested in something like that. Also, we have tons of podcasts around YGist and the ecosystem that we created around this framework. There's just one link.
15:01
Yeah, that's basically it. Really happy you were there and hope you enjoyed the demo. Are there any questions? I think we have eight seconds. Thank you for your talk. Thanks, too.