Trixnity: One Matrix SDK for (almost) everything written in Kotlin
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 542 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/61778 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
00:00
Matrix (mathematics)Information securityInformation privacyDiagram
00:22
Information securityInformation privacyMatrix (mathematics)Software maintenanceSoftware development kitClient (computing)Server (computing)Cross-platformDisintegrationSoftware testingRepository (publishing)Computing platformMultiplicationEvent horizonType theoryGeneric programmingComputer programmingFormal languageKeyboard shortcutCodeCore dumpInterface (computing)Message passingData modelString (computer science)Context awarenessObject (grammar)Client (computing)Computer-assisted translationLevel (video gaming)ImplementationAeroelasticityForcing (mathematics)Axiom of choiceContext awarenessField (computer science)Server (computing)Software testingSerial portEndliche ModelltheorieType theoryContent (media)Point (geometry)RobotMatrix (mathematics)Cartesian coordinate systemWritingLatent heatAbstractionFormal languageExtension (kinesiology)Limit (category theory)CodeKeyboard shortcutoutputBitCryptographyDependent and independent variablesCross-platformWrapper (data mining)Execution unitSoftware frameworkAndroid (robot)EncryptionComputing platformFamilyCore dumpMobile appKey (cryptography)AlgorithmEvent horizonIdentifiabilityInterface (computing)Problemorientierte ProgrammierspracheIntrusion detection systemSign (mathematics)Integrated development environmentLibrary (computing)Object-relational mappingData structureComputer animation
07:51
Human migrationEncryptionCryptographyWrapper (data mining)Binary fileCustomer relationship managementEvent horizonCore dumpEndliche ModelltheorieData modelClient (computing)Server (computing)DatabaseHypermediaCache (computing)BackupFormal verificationThumbnailDatabase transactionSynchronizationInstallable File SystemRead-only memorySoftware testingWeb browserSoftware development kitReading (process)WritingMobile appFile systemWrapper (data mining)Complete metric spaceDatabaseMatrix (mathematics)Client (computing)Data storage deviceLibrary (computing)Subject indexingProjective planePlanningRelational databaseWeb browserMathematicsBuildingoutputCryptographyThumbnailHypermediaRepresentation (politics)System callStability theoryRobotEncryptionCustomer relationship managementSynchronization2 (number)BitView (database)AdditionEndliche ModelltheorieCache (computing)ImplementationUser interfaceDataflowService (economics)Data structureInterface (computing)Computer animation
15:19
Process (computing)SynchronizationDatabaseDatabase transactionEvent horizonContent (media)Open sourceMatrix (mathematics)Control flowExtension (kinesiology)Client (computing)MereologyDatabaseKey (cryptography)Multiplication signEvent horizonDataflowDatabase transactionWritingVirtual machineCache (computing)DeadlockRobotToken ringSynchronizationMathematicsCASE <Informatik>Projective planeSoftware testingBlock (periodic table)Type theoryMedical imagingExtension (kinesiology)Operator (mathematics)EncryptionNormal (geometry)Content (media)outputAndroid (robot)Message passingINTEGRALServer (computing)Computer animation
22:48
BitMatrix (mathematics)Pairwise comparisonVirtual machineComputer animation
23:22
SynchronizationMathematicsLocal GroupServer (computing)Dependent and independent variablesElement (mathematics)2 (number)Open setZoom lensCache (computing)
24:34
Message passingGamma function2 (number)Element (mathematics)DatabaseMessage passingDatabase transactionFormal languageInsertion lossZoom lensProblemorientierte ProgrammierspracheEvent horizonBitOpen setSmartphoneWeb 2.0Multiplication signClient (computing)WindowThread (computing)Computing platformMobile WebMatrix (mathematics)Level (video gaming)Block (periodic table)Token ringComputer animation
28:45
Program flowchart
Transcript: English(auto-generated)
00:05
So, welcome to my talk about Trixity, one matrix SDK for almost everything. I added written in Kotlin a few days ago, so maybe there are some Kotlin fanboys here.
00:21
Yeah, let me first introduce myself. I am Benedict and my friends often see me as a killjoy when it comes to data protection and data security. But I convinced them to come to Matrix anyhow, so I have 20 users, family and friends on my own Matrix home server.
00:42
My first contact with Matrix was four to five years ago and I gained a lot of experience with it since then. And so much I founded Connect2X and this is just a company that is developing Timmy and that is a TI messenger for the medical health sector in Germany.
01:06
Now let's start with Trixity. Trixity is a matrix SDK and it is for developing clients, bots, app servers and servers. It is multiplatform capable, so everyone thinks Kotlin is JVM only. It is not. You can compile it to JS and you can compile it to native.
01:31
That's important for iOS. So we all have that targets with Trixity. And it's also developed test driven, so we have a high test coverage and it is licensed under the Apache 2 license.
01:52
You may wonder why another SDK. So back in January 2020, there were only a few multiplatform SDKs to choose from.
02:03
If I remember correctly, there was the Matrix Rust SDK, but it was in a very early stage. And there was the Dart SDK, but very likely this forces you to use Flutter in the UI.
02:21
So there is no real free choice which UI framework you want to use, especially when you want to use native UI technologies, for example on Android Compose or on iOS Swift. Additionally, most SDKs didn't have a very strict typing of events and the rest endpoints.
02:43
And also the extensibility was a bit limited. Even if the next point is not that important, SDKs were really bound to its purpose. So you've had SDK for a client, for a server, for a bot and so on.
03:00
So you have to learn a new SDK for each target, each purpose of application for Matrix. Why I chose Kotlin? I chose Kotlin because it is a statically typed language which compiles, as I mentioned, to JVM, JS and native.
03:21
And you don't need bindings. Like in Rust, when you use JS, you get JS. You don't need to make bindings over RASM or something. And on native, you can just call it from your Swift or Objective-C code in Xcode and have access to Trixity.
03:44
Moreover, besides shared common code, it is possible to write platform specific code. You just define a common interface and depending on the platform, the actual implementation can be different. This way you have access to platform specific APIs and libraries which can be very helpful when implementing encryption like AES for attachments.
04:09
So you have on each platform, can you use the native encryption algorithms the platform gives you already. And last but not least, you can define your own domain specific language. You will see later what I did with that.
04:26
So let's start with the core of Trixity. The core contains all basic data structures of the spec and its serialization algorithms. This includes events, identifiers like user IDs, event IDs and so on and other things like cross signing keys, device keys.
04:51
One goal of developing Trixity was the ability to add custom events which are strictly typed. So this is achieved by mapping event types to just a serializer.
05:04
In this example, we add a new type of m.room.cat which is of the Kotlin type cat event content. So you have access to all fields of this cat event content and don't have to mess around with the JSON.
05:24
The next layer of Trixity is the API layer. Each API has its model which defines all endpoints of the API. The actual client and server implementation just use these endpoints. And so as a consequence, there is no need to be defining things twice. They are using the same Kotlin object.
05:47
So a Kotlin object represents an endpoint on the Kotlin class, not Kotlin object. The best way to show this to you is with an example. This example is the endpoint leave room. You just implement matrix endpoint, give them the types and in this case, unit is the response.
06:13
So we don't get a JSON as response, just a HTTP OK or an empty JSON.
06:20
And you can also define a request, a URL, HTTP method and all that. And you can use this to call, to use a matrix client on the client side to call these endpoints.
06:42
So you create a leave room object, a request and you get the response. That's all on the client side. And the same thing on the server side. So you define an endpoint, give it the type you expect as a request and in the context object, you have access to the request and can answer with the response.
07:07
To make it a bit more easier for developers, there is a bit of abstraction on top of that. So you can also just call leave room. So you don't have to know which endpoint are there existing. You just type point on your IDE and see, OK, there's a leave room, I can leave a room.
07:26
And the same on the server side. So you just need to implement an interface and see all your endpoints you have to implement to be a fully featured matrix server API. Regardless of the API, there is Trixity ORM and Trixity crypto. Trixity ORM is just a wrapper for libORM.
07:50
As mentioned, a platform independent implementation doesn't need to worry about the actual platform specific implementations. So you have, when you use Trixity ORM, you don't need to know how libORM is accessed. So on the JVM, I use JNA. On JS, this is done via VASM.
08:14
And on native, just Cinterop from Kotlin.
08:21
LibORM is also packaged into Trixity ORM. So as a developer, you doesn't need to ship the build C library and it is just loaded automatically. So you just, you don't need to init your encryption like in other libraries as it's just loaded. My plan is to migrate that to Volosémark, but currently UniFFI, we heard of that in another talk, does not support VASM targets.
08:55
So currently I can Volosémark just only use in Kotlin JVM and native, but I also want to use JavaScript. So this project is currently on ice.
09:09
Trixity crypto currently implements the key management and allows to decrypt and encrypt events. And in the future, it will be more so you can reuse the complete crypto stuff, for example, in app services.
09:28
Trixity client allows you to, oh, sorry. The most abstract layer are Trixity client and Trixity app service. While Trixity app service is still very basic and does not have a persistent layer, Trixity
09:45
client allows you to choose which database and which media store implementation you want to use. And on top of that, there is something that isn't released yet. We are not sure how to release because we have to make money with our company.
10:03
It is Trixity messenger. This is just the view model representation of a messenger. So you only have to implement a thin UI layer where when the user clicks a button, the UI sends this
10:21
to the view model and the view model says, okay, send a message or go to this room or any other stuff. And with this approach, we have implemented an iOS client in a few weeks with one person.
10:40
So Trixity client allows you to implement a nearly fully featured matrix client or bot. So if you were at the matrix Rust SDK talk, you can just use their representation and instead of Rust, you write Kotlin. So everything that matrix Rust SDK does also does Trixity.
11:05
Some features like sliding sync aren't there because we want to follow the stable matrix specs. So we don't implement any MSCs. So we have all the E2E features, the exchangeable data stores and media stores.
11:25
We have reactive cache on top of that, notification, thumbnail generation, all that stuff you need to implement a client. There are already some media store wrappers that we implemented for all targets, targets expect browsers.
11:47
We just use the file system and on browsers we just use index DB. Next, I want to talk about how I accidentally created a cache. So on the left side, you see the relation between the UI, Trixity and the storage layer.
12:06
And because reactive UIs are really common, I wanted Trixity to give the UI access to the data in a reactive way. So if anything changes, the UI should immediately know about this.
12:20
But the question is how? On the one hand, there are a few databases which support listeners to react to changes to the database. But on the other hand, this would limit support for multiple supported databases because finding a common interface for listeners would be hard.
12:44
So I started implementing an intermediate layer based on Kotlin flows. The flow in Kotlin is a reactive data structure. So you have a producer on the one side and a consumer on the other side. So if the producer changes anything, the consumers immediately know about that.
13:08
And what does this intermediate layer? It talks to a very thin database layer which only knows about save, read and delete data.
13:21
And if someone wants data from this layer, it just reads it from the database. Or if someone changes something in this layer, it just writes it to the database. And the values are kept in this layer as long as they are subscribed from anyone.
13:41
So this means that if anyone else subscribes to a value, he will immediately get the current value because there is no additional database call needed because it is persisted in the intermediate layer. This goes so far that even if there are no subscribers anymore, I just keep the value a bit longer in this layer.
14:03
So if someone asks for a value, for example, 10 seconds later and the value is still stored, he gets the value and there is no database call needed. And you can now guess what I implemented. It's just cache.
14:28
So as you see with this cache, everything in Trixity is reactive. These are just a few examples, so we can just get all users or check if a user can invite another one.
14:40
You immediately get the change, get a notification if anything has changed. As mentioned, the database layer is really thin, so we implemented many database layers. So SQL-based one via Xposed for JVM-based targets.
15:04
We implemented one with Realm that can be also used on native targets like iOS. And for browsers, we have IndexedDB. So most of the data changes when a sync is processed.
15:22
Most of the data changes when the sync is processed. So it is way more performant to make a large transaction around the sync. So you don't have a transaction every time the cache writes something into the database.
15:40
Trixity just spends a large transaction around sync, so you have thousands of writes in one transaction. So everything fine? No. Then there was Realm. And Realm is just a really fast database, but Realm only allows one write transaction at a time.
16:03
So if another one wants to write to the database, he needs to wait until the first transaction ended. And the problem is that while the sync is running, it may be needed that we have to wait for outdated keys to be updated to decrypt all stuff.
16:25
So if the outdated keys part of Trixity want to write something in the database, he needs to wait until the sync is ended, but the sync waits for the keys to be updated, so we have a deadlock there.
16:40
This is one of the reasons why I introduced async transactions. The other reason was that most of the time the sync processing, as I find out with some benchmark, was wasted due to writing to the database. So processing a sync takes a long time because there are so many IO operations that the user has to wait until all operations are done.
17:10
So what does async transaction in Trixity mean? It means that all changes to the database are collected and processed in the background.
17:22
So database operations are decoupled from the cache layer and they are just written in the background. If everything fails, it is rollbacked, but that's irrelevant in the normal use case. So we can process even more syncs in the same time as if we would wait that the transaction has finished.
17:47
And this gave Trixity a huge performance boost. Actually, I released it last week, and I wrote the integration test which just fails if it is not 50% faster.
18:05
So it is always green. The next thing I did completely different in Trixity are timelines. So normally syncs are sent as fragments from the server to the client.
18:22
So one fragment contains a few timeline events and if there is a gap, you get a token. So you know as a client, okay, there is a gap, I need to fetch to fill that gap and so on. And these fragments normally are saved as is to the database in clients.
18:40
In Trixity, I use another approach. There I have each timeline event pointing to each other. And if there's a gap, the timeline event knows about this. So this allows Trixity to again benefit from Kotlin flows.
19:04
So we have a producer that is the room starting from a timeline event and a subscriber who wants the next timeline event to fill its timeline. So this allows us to go really fast through the timeline and build the timeline under the top.
19:25
And it makes it easier to fill the gaps because we don't have another layer fragments, we just have timeline events. And this way, it's also possible to very easy connect upgraded rooms.
19:43
So that one I released yesterday I think or two days ago. So the timeline event just shows to another timeline in another room. So timelines with room upgrades are invisible for users of Trixity.
20:01
You just get an infinite timeline until you reach the oldest room and the first timeline event. And finally, a small example. So if you want to write a bot, that's a good start to use Trixity just to get a feeling about it, how it works.
20:22
You can just call get timeline events from now on. And what this does is it subscribe to the flow that I mentioned which built the timeline and waits until the timeline event is decrypted because the timeline itself also is a flow. So if everything changes, it is redacted or there's a reaction or a replacement, the timeline event flow changes.
20:49
So this get timeline event from now on just wraps it down so you get a timeline event that is decrypted. And you can see we can just check what type it has and when we have checked the type, we have access to buddy.
21:07
And then we have send message. So when you call send message, you don't have to worry about if the room is encrypted or not. You can just use the DSL that I created to write text messages, image messages and so on.
21:23
And you can also form relations with that. So you can say like here, yeah, this is a reply to the timeline event I just got. And this has extensible events in mind. So if in the future there are other content blocks that are added, we can just extend the
21:44
DSL and you can very, very intuitive write your content with Trixity into an event, into an agent. An extensible event. So here are some projects that are using Trixity and that I know about.
22:03
There is a Spotify bot, a Mensa bot. Someone has created some extensions to better use it for bots and so on. And there is Trixity examples. That is from me. This is just a ping bot. Part of it you saw here.
22:23
It is E2E enabled. You can just run it in your browser, on your Linux machine or on your iOS client or via the JVM on Android. And there's also Timmy messenger. That's our messenger from our company, but it's not open source yet.
22:41
We plan to, but we don't know how because licensing. Yeah. Just try it out and come to me if you have questions. I'm a bit around. This is the matrix room. This is my matrix ID. And if we have a bit time, I just can show you a small demo, I think.
23:07
Yeah, I made a small performance comparison. It's not representative because it just runs once on my machine and there was no warm up or multiple runs.
23:23
Yeah. On the left side, you see our Timmy client, but basically it's just using Trixity. On the right side, there is Element and in the middle is FluffyChat. And now you can give me your bets who is the fastest. Yeah, let's see.
23:41
When the red zoom comes, the response from the server reached the client. So I just looked into the synapse logs when the response was sent. So we just wait a few seconds and then we see who is first. And you can look into opening rooms because we have this caching.
24:01
It is very fast in our client, but I must say FluffyChat is also very fast regarding opening rooms. So, oh, Trixity was the fastest and we can open rooms and you see open rooms is also a lot faster than on Element.
24:21
And there was FluffyChat and FluffyChat also is very fast. Yeah. I also have a desktop demo, but there Neko is the fastest. This is Neko, this is Timmy, Element on the web. It's a bit hard, this comparison, because Element runs in the web and does not have the mighty threading other clients have.
24:49
So Neko, just three seconds. I can just chat around. And the next is Timmy on the left, top left side.
25:00
Also very fast opening rooms and switching rooms because it is cached all the time and events. Then there was Element and I think also FluffyChat. Yeah, FluffyChat also. Yeah. Okay. That was my talk. Thank you.
25:25
Questions? Yeah. How do you prevent data loss with your async transactions? The transactions are run each after.
25:40
If one transaction failed, the other transactions aren't just run and you can, the next thing starts with the ALT token. But what happens if say your battery runs out whilst a bunch of transactions are queued? Sorry, I didn't answer. If your battery runs out when all those transactions are queued, so they haven't been written to the database. Yeah, then they are gone.
26:02
There your client have to do the work again, but mostly this doesn't happen. If you close your client, all transactions are written that are just opened. But it depends on your platform if it is killed hardly or a trick that you have a bit time to write the transactions back to the database.
26:20
But it's still very fast to write. So it's just a bit snappier on mobile devices which are not that fast. Like my smartphone from 2016, Element, I can't run Element on that because it's too slow. And sending messages 10 seconds later, the message is, oh, okay, yes, yes, now we send the message.
26:44
I don't have this problem because zooms are just faster than the slow I owe writing to the database we have on old smartphones, for example. Another question? I go for a domain specific language over just chaining together methods to say add a new content block.
27:09
Is there any benefits to that DSL? It's nicer to write. I like DSLs. In Kotlin we have them all over the language and it feels very intuitive because your
27:22
IDE gives you suggestions what methods there are and it's a lot easier to read, I think. To the character.
27:42
There's Rust and there's Kotlin. Is there any way to minimize the amount of things that the user has to learn? I didn't understand the question acoustically.
28:12
To be honest, I don't like Rust. I just like a higher level of implementing stuff so we didn't spoke, I didn't spoke with the Matrix Rust team.
28:27
We are done most of the time. The last question from the audience would be that we can open the windows and the doors a bit to get new air in. Thank you very much.