Matrix Widgets in the "Sovereign Workplace" for the German public sector
This is a modal window.
Das Video konnte nicht geladen werden, da entweder ein Server- oder Netzwerkfehler auftrat oder das Format nicht unterstützt wird.
Formale Metadaten
| Titel |
| |
| Serientitel | ||
| Anzahl der Teile | 542 | |
| Autor | ||
| Lizenz | CC-Namensnennung 2.0 Belgien: Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen. | |
| Identifikatoren | 10.5446/61790 (DOI) | |
| Herausgeber | ||
| Erscheinungsjahr | ||
| Sprache |
Inhaltliche Metadaten
| Fachgebiet | ||
| Genre | ||
| Abstract |
|
00:00
Streaming <Kommunikationstechnik>WidgetMatrizenrechnungProjektive EbeneComputeranimation
00:34
SoftwareEDV-BeratungMatrizenrechnungKollaboration <Informatik>Element <Gruppentheorie>PunktwolkeWidgetSuite <Programmpaket>Stochastische AbhängigkeitQuellcodeKomponente <Software>Notepad-ComputerSystemplattformCloud ComputingSpeicherabzugSystemverwaltungProdukt <Mathematik>DifferenteIntegralStochastische AbhängigkeitOpen SourceProjektive EbeneWidgetCASE <Informatik>Dienst <Informatik>MatrizenrechnungSoftwareSuite <Programmpaket>Weg <Topologie>Computeranimation
02:10
MatrizenrechnungTOEElement <Gruppentheorie>PunktwolkeWidgetSuite <Programmpaket>Stochastische AbhängigkeitQuellcodeKomponente <Software>Notepad-ComputerServerKollaboration <Informatik>TelekommunikationSystemplattformDesintegration <Mathematik>ServerSchlüsselverwaltungIdentitätsverwaltungInformationsspeicherungOffice-PaketVorlesung/KonferenzComputeranimation
02:38
MatrizenrechnungKollaboration <Informatik>Element <Gruppentheorie>WidgetSuite <Programmpaket>QuellcodeTelekommunikationKomponente <Software>Notepad-ComputerSystemplattformServerClientBrowserBitIdentitätsverwaltungElement <Gruppentheorie>Dienst <Informatik>Message-PassingClientRelativitätstheorieWidgetVideokonferenzUmwandlungsenthalpieFunktionalDatenbankWeb-ApplikationKartesische KoordinatenLesen <Datenverarbeitung>MatrizenrechnungGebäude <Mathematik>IntegralMereologieStreaming <Kommunikationstechnik>ServerRechter WinkelAggregatzustandWeb-SeiteDatensichtgerätKonfiguration <Informatik>DifferenteTouchscreenATMFront-End <Software>App <Programm>Protokoll <Datenverarbeitungssystem>MinimumSichtenkonzeptCASE <Informatik>Zusammenhängender GraphEreignishorizontGruppenoperationInteraktives FernsehenZweiGeradeInverser LimesWeb SiteTermPerspektiveInformationsspeicherungEinbettung <Mathematik>TelekommunikationEchtzeitsystemChatten <Kommunikation>HydrostatikTexteditorVorlesung/KonferenzComputeranimation
09:13
WidgetHumanoider RoboterDrucksondierungBitWidgetHumanoider RoboterMailing-ListeMessage-PassingWeb-ApplikationApp <Programm>SystemverwaltungAggregatzustandMechanismus-Design-TheorieWort <Informatik>Gemeinsamer SpeicherComputersicherheit
11:14
Element <Gruppentheorie>MatrizenrechnungAggregatzustandWidgetMessage sequence chartHypermediaInhalt <Mathematik>DokumentenserverImplementierungSynchronisierungGerade ZahlClientWidgetMereologieInteraktives FernsehenHydrostatikServerElement <Gruppentheorie>Ein-AusgabeBenutzerbeteiligungMatrizenrechnungInformationInverser LimesProzess <Informatik>Rechter WinkelRechenschieberTeilmengeWeb-SeiteImplementierungElektronische PublikationToken-RingHumanoider RoboterGerade ZahlDokumentenserverHypermediaInhalt <Mathematik>CASE <Informatik>Computeranimation
17:09
WidgetSynchronisierungGerade ZahlInformationsspeicherungReelle ZahlTelekommunikationInverser LimesMarketinginformationssystemNotepad-ComputerMatrizenrechnungWhiteboardSpieltheorieRichtungDemo <Programm>WidgetRoboterCASE <Informatik>Inverser LimesKonfiguration <Informatik>Peer-to-Peer-NetzElement <Gruppentheorie>Einfach zusammenhängender RaumEchtzeitsystemTelekommunikationPuls <Technik>WhiteboardVollständiger VerbandEreignishorizontSchedulingVerschlingungCAMPunktVideokonferenzMatrizenrechnungChiffrierungOpen SourceKartesische KoordinatenIdentitätsverwaltungATMAggregatzustandToken-RingClientComputeranimation
23:05
QR-CodeWidgetBildschirmsymbolEreignishorizontMathematikMultiplikationsoperatorWeg <Topologie>Element <Gruppentheorie>RoboterSystemplattformBenutzerbeteiligungDeskriptive StatistikRechter WinkelComputeranimation
25:07
EDV-BeratungWidgetDatenverwaltungIntegralServerGüte der AnpassungDimensionsanalyseAggregatzustandURLBenutzerbeteiligungEreignishorizontQR-CodeRichtungAliasingRechter WinkelComputeranimation
26:49
WidgetChiffrierungWidgetKonfiguration <Informatik>DatenverwaltungHilfesystemPunktIntegralInformationVollständiger VerbandSichtenkonzeptCoxeter-GruppeElement <Gruppentheorie>AggregatzustandSystemverwaltungTouchscreenURLVorlesung/Konferenz
28:35
WidgetMatrizenrechnungWidgetDimensionsanalyseTeilmengeIntegralDatenverwaltungSystemaufrufRechter WinkelEreignishorizontMessage-PassingClientWeb SiteMailing-ListeApp <Programm>Element <Gruppentheorie>InformationsspeicherungComputeranimation
30:55
ComputeranimationFlussdiagramm
Transkript: Englisch(automatisch erzeugt)
00:05
Welcome everybody. Good morning to our talk. All the people on stream on video on demand and of course here in the deaf room. My name is Kim and I'm here together with my colleague Oliver. We are from Nordic and we are going to talk to you about the matrix widgets
00:23
and particularly those we develop for the sovereign workplace project in the German public sector. So at first, quickly about us. Nordic is an IT consulting company based in Hamburg, Germany.
00:41
We are about 40 IT professionals and we, among other things, we develop first matrix integrations for productivity software in the public sector in the so-called sovereign workplace project. We are here today to provide you some insights from that we gained in the last couple of years.
01:06
So what's the sovereign workplace? Let's take a step back and introduce you all to it. It's a project with the goal of providing IT services to public administration in Germany and potentially also Europe.
01:21
It's founded by the German Federal Ministry of the Interior and Community and one of the core aspects is to gain independence from US cloud services and to retain full control over your data. And there's this suite we call sovereign workplace which covers many use cases for productivity
01:45
and it's achieved by combining the products from many different vendors which you can see on the right here. But all of this is open source software. So you could say this whole project also supports and maybe funds some of the open source projects here.
02:03
We could talk a lot more about the whole sovereign workplace project and that could be a whole talk for itself on another track but of course we are here to talk about matrix and the matrix death room. So quick overview here. Here we can see the Univention corporate server and key cloak which implements identity management in the stack.
02:24
There's groupware which is done by OpenExchange. There's cloud storage by Nextcloud. There is an office suite which integrates with Nextcloud very tightly which is Koloboro online. And of course there's video conferencing which is done with Jitsi right now
02:45
and perhaps in the future using matrix natively. And of course there's messaging which is done natively on matrix and particularly using Element as a client. How are we involved in this?
03:01
Well, together with Element we provide this real-time communications component which is messaging and also video conferencing. So there's chat and video. And in particular we are extending this chat use case using widgets for some specific use cases.
03:23
So this is the crucial part of this talk. The idea is to provide better integration with other components in the stack and build new work solutions. So you might be familiar with the concept of a widget and we've seen it in the previous talks but still I want to summarize a bit.
03:43
As I said, we can use widgets to extend the regular chat client functionality for specific use cases and I brought you some examples here. So essentially it's an option and also this was said before but it's an option to embed some kind of web app into your existing matrix client
04:03
and display more functions right there. So for example you can put it in the right bar like Hookshot does here or you can pin it to the top of the room and it will even adjust to your theme like this countdown. And there's more there.
04:20
You can add a lot of widgets at once and see all of them at once in multiple places. Of course there's video conferencing which you can also maximize to view all of the people in the video conference rather than having chat.
04:40
And there's also this half full screen mode where the chat moves to the right side and you can see the widget in a full screen-ish style. And there's more. So actually if you're following Fostum remotely you might find this a familiar site. So you are actually using widgets right now to watch this.
05:02
And there's this schedule, there's the live stream and also from the editor side there are widgets to support us. So if you followed closely or you are very familiar you might have noticed wait a minute, this is a bit more than just static web pages.
05:24
So actually there's a thing called the widget API and that allows your embedded page to interact with the client but also with the matrix. So for example from our widgets we can also send messages, read messages to the room.
05:43
So one way to explain widgets would be you could say these are some kind of limited matrix line for a specific use case. And you can build a lot of things. Really there are endless possibilities and now Oliver is going to show you a bit more about how that works.
06:06
Thanks Kim. So let's head a step back. We're now back. It's a concept that it's a website that's embedded. So you have an iframe, you'll see it soon. And the iframe is just any static website
06:23
and that website has an API that can be used to communicate with the client. So for example in this case Element. And then you have the other option that the Element client could be connected to your home server. So it's like I would say a pass-through API
06:43
which gives the comments of the widget to the home server via the widget API. And with this API you can do a lot of stuff. So one thing that is important is that this iframe also allows to have like an isolation. So even though you embed a website there
07:01
it's not like having full access to what the Element does. Everything that you do has to be done via the widget API. So from the feature perspective of the widget API you can separate it into two parts.
07:20
So on the one side you have everything that is related to displaying widget, interacting with the client. For example, Kim showed you that you can have different display modes. There are also even more. For example, you can open modal widgets that then display it in a full screen view. You can set things like always on top where the widget is displayed in the right bottom corner.
07:46
And it's always there even if you switch to a different room. And you can do capability requests. I will come to a second what that means. And then you have the second group of features which is like matrix or room interactions.
08:02
So that is everything about sending events, receiving events, state events, room events or two device messages as Florian also mentioned today. Some special things like reading relations and requesting OADC identity tokens
08:21
that you can use at another service to identify yourself. So if you go now back to this reading and sending messages you notice that actually you could use matrix for non-chat application where you store your data in the room. You can see it as like a return database.
08:42
We had the term today. And store your stuff there and have all the benefits of the matrix protocol that you would have for chat two. For example, you can store your data and have it enter and encrypted. You can collaborate with others via federation.
09:03
And all these features are already there and you can see it as a nice backend for building collaboration apps. So I already mentioned this word widget capabilities. So one issue that you would have if you provide all these features
09:23
just to an app without having any security, they could do a lot. So actually I could build a widget that reads all your data and post it somewhere else like which chats you write or does stuff in your name for example. It could send messages in your name.
09:42
And therefore there is a security mechanism with these widget capabilities. And it's actually quite similar to what Android is doing with permissions. So your web app once it is embedded and once you started it so every administrator can embed widget into your room.
10:01
Once it's embedded you have the chance to request specific permissions for your app. And then the user gets a screen like this. It's quite a long list and can explicitly allow data to be shared with the widget. And that way you have your control over your data.
10:21
You don't share anything that you don't want to share with the third party side. And only then the widget has access to it.
10:59
But maybe grouping and writing better text can help here
11:04
to avoid this situation where users just click accept without thinking about what they are sharing. So now I want to talk a little bit with you about the state of the widget API. So right now it's only supported completely in element web and desktop.
11:26
So there is support for widgets in Android and iOS. But it's more like static web pages without the widget API. And the widget API is what it gives is this interactivity stuff
11:43
that you wouldn't have with just a static web page. Why is it only supported in the two clients? I think there are at least two reasons. One reason is that it's not part of the magic spec yet. So there is like a draft here which is the spec.
12:01
But the draft is a bit outdated. It collects some of the MSCs around it. But a lot of the MSCs are not part of the spec yet. So it makes it quite hard to develop based on it. So it makes it hard for consumers of the widget API to develop it. But it would also make it hard as an embedder
12:21
because you always have to go through the MSCs, look for stuff that you need and build it. But I think it's not the only reason why it's not yet fully supported by every client. I think it's also the situation that it's maybe not the perfect implementation yet.
12:43
And what I mean with it, I want to talk about it in the next slide. If right now I want to extend the widget API with a new feature, it's not feature complete so it doesn't support everything you have in the matrix JS SDK. It has a subset of features. But if you want to bring a new feature into it,
13:01
right now we have to follow this process. So let's assume that I want to support uploading content into the media repository. So it would be quite nice for some use cases. Then I would create an MSC, which I think itself is not a problem. You look into the current spec of the client server API,
13:21
look at the behavior. For example, uploading requires thinking about quota, thinking about limits, size limits and stuff like that. And then how the API responds this information back to you. And so you look at the spec, copy that behavior.
13:42
So it's the same. And think about how you integrate it into the widget API. Then the next step, you think about like capabilities. How can I prevent users from doing stuff that they shouldn't do with it? Or how can I keep control about it? And then you have like a base to start implementation.
14:02
You would probably implement it in the matrix widget API repository, which is the API that both embedders of widgets use, but also the widget itself. And the next step, you would implement it in element. And if there would be more clients that support it, in all other clients that you want to support it.
14:23
And that's actually a lot of work. And just for something that was already there, right? So, if you wouldn't use widgets, you could upload files with the client server API. And then you notice that you can upload files, how to download them,
14:41
start the process over again. So I think it's not the problem with the MSC process, but the problem is that you have to copy stuff that is already there. So behavior that is already there, specs that is already there, it would be much better if I could just use the client server API from my widget.
15:03
And there is already like the idea of doing that. So there's this MSC that thinks about how can I share like the client server API with my widget. And it's done in this example, or in this specific MSC,
15:20
by sharing the access token with the widget, which brings like new challenges. For example, you have to think about capabilities again, because if you share the whole access token, then you can do everything. So you would have to think about how can I restrict the access again.
15:40
And one idea is there to use scoped access tokens that have, there's actually an MSC and I missed it here. You use scoped access token to create a token that is like only possible, well, it can only access the stuff that the user previously allowed the widget.
16:01
So that's something where you can mirror these capabilities that you've previously had with the widget API with this approach. So that would bring us a lot of benefits. So we would actually directly have this feature parity with the client server API. We can do, yeah, it may be also more performant
16:22
because now we rely on element loading all the data and also relaying all our requests to the home server. We could directly talk to the home server and optimize our tools. And we would also assume that it's a lot easier to implement because widget and better,
16:42
so a client like element only has to implement the exchange of the credentials to the widget, but not all the API calls that are available that have to be relayed and capabilities have to be checked and all that stuff. So it would make the implementation a lot easier
17:01
and hopefully also bring it to more clients. There are some challenges. For example, it's actually quite good that element does all this stuff because as a widget author, you don't have to do, have to think about sync. You don't have to think about E2E, E2EE, so end-to-end encryption. So it actually makes it quite simple,
17:22
but probably that's challenges we have to solve then. And there's also like this MSC proposed to bring the access token via the URL into the widget, which might also not be the best way. Maybe OIDC, once it's there,
17:43
can help us here to delegate the identity and access into the widget. In the morning, Florian already talked about this Matoska mode for element call, and there they're using the,
18:01
I think it's called Widget Room Client from the Martex JS SDK, which is quite cool because it allows you to already start using the Martex JS SDK in your widget, and it feels like it's a Martex JS SDK, but it's relayed over the widget API and providing you later on maybe a better way
18:23
to migrate to this state or to this style of API. So what would these features bring us, or what can you already do with existing widget API? So you could build really cool equilibrative tools
18:41
because you have like a real-time communication channel. You can build, you will give some examples later, sorry, I don't give them. You can build stuff where you would actually normally have to build a backend, a communication layer, and all this stuff,
19:00
and think about a lot of stuff that Martex already has and brings. You can use the rooms for data storage. There are actually some tricks needed to do that efficiently. That would be like the talk for itself. But you have the idea that all these applications
19:24
that you build could just use it. I talked about real-time communication before, so actually if you use Martex, you have some kind of very slow real-time communication. It's not suitable for building more complex
19:43
or more quicker stuff like, for example, a game or a whiteboard or stuff like that. That's where, for example, Martex FDC comes into play, where you have direct peer-to-peer connections or via SFUs.
20:00
And if you have access to that in your widget, that would allow you to build really great stuff. And actually, if you reach your limits with widgets, you always have the options to switch to more components, for example, like a bot that is invited into your room
20:20
and helps you to do stuff that you could actually not do with just the widget at once on the user side. Kim will give now some examples for widgets that we built and also does a quick demo. Thank you, Oliver.
20:45
Here are our use cases, the widget that we built. As you can see, there are four of them. The first one we built is the pulse widget. As you might know, there are pulse in element now,
21:01
and I believe they are coming to the spec. I think they're still in MSC. These allow you to do some simple pulse, but you might have some cases where you actually need to do some more fancy things, like you want to even use parties, for example.
21:21
And so we built this pulse widget that allows you to cover many more advanced use cases. And in fact, this is already open-sourced at the end of last year, I believe, in November, December sometime. And you can find it online under this link on our GitHub.
21:41
So the next one is the bar cam widget. If you are unfamiliar with the bar cam, the idea is to meet in a group, collect some topics, and then build a schedule right there and then have an event based on that schedule. I'm going to show it to you in a minute. So this is our second widget that is also now open source.
22:05
And further, we are also developing a meetings widget, which allows you to create appointments within the widget. And it will set up rooms for you and set up the possibility to have a video call right there.
22:22
This is a work in progress, but will also be open source. And then there's also the whiteboard widget we are building. And again, this is going to be open source at some point. When it's finished. Or in a beta state.
22:45
Right, so as I said, I want to show you the bar cam widget for a bit. And we actually got the chance to use it productively on Friday at our matrix community meetup. And yeah, we have prepared a quick video for you.
23:06
So right here, on the left hand side, you have the grid. And you can, for example, add tracks, you can edit the track names, you can choose some icons on that axis. And of course, you can also modify the other axis,
23:22
which are the different time slots. You can move stuff around, change the length of stuff. And then once that's set up, you can enable the topic submission. And you and all the other users in the room can create this kind of post-it cards, where you enter your topic and maybe a short description.
23:44
And then once you send it, it will appear here on the right in the parking lot. But yeah, because this is not yet quite supported on other platforms than Element Web Desktop, we also built a bot as a compatibility layer.
24:05
Which allows you to also submit topics through the chat right here. You write a bot command, the bot will convert it to the event that's read by the widget. And you see a tick, and it also appears right here in the select your topic button.
24:23
Here's the first one we created. At that point, you can even edit as a moderator. And then you just move it into the schedule. And then you can select the next topic and also review it. Maybe you edit it, maybe you don't. And put it on your schedule.
24:46
Yeah, you have the feature of locking and unlocking submissions for the non-moderator users in your room. And I believe that's it. All right. So, yeah, thank you very much everybody.
25:01
And it's now time for QA. We also have this QR code. If you want to find us on Matrix, you can use the QR code. You can use this room alias and come talk to us. Of course, you can also find us in the dev room online.
25:20
We have to order the members in our room to find out about the widget.
25:43
There was something called a dimension, some place to post the widget. How does that work? Right. Do you want to answer? Okay, repeat the question. So I think it was kind of two questions. So one question was how do users find the widgets that are installed in a room? And then I think the second question goes more into the direction of integration managers with dimensions.
26:03
So there's something like that available. So widgets are like state events in a room. Once you added it to the room, you have like an event with a URL. So it could be hosted on any web server that's then embedding it.
26:21
And you have a question about discoverability. So yeah, there are integration managers. For example, dimensions that you can use to add widgets. But at least I mentioned doesn't have such good support for widgets that use the widget API. So you would probably need something else.
26:42
I don't know about any integration manager that supports them very well. Let's just take one of these bar examples. How does that get into... Let's say I wanted to have that widget and host it and my thing is easy enough to do. So right now you have to have the URL of the widget.
27:04
And then you can just use the add widget command and add it to your room. But an integration manager would be great. So I could just click in the bottom right of element. And there is already this integration for it. And then as a room admin or moderator, you even have the ability to pin the widget to the top of the room, for example.
27:27
Or to maximize it and then to save that view state for the room. And then everybody else in the room will automatically also have the widget opened for them.
27:42
So just added additional info. So in the sovereign react place, we have the meeting widget which is like the start point for creating meetings with widgets. And there we have some kind of integration manager built in. It doesn't help for other rooms, but there the user already has the option to just take the widget that they want for the meeting.
28:03
I'm on my screen in this virtual presentation of the dev room, the widget. Which is showing me all the questions where the people ask the questions in there which are most upvoted. And there is a question. They ask me, how do widgets manage the end-to-end encryption?
28:21
So does widgets have access to the encrypted rooms? So the question was how do widgets manage end-to-end encryption. And right now it's actually quite transparent to widgets because they don't know about end-to-end encryption. So the client itself does everything and just returns the already decrypted events to the widget.
28:43
And the other way around. So as a widget, I just send a message over the widget API. And Element, for example, does a heavy lifting of encrypting it and sending it to the room. Any more questions?
29:05
Is there a matrix RTC widget? Yes and no. So Element call itself is a widget too. So it uses also the widget API and implements all its stuff, also the matrix RTC stuff, via the widget API.
29:24
So it's currently I think the code for that is mainly the matrix.js SDK and then they use this room widget client to communicate via the widget API with the client and then the room.
29:41
Is there a place to discover widgets or is there a collection of widgets? Where to find them? Actually it's quite hard right now. So the question was how to discover widgets. Is there a place, a central place where you can find them? I would say yes and no. So there is a widget or integration manager like dimensions.
30:03
But it only has a subset of all the widgets that are available built in. And there's right now not something like a store or collection where you can easily choose all of them. I believe the matrix.org website either is already or is going to collect a list of all the available widgets.
30:24
So you can browse them there. Or yeah of course if you create a widget or any other app yourself you can make a pull request against the matrix website.
30:42
So please let everybody know about anything you build.