Talk to DBus from a Python application
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Title of Series | ||
| Number of Parts | 542 | |
| Author | ||
| Contributors | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/62027 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
00:00
Multiplication signMiniDiscSoftware engineeringKeyboard shortcutDemonCommunications protocolProgrammer (hardware)Expert systemInterprozesskommunikationCartesian coordinate systemPhysical systemLaptopMereologyLevel (video gaming)Bus (computing)Library (computing)Computer animation
01:12
Online chatPhysical systemInterface (computing)Peer-to-peerBus (computing)Physical systemObject (grammar)Library (computing)Service (economics)Representation (politics)Computer animation
01:41
Keyboard shortcutLevel (video gaming)Service (economics)MereologyLibrary (computing)AbstractionComputer animation
02:05
Proxy serverOnline chatMessage passingLetterpress printingLoop (music)Message passingService (economics)Event horizonObject (grammar)Multiplication signProxy serverBlack boxConnected spaceBus (computing)Loop (music)Cartesian coordinate systemComputer animation
03:41
Loop (music)Interface (computing)Physical systemWorld Wide Web ConsortiumNewton's law of universal gravitationCategory of beingDemo (music)Game theoryMaxima and minimaGamma functionMessage passingEvent horizonService (economics)CodeNumber2 (number)Server (computing)Slide ruleClient (computing)Loop (music)Signal processingOnline chatInterface (computing)Letterpress printingComputer animation
06:36
Online chatLoop (music)Service (economics)Event horizonMereologyProcess (computing)Cartesian coordinate systemLoop (music)Computer animation
07:20
MathematicsObject (grammar)CodeMultiplication signProjective planeDeclarative programmingLatent heatComputer animation
08:15
Bit rateOnline chatSocial classString (computer science)Letterpress printingCategory of beingData typeLatent heatMultiplication signObject (grammar)MappingStandard deviationParameter (computer programming)Interface (computing)Social classMessage passingImplementationType theoryCategory of beingInternet service providerComputer animation
10:10
Information managementMassAliasingMaizeGame theoryDedekind cutOpen setWorld Wide Web ConsortiumInterface (computing)Demo (music)Context awarenessEvent horizonServer (computing)Connected spaceBus (computing)Interface (computing)Projective planeMessage passingImplementationSocial class
11:59
Online chatProxy serverError messageLetterpress printingData structureComputer configurationComputer fileDifferent (Kate Ryan album)Default (computer science)Video gameMappingData dictionarySocial classBus (computing)ImplementationError messageServer (computing)Client (computing)QuicksortSystem callInterface (computing)Uniqueness quantificationObject (grammar)NamespaceGroup actionType theoryData structureData typeException handlingLibrary (computing)Latent heatData managementService (economics)CodeMessage passingMathematicsPhysical systemAttribute grammarResultantString (computer science)Computer animation
18:21
AdditionLibrary (computing)Position operatorCodeBus (computing)ImplementationPatch (Unix)Projective planeComputer animation
19:56
Library (computing)Open setDemo (music)Link (knot theory)Computer animation
20:22
Online helpCartesian coordinate systemMessage passingOnline chatComputer animation
21:17
HexagonRight angleHypermediaCodeScripting languageCartesian coordinate systemPlanningEvent horizonAdditionMathematicsQuicksortAbstractionLibrary (computing)Multiplication signLoop (music)Default (computer science)Flow separationServer (computing)Level (video gaming)Category of beingBus (computing)Client (computing)Computer animation
25:41
Program flowchart
Transcript: English(auto-generated)
00:05
And it's time for our next talk. I would like you to welcome Vendula. The stage is yours. Hopefully, oh, I can talk, okay. So hello, I'm Wendy.
00:21
I'm a software engineer. I work at Third Hat for six years now, and I'm part of the system and star team. And today, welcome on my talk about the communication with DBaaS from a Python application. So first of all, I would like to clarify that I'm in no way an expert on DBaaS.
00:41
I'm just a very lazy programmer who wrote a library to make her job easier. So what is DBaaS? It's a shortcut for the disk to bus, and basically it's a system for the inter-process communication. It consists of two parts, the protocol and the bus daemon and on a typical Linux distribution,
01:03
you can usually find the two bus daemons, the session bus and the system bus. So for example, this is a screenshot from my laptop. This is the visual representation of the system buses and services that you can find.
01:20
And for the demonstration purposes, I've created the example chat service. And you can see on the right side that this service provides four objects, and these objects implement some interfaces. So how do we talk to this thing from Python?
01:40
We will use the DaaS bus library for that. It's a library that I wrote some years ago, and basically it's an abstraction level above Python bindings for the GDB library. Okay, so let's jump in. Let's start with the client part. So we know that there is a DBaaS service that you want to talk to,
02:01
and how do we do it from the Python application? First of all, we need to establish the connection to the message bus. In this case, we know it's the session bus, so we will use the session message bus. And the other thing that we have to create is the proxy of the remote object we want to talk to. For that, we need to know two things.
02:22
The first one is the name of the DBaaS service. The second one is the object path of the remote object. After that, we can use the proxy like any other Python object, so we can get and set properties. We can call methods. And another thing we can do is to watch DBaaS signals.
02:44
DBaaS signal is something you can connect to. You will create a callback, connect this callback to the signal, and every time this service emits the signal, your callback will be called and run and processed.
03:00
So this is how we do it. The service or this room proxy has one signal called message received signal, and you can connect the callback that will just print the messages that you will receive, and that's it. This requires one more step, and that's to run an event loop.
03:21
Basically, it's a little complicated to explain. It's just a black box that runs forever and handles any asynchronous events that can come up, like emitting of a signal. So the things runs forever unless you stop the loop or kill the application.
03:41
Yeah, so let's actually do some demonstration, because I think the demonstrations are doomed to fail, but let's try. So here I need to start my server. Let's not dive into that yet. It's just I need it to have a training, and we can actually check the defeat,
04:03
and on the session bus, you can look for my example chat, and you can see what you saw on the slides. Sorry, it's so little. And there's my Room 3, and there are some interfaces, and this is the interface I'm interested in. So let's do some stuff with it.
04:21
The first one, we will ask for the name of the room, and that's it. So here I can just write the first one, and it just prints three, because we asked the room number three for the name. The second thing that we can do
04:41
is to send a message to the room. So this is the number two. Yeah, this doesn't print anything, but as you can see here, the server received a message for this room, and it printed. So if I call it several times, it prints the stuff, and if I change it to something else,
05:07
I can actually talk to another object and send another message and stuff like that, and it just will send a different message to a different room. Yeah, it's a very primitive chat. Don't try to find something clever about it.
05:24
And the last thing I want to show you is the signal processing. So this is how it looks like. The callback will just print the received message with some additional stuff, and this is where I connect the callback to the signal,
05:42
and then I will just start the event loop. So for that, I actually have to... Yeah. So this thing is listening. It's waiting in the event loop, and it's waiting for any events. So if I, again, send some messages to the...
06:03
Oh, and something. Yeah, because I changed it. That's why you shouldn't change your code. It should work now. Yes. Okay, so because it's listening on the room three,
06:21
so if I send a message to the room three, it will print it here, and this service behind it, it's still running. It receives the messages. So okay, that was the client side of the things. Let's be the service. How do we do this?
06:40
First of all, we need to register the DBAS name of the service. That's basically you announce the name of your process. So the other applications can find you and reach you and talk to you. The other thing that you have to do is to publish some objects. So other applications can actually use some API
07:01
that you provide and do some stuff with your service. And the last thing, again, is to start the event loop, and the event loop handles the incoming DBAS calls, calls with relevant handlers and callbacks, and sends the return values to the callers. So the last part that is missing
07:21
is how do we create this DBAS object? Every DBAS object needs to provide something called the XML specification. It's a declaration of all interfaces, methods, properties, and signals that this DBAS object implements
07:42
and that you can call. And when I saw this for the first time, I thought, oh my god, my colleagues will do some metai posts and things, and we will have so many bugs. So the first mission I had was actually to get rid of this and make it generated automatically from the code, because I knew that we are going
08:02
to do a lot of changes all the time because it was a huge project. And in no way we would keep this same as the code. So let's look how it can be done with DBAS. All you have to do is to just use the DBAS interface decorator and provide the name of your interface.
08:23
And this decorator just looks at the class members of the created class, and for every member it will generate this piece of the specification and it will create this whole specification. Sometimes it collects more interfaces,
08:43
it's a little more complicated. But at the end, you don't have to do or write any XML, but you can have access to it and use it to publish your object. So we will start with this decorator, and then you can just define a DBAS method.
09:00
This is definition and at the same time the implementation of the method. So you can see it prints the message. One thing that you have to do is to provide type hints for the arguments and the return values, because DBAS needs to know about the data types of the arguments.
09:21
And another thing, yeah, everything is camel-cased. I'm sorry about that. It's a standard for DBAS and it didn't make a lot of sense to try some mapping from the traditional Python to this camel-cased thing. So it's easier to just try the camel-cased. So this is the method, and this is how to define
09:43
a DBAS property, it's just a Python property with a type hint again. And last but not least, DBAS signal, you need to use a special decorator for that. And if this signal emits some additional arguments, you need to specify them as arguments of the method,
10:03
kind of, yeah. This method is never called, it's just used for the definition of the signal. So, that's it. Let's have a look at the implementation of, how it looks like when you put it all together. So this is the classroom.
10:23
There is the decorator, and as you can see, there is just all the definitions and implementation that this class needs, and the XML is just generated by the decorator. You need to, you don't need to care about anything. The chat also has a DBAS interface, but it doesn't contain anything.
10:42
Here we can see later how the XML looks like, and you create a connection to the message bus, you register the name of the service, you publish some objects, and you start the event loop, and that's all. In this case, it's good to disconnect from the bus.
11:02
There's an open for request to use the session bus as a context manager, because it's nicer, but it's not matched yet, because it will unregister everything that you did here, so that's a good thing to do. So, and here we can see that when the server was started, it printed the interfaces that were generated,
11:22
so the first one is empty, there's nothing there, but with the room, it contains everything that was inside of that class. So, yeah, this was completely generated, you don't have to care about it, you don't have to figure out what type S is. In this case, it's simple, but sometimes it gets a little more not so pretty.
11:44
So, yeah, you don't have to care about it, which is, I think, great. Features. So, this bus has a lot of features, because the project I was working on was big, so we wanted to do as many simplifications as we could. So, one thing that we did,
12:01
oh, okay, I want to mention one thing. I decided I will focus on the end result of these features, because it's a little difficult to explain the definitions, and all the steps you need to do before that, so I will just show you what you will end up with in your code. So, the first thing I want to mention is the management of the bus names and paths,
12:22
because you could see that there are a lot of strings you need to handle, and it's very easy to make typos in this again. So, I can, okay, yeah. So, it's very easy to make typos in that as well. So, basically, this bus provides a system
12:40
that allows you to define namespaces and objects that are published in these namespaces, and at the end, you will have these very simple objects called chat and room3, and you can just use these objects to create proxies, or ask for the interface name,
13:00
or ask for the service name, or get the object path, and you don't have to care about the strings behind it, because they are created from what you defined earlier. Yeah, I can get to that later, if we have time. So, another thing that this bus provides,
13:20
it's management of a group of publishable objects. So, let's say that the chat is not static, it doesn't have only three rooms, but you can ask the chat to create a new room, and you want to get the bus path of that room, so you can connect to it. So, yeah, you can implement it manually on your own,
13:43
and make sure that every room has a unique path, and if someone wants to do something with that room, you can just figure out, again, what was the room, or you can just use the room container. It's very easy, you just provide the namespace that the container can use, and you will specify the message bus
14:01
that can be used to publish these objects, and the whole purpose of this container is just to give it a Python object, and get a D bus path, and it works the same way backwards, so if you receive an object pass, you can get room. So, with this mapping,
14:22
you can deal with this mapping very early, and in your code, you only have the objects, you don't have to care about the D bus implementation behind it. So, yeah, it's a little difficult to explain, but it can simplify your life load.
14:43
Yeah, another thing I want to talk about is how to handle D bus errors. So, this bus raises a default exception by default, but sometimes you want to handle a specific D bus error, or maybe you want to raise a specific D bus error from your service.
15:02
It's a very easy thing to do. There's a special decorator for that that you can use for your exceptions, and in the decorator, you specify the D bus name of this error, and that's all you have to do to be able to use this exception in your code. So, once you decorate it, you can raise it in a service,
15:20
and you can accept it in the client, and you don't have to care about the magic between that. Yeah, that's also a cool thing to do. D bus structures. So, this is very, yeah, this is a funny thing. D bus doesn't have native support for structures, so what everyone does is they send dictionaries
15:43
that map attribute names to attribute values, and since these values can be of different types, you have to wrap them inside variants. Variant is a special data type that basically couples the data and the type together, so when you send it from your service,
16:04
the client is able to interpret the data even though it didn't know the type of this data before. So, this is pretty horrible thing to work with, especially when you need to receive the structure, change something, and send it somewhere else,
16:21
because creating these variants is not easy. Variants are not changeable, so you have to always create new ones, and yeah, it's not pretty. So, with this bus, you can actually describe the structure using data classes, and these classes just have some properties, and there's a lot of automation that allows you to basically give it the dictionary.
16:43
It will transform the dictionary into a Python object, then you can just play with the Python object, and when you need the structure again, you will just go to structure to get the structure that you can send on D bus. Yeah, so that's another thing you can do. So, lastly, this is a new feature
17:03
that I was working on with some people last year. Yeah, I would like to think this way. I would like to use this to thank everyone who was involved in this, because it was bigger issue than I expected, and basically, you can send Unix file descriptors
17:21
through D bus. It works only on Unix systems, obviously, and it's optional feature. It's disabled by default because there's some overhead, and I didn't want it to slow down everyone's services. So, all you have to do is when you create the proxy,
17:41
you will specify a little different client library or the server library that will be used to process the incoming calls or the requests, and it basically means that if, yeah, it's sort of complicated,
18:00
because D bus has a special support for Unix file descriptors, but it's very messy when you have to deal with it specifically, and with this extension, you don't have to care about it, basically. If you want to send a file descriptor, you will just send a file descriptor and receive a file descriptor on the other side.
18:21
Yeah, so these features can be very hard to understand, and I get it. So, I want to mention that this is optional. It's not something that you have to use if you want to use DAS bus, and I would suggest if you don't have the needs to use them, don't use them. Keep it simple. Do whatever is easiest,
18:42
because there can be a lot of additional code that can be hard to understand. Another thing I want to mention is that I acknowledge that every project is a little different and has very different needs, and sometimes you can make a lot of assumptions
19:00
about your service, like you might not need half of the D bus support that there is, so you can simplify some stuff a lot, and that's great, but that's often not generic enough to be implemented in a library like DAS bus. So, what you can do, actually, is to take any piece of DAS bus,
19:21
or implement it to fit it to your needs, and propagate it in the right places so DAS bus will use it instead of the original implementation. I want to mention this because we were in this position at the beginning of our project, and we had a lot of troubles with the library that we used back then,
19:42
and basically we had to patch the whole library because we were not able to get there the stuff that we needed, and it wasn't easy to just change it, so we have to patch it. So, this is the link to the library. There's an open discussion session,
20:03
there's an issue tracker, so if you have any suggestions, questions, you can find me there, reach me there. Yeah, don't hesitate to ask. There's also, there are some examples. You should be able to find the examples that are similar to the one that I showed you.
20:20
I think I will post there also the demo stuff because it's easier to understand. Yeah, and there's also documentation that might help you, because maybe this talk didn't help you so much. Yeah, so that's all from me.
20:40
Thank you so much for coming. Yeah, does anyone have any questions?
21:01
Hi, I just wanted to ask, where do you find people are using this? Is it in chat message applications, or what are the applications of this for most people? What are the applications who are providing the D-Bus API that you can, ah, okay, so like on the UNIX system, it's basically any,
21:24
like there are printers, or you can control the media player, or you can set up your firewall, or on the session bus. I think HEX chat actually has their own D-Bus API. So, it's more like the applications
21:40
that are running on your desktop often provide this D-Bus API, so you can write some scripts to tweak them, and to control them. So, yeah, that's it, I'm sort of, yeah. Does D-Bus support properties and annotations?
22:05
Properties, yes. Annotations, do you mean like properties changed annotations, or stuff like that? Like, emits changed signal, and deprecated, and stuff like that.
22:21
I'm not sure about it. I think it's not like needed, it's just like a recommendation for the documentation, but it's not something that... Otherwise, the client can't see that the server may support an API call, but it's a mark that's deprecated. Oh, okay. Yeah, I think it doesn't support custom annotations,
22:44
but that's definitely something I can look at, if it's like... So, there's one that you can add them, that's fine. Yeah, yeah. So, I don't think that this is supported, but it's definitely supportable, so, yeah.
23:03
Hi, why would someone want to use D-Bus, I mean, Daspus, versus some other D-Bus library that's out there, for Python? Yeah, okay. So, this library was actually inspired by PyD-Bus, which is also very popular.
23:21
It just, we hit some issues, and it's complicated. Like, you need to think a lot for us. So, at some point, I just got so frustrated, I decided to rewrite it, and create Daspus. But, yeah, there are a lot of interesting libraries, and sometimes they are a little simpler, and it might be enough. So, like, you don't have to use this one.
23:43
It's just, this is much easier if you, if you have a lot of D-Bus API, because, like, with our project, there are several D-Bus services, and it has a lot of objects, a lot of interfaces, and it would be very difficult to deal with it
24:00
in a, with a library that operates on lower level. So, we needed a lot of abstraction to make sure that the code is okay. We have time for one last question. Who was first? Maybe two questions.
24:24
Hello, thank you for the library. I've been trying it, and it's great. I wanted to ask a question regarding the event loop. Lately, I've been doing some work with D-Bus, and I find it very painful that most libraries rely on the glib main loop,
24:45
rather than the default event loop coming from async.io. I saw that in the code base, there is an abstract event loop that could become something else, but do you have any plans about that? Yeah, so, right now, I don't know about the demand
25:03
for additional support for other event loops, and, like, backends, but the code is implemented in a way that it should be possible to do it. So, if there is, like, enough people who would be interested in this, that's definitely something I would like to look at. It's just, yeah, it's no demand right now.
25:23
Thanks a lot, thank you again. You probably have a great audience for people interested in Linux desktop here.