Say hi to Avahi
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Title of Series | ||
| Number of Parts | 33 | |
| Author | ||
| Contributors | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/54684 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
2
3
5
6
7
9
13
14
18
19
21
22
24
26
28
29
30
32
33
00:00
Local ringComputer fileVirtual realityMusical ensembleBitComputer fileGoodness of fitConfiguration spacePresentation of a groupCodeComputer animation
00:35
Computer networkConfiguration spaceCASE <Informatik>MathematicsCodeCodeSharewareConfiguration spaceCASE <Informatik>BitLimit (category theory)
01:27
Programmer (hardware)Direction (geometry)Digital signalPascal's triangleAssembly languageJava appletScripting languageOpen sourcePoint cloudScale (map)Computer networkLaptopBackupSystem programmingSoftware developerProduct (business)Revision controlDirect numerical simulationService (economics)Local ringLocal GroupAreaLocal area networkType theoryPersonal digital assistantComputerRouter (computing)Broadcasting (networking)Latent heatUnicastingverfahrenAddress spaceBefehlsprozessorBroadcasting (networking)WindowLocal area networkBitRouter (computing)MultilaterationNeuroinformatikOpen sourceComputing platformSinc functionData miningType theoryCodeDirect numerical simulationProduct (business)ImplementationAreaCommunications protocolPlastikkarteOperating systemComputer architectureInstallation artMultiplicationModulare ProgrammierungDifferent (Kate Ryan album)Data centerQuicksortData storage deviceEnterprise architectureChaos (cosmogony)Musical ensembleIntegrated development environmentBefehlsprozessorMultiplication signSoftwareUnicastingverfahrenTime travelSimilarity (geometry)Computer hardwareService (economics)Physical systemFirmwarePoint (geometry)Interface (computing)Address spaceSlide ruleInformationCartesian coordinate systemSoftware developerComputer programPoint cloudScaling (geometry)Assembly languageDirection (geometry)Public domainLaptopBackupCASE <Informatik>Computer animation
11:19
Address spaceBefehlsprozessorChaos (cosmogony)Dew pointStandard deviationWorkloadClient (computing)Data managementSystem programmingComputer hardwareDatabaseComputer configurationImplementationLocal area networkWide area networkCASE <Informatik>Query languageDirect numerical simulationRegular graphComputer networkService (economics)Server (computing)Musical ensembleUniform resource locatorMiniDiscWorkstation <Musikinstrument>Address spaceOpen sourceServer (computing)Virtual machineInformationQuicksortDirectory serviceMathematicsData managementGoogolExistential quantificationBroadcasting (networking)Flow separationAuthenticationRootEnterprise architectureElectronic mailing listRepresentation (politics)Software developerIP addressSoftwareIntegrated development environmentConfiguration spaceService (economics)Library (computing)Inheritance (object-oriented programming)Cloud computingLocal area networkDirect numerical simulationSynchronizationPublic key certificateComputer hardwareMacro (computer science)Query languagePay televisionData conversionClient (computing)Computer programAbsolute valueChaos (cosmogony)InternetworkingBridging (networking)Router (computing)NeuroinformatikRow (database)Key (cryptography)Local ringStandard deviationMultiplicationBitWeb pageRobotCASE <Informatik>Water vaporCodeRegular graphEncryptionBuildingRoutingComputer animation
21:11
Service-oriented architectureWebsiteLocal ringInformationServer (computing)SynchronizationExecution unitWeb pageClient (computing)Computer iconRippingChemical equationWeightNumberServer (computing)Electronic mailing listClient (computing)Maxima and minimaSocial classComplex (psychology)Local area networkMereologyInformationSynchronizationService-oriented architectureDefault (computer science)AreaDirect numerical simulationMultiplicationFunctional (mathematics)Set (mathematics)CodeDescriptive statisticsPhysical systemWeb pageComputer programLoop (music)Local ringService (economics)Greatest elementPoint (geometry)Mixture modelUniform resource locatorControl flowIP addressTouchscreenPointer (computer programming)Musical ensembleWeightWebsiteConfiguration spaceEnterprise architectureBlock (periodic table)Message passingIntegrated development environmentIdentifiabilityInstance (computer science)BitSingle-precision floating-point formatCountingZoom lensDatabase transactionDot productSource code
28:33
WeightLocal ringClient (computing)Computer networkSynchronizationServer (computing)Web pagePresentation of a groupTheorySynchronizationServer (computing)SoftwareError messageRevision controlService (economics)Client (computing)Pay televisionWebsiteTerm (mathematics)LoginMathematicsPrice indexInformationStatisticsMusical ensembleBasis <Mathematik>Physical systemComputer animationSource code
32:03
SynchronizationComputer networkComputer iconClient (computing)Web pageNormed vector spacePresentation of a groupMusical ensembleSource codeComputer animation
32:36
VideoconferencingVirtual realityHypermedia
Transcript: English(auto-generated)
00:08
Good afternoon, evening or morning for everyone. I'm Patrick Fitzgerald and my little presentation
00:22
in the tiny little bit of code is all about zero configuration of whatever you want, not just files and printers. So what the hell is a zero configuration?
00:40
What's DNS, what's Avahi, what's multicast DNS? And to get to that, we're gonna have to go through a couple of steps. So if you know a lot about networking, you're gonna get bored. If you don't know anything, hopefully it'll be helpful.
01:01
And our use case of why we needed it, what we wanted to do with it, and how it's actually working very well. There are some limitations and there's a bit of code we're gonna go through. And we're gonna do a very small demo because, well, that's what I promised,
01:21
but actually having done it, it's not this, it's underwhelming because it just works. So this is me, I'm Patrick Fitzgerald, CEO of Required Magic. We specialize in large scale Linux deployments. Been a programmer since forever.
01:42
Spent about eight years working in film and television before it all went digital. And a friend of mine once described me as being creative in all directions, but I'm not sure if he was insulting me or complimenting me, I'm not sure about that. Some of the things I've done in 2010,
02:00
we built with a colleague, we built an open source cloud based on SUSE Linux. And it's still running in data centers in both Zurich and London, and it's got some large financial institutions using a third party software package that are basically the people we're hosting for.
02:24
And I'm also a refugee because of Brexit, because about six months ago, I realized it was all turning quite sour and the relationship from Britain to the rest of the world is troubling,
02:42
I mean, to say the least. So I've moved to Germany and enjoying it very much. And I'm a survivor of many things, not including a cardiac arrest that happened around this time, two years ago.
03:01
But that's something for a drink later. So, bonjour, as Apple like to say. It was actually, I think it was originally designed by Apple, and when you connect your laptop, you can see a printer, and you can also see other devices
03:22
that are on your same network, on your same network segment. If you've got a Mac, which I don't, but I understand that you turn it on, and if you've got a time machine backup, it finds it automatically. And how does your computer know
03:44
where your another device is on the network? And of course, you plug in a Linux or window or Mac system, and you can see other hosts on the same network. What's that? How does that little piece of magic work?
04:00
Because it's not necessarily an obvious thing for it to be done. And if you're, you know, why do you care? Well, if you're not a developer or a product designer, you've probably got no interest in this, so you should head to the bar anyway. But it's just a bit of interesting work that we had to do because we're doing
04:22
a large-scale implementation of Linux across thousands of desktops. So the way this works is two technologies. One is called multicast DNS or mDNS,
04:41
and the DNS service discovery or the DNS-SD. That's been implemented in all the major desktop platforms for some time, way back in 2002 with the Mac. And Windows is taking its time and variously implementing it.
05:01
I can't imagine why it's taking so long for Windows to get everything working. It's a very simple protocol. They probably had their own ideas as to what to do and how to do it, like Microsoft tend to do. But the protocol itself is very simple,
05:22
and it works. Similar things have been developed to do similar things in different environments. For example, your computer will also, depending on how you've got it all set up, may discover your TV. Your TV might discover your storage device
05:44
and offer to connect to it or to your music players. And that's, they're all different implementations of the same thing, which is I've got something to offer you. I've got a service I need to offer you. How can I tell you about it?
06:07
So this is just a bit of a detail and a simple trip down the network lane. So a local area network is defined as, well, it's technically defined as a broadcast domain,
06:24
but that's defined by a bunch of computers networked together with a router at one end, and everything that has to go off the network goes through that router or multiple routers in an enterprise situation.
06:45
A LAN, and I know this because I was programming network code 20 years ago in assembler, is different sorts of packets are sent across a local area network. And I'm simplifying here because there's a,
07:03
this is all based on TCP IP. There are similar architecture that existed at layer two, but the network layer that we're interested in is TCP IP. So it does broadcast,
07:22
and the broadcast is essentially sending a packet to every machine on the network. Now that, if you're in a switched environment or in a hub, if you're in a hub, you're connected to a hub, not a network switch, which most people have switches now.
07:41
In fact, 99% of I think networking installations are switched. But if you're a hub, I send you a packet as a broadcast and every machine on the network gets that broadcast. Similarly, it works the same way in the switch.
08:03
At the broadcast is something that the machine and the network card is told to listen to and listen for, because it's something important. And it might be something similar, simple, like I wanna get,
08:21
in fact, I can't tell you the circumstances where they use broadcasting, but it's something like when you join a computer, when you start a computer up, it might send a broadcast to get a pixie information or something like that. Multicasting is similar to a broadcast.
08:43
And it's sent to a certain address, but that certain address is then distributed across all the connected hosts that are turned on. Now, the reason why there are two different types of that is because a broadcast is almost always a non-routable. So your broadcast will get to the router or router.
09:03
I'm Australian, so I can say it either way. And it stops at the router because, there's a slide coming up, right? Yeah, there we go, go chaos. But yeah, it's typically is non-routable
09:24
because you don't want those kinds of broadcasts to escape to other networks. Multicasting is also similar, it's pretty well, typically non-routable. And a unicast is when you send one packet from my machine to your machine
09:42
and no one else knows about it. So, but in all cases, there has to be a service, a network service or an application sitting on the system that is receiving the information, receiving the packet.
10:00
Otherwise, it is accepted by the network card. And rejected at some point. So yeah, broadcast packet sent all addresses. And the thing is about that, it consumes CPU. So even if there's nothing running on that machine
10:20
that accepts that packet, then the network interface card has to accept that packet and then pushes it up from the hardware into the software, through firmware into the software that the operating system is. And the operating system says, I've got nothing to do with this packet, I don't know what it is. Or it says, oh yeah, I've got something
10:42
that will accept that. So a broadcast is quite, well, back in the day when I was doing it, you could have a, you know, if everyone's broadcasting, it would physically slow down the system. It also, a lot of network protocols require
11:02
a broadcast to be answered by another broadcast. And if you get into that kind of situation, it ends up in packet hell. So the same thing happens with multicast.
11:22
And again, it goes up to the network stack to discover if something's working. Now, routable or routable broadcasts, I mean, what happens, why would you stop that, the router? And why would you stop a multicast packet at the router? Because perhaps there are things,
11:40
you know, you could send to everyone, and you don't have to send it. You don't have to get a list of everyone. You send it once. And if you're paying attention to what I've been saying, then you'll end up with absolute chaos. Because if network routers, and your routers connected to your broadband networks
12:05
could see every other computer on the internet, no one would get anything done, literally. It is like everyone's yelling at you, broadcasting information at you, and you're trying to listen. That's the worst thing is that every machine, every node on the network has to listen
12:24
to what everyone else has got to say. And pretty soon, you know, you can't actually communicate with anything. You can go to the Broadcast Storm page on Wikipedia. It's a well-known thing.
12:40
And of course, so there are multiple standards. It's just a bit of a comment there from XKCD. It's, as I said before, there are multiple standards for multiple industries that have all had their own ideas as to what is, you know,
13:01
they've got something they want to plug on, plug into a home network. How do we make that? Not a lot of it is probably proprietary and not open source. And gradually they're coming down into a common standard, which is actually called Zeroconf.
13:27
Let's get some water. So how did we get to this?
13:41
We've got a Linux deployment tool called Snoop here, because we can't think of anything better for it to call it. And one of the use cases was rather than just doing a deployment of Linux onto 1,000 or 10,000 workstations. If we're doing that, then we could inject
14:02
a monitoring tool and return information about all sorts of things back to a centralized node. So it could be managed better. We could get disk-based wellings, all these kinds of things that every other management tool has.
14:22
But this is usually for edge devices and workstation devices. And, you know, we saw some very interesting ways that we could use an agent on the machine. So we started writing one.
14:40
But one of the things is, how does... So on every network that we install onto, you have to have a, it could be a virtual machine or just a server node or sync node that then talks to the cloud service that we offer,
15:00
or if you're doing it locally at the local service. Instead of making everything hand-editing, the location of the server's IP address or the sync server's IP address, we just figured there's gotta be a better way,
15:21
because in a corporate environment, in an enterprise environment, we wanted to slip in the software and the ability to do what we're trying to do, not under the radar, but might as well be. The harder it is for someone to deploy a tool
15:41
in an enterprise environment, the less likely it is to be accepted. And that was one of the things that was driving the development, is how do we make everything as easy as possible? So instead of doing a deployment and then running another tool that will then write its own configuration file
16:04
that points the node management software to the appropriate IP address, we just thought, well, maybe we should look at zero conf. And everything we've written is in Python,
16:20
and we came across, I think there are multiple libraries, there's one called zero conf, and it's super simple to use. So what is multicast DNS and DNS service discovery?
16:44
So multicast uses a multicast packet, or several packets, to query host names on the same subject. Usually that is provided by a DNS service. I'm thinking, bearing in mind the whole thing
17:04
that I just said before about enterprise and trying not to change anything, or as little as possible, if you're in a, I mean, one of the major customers is a bank,
17:21
and they had, whenever we're talking to them, we had to get three different representatives from three different departments. So it was the hardware people, it was the authentication and active directory people, and the client manager,
17:41
and if we needed to, the networking people as well, who are generally in charge of DNS, although if you're an active directory, but you're making changes to active directory, making changes to DNS, if everyone, if it was, not everyone wants to do it,
18:01
in fact, everyone wants to do as little as possible, to be honest. So we had to find a way to provide, to get the host name of every machine that is running our software, or at least the other way around, to find the host that every machine needed to talk to.
18:23
They don't need to talk to each other, they just need to talk to our service that's sitting on their network. So we ask the question and the network tells us, and then we ask the question service, what the DNS SD does,
18:40
which is basically says, are you running this service? And can I use that service? And what port are you running at? And what's your IP address? Now, DNS SD works with regular DNS and multicast DNS. In all these scenarios, you'll have a DNS server,
19:04
but for example, if you've got 8.8.8.8, as your DNS server, there is no way that you're gonna be able to convince Google to tell your local network where your Sniffy Sync server is, for example.
19:21
So we put a text file or can be built into the code. At the server end, we can say, we're doing this kind of, we're offering this service at this IP address, come and find it. DNS works the same way with, if you know anything about DNS, there are such things as SRV and text records.
19:46
And they're things you send, you apply to your DNS server and say, this is the location of an active directory server, for example, or a configuration information
20:04
which is stored in a TXT record. And that's one of the things that you come across if you're doing, what's it called, the certificate certbot configurations for Let's Encrypt.
20:21
If you're doing a domain-wide, building a certificate for a domain-wide thing, so like star.requiredmagic.com, you have to put some text into your DNS record
20:40
so that it can look at it and go, oh, okay, so that's the key that I need to build to build the certificate. So then, as I said, the micro DNS or the macro, sorry, the multicast DNS program,
21:01
ask the network who the host is, and then it has this conversation about what it's got and how do I connect to it. So we're rapidly, a lot faster than I expected, getting to the code.
21:22
So how simple is the code? I mean, the server is, yeah, I mean, that's all you do in a Python program to say, this is what I'm offering. You can go through it. I'm not sure if you can see the pointer.
21:41
I probably can't, and I can't see any comments because my screen's too small. But you import the zero conf service and you create an instance of it, and then you say, this is what,
22:02
there's a, we've got a settings file, which basically says, this is the site UUID, which is set in a, it's put into a settings file and a label, which is broker host. And the service info is magic-sync,
22:23
and that, what follows that bit is specifically, is specific to every single MDNS entry or zero conf entry has something like that, and that says that this is a TCP,
22:41
it's an exchange, a transactional message or packet sending, and the identifier, and the other thing there is the dot local. Dot local is one of the key things of that because it's, dot local is actually for your local area,
23:04
your local area network and your home or any other things. People must make mistakes when they think that defining it in their DNS settings in say, in an enterprise environment is helpful, and it actually isn't, shouldn't do that.
23:23
So, so we'd look up some of the details. Now, what our client software, the client software and the server software is trying to send information backwards and forwards using, let's just, let's say port, what's it,
23:42
I've just had a mental block about the MQTT, that's it. So MQTT is a very lightweight message passing system, message queuing system, and that's how we send messages backwards and forwards
24:02
from the client to the server and backwards and forwards from that, and that's the port number 1883, that's the default port. We could change that in the settings we wanted to.
24:20
And the description is magic sync dot local. So then we have these two functions that are defined at the beginning of the program loop in the server. One is init zero. And that's basically in it,
24:40
in knitting the zero conf service. So I registered it, and if there's a problem, of course you log that. And then at the end of the program loop, when you're closing everything down,
25:01
then obviously you'd be nice and unregistered the service from the system. So that, I'm not sure if anyone can see that. Zoom in if possible, I think I can zoom in.
25:23
But there are two pages to this code, and a lot of it is actually just comments. And it's basically super simple, get a list of devices, get a, remove the service, start the service, count the number of servers that are offering the service,
25:42
because it might not be just one. And then you add the service to the return that what the service details are having been asked. And this multi magic listener class is how, this is as complex as it gets to start the service.
26:05
And then this is where we call that magic listener class, and we say maximum number of servers is five, and it's a five second timeout. And it just basically says, start the listener,
26:22
and then come back if there's nothing, or come back if there is something that's returned. And that's as complex as it seems to get. And then there's a mixture of stuff here that you've probably, you may have not seen.
26:43
And so the client makes for a question that comes back as a JSON packet, just saying, this is the location for the service that you need to contact. This is the port that you need to talk to. Waiting priority if they're different services
27:02
that I haven't actually got to the bottom of those yet, because this is working perfectly when we first did it. So there's no point to go any further. And the X35BCU is actually the host name, and the broker host is its IP address.
27:24
So it finds all that information just from starting up with no configuration whatsoever. And it sets things that you would normally have to put into a text file. And then our code then writes the configuration file,
27:42
so it doesn't necessarily have to refer to it again by doing the service, the zero conf stuff, but it does anyway. And we can change, this means also that we can change the location of the server.
28:01
We can change the port number. We could put in multiple servers or multiple, on multiple IP addresses or multiple ports. It's all just, you put this kind of, these couple of pages into your Python code. And you'll end up, and you've got this,
28:21
the similar service sitting on the server side, you will then be able to do everything in a zero configuration like this, and you'll get the information back. I love it. And that's actually the end of the presentation, but it's not because, in theory, if I go over here,
28:42
I can give you a very brief demonstration of this. Let's see if we can make that larger. So we've got the, this is the server
29:04
and this side is the client. And unfortunately, due to a, out of all things to happen, a networking error or a network problem
29:21
that I've got at home, I have to connect to a client site to tell them to do this, to demonstrate this. This particular version of Snippy Sync doesn't have any, any, oh, I'm not here.
29:42
There's no logging. So it's almost pointless not showing to you because the log doesn't come up. I can go there, let me see, there. So you won't actually see anything change at the server side.
30:10
There'll probably be some errors that pop up, but they're irrelevant. There's nothing really, but what you're seeing there are things that are being sent from different, I don't know if there's an error,
30:24
but this is a client there. Actually, this is the sync server subscribing to the MQTT server. They're just running on the same machine. So it's not very dramatic, not very interesting.
30:41
But here on the client side, it's kind of where it gets, it gets cool. So we'll restart that and then immediately call the log and you'll see there, it's checking for zero-conf services
31:02
and it's found them. It's found one because there's one and it's there at that and it writes the, and it's already sending this information to the server. As I said, there's no, I don't think there's any indication that's receiving that, maybe. Published stats, I think, is one of the things that goes up upstream.
31:22
But it sent all that information immediately upon subscription to the service. And it does this on a regular basis, which is, on our system, is hand. And it's configurable so it can do it. And you'll also see that they're running
31:42
an old version of Leap, so we'll have to fix that. And yeah, so that's kind of it in terms of the way it all works.
32:03
And so yeah, that's kind of it. So I'm just going back to the, so I think that's kind of, that's the presentation.
32:22
And I can answer some questions in the chat room. So I'll see you there.