Building an embedded VoIP network for video intercom systems
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Title of Series | ||
| Number of Parts | 490 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/47461 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSDEM 2020243 / 490
4
7
9
10
14
15
16
25
26
29
31
33
34
35
37
40
41
42
43
45
46
47
50
51
52
53
54
58
60
64
65
66
67
70
71
72
74
75
76
77
78
82
83
84
86
89
90
93
94
95
96
98
100
101
105
106
109
110
116
118
123
124
130
135
137
141
142
144
146
151
154
157
159
164
166
167
169
172
174
178
182
184
185
186
187
189
190
191
192
193
194
195
200
202
203
204
205
206
207
208
211
212
214
218
222
225
228
230
232
233
235
236
240
242
244
249
250
251
253
254
258
261
262
266
267
268
271
273
274
275
278
280
281
282
283
284
285
286
288
289
290
291
293
295
296
297
298
301
302
303
305
306
307
310
311
315
317
318
319
328
333
350
353
354
356
359
360
361
370
372
373
374
375
379
380
381
383
385
386
387
388
391
393
394
395
397
398
399
401
409
410
411
414
420
421
422
423
424
425
427
429
430
434
438
439
444
449
450
454
457
458
459
460
461
464
465
466
468
469
470
471
472
480
484
486
487
489
490
00:00
Computer hardwareVideoconferencingStandard deviationGoodness of fitSoftware engineeringCore dumpTouchscreenProjective planeBuildingPoint (geometry)Multiplication signProduct (business)VideoconferencingElectronic visual displayRepository (publishing)Ring (mathematics)HypermediaMereologySoftwareWhiteboardInformation securityPresentation of a groupCoefficient of determinationCASE <Informatik>RingnetzRow (database)Line (geometry)Interface (computing)Different (Kate Ryan album)Key (cryptography)InternetworkingInternet forum1 (number)Open sourceComputer-assisted translationPlanningProxy serverBitDistribution (mathematics)AnalogyDemonMobile WebArmDigitizingSession Initiation ProtocolSystem callMultiplicationMessage passingAsynchronous Transfer ModeClient (computing)Physical systemINTEGRALConnectivity (graph theory)Computer hardwareStandard deviationInternettelefonieReal-time operating systemMobile appCommunications protocolDerivation (linguistics)Lecture/Conference
09:12
Mobile appPersonal area networkPresentation of a groupOrder (biology)VideoconferencingElectronic mailing listProxy serverConfiguration spaceRow (database)Projective planeIP addressProduct (business)Form (programming)Wave packetCartesian coordinate systemInheritance (object-oriented programming)InternetworkingTouchscreenService (economics)Information securityRevision controlGame controllerCoefficient of determinationLibrary (computing)RingnetzCategory of beingCore dumpMultiplication signInstance (computer science)FingerprintMereologyRing (mathematics)Group actionSmartphoneComputer-assisted translationComplex (psychology)RoutingPhysical systemSystem callBefehlsprozessorFormal languagePasswordComputer fileDomain nameAuthenticationSheaf (mathematics)State of matterProcess (computing)Hacker (term)Local ringAddress spaceTransport Layer SecurityLevel (video gaming)HypermediaSession Initiation ProtocolPoint (geometry)Line (geometry)DemonCommunications protocolStandard deviationBuildingSoftwareStreaming mediaLecture/Conference
18:52
Mobile appSystem callImplementationCodecCodierung <Programmierung>Proxy serverTablet computerSession Initiation ProtocolSoftwareInternettelefonieVideoconferencingStreaming mediaComputer hardwareMultiplication signCommunications protocolOperational amplifierWave packetOpen setOpen sourceTouchscreenProduct (business)WhiteboardLecture/Conference
20:48
Speech synthesisComputer hardwareService (economics)Internet service providerFamilyInternettelefonieTerm (mathematics)SoftwareSystem callProjective planeMultiplication signRegular graphNumberQuicksortBitConnected spaceType theoryOpen sourceLevel (video gaming)Proxy serverTelecommunicationProduct (business)Client (computing)Standard deviationInternetworkingLocal area networkBit rateGoodness of fitMaxima and minimaCoefficient of determinationRight angleMixed realityVideoconferencingLabour Party (Malta)Order (biology)Presentation of a groupLecture/Conference
23:43
VideoconferencingStandard deviationAddress spaceInternet service providerInternettelefonieSession Initiation ProtocolSystem callSoftwareElectronic signatureLine (geometry)Lecture/Conference
24:17
Standard deviationInternet service providerClient (computing)Multiplication signComputer animation
24:43
Point cloudFacebookOpen sourceComputer animation
Transcript: English(auto-generated)
00:05
Good morning everybody, my name is Jean Monnier, I'm a software engineer since 1999 and I'm mainly involved in the Linfone project since 2010, so in a few words, the Linfone project
00:25
is a voice over IP software developed for Linux in very early 2000. So what I'm going to be talking about is how to use the voice over IP technology to build
00:40
an intercom system for the entry. So the presentation will first start to introduce the use case, so what it is about when you want to build this kind of system. So what is the voice over IP technology that we suggest to use and how to build a very
01:06
simple voice over IP network using Raspberry Pi as an example and with the Linfone software and what would be the next things.
01:21
So the simple use case is like this, so you have an intercom doorbell in front of your building and you want someone to open the door. In the past, the very simple solution is to have a camera in the front of the building
01:46
and to use a simple coax cable to link the camera to the home screen. So it's pretty cool but now if you want to have multiple displays at home like the
02:03
home screen but also why not a mobile device or a PC. So we start to be in the multi display use case. Okay, so same thing, I have the coax but what if I want to bring the video to my mobile
02:21
device for instance, so it's become much more complex and what we can also imagine that the same kind of use case, so you want the video to be displayed at home but also in the street on your mobile device if you want to know if something is happening
02:43
in front of your home. So the good thing is that we can leverage on digital infrastructure to do this kind of thing. So no more analog audio or video but we can use IP to carry audio and video packets
03:04
from the door entry panel to the home screen or even a mobile device or even a mobile device connected to the public internet. And when it's about IP, voice over IP, it's a good idea and especially two protocol
03:28
which are SIP, SIP and RTP for real time transport protocol. So voice over IP technology that I'm going to use for this presentation are based on
03:47
two main IETF standards. The first one is SIP, session initiation protocol and the second one is real time transport protocol.
04:02
So SIP, in a few words, it's a text-based protocol inspired from HTTP, standardized in 2000. There are two main components in a SIP network, the SIP user agent and SIP proxy which are
04:23
about writing SIP messages between different clients. And RTP in short, basically the idea is to send audio and video in a packetized way
04:41
over internet. Okay, so now we want to use IP, we will use SIP and RTP. So what we need is a SIP user agent located in the door entry camera and a SIP user
05:01
agent in the home screen for the software. For the hardware, at the door entry camera level, what we need is to be able to capture audio, video and for the home screen, we need just to be able to display video
05:23
and capture the audio because most of the time there is no screen in the front door. So for the software, what we need is, what we propose is to use Linfone which is a SIP
05:42
user agent with RTP capabilities and for the home screen, so we can do exactly the same. Okay, so now we have the hardware can be a Raspberry Pi or any small hardware running
06:04
ARM or any other processor, a display, a cable and on the software base we can use the Raspbian distribution of the Raspberry Pi with Linfone on both parties.
06:28
Okay, so at the door entry camera level, what we can use is Linfone Demon which is a command line interface for being able to place or receive call.
06:44
With early media feature, I'm going to explain what it is about. So, early media is a specific way of initiating calls. For a regular call, you wait for the call to be established before exchanging media
07:07
between the two endpoints. In the case of early media, the idea is to be able to send the video before answering the call because when someone is in front of the door, you want to be able to see
07:25
his video. It's a kind of preview but not necessarily accepting the call. So Linfone Demon, it's an open source project derivative from Linfone.
07:40
It's available on our public Git repository and also something interesting for the embedded case, we provide some Yocto receipt to integrate Linfone Demon within a Yocto distribution.
08:02
Okay, so how it will work? Just a button connected to the GPIO. I just start Linfone Demon in the background with a Unix pipe and with a couple of lines of Python, it's pretty easy to handle the button and just to open the socket,
08:30
communicating with Linfone Demon and to initiate a call. That's it. On the display side, it's almost the same.
08:45
You can just start Linfone Demon in auto answer mode and it will automatically display the video. So, it's for the very simple use case. So now, if you want something a little bit more complex, you want to be able to distribute
09:05
the video not only to the home screen but to the home screen and to another equipment which can be, for instance, a mobile application. You need to add another equipment in the SIP world which is the SIP proxy.
09:26
The purpose of the SIP proxy will be to route the call on both the CPU agent which is in the home screen and on another user agent which can be a mobile application for
09:45
instance. So, now we suggest to use another equipment which is called Flexi SIP proxy. As I said in the beginning, it's the matter of SIP proxies to route the calls and with
10:03
a special feature which is the early media call for hacking because as I said at the beginning of the presentation, what we would like is to have the video coming from the door entry system to the home screen or to the mobile application before establishment
10:23
of the call. So, we need a SIP proxy which is able to duplicate the audio stream in order to be displayed on both devices. It's what is called the early media call for hacking.
10:44
And the video from the call initiator to all ringing devices. Okay, so Flexi SIP, it's a simple process running on Linux with a very small configuration file, so just a couple of sections to authenticate.
11:01
I need to set the domain of my house and a smaller configuration file with the password in order to authenticate the user.
11:24
If we want to go in deep into the SIP protocol, so I have the smartphone app which was registered to the home screen, to the home screen SIP proxy, the home screen panel which was registered as well.
11:40
And when the door entry camera want to invite call, the invite goes to the SIP proxy which fork the invite to both the smartphone application and to the home screen. And the RTP with the video can be displayed on both device at the same time.
12:01
Okay, so now if we go a little bit deep in the complexity, I want the video of my door entry system to be seen on my own, but also if I'm outside, it would be cool if I could be able to see who is pressing the button in front of my house.
12:24
So now what we need is two Cascades, two SIP proxies, one at home and another which can be located anywhere on the internet. So we can do it with FlexiSIP the same way.
12:42
So we had a small configuration file which explained, which state where to fork the call on the public internet. So as you can see, we have a first SIP address which is everyone at my house which
13:05
is fork2 home screen and to bob at sip.infone.org which is an IP address available from the public internet. Okay, so more or less it's the same.
13:22
Just the end is different. The SIP proxy from the home screen is forwarding the invite to the public internet SIP proxy. And at the end, the video can be seen on the local network and on the device connected
13:42
to the public internet. Security consideration. So it's very sensitive data which is coming from your apartment and from the doorbell. So make sure to use SIP TLS and SRTP everywhere to secure the communication.
14:06
And also if you put some password in the local network, it's still better to hash the password.
14:22
Okay, so it was for the presentation of the VPN network for the video interconnect system. What next? We can also imagine to do the opposite. I mean to call the door entry panel from a smartphone.
14:45
It's also something which can be doable. It's also possible to add action to the system. I mean if you want to be able to open the door, you can as an example use GTMF
15:06
to pilot some switch at the door entry panel level. An interesting thing that you can also use is MDNS to able to automatically discover
15:24
the door entry panel or the home screen without having to configure IP addresses. Another interesting topic that we could discuss is how to use push notification to be able
15:44
to wake up mobile application when someone is pressing the home entry button. This is something which also is available from the FlexSIP proxy.
16:03
Another interesting point is interworking with existing door entry camera. I'm pretty sure that on your building there is a door entry panel and what we experience is that most of them are using SIP as the protocol
16:23
to bring the audio and video from the door entry panel to an equipment in the house. So if you want to be able to use existing equipment and just to change the display, it's something which is possible as they follow the SIP standard.
16:47
And the last thing is in my presentation I introduced Linfond Demon, which is a common line tool, but if you want to have deep control of the application,
17:00
you can use the library version of Linfond Demon to do the same. It's available on C, C++ and on many other languages. Even on Python. Okay, thank you. I think I'm three minutes ahead.
17:21
Thank you for your questions.
17:49
Okay, any questions? Thank you for the presentation.
18:03
So, is that a commercial project or is that your personal hobby project? In fact, there is no real project for this kind of application yet. I would say that I'm part of the Linfond team and there is a company backing the Linfond project,
18:26
which proposes this kind of service, but there is no project dedicated to this kind of application.
18:40
It's more a usage of Linfond and TextySIP for this kind of application than a real project dedicated to the entry system. Was it your question? The question was really, you were talking about having your proxy
19:00
and then SIP on the stationary tablet to display the incoming call. So, would you put proxy on the same tablet? Oh, the proxy which is in the home? Yes. The idea is to use the same hardware for both the home screen and the SIP proxy.
19:28
Okay, so let's say you're streaming the video to proxy and then it goes to the tablet itself and mobile phone. How do you deal with video decoding?
19:41
Is that hardware supported on both phones and tablet? Can proxy do recording or how do you do this? Video decoding? Video decoding is done on software, is it hardware decoding? The proxy is not decoding the video at all. It's the same RTP stream which is forked to both the home screen
20:05
and to the mobile application. And the mobile application is supposed to be able to decode either H.264 or VP8 depending on the protocol which is used. Most of the time it's H.264 because it's possible to leverage on the hardware
20:22
So, we have implementation of this kind of codec which is available either on Raspberry Pi or on many other embedded hardware. Okay, cool. Thank you. You're welcome.
20:46
So, how is Lynfone compared to other VoIP softwares like Zivo and Elastix? The question is, is there any...
21:01
Zivo. Zivo, yes. It's an open source project around Asterix. I would say that we have no experience in this kind of scenario with Zivo but Lynfone works with Zivo for regular phone calls. So, as Zivo is based on Asterix and Asterix is following the same protocol,
21:24
I don't see any kind of issue to use Zivo with this kind of deployment of Lynfone within the doorbell.
21:50
On the type of hardware that you've been talking about here, what sort of latency do you get between somebody actually pressing the doorbell and a mobile phone getting an image?
22:00
And then what's the quality of communication between the two ends like in terms of delay between one speaking and the other hearing? Oh, I would say it's a matter of network quality. So, if the quality is good, if it's a wire cable, if it's cable, there is almost no delay.
22:24
You have the encoding time at the doorbell level but if it's hardware encoding, it's good. And it's a ping time more or less.
22:41
There is no 50 milliseconds, 100 at maximum, but not more on the local. If you are speaking about the public internet, it's a little bit more complex. But on local network, there is no problematical latency.
23:18
Thank you for the presentation. My question is, I use a voice over IP product like a service from a service provider a few years ago
23:26
and I have it always connected in my zoiper or something. Have you considered giving the doorbell, for example, a connectivity directly to the internet or through some proxy with some general voice over IP telephone number
23:40
so that I will only set the telephone numbers of me and my family members and it will act as a standard voice over IP video call in their mobile clients? Why not? I don't see any issue with that. As I said, it's standard SIP.
24:01
So, if your SIP network provider follows the standard, you can simply configure the SIP address that you have and directly connect the doorbell to your public.
24:22
So, if the provider keeps all the standards and supports DTMF, everything would be possible using a standard client. Okay, thank you. Okay, this has all the time we have now, so thank you very much.