OK, thanks for the introduction. So I'm Lorenzo. I'm from Napoli in the south of Italy, which means that pretty much the only thing that you really need to know about me is that this is a big no, OK? Now that we have settled this for good, once and for

all, let's start with the presentation. So I don't know how many of you actually know about Janus already. This is going to be just a very quick recap about what it is. Basically, it was conceived as a general purpose WebRTC server. I'll try to explain what we mean by general purpose in this sense. It's, of course, all open source.

There are demos and documentation available there. And the idea was that we would write a core that would be responsible for all the WebRTC stack. So Stephen just explained how WebRTC works, what it is. Basically, this WebRTC stack allows Janus to act as a browser in the communication, in that peer-to-peer communication that you saw before.

So this WebRTC stack is responsible for all eyes, detail eyes, all the protocols that relate to WebRTC. And then we have different models to take care of the rest. So we do have a Janus API, so a way to communicate with Janus to control what it needs to do and how. And this is available. Different models can implement different protocol transports for this.

But more importantly, we have all the media logic implemented as different plugins as well, which means that this way, you can just write a new plugin to have Janus behave in a different way. So handle the peer connections that you created with the core in a different way. So by default, the stock plugins allow you to have an SFU plugin, something that allows you to do some basic

video conferencing, let's say, an audio mixer, ways to talk to the C port, ways to talk to RTSP cameras, or whatever. So all the logic belongs there. So if there is a new need that you have that is not covered by the existing plugins, you just need to write a new plugin to handle media, add it to Janus, and

it will theoretically work. The only caveat here is that Janus is written in C, which means that if you want to write a new plugin, you typically have to write it in C as well, which may or may not be an issue for you. I mean, for some it might be, because you need to know how to code in C and make sure you're not screwing up and causing issues of any sort.

And just to give you an idea of how the plugin API looks, and this is important just because I'll get back to it in a few slides when I address the purpose of this talk here, is that typically a plugin has to implement a set of callbacks in order to talk to the core.

Because of course, if you want to be plugged as a model to Janus as a core application, you have to respect some APIs in order to communicate with it. And typically, it's just, let's say, some ways to register at the core. Let's say this is my package name, this is my version, a couple of meters to initialize the plugin and

destroy it. And typically, init is called whenever your plugin is loaded, which happens at startup. So Janus starts, starts to load the plugins, you will get notified whenever your plugin starts. And destroy instead is called when Janus is shutting down, so that you can keep track of the life cycle of Janus itself that way. Some versioning information, as we said, but most importantly, some ways to handle users, so to interact

with sessions. Because typically, the Janus core will notify you any time that a user attaches to your plugin. We call this attach as emitted. So any time that a user joins your plugin, you are notified about this so that you can start keeping track of this user. You can start receiving messages from the user.

You are notified any time a peer connection for this user on this specific connection came up or came down. And of course, most important of them all, you are notified any time you have incoming data from that user. So if the browser is sending RTP packets via the WebRTC peer connection, these RTP packets get to you in the plugin, so that you can then decide what you want to do with it, which may be, I'm going to send this to all of

the other participants in a conference, or I'm going to decode it, add it to a mix. I'm going to send it to a CPUs or whatever. Same thing for RTCP statistics and data channel messages as well, because you may want to handle just data channels or data channels in conjunction with media. And in general, this allows you to track all the life

cycle of a specific peer connection for a user, which is quite useful. And finally, of course, the core itself also exposes an API by which plugins can then start sending info back to the user. Stuff like ways to send messages to a user via the Janus API. And this is important, because each plugin has its own API.

You can consider each plugin in Janus as a separate application living within Janus itself, meaning that each plugin can actually implement their own messaging that then gets transported over the Janus API. And you can do that via, I'll show a couple of sequence diagrams that show how this works. And at the same time, you can also tell Janus when you

have an RTP packet to send back to the user, when you have statistics to send, a data channel message to send. And so these two sets of APIs allow a plugin to both receive and send media at any time with a connection, which gives you complete control over what peer connection will transport within the context of a session. And then you have some additional meters, let's say,

close up your connection, because you don't want it anymore, and stuff like this. And just to give you an idea of how this works from a sequence diagram perspective, the idea is that the user decides to attach to a specific plugin. And typically, the user just specifies the plugin namespace, which is the unique identifier for a plugin.

The Janus core knows that the plugin is registered somewhere, and so uses the create session callback to add the new user to the plugin. Then it's up to the plugin. It's up to the responsibility of the plugin itself to take care of this user and do what we've seen before. So keeping track of the user, maybe create a new structure

for this user as a conference participant or whatever. And then any time that the user sends a message via the browser or whatever meant for that specific plugin, the Janus core will strip everything else and will just give the message itself to the plugin so that this can be processed somehow.

And the way that this messaging works is that typically, plugins can handle messages in two different ways, in a synchronous or an asynchronous approach. When you handle messages synchronously, then you just send a response right away to that callback that we've seen before. So you receive the message, you process it, you send a

response back. And this can be useful, for instance, like when I request this for give me a list of conferences that this plugin handles and stuff like this. In this case, it's pretty much synchronous, as you see. But more often than not, plugins actually communicate asynchronously instead, which means that users send a message.

The core passes the message to the plugin. The plugin just sends an hack back. And then later on, we'll send you one or more events notifications to take care of this. The asynchronous messaging is particularly important because it also takes care of the WebRTC negotiation by itself. So Stephen has anticipated how SDP offers and answers

are involved. Plugins are responsible to prepare those SDP offers and answers, depending on who originates the session. And this is all done in an asynchronous way. So maybe a user sends an offer, it gets to the plugin. The core will strip all the WebRTC stuff. It will just give you a bare bone SDP. The plugin prepares an answer and sends it back.

So same thing when the plugin is actually offering something. And finally, of course, as we've seen, media will be just bridged by the JANOS core. So the users will send something via secure RTP. The plugin will strip the security part. And the plugins will have access to the unencrypted traffic for their needs. As we've seen, as I anticipated before, all of

these plugins and meters need to be implemented in C because that's how JANOS was conceived. So JANOS loads a shared model and expects a C interface to communicate with. But if we think about it, we don't really need the whole plugin to be in C. As long as the interfaces are something that looks like C and the core is happy, then

we can implement all the logic in whatever language we want if we then implement the hooks somehow so that we have some stubs that then delegate all the logic to something else, which in principle is exactly what we did with the Lua plugin, which at the moment is a pull request that is available on our JANOS repository.

I also wrote a small tutorial about how to write a Lua script from scratch to use this. I'll go a bit on this later on. And the basics are that typically there are a few things that we need to consider. So for this Lua plugin, we basically decided to, of course it's still a C plugin because we need a C interface.

But then this plugin loads a user provided Lua script when you initialize it. So you say open this Lua script, which can be maybe different scripts for different applications. Then all the plugin callbacks are implemented in C, but they then call a Lua function so that the Lua script can handle those requests that we've seen. And at the same time, this plugin also implements all

the core methods that should be needed to communicate with the core and exposes them as Lua functions. So that when I call push event from a Lua script, the C plugin will transform them in something that the core will be able to digest. And in order to track users and sessions in a way that both

the C and Lua code can identify, we chose just for simplicity to use a unique numeric ID to do that. And in principle, this is exactly what we did. So some kind of a C to Lua proxy communication, even though there are some problems that we needed to address that I'll talk about in a minute. Which means that if you look at that API that we've seen

before, this is the C version of these callbacks that the plugin implements, because the core needs them. But they are in turn then all passed to Lua functions on the other end. The only two important ones in this case are init and destroy, because for the versioning information, the Lua plugin can answer it itself.

But if you want to override that information as well, you can, basically. And more importantly, for all the session management, we have alternatives for that as well. So whenever a user attaches, there will be a create session callback called on the Lua script and so on, with the only exceptions of incoming RTP, RTCPN data, which you

might implement, but you probably will not want to. And typically, you will not want to, as I'll explain in a minute. And of course, for sending stuff back, it's pretty much the same thing. So the Lua script will call push event or end session or whatever. This will be intercepted by the C plugin via this Lua C

communication, and it will be sent to the Lua plugin itself, which means that very simply, this plugin then acts as an intermediate layer between the C word and the Lua script. So anytime something comes from one direction, it translates it to the other part and vice versa, typically.

There were a couple of issues that we had to implement to take care of those. Most importantly, we've seen how asynchronous events are very important in the plugin's communication, because that's also how negotiations happens. And the problem was that the Lua is typically single-thread. I mean, they have a concept of threads, but they are not

really threads. They are mostly called coroutines, typically. And besides the access to the Lua state, which means the Lua script that we loaded that acts as the, we might call it the virtual machine for the Lua script that we are handling is not thread-safe, so we cannot access it from multiple threads without causing trouble, basically. So we had to, first of all, lock the Lua state any time

that we wanted to access it. And most importantly, we had to implement some sort of a C scheduler that we could implement to take advantage of any time that we wanted to have some kind of asynchronous communication. So any time the Lua script wants to have a message handled asynchronously, it will basically just say, create a coroutine, keep it in a task list, and then notify

the scheduler in C to say, I will need this to be handled in a coroutine later on, as soon as there is time for that, which is what we implemented via a method that is called POCScheduler. So the Lua plugin will just call this method. This method will handle a thread internally that then keeps a queue in order to then call the coroutine as if

it were a generic Lua function later on. And you can see it pretty much as exemplified here. I'm not going, I will not go too much into the details due to lack of time. But you can ask for more information later if you're interested in this.

And this is a very simple example of how you can do that. So you have a message. You want this to be handled asynchronously. You add it to your own hash map or whatever. You call POCScheduler, and then a thread will wake up in the C code to then invoke later on the

viaResumeScheduler, the task list, so that you can do it again. We also have an alternative to, let's say, time callbacks on your own schedule rather than, let's say, call this method as soon as it is possible. And you can do that via a method that is called Time Callback, which basically means you say, call this

callback with these arguments in five seconds, for instance. And typically, the approach is exactly the same. We have a thread back in the C code that takes care of these. As soon as the time pass, we invoke the function that you want it to be invoked, which makes for an easy way to implement this asynchronous behavior that we needed. But this is actually more important than the rest.

We wanted to have also a way to handle RTP, RTCP, and data in a way that would not completely block the LuaState machine. Because as we've seen, we have meters to handle incoming traffic, so an incoming RTP meter to handle incoming RTP traffic, for instance. Let's say the normal way you think you could do this would mean take this RTP packet and pass it to Lua exactly as it

is, and let Lua worry about it, which is actually not an option. Because as we've seen, Lua is single-threaded. It will completely kill performances if you try to have Lua take care of this delivery itself. So the way that we handled it was to basically handle this

in a different way. This was because for RTP and RTCP and that as well, the traffic may actually be very frequent, which means that you may actually kill performance if you did it directly there. So we basically implemented some meters that allow you to configure the routing of the media, depending on how media is flowing in, and then let the C code worry

about actually delivering these packets in either direction. And the concept was quite simple. So we chose to add a couple of meters that are called add recipient and remove recipient that basically allow you to say for these specific users, all media coming from this user, send it to these other users instead.

And of course, you can call these multiple times for the same user so that you can have a one-to-one conversation or a one-to-many conversation if you wanted. And then the C plugin takes care of all the rest, so they're overwriting the RTP headers if needed, sending packets around as they are needed, and so on, without the Lua plugin needing to worry about anything at all. So it just, let's say, turns some knobs and some buds here

and there to say this is where the media should flow. And this is flexible enough to allow you to do any kind of communication from one-to-one to actually complete a few if you wanted to do that. And this is a very simple example where, for instance, Alice is starting to send media, just Bobby subscribed to it.

And then we receive a new meter that says, send this media to Tom as well, which means that the packets that Alice is sending will be sent to both eventually. We can see a couple of examples here. So for instance, in that pull request that I've shown before, we have a couple of examples out of the box.

So one of the examples was we just replicated the exact echo test that we have implemented in C as the Lua plugin itself, which means in this case it is really easy, we replicate the same API, and then we just make sure that whatever the user is sending, we send back to him. Which, as you may expect, just means that as soon as a WebRTC connection goes up, and we know it because we are

notified about this by the C plugin, we just call an adrecipient meter for this session, and we pass it to the same session itself. So that we say, any packet that is coming from this session, send it back to the same session. Which is kind of easy, but already shows the flexibility that you can do with it.

But of course, this is more almost done. OK, sorry, I'll fly with Don Dylan's slides. So we also have an SFU clone, so there is also a small video conferencing application written in Lua that is kind of cool, where you actually do this adrecipient stuff for one-to-many kind of thing. But I also wrote a complete tutorial to write a plugin from

scratch using a video call example. And you can see on our blog, we have a complete post over here that goes all over the basics. So it starts from the very beginning, and it will drive you in explaining all the steps that you need to do in order to do that. I also implemented a small demo at Astricon a few months ago that took advantage of that.

And I also wrote a demo specifically for this session here. Unfortunately, there is no time for that. But if you go over that link over there, there is something that looks like the video call, but is actually a chat roulette. So I assume that you know pretty much how chat roulettes work. So you start, you are not connecting to anybody, then somebody randomly may join in in this completely unedited

and unaltered picture, as you may see. Then people may go away, and then another match may be coming for me, which I may not be really comfortable with, so I may decide to drop the communication instead. And all of this happens within the context of the same peer connection. So I connect to the plugin once, and then I just send

buttons to say, I accept this match, I reject this match, and so on. And the plugin in the background decides who should receive my media, and dynamically so in this way. And I'm almost done here. So what to do next? I mean, there are some things missing. So typically, there are the most advanced stuff that we have in Janus are not available in the plugin yet, like simulcasting and stuff like this.

And besides, this Lua plugin is also based on a branch that is experimental by itself, because it's the reference counter's branch. But there are some things that we may want to do in the future. Like for instance, I mentioned that we have a couple of other type of plugins, like transports and event handlers. So it might make sense to start to think about Lua for

those as well, if it makes any sense. But actually, something that is more interesting to me is start experimenting with other programming languages as well. Because Lua may be just an opening door for this. We already have all the hooks in place to decide how to then communicate with a different language, and have all the hooks and stubs in the C code. So it should be easy enough to, let's say, use something

like duct tape or something else, to communicate instead with other programming languages, like JavaScript or Python or whatever else. I mean, this is something that I'd like to experiment in the future. And I hope this was interesting enough for you to play with them. So if you want to start playing with these, or maybe also start implementing a new programming language by yourself, just play with it and let us know.

It would be really cool. So that's all. I don't know if we have time for questions or if we're done. OK, sorry about that.