WPE, The WebKit port for Embedded platforms
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 490 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/47467 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSDEM 2020237 / 490
4
7
9
10
14
15
16
25
26
29
31
33
34
35
37
40
41
42
43
45
46
47
50
51
52
53
54
58
60
64
65
66
67
70
71
72
74
75
76
77
78
82
83
84
86
89
90
93
94
95
96
98
100
101
105
106
109
110
116
118
123
124
130
135
137
141
142
144
146
151
154
157
159
164
166
167
169
172
174
178
182
184
185
186
187
189
190
191
192
193
194
195
200
202
203
204
205
206
207
208
211
212
214
218
222
225
228
230
232
233
235
236
240
242
244
249
250
251
253
254
258
261
262
266
267
268
271
273
274
275
278
280
281
282
283
284
285
286
288
289
290
291
293
295
296
297
298
301
302
303
305
306
307
310
311
315
317
318
319
328
333
350
353
354
356
359
360
361
370
372
373
374
375
379
380
381
383
385
386
387
388
391
393
394
395
397
398
399
401
409
410
411
414
420
421
422
423
424
425
427
429
430
434
438
439
444
449
450
454
457
458
459
460
461
464
465
466
468
469
470
471
472
480
484
486
487
489
490
00:00
ArchitectureMultimediaDemo (music)Computer fontService (economics)FreewareSoftwareBitCASE <Informatik>Cartesian coordinate systemTelecommunicationCommutatorNeuroinformatikWeb 2.0Demo (music)Goodness of fitPresentation of a groupComputer animation
01:19
Cross-site scriptingEinbettung <Mathematik>Process (computing)Information securityArtistic renderingEvent horizonoutputGUI widgetExtension (kinesiology)Front and back endsBinary fileWeb browserDevice driverMobile appGraphical user interfaceSingle-precision floating-point formatView (database)Digital rights managementBuffer solutionElectronic visual displayObject (grammar)Position operatorTouchscreenFocus (optics)Order (biology)Computer architectureBoss CorporationSoftware development kitOpen setResultantCASE <Informatik>RoboticsRemote administrationRun time (program lifecycle phase)Multiplication signCAN busCartesian coordinate systemVolumenvisualisierungMereologyProjective planeProcess (computing)Intelligent NetworkHypermediaoutputView (database)Range (statistics)SoftwareQuicksortState of matterSet (mathematics)Electronic visual display19 (number)Buffer solutionObject (grammar)Direction (geometry)Web 2.0Data storage deviceWave packetTelecommunicationExtension (kinesiology)Web browserFront and back endsArtistic renderingDigital rights managementSingle-precision floating-point formatCuboidDevice driverWindowEvent horizonEncryptionShared memoryComputer animation
06:36
Client (computing)Server (computing)HypermediaVideoconferencingStreaming mediaExtension (kinesiology)Content (media)Electronic visual displayElement (mathematics)Demo (music)Module (mathematics)Integrated development environmentInformation securityMultiplication signDirected graphRun time (program lifecycle phase)Server (computing)Open sourceGraph (mathematics)Computer fileArtistic renderingVideoconferencingPlug-in (computing)MathematicsCodierung <Programmierung>Module (mathematics)Overlay-NetzWeb pageVolumenvisualisierungSoftwareLine (geometry)Web browserMultimediaElement (mathematics)Web 2.0Process (computing)BitExtension (kinesiology)DataflowLibrary (computing)HypermediaCartesian coordinate systemExterior algebraInstance (computer science)CASE <Informatik>CodeSoftware frameworkInformation security2 (number)Computer configurationGraph (mathematics)View (database)Demo (music)Hecke operatorGrass (card game)Different (Kate Ryan album)Uniform resource locatorCodeTerm (mathematics)TwitterService (economics)Heat transferQueue (abstract data type)Lattice (order)Video gameSound effectStreaming mediaWindowPlanningMultilaterationInternetworkingComputer animation
11:45
Open sourceStreaming mediaElement (mathematics)Point cloudPersonal digital assistantView (database)Web browserVideoconferencingKeyboard shortcutOverlay-NetzInstance (computer science)Keyboard shortcutDemo (music)Context awarenessWeb browserVideoconferencingCore dumpWeb 2.0Broadcasting (networking)View (database)Semiconductor memoryEvent horizonStreaming mediaCASE <Informatik>Type theoryBitHTTP cookieComputer animation
13:12
Overlay-NetzMeta elementCodeCodeMixed realityComputing platformPhysical systemMultiplication signComputer animation
14:27
Overlay-NetzCodePartial derivativeMeta elementCurvatureComputer animation
15:08
Multiplication signComputing platformDevice driverDot productPerfect groupRight angleComputer animation
16:51
Open sourcePoint cloud
Transcript: English(auto-generated)
00:16
All right. So I get started.
00:22
Yeah, good afternoon, everybody. I have this presentation about WPE, which is a web engine based on WebKit and designed for embedded platforms. So during the talk, I'll give a short introduction about WebKit, a very short overview.
00:42
And then I will more focus on applications, how you can use WPE in your application, what are the use cases for it, and I probably, if I have time, I'll give a short demo about something I wrote. All right, so I'm Philippe. I'm WebKit reviewer, just from my computer.
01:03
I work for a company based in Spain called Igalia. We are working on cooperative, and we are currently about 90 IP Igalians across the world. So what's WPE?
01:23
So it's based on WebKit, which probably, as you know, was a project started by Apple as a fork of KHTML around 2004. And the main goal for them at the time was to build Safari. And since then, other people joined the projects, other companies, and the community added new ports.
01:45
I will talk about that later. So I briefly introduced the WebKit 2 mutiprocess architecture. On that diagram, you can only see two processes, but nowadays there are more.
02:00
There's one about network nowadays, one about storage, and there's recently introduced a GPU process as well. So the application is called the UI process in that architecture, and it communicates with other processes using IPC. And all the DOM passing, all the things internal
02:21
to the web engine are done in those processes, like JavaScript and all those things. So WPE is a port aimed for embedded platforms. Traditionally, WebKit ports are built on top of a toolkit widget, a widget toolkit, sorry.
02:45
But this port is a bit special in the sense that it decouples rendering to dedicate it what we call rendering backends. That means that it gives more flexibility to the application developers, and there's more openness
03:02
about the platforms that you can support using that model. So that means that the rendering is deferred to another part of the port that is loaded as runtime, basically. And the input events are also entered using that model.
03:25
So those rendering backends usually require a thing called Wayland EGL, which is a way for applications to share graphics resources between processes.
03:41
And they have extensions dedicated to that. So there are quite a few backends nowadays. I will just focus on two. And the first one is the FDO backend, which I will talk about in a minute. And the other one is the RDK backend, which is developed by companies such as Comcast.
04:03
And they have deployed it in a wide range of set of boxes actually in the world. So the FDO backend, it depends on the ETL. So that means usually if you have a GPU driver
04:22
that's using MESA, or probably something that provides EGL, you can have binary drivers. So that backend provides a high-level API for applications to be able to get EGL images from the web engine so that they can do
04:42
whatever they want with that. And then that's also what we recommend for the WP community. It's the backend used on the billboards. And it's the backend we use the most currently in the community.
05:00
Then obviously you need a browser or some kind of application. So I think we are working on a minimalistic browser called COG. It depends on the WP backend FDO. And it's really minimalistic. There's actually no window decryption or nothing.
05:20
And single web view, although we are working on multi-web view support. And it also can be controlled using DBUS. So in your application you can remote control the browser using other applications.
05:41
So as a basic, you need to have a Wayland compositor to be able to run COG. But since recently we wrote a new backend that leverages the DRM architecture. And thus we don't really require running Wayland compositor.
06:00
So that's quite cool because it reduces the dependencies that you need for the application. And if you need a full screen application you can use that backend. Like kiosks, set the boxes, UI, any full screen display.
06:21
And then the rendering is done with DRM. So the Wayland buffers are imported as GB and resource buffer objects. And then rendered using DRM. I have a small use case for, small showcase for that.
06:44
It's basically a thing I did in my free time. I was a bit tired of using Kodi at home so I wanted to have a minimalistic media center. Self-contained. So I've built a web extension for WP. And that can be loaded at runtime by the web process.
07:03
And in that web extension I use a UPnP library I call the UPNP. So I use that library to discover
07:20
the media servers on the network. And then for each server I modified the DOM and add these elements to the HTML page rendered. So then I make a little web page based on what's on the network basically. And then I can do the video playback with the normal video element.
07:41
That's standard nowadays in WebKit. So there's a demo. It's really 30 seconds if you want to look at it on your phone. You can scan that code. Otherwise I can just move on. But yeah, it's an interesting showcase of what you can do with code.
08:01
And it gives an overview of really what you can do. Another thing I wanted to discuss about is Qt applications. As you probably know, Qt is used, widely used in the industry nowadays.
08:20
And if you have a web engine in your application such as Qt WebKit or Qt WebEngine nowadays, you probably would like to have an alternative because Qt WebKit is not maintained anymore. It has a lot of security issues. And some people don't like Qt WebEngine because it's too big.
08:43
So this is an alternative to those two options here. So what we did in upstream WebKit, we've built a QML module that can be swapped in instead of the Qt WebKit based QML module. So if you have an application using Qt WebKit,
09:01
you can directly use that module. And it will use internal AWP instead of Qt WebKit. So what you gain is a maintained WebEngine with security releases. So in that sense, it's quite a good advantage. There are some drawbacks though.
09:23
It works only on Linux. So if your application needs to run on Windows, you're a bit screwed. And then there's dependency on Wayland, EGL, but that we can't really work around. And it works currently in the EGLFS, QPA,
09:41
and Wayland EGL as well. So that means it can work on desktop and on embedded platforms. So to enable it, you just need to download QT WebKit, WP WebKit, sorry, and turn on that CMake option called enable the WP Qt API.
10:01
And then over time, you just need to make sure that the SO file is in your import path. And then I have a small code snippet, QML snippet there that shows how to basically use that WP view in your application. So the changes compared to Qt WebKit
10:21
is that you just need to change the import line and the module name, it's WP view. But in the Qt WebKit world, I think it's a different name. Otherwise, the API is similar. So on title change here, that is already available in Qt WebKit, can be used as it is.
10:43
All right, so, and then the last thing I wanted to talk about is how to use WP in multimedia application using Gstreamer. That's a bit of an over way. So basically, think about streaming browsers or HTML overlays in live TV, for instance.
11:06
So that could be a use case for WP. Just a brief overview about Gstreamer. Who hasn't read about Gstreamer? Nobody, yes? Okay. So basically, it's a multimedia framework based on graphs.
11:23
So in your application, you build a graph and the data flow will flow from the source to the sink and there will be data processing like decoding and rendering. There's many, many plugins. I won't go into details, but what I did is write a new plugin that uses WP
11:40
and I built a source element for that so that the video, the web view can be outputted as a video, basically, as a video stream. And it's quite cool because we have zero copy from WebKit world into Gstreamer world
12:01
using the charge GL context. So there's no memory copies in there. And these cases, as I said, HTML overlays and streaming browsers. I started, so that's a demo I wrote. On the right side, you can see some HTML
12:20
and on the left side, the preview of it with a live video behind. And then you can stream that. I can show it later, maybe. And when you update the HTML, it's updated in the video, of course. So that's for the broadcasting world, it could be useful, for instance.
12:45
Some ongoing work I've been doing lately on that plugin, adding audio support. I have a prototype already. I need to upstream it. And then navigation events in Gstreamer are not really, have been designed many years ago and it needs to be modernized a bit,
13:01
so I started working on that as well, mainly adding mouse core support to events. Improving keyboard support as well. So the code is in wpwebkit.org. We have a Yocto layer.
13:21
The support is upstream in build root as well. This has been working quite well on the MX6, MX8M platform, Raspberry Pi 3 and 4. And it's deployed in the wild already, so you can use it now. I guess I have time for a demo, maybe?
13:44
Okay. If it works.
14:30
Okay, it doesn't work, but. But yeah. But you can install it with, there's a flat pack available,
14:41
and you can install it quite easily on your desktop. I can provide the flat pack ref later on if needed. But I don't have it installed here, sorry.
15:01
No, it's not working. Anyway. So yeah, that's it. If you have any questions, I will be happy to answer.
15:33
Yes, a question? So how's platform support? I've seen some support for MX6, MX8, right?
15:41
Anything new and notable in that? Sorry, can you? Anything new and notable from platforms supported by WPE? Yeah, so I can say that I've been working on, more especially on MX8M lately, on making sure that WPE works with the native driver,
16:04
prefix driver, instead of the Viviente. And apart from that, I'm not sure what you're asking about. Yeah?
16:30
Sorry, quick question. Sorry if it's basic. Just wanted to know what the license for WPE is. Yeah, so the license is BSD and LGPL2.
16:43
Okay, thank you.