The Chromium/Wayland project
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Alternative Title |
| |
| Title of Series | ||
| Number of Parts | 644 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/41658 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | |
| Genre |
FOSDEM 2018394 / 644
2
3
5
6
7
8
10
12
16
18
27
29
30
31
32
34
39
46
47
48
55
57
58
61
64
67
76
77
80
85
88
92
93
98
101
105
110
114
115
116
118
121
123
128
131
132
133
134
140
141
142
143
146
147
149
162
164
171
173
177
178
179
181
182
184
185
187
188
189
190
191
192
200
201
202
204
205
206
207
211
213
220
222
224
229
230
231
233
237
241
242
243
250
252
261
265
267
270
276
279
280
284
286
287
288
291
296
298
299
301
302
303
304
305
309
310
311
312
313
316
318
319
322
325
326
327
329
332
334
335
336
337
340
344
348
349
350
354
355
356
359
361
362
364
365
368
369
370
372
373
374
376
378
379
380
382
386
388
389
390
393
394
396
400
401
404
405
406
407
409
410
411
415
418
421
422
423
424
426
427
429
435
436
439
441
447
449
450
451
452
453
454
457
459
460
461
462
464
465
470
472
475
477
478
479
482
483
486
489
490
491
492
493
494
496
497
498
499
500
501
503
506
507
508
510
511
512
513
514
515
517
518
519
522
523
524
525
527
528
534
535
536
538
539
540
541
543
544
545
546
547
548
550
551
553
554
555
559
560
561
564
565
568
570
572
573
574
576
578
579
580
586
587
588
590
593
594
596
597
598
601
603
604
606
607
608
610
613
614
615
616
618
619
621
623
624
626
629
632
633
634
635
636
639
641
644
00:00
Game theoryOpen sourceCompilerComputer networkKernel (computing)MultimediaPoint cloudVisualization (computer graphics)System programmingMobile WebUsabilityIntelAbstractionSurfaceoutputEvent horizonDigital rights managementSoftware maintenanceAsynchronous Transfer ModeDisintegrationGraphical user interfaceGamma functionService (economics)Front and back endsPhase transitionWindowElectronic visual displayEmbedded systemTouchscreenPhysical systemDemo (music)Goodness of fitLevel (video gaming)WhiteboardINTEGRALData managementMultimediaConnected spaceWindowLimit (category theory)Multiplication signFunctional (mathematics)Wellenwiderstand <Strömungsmechanik>InterprozesskommunikationSoftware engineeringPoint cloudKernel (computing)TouchscreenRevision controlDifferent (Kate Ryan album)User interfacePhysical systemMereologyCuboidFront and back endsMobile WebSurfaceStaff (military)MassAbstractionAsynchronous Transfer ModeVolumenvisualisierungView (database)Digital rights managementModule (mathematics)Open sourceCross-platformFormal languageGUI widgetGraphical user interfaceBitSoftware developerVisualization (computer graphics)Information technology consultingComputing platformProjective planeWeb browserAreaProcess (computing)World Wide Web ConsortiumExpressionDemo (music)RobotCore dumpStandard deviationWeb 2.0Service (economics)Interactive televisionRun time (program lifecycle phase)Software maintenanceMaxima and minimaEvent horizonServer (computing)Reduction of orderInformationInformation securityInsertion lossOcean currentCompilation albumAnalogySoftwareDevice driverStability theoryDiagramTrailMachine visionGreatest elementStack (abstract data type)Latent heatNeuroinformatikSelf-organizationDrag (physics)QuicksortComputer animation
09:39
Demo (music)Phase transitionFunction (mathematics)Extension (kinesiology)Graphical user interfaceAsynchronous Transfer ModeWindowCodeParameter (computer programming)Event horizonTouch typingCollaborationismKeyboard shortcutCursor (computers)GUI widgetStability theoryNP-hardObject (grammar)Data modelHierarchyDisintegrationNetwork topologyProcess (computing)Alpha (investment)Software testingService (economics)Demo (music)Theory of relativityWeb browserServer (computing)WindowAsynchronous Transfer ModeKeyboard shortcutStability theoryMassTouchscreenBound stateGraphical user interfaceGUI widgetMeta elementData managementMultiplication signWhiteboardOnline helpFluxElectronic visual displayPhysical systemNP-hardLevel (video gaming)Revision controlGastropod shellVideoconferencingCursor (computers)Order (biology)Module (mathematics)Representation (politics)1 (number)Functional (mathematics)Patch (Unix)Endliche ModelltheorieProjective planeDifferent (Kate Ryan album)Insertion lossEvent horizonObject (grammar)ImplementationClient (computing)INTEGRALClosed setLatent heatRoutingTouch typingFlagSource codeCodeUser interfaceMultiplicationNetwork topologyHierarchyState of matterMenu (computing)Computer animation
14:23
Front and back endsSoftware testingBranch (computer science)Computing platformSoftware developerMassRevision controlComputer animation
15:08
GoogolTwin primeAlpha (investment)Software testingAsynchronous Transfer ModeWindowBit rateVideoconferencingDemo (music)Computer hardwareFront and back endsWhiteboardService (economics)Binary codeWindowComputing platformMultiplication signFlagCase moddingWeb browserFirst-person shooterComputer animation
16:31
WhiteboardSymbol tableSoftware developerCore dumpComputer animation
16:54
Bit rateGamma functionSoftware testingWeb browserStrategy gameGoogolFrequencySynchronizationContinuous functionMultiplicationTouchscreenKeyboard shortcutInsertion lossAsynchronous Transfer ModeControl flowDisintegrationGame theorySoftware testingMessage passingPresentation of a groupINTEGRALThread (computing)Visualization (computer graphics)Unit testingService (economics)Projective planeLinear regressionWordRobotWeb browserCommitment schemeMassMathematicsFlagPatch (Unix)Process (computing)WebsiteExecution unitPlanningInstallation artContent (media)FrequencyImplementationWindowRepository (publishing)Basis <Mathematik>Keyboard shortcutOnline helpInternet der DingeInsertion lossBitMaxima and minimaDirected graphUltraviolet photoelectron spectroscopyBit rateStrategy gameInternet forumComputer animation
20:48
CollaborationismService (economics)Program flowchart
Transcript: English(auto-generated)
00:07
Good evening, everybody. My name is Maxim, and I'm a software engineer at Igalia company, and today I'm going to present you our project, which is called Chromium's Way to Welland. First of all, I'm going to talk about Igalia company.
00:23
I will give some information about us. Then I will proceed with the goals and the motivations behind the project. I will give some background, give the developments overview, what we have done, what we are doing now, what we're going to do next, and of course, I will do some demonstration.
00:42
So Igalia is a private worker-owned company. We run a consultancy business in open source area, and we are based in Galicia, Spain, in a city called Caruña. It's in the northwest of Spain, and currently we are 62 employees around the globe.
01:01
Half of us work from Spain, and another half work remotely from other parts of the world. And the main areas that we work on are Chromium, Blink, WebKit server, we're also a member of W3C community, and we contribute to web standards.
01:21
We do work on compilers, JavaScript engines, like V8 and JavaScript core runtime, language runtime. Also, we do multimedia stuff, like Gstreamer multimedia toolkit, kernel development, networking, graphics like method drivers, accessibility, visualization, and cloud.
01:41
And on the picture, you can see our web engines hackfest that we host every year, and this one was hosted last year in October. So this map shows how we are distributed. Some of us work from Europe, Spain, Finland, Denmark, UK, also from USA, Brazil, Chile, and South Korea.
02:06
So what's the goal behind the project? The goal is to be able to run Chromium natively on any Wayland-based system. So if there is a system which runs Wayland, there should be no limitations or impedances to run Chromium on it.
02:21
And what motivated us to do that project is that Wayland is actually quite a major solution nowadays, and we think it has better design in a sense of security, better design stack support, and what is more, it's a, well, nowadays, good solution to the X Windown system.
02:43
Also, there is a quite big demand from different industries, like automotive or mobile and desktop. And to name a few, those are from the automotive side, or, for example, automotive great Linux, Genevieve, Volvo.
03:01
From the mobile side, there are a Finnish company called YOLA, Tizen, as well as QT, Toki, GDK, and on desktop side, there are Ubuntu, Debian, which started to ship Wayland. As the background, so where the Wayland, Ozone Wayland project come from?
03:21
So first of all, it was done by Intel company, 01.org organization, and it used, back at the time, Ozone project, which was and is still part of the Chromium project, and it's an abstraction layer underneath the Aura toolkit to construct the accelerated surfaces.
03:44
And the back ends, it supported the DRI, which became DRM utilizing the GBM for the Chrome OS operating system. So Wayland took this approach, this project, and started to develop Wayland support off the trunk for Linux operating system.
04:03
And to give you a minimalistic view how desktop integration looked like for the stock Chromium, I have a diagram here. So there is a browser process and render process which has some kind of JavaScript, HTML, staff computation, and the sandbox GPU process.
04:23
And inside the browser process, there is a so-called desktop integration which has toolkits and graphics platform related stuff like X11 bits and Windows. So they took that approach and added Ozone,
04:40
Wayland, Siblin inside the desktop integration. On the other side, there was a GPU process which had Wayland connection. And everything was communicated using old IPC APIs. So we can treat this project quite successful because there was a good community adoption.
05:00
Many companies took it to ship in their project as well. But eventually, it came to the maintenance mode in December 2015. And back at the time, the Chromium version was M49, and today is M66, which is 17 miles cap.
05:21
So it means that that lacks of some security, box fixings, all other box fixings, and some of the functionality losses. So in the meanwhile, the Ozone layer became to something else,
05:41
and it supported, and it get new backends like X11 and Wayland. But was the problem solved? No, it wasn't. Because the original desktop integration, which was taken, didn't comply with the foreseen desktop integration in Google.
06:01
So as you remember, this is the approach that Intel took, but now there is another design called mass Linux integration for desktop. So in the desktop integration, there is now our mass platform-independent integration,
06:21
which communicates with a new service called a UI service using the module APIs, which are the newer IPCM APIs in the Chromium browser. Inside that, we can see the GPU service hosted by the UI service, and the Ozone layer is now underneath the UI service,
06:41
and it has Wayland and X11 backends. And to see more, nowadays there's another work happening in Chromium, so-called visual service, which is about to have GL, heat-tasting, and compositing in a different process.
07:00
And it will also host a GPU in another process as well, which will give additional 15% of performance. So Ozone project evolved to the abstraction layer underneath the UI service instead, and it supported such backends as GRM, X11, and Wayland,
07:22
but for the Chrome OS. For the Linux, we still needed to do something else, to be able to run it just out of the box. So how we proceed with the project? So in September 2016, we brought the Wayland backend
07:40
to the tip of the trunk, and started experimenting so that Ozone is not Chrome OS. On the picture, you can see how it looked like. So at the bottom, there is a Chrome OS-specific widget, which is not acceptable for the Linux version of Chromium.
08:00
We also added documentation for the Ozone and set up a build bot in the upstream. Then in the cooperation with Google, we came up with two expressions. The first one was internal window mode, which is about having Chromium for Chrome OS,
08:21
which runs a window manager inside and the screen manager. And that window had only one accelerated widget, and then it was Aura, which drawed different windows inside that one accelerated widget. So user didn't interact with the host operating system
08:41
window as itself. And then there is another expression called external window mode. It's the mode when there is no internal window manager, and there are different accelerated widgets created for each new Chrome window. Let it be, for example, tool tips, menus,
09:02
and any other Chromium widgets, Chromium windows. And now user manipulates the accelerated widgets via the host operating system window to maximize them, minimize them, resize, drag, or full screen. In internal window mode is the internal window manager,
09:23
which does all this stuff. And of course, in external window mode, there is no such thing as a screen manager. So to give you an overview how it looked like in the beginning, here's a demo how it run on the Renaissance M3 board back at the time in 2016.
09:45
So as you can see, the bouncing balls demo is not smooth enough, and it lacks of performance. Here's also scrolling, which is not that smooth enough.
10:02
And of course, the work still needed to be continued. We also for the meta browser and implemented our meta browser recipes so that any Yocto-based system could build the Chromium and use it there.
10:24
So what we needed to do in order to bring external window mode. So we needed to modify internal middle mode so that it created for each new top level window an accelerated widget. And we also needed to extend the mass
10:40
so-called module UI service and ozone in order to support external window mode. And we needed to ensure that there were no major functionality loses or performance loses compared to the stock Chromium. To start with, we begin with mass demo. So this is a demo which actually exercise the UI service.
11:04
So we needed to extend it in such a way so that it created different windows with different accelerated widgets and draw some content inside that. We also needed to rework the internal middle mode code assumptions.
11:22
For example, in Chrome OS, there is one to one relationship of the display on the UI service side and the display on the Aura side. For the external middle mode, the display display on the Aura side actually represented a physical display.
11:40
But the display on the UI side represented an actual window, Chromium window or menu window or any other window related to Chromium. So the relation became from one to one to one to many. We also extended mass and ozone to support external middle mode as I previously said.
12:02
And what we needed to do there is to properly handle, for example, bounce and route the events from them, from ozone to up to the Aura. Also, we made it possible to run Chromium with the existing dash dash mass flag
12:20
so that UI service would came up. We also added XDJ version six support because before that it was only XDJ version five support. Keyboard events, auto repeating, mouse course source because before that it was the internal window monitor which did the stuff.
12:40
Also added to support for the touch events. This patch was provided by our friends from Calibre. We added multiple window support. We started to use built in window decorations instead of the host provided ones. Implemented window closing support
13:00
because before that, that was the window manager with internal window manager which actually did the stuff. We added menus, widgets and tool tips support and allowed user to interact with the windows like maximizing them, minimizing, restoring,
13:20
making them full screen and so on. We also needed to change the ownership model of some objects, for example, representing the states of the windows on the UI side, UI service side. We also implemented keyboard like any service integration.
13:41
We did some slightly custom window tree hierarchy so that representations of the window on the Aura side would comply with the representation of the windows on the UI service side. We also needed to rework the access policies so that the clients on the Aura side
14:00
would access only the specific widgets, windows. It is allowed to access. And of course, we work on stability and hardness of our implementation and made the content shell to be run with mass on Linux. And what the status of the project today? I will show you the video,
14:22
how it runs on Debian with GNOME Shell on Wayland. So let's have a better quality. So as you can see, this is mass with ozone platform, Wayland backend, and it's actually version 66,
14:43
the most latest one by this day. It supports many tabs. Here's actually an HTML5 test, and we score 526 points out of the 555.
15:04
And all the features which are supported in the development branch in the TUT are supported by our browser as well. And we score around 60 FPS in the WebGL demos, so everything is hardware and GPU accelerated.
15:20
So you can see two windows working at the same time. And you can ask why I'm showing it to you because it's the same Chromium. Well, I want to say that it's actually the same Chromium, but it uses another design, and it uses Wayland backend,
15:40
which was impossible before that. Here is also another demo with X11 backend, but I'm not going to present it here. The idea is that while we were working on extending UI service to be run on external Linux mode,
16:00
we also added support to run both Wayland and X11 backends, which means that you do not need to get another Chromium binary for that one. You will just need one binary, and using the Ozone platform flag, you will specify which backend you are going to use.
16:25
And if you remember, there was another video which I showed previously, how it run on the Renaissance Aircar M3 board using the bound symbols. So nowadays, it's pretty smooth, given that nowadays, Renaissance M3 board
16:45
runs only on two cores because other four cores are blocked, and that's under development. So a few words about the project itself. Currently, the project is hosted on GitHub
17:02
in the up and down strippers repository. We have a well-defined contribution policy, like we do peer reviews on each commit. Also, we have an internal build bot running our existing tests. For example, the services unit tests, like some UI-related tests, Ozone unit tests,
17:24
browser tests, content browser tests, and we have 98% of the pass rate. Also, we run content unit tests. And why we need internal build bot is that because we want to ensure that whenever we implement new feature
17:41
or do any changes, there are no regressions. We also have well-defined weekly rebasing strategy, and we continuously clean our GitHub history. For example, when we implement a new feature, and we need to have some kind of change for that one,
18:04
we commit a new fix-up. And whenever we do rebasing, we squash all the fix-ups so the history keeps clean. And when we upstream patches to the upstream repositories,
18:21
we do not carry forward flags so that next week when we do rebasing, we do not add this commit, which might have been changed in the upstream while we were upstreaming it because a hormone community might have requested to change the patch a bit. Also, we do periodic sync-ups with Google
18:42
so that we comply with their plans. If you are interested in more design and more deep, you can follow the documentation in the Google Docs. And this presentation is available on the Fosnung website so that you can follow from there.
19:00
So what's left to do in this project? We still need to implement drag-and-drop support and we lack of clipboard support between Chromium and Host. For example, you can do copy-pasting inside the Chromium window, but not outside that one. Also, we still lack multi-screen support
19:20
and we do not support non-English layouts. Also, we continuously ensure that there are no feature loses and a major performance penalty if we compare to the stock Chromium. Also, we do integration inside Automotive Great Linux. And this job is started and done by IoT,
19:42
but whenever they request our help, we provide it to them. Also, we plan to release desktop installers in the sense of RPM packages. And one of the important things is to continue upstreaming our projects to the tip of the trunk.
20:01
Also, we are planning to enable more tests in the upstream buildbot along with upstreaming. And as I previously said, we still need to decouple mass from visual service so that the GPU process will be run not as a thread inside the UI service,
20:22
but as a separate process. Thank you for your attention. The project is done by Igalia and sponsored by Renaissance. Whether you have any questions, you can contact me, Maxim, or my colleague, Antonio from Igalia.