Classic Applications in a Confined Ecosystem
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Title of Series | ||
| Number of Parts | 611 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/41925 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language | ||
| Production Year | 2017 |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSDEM 2017116 / 611
10
14
15
16
17
21
22
24
25
27
31
36
40
42
46
50
55
56
63
70
73
78
84
94
101
102
104
107
108
109
110
111
112
113
114
115
117
119
122
123
126
127
128
130
131
132
135
136
137
138
141
142
144
145
146
150
151
157
158
159
160
162
163
164
166
170
171
173
175
176
177
179
181
184
187
189
191
193
194
199
200
205
207
208
209
211
214
218
219
222
223
224
225
226
229
230
232
234
236
237
238
239
245
248
249
250
251
253
255
257
258
259
260
261
264
265
266
267
268
271
272
275
277
279
280
282
283
284
287
288
290
292
293
297
302
304
305
306
307
309
310
311
312
313
314
316
317
318
319
321
322
327
329
330
331
333
336
338
339
340
341
346
348
349
350
352
354
356
358
362
363
364
367
371
372
373
375
380
384
385
386
387
388
389
390
391
392
393
394
395
398
400
401
402
405
407
409
411
412
413
416
417
418
420
424
425
427
428
429
431
435
438
439
440
441
443
444
446
448
454
459
460
461
462
465
466
468
471
473
477
478
480
483
487
488
489
491
495
498
499
500
501
502
503
504
507
508
510
511
512
514
518
519
520
522
524
526
528
530
531
533
535
536
549
550
554
555
558
560
563
564
573
575
578
579
582
585
586
588
589
590
591
592
593
594
595
596
600
603
604
605
609
00:00
Percolation theoryThermodynamisches SystemSpacetimeRevision controlDifferent (Kate Ryan album)Cartesian coordinate systemLine (geometry)Statistical hypothesis testingPatch (Unix)SoftwarePhysical systemMathematicsDirectory serviceSlide ruleConnected spaceOperating systemColor confinementCanonical ensembleServer (computing)Differenz <Mathematik>Video gameMereologyVirtual machineProcess (computing)Mobile appDatabase normalizationContent (media)BitTouchscreenNumberIntegrated development environmentClassical physicsBinary codeMalwareRootkitElectronic visual displayComputer architectureChainMultiplication signFile systemSystem administratorVisualization (computer graphics)Loop (music)Uniform resource locatorData managementTask (computing)Interface (computing)Library (computing)Information securityResultantComputer fileElectronic mailing listComplete metric spaceUsabilityNeuroinformatikVector spaceQuicksortFlow separationKeyboard shortcutIterationContext awarenessExtreme programmingSuite (music)Radical (chemistry)1 (number)Form (programming)Hybrid computerSpectrum (functional analysis)Software frameworkAdditionMaxima and minimaMusical ensembleEinstein field equationsGoodness of fitDecision theoryComputer animation
08:24
File systemSet (mathematics)Physical systemComputer programmingData managementDirectory serviceCartesian coordinate systemWindowOutlierIntegrated development environmentProcedural programmingData dictionaryArray data structureDatabaseImplementationGoodness of fitDifferent (Kate Ryan album)CodeType theoryProcess (computing)NumberCASE <Informatik>GradientFlagSocial classLimit (category theory)MereologyLibrary (computing)Raw image formatLine (geometry)Mobile appComputer fileException handlingKernel (computing)NamespaceFunction (mathematics)Group actionFocus (optics)Server (computing)Keyboard shortcutTrailSoftware frameworkScripting languageUsabilityRootkitOnline helpDomain nameDefault (computer science)Binary codeAbstractionFunctional (mathematics)Link (knot theory)Utility softwareMultiplication signSampling (statistics)Patch (Unix)Instance (computer science)Traffic reportingMusical ensembleAddress spaceKey (cryptography)
16:44
Socket-SchnittstelleElectronic mailing listIntegrated development environmentMobile appPlotterDemo (music)Automatic differentiationCartesian coordinate systemException handlingClient (computing)Regular graphPresentation of a groupSystem callLibrary (computing)CASE <Informatik>Line (geometry)InformationEvent horizonPhysical systemMathematicsProduct (business)Structural loadOrder (biology)Information securityRootkit1 (number)Table (information)Computer iconFlow separationWrapper (data mining)Default (computer science)ImplementationFamilyClassical physicsInstance (computer science)Virtual machineMultiplication signSeries (mathematics)Data managementWindowGraphical user interfaceTelecommunicationKeyboard shortcutNumberElectronic visual displayFile systemVariable (mathematics)CodeThermodynamisches SystemRevision controlBitBinary codeCompilation albumDirectory serviceMappingServer (computing)Touch typingGraphics tabletComputer fileKernel (computing)Front and back endsIntrusion detection systemThread (computing)Interrupt <Informatik>Computing platformGroup actionAreaScheduling (computing)Computer programmingLetterpress printingMusical ensembleOcean currentWeb pagePower (physics)MereologyPressureView (database)State of matterSource code
25:03
Cartesian coordinate systemReal numberComputer fileRootkit
25:35
Computer fileSoftware testingState of matterGreen's functionPosition operatorProgram flowchart
26:03
1 (number)
26:26
Data recoveryState of matterWebsiteAreaMathematicsMobile appRow (database)RoboticsFront and back endsFlow separation1 (number)BitPhysical systemVirtual machineInterrupt <Informatik>QuicksortInformationLinear regressionInstance (computer science)Right anglePoint cloudCartesian coordinate systemClassical physics
29:07
Electronic visual displayFile formatServer (computing)HorizonComputer animation
29:32
Computer animation
Transcript: English(auto-generated)
00:08
Are we just starting? So hi, everyone. Bonjour à tous. Thanks for joining me. Like this slide says, my name is Larry Price. I work at Canonical on some of the new English Chinese
00:20
perks of popular living space operating systems of Bluetooth. This afternoon, I'm here to talk to you about how we run classic applications in a confined ecosystem in Ubuntu. So what do I mean by a classic application? And furthermore, what do I mean by a confined ecosystem?
00:41
Well, so let's start with a little bit of background. My job is focusing on the idea of convergence, bringing the same as Ubuntu on any platform, regardless of screen size, computing power, stuff like that. Convergence has driven us to build a lot of really cool software over the years, discovering new concepts of security and usability
01:02
from a computer spectrum. While we want to take advantage of these new architectural tool chains that we built, we also want to take advantage of a stable, mature software available in the archives. And we want the use of that software to be seamless to our existing users.
01:21
So a classic application in this context is any application that I can currently install with a traditional package manager, with a package or a software center, and then run using either the terminal or a graphical discovery interface, such as Unity Vetch. These applications are terminal-based for Graphi3 CDX11.
01:40
Their application we'll be using for years to make our Linux desktop experience complete. Actually, Firefox is your office, again, and D is your ID, PINGAS, and many others. Without these apps, it would never be your Linux desktop instead of every year being your Linux desktop.
02:03
And though some applications have already been migrated to new display servers and new packaging systems, this talk will be focused on those which have got to be migrated. Classic applications were designed to be installed and used in an unconfined ecosystem.
02:20
The way that applications have been traditionally installed and accessed resources has grown up organically alongside the tooling and other parts of system management. And throughout this process, applications can or must access parts of the system that they are never meant to access. And malicious software could easily snoop on the private business and deny applications.
02:41
This is not only a shown desktop where you're likely checking on the machine once a day or so, but also a potential threat to internet-enabled devices which are intricately outdated or checked for health issues and could be scattered throughout your home. Undependencies are another place where classic applications run the problem.
03:01
Typically, it acts to pin on several other packages and sometimes those packages can conflict with other versions of packages installed by other applications. Worse, sometimes applications need newer versions of packages that aren't available on the system or similar packaged versions like different patches on your distros. Putting the application and pulling back changes until a newer application version becomes available
03:20
which may not be until the distro makes a new release. This could be preventing users from running the best version of the PCOS software at any given time. On top of that, we also see an increase in the pollution of our root filesystem and this is hard to read but this graphic is a great visualization
03:40
of the way that the Linux filesystem looks. You may notice that it's pretty complicated and some of these things are a little bit redundant. For instance, this is the contents of my add variable. We mean they're user-runable binaries that's bin, sbin, userbin, user sbin for games,
04:01
user-local bin, and any number of doc directories in my home directory, as well as Heroku sneaking in some random user-local root event. Similarly, library and data files appear wherever they want these are the results I'm bringing out so I'm not considering this complete list
04:20
and data directories are kind of wherever they need to be. So, to generalize the goal of confinement, we want to move away from letting apps loop on each other without permission and we want to take back control of our filesystem. Putting these applications in a known location and giving each application the appropriate tendencies.
04:43
Our traditional package manager and definitely based operating systems is over 20 years old and appears to be one of those years on its shoulders and because of this, we thought it would be best to introduce a new packaging toolchain.
05:04
Historically, a couple of years ago Canonical announced click packages which were kind of a solution to this problem. An application packaged in click is bundled with any specific dependencies that it needs and it takes advantage of frameworks to access any shared resources
05:20
that it might need. It also uses AppArmor to create a confined application and specify any security feature that you need. Clicks were used to package applications for DNS where I would do test devices giving them a little bit of a headline. These devices ran here instead of X11 as well as UnityA
05:41
as a display server which sort of gave us our combined ecosystem but these toolchains were missing some important things and lack maturity. Fast forwarding to today, we've seen the emergence of snap vectoring
06:00
also referred to as snappy. Snappy is essentially another iteration on clicks giving better version management sleeker applications with finer definitions and new ways to interface with system resources. You can use snaps today on Modern Ubuntu or several other distros snaps will all live in a special root directory and can be completely confined when installed from stable channels
06:21
or unconfined when installed from unstable channels if you must. Snap apps come bundled with all their connections meaning that all the apps will have the connections they need and upgrade whenever they want. No longer do you need to wait on package containers to get them into the distro. Snaps are automatically updated once a day
06:40
taking the pressure off of the user to keep their system up to date. Snaps come with the option for delta updates which means that the toolchain can validate a diff instead of an entire package potentially making downloads smaller and faster. And as confinement is concerned snap apps are relegated to their root only piece of the process requiring interfaces to be connected to perform system tasks
07:01
such as reading the display, using audio and using the network. Instead of using a hybrid or classic environment in addition to a traditional package manager we can continue to pull in and use our classic applications while enjoying confinement security in our migrated apps. However, one exciting premise is
07:21
Snaps is the ability to run in an all-snaps environment which we, I think, call a bit diverse. But this puts us in a bind we can't run classic apps in this all-snaps environment. Our little classic applications are looking for an unconfined system and looking for the opposite of x11 and this all-snaps environment
07:41
is prepared to use in neither. We need a way to round this issue to allow users to run applications just like they would in a classic system. So to resolve this issue we've been working on a suite of tools which we call Libertine. Libertine is a person who acts without moral strength.
08:01
Libertine is an extreme form of hedonism living one's life for pleasure while only thinking of oneself. In a sense, our classic applications are practicing Libertines. And it's our job to pull them back and to more on my standing. These applications need a safe place to go while the rest of the world falls in line
08:21
and we provide the safe space as a Linux container. Containers are generally unrecognizable from a traditional Linux desktop experience. A Libertine container has some file systems to work with and binds to system resource directories to access things like home and some system resources. Internally, we continue to use apps and de-package to manage our packages
08:41
even when the host system is not allowed to use de-package, even if de-package is not installed on the host system. Libertine also connects to a program called Exmir which allows us to run an X11 window isolated on top of Mir. Every installation, removal, and system modification is done internally
09:03
protecting the outside world from whatever these classic applications may be doing to the classes. In the Libertine suite, it has many parts to extract away the underlying limitations of the classic printer as best as possible. C++ library including Python launchers
09:21
and command line interfaces and tools to control other aspects like watching Exmir and other people on your clipboard. I'll go over a few of these now. Again, a little difficult to see. This is just JSON dictionary and arrays
09:41
and all the good JSON stuff. So we use Libertine uses JSON as a database file keeping track of all installed packages creating containers modified bind mounts and we just use it as a database maintained individually for each user on a system
10:01
and using JSON we found lets us not handle any database frameworks which is good for our use because then we can also just ask users for the raw database file and they don't mind because 30 or 40 lines of JSON. The crux of our
10:21
container management is our Python library. Of course it contains an abstraction around the database that we just saw and utility functions but the primary use is to extract away the process of handling different implementations of Libertine containers of which we support three types JARU, LXC, and LexD.
10:43
JARU is a special case maintained only for use on devices with kernels too old to support LXC or LexD. For the JARU back then we used a finger cutting line tool to create a base domain system in the user's home directory as a JARU jail from which we'll run all of our commands and applications
11:04
and when running commands within a JARU we'll use a tool called help keyword and we'll generally try to run a keyword on privilege to run that command within our JARU but every now and then we have to run privilege keyword commands and so we prefer not to use root command containers instead of them completed though it's still useful.
11:25
But LXC containers have been our preferred container type for most of Libertine's lifetime. LXC combines the kernel C groups and support isolated namespaces to provide an isolated environment for applications.
12:00
We're capturing a song from the camera but it's just not perfect. We have a container inside the container that gives you access to home and system directories and setting up how to leverage LXC to integrate additional best of features and we've added great links to ensure that we can access things like audio through the system.
12:26
Also note that the LXC Python API is quite mature and works relatively well for our needs for instance. This is a code sample showing how we use the LXC container on the patch method to run our command into your Python but the newest shiny back-end
12:45
for Libertine is LXC containers. LXC is a new face on top of the LXC implementation, simplifying container management and frequently providing speed and usability improvements as well. Libertine LXC containers are outliers in that we depend on the LXC back-end to
13:05
maintain the root file system though we still maintain locally they use home directories and mobile applications directories. For the most part setting up LXC is just like every set of an LXC container we biomass it with devices mapping our own procedures in the system and it's also going to be okay.
13:25
I guess one weird thing about LXC is how we have to set up slash run slash user slash run slash user is a system directory for storing files and running processes when LXC finish starts. This directory unfortunately is overwritten completely when it starts, so we need to
13:40
remap it using the sound and stutter scripts. To work around this we create a custom script and copy it into the container. The script creates missing directories, faces and permissions and remaps things like slash run slash user since here the system is only reordered. We add the files to the container using the files API
14:01
which is back there which is part of the Python LXC package. But the Python API for LXC is not quite mature enough that we can use it for all of our use cases. So we use a hybrid solution that you can play the API and calls to be open to start processing manually. As an example here, we use the
14:21
Python API to update those devices. It's just a dictionary and we use the Python pylyc API to create this profile, update the dictionary and say using LXC exceptions to tell us when there's an error. On the other hand, when we run applications we have to use psc.py open
14:40
to launch applications because that's the best way for us to run LXC applications in the background while still getting all the output in the foreground. So we also have these services that are responsible for making sure the containers are
15:01
started and stopped appropriately, LXC and LXC. This is dialed with a safe battery life so that the containers aren't started all the time. And it ensures that when a user stops one container or stops one application within a container, the container doesn't stop, but we wait for all applications to be closed. Moving on,
15:22
we use the run with binaries Python library, or a Python binary called the original container manager, which allows you to essentially create and manage containers and excel packages and modify them by amounts.
15:40
This is a basic overview of using the original container manager to create a container. It's fairly straightforward. We just call them grade sub-command with any number of flags all of which have intelligent defaults except for ID so the user doesn't have to know what kind of container backing we're using. Here I've shown that as just a black box, and then it just
16:02
updates the database and adds all of the file systems using the given containers backend. And this is similarly just what installing a package would look like. Note that a package name may not line up with the binaries installed, but we're only
16:21
keeping track of what the user is supposed to be asked to install. Because the purpose of using the container manager is to prevent the user from having to deal with the low-level concepts of any particular container type or package management system and focus on the key concepts of creating and installing.
16:42
We also have U.S. servers within Python to replace the work being done by the last application we installed. All of this has been fully implemented yet, but many of the commands that we use to replace the container manager takes a very long time to arrive, and bad things will happen when those commands are interrupted. For instance, stopping bpack, which of course will cause, or can cause, broken
17:01
packages to appear in the system. Libertine being able to solve that by being a completely asynchronous method where we use Python threads and send updates to U.S. signals. And although we would like to replace the container manager backend with this in the future, we currently just use it to list container ads and application IDs.
17:22
Which brings us to libLibertine, which is the root of discovering container information for several external libraries. It's currently a wrapper around a client for Libertine-D, and many of the low-level ads libraries that access Libertine are written in C++, and Libertine gives them a convenient
17:40
way to access Libertine information without having to talk directly to D+. The code that we're listening to is just us making a simple D++ call from C++. We also have a GUI written in C++ with QOL. This is an easy way for us to manage containers and even allows us technical users to be able to
18:02
easily manage containers and install packages. It's mostly a series of lists showing you what you can do in a list of containers in which you click through the list of packages. For the most part, my family GUI is the same as it is in the mainline tool.
18:21
And now the real neat thing is that it can launch, which is a Python tool that we use to actually launch applications. What Libertine launch will do is connect the environment for running apps and then forward that environment to the container implementation to actually run the application. Libertine launch can be used for a terminal, but we generally prefer it to be used from a launching tool,
18:40
like in Ubuntu and Unity 8 we use at the draft launch. Hasty is a daemon. It's a convenient binary to help applications use the clipboard. We just use huge signals and slots to copy and paste events from the underlying Comp Hub and then we forward those events to our excellent maps.
19:02
Doing this, we can use copy and paste pretty much like we would in these applications in a classic desktop. So what actually happens when we launch applications through Libertine? By clicking our desktop icon, we'll call Libertine launch with a container ID and application event line
19:21
which was properly found using Libertine which found that information actually from Libertine. Libertine launch will then start utilities and paste it next year. It'll set up a session by coming up the environment and setting up a whole bunch of sockets to communicate between the two. And then it will launch a container instance
19:41
based on the Python library which will then do any number of things based on the container type. Update bind mounts, launch the window manager to connect to an instance with X, and finally launch the app. And that's most of the full picture. To review and build a container using one of the living machines and set it up to install applications through your app
20:02
even when the app is not a system. Libertine keeps your container and its classic application is confined to their root file system and a small folder of code to let them access things like the home directory and its resource directory. At which point, something or someone can launch the application by leveraging launch which will prepare the
20:20
container environment to mirror the host, show the window manager and set a container connected to the host display variable and launch applications. Given all of that, where can we actually use Libertine?
20:41
The answer is well in most of our modern movie systems. Unity 7 is current in the desktop session in the Bluetooth and using Libertine in Unity 7 is a little bit silly because Unity 7 is not a confined environment. Unity 7 already runs X11 and already installed the package to be packaged
21:02
but I think we are learning that we can use Libertine just as well here in order to allow users to run their applications in a confined container without clocking up their system. Because Unity 7 is already running X11 Libertine's classic app can use the running X server to connect to their window manager so we don't need any special tools for that.
21:23
And because Unity 7 is not a target platform applications in machine containers are automatically discovered by Unity Dash so we have to jump through some hoops to get them to show up either by writing some final desktop files or by manually launching Libertine launch.
21:45
Unity 8 is a new and improved version of Unity 7 running Mirrors display server instead of X and creating thousands of users. Desktop Unity 8 is also not particularly confined, but running Mirrors instead of X, we needed some of our previous security things.
22:01
Mirrors Unity 8 can run some applications natively, such as modern TDAX but are unable to run all of our classic apps natively just yet. And Libertine works well in Unity 8. And it's built into the application discovery logic. You guys can see here, search for pingas and it should come up. Click on icons to open classic app
22:21
and it's a regular version of launch except in this case there's no X11 so the system will also start instances with Xmir or there are four applications to attach to the window manager tool. Xmir allows us to run X11 apps in an X11 window on top of Mirrors. Each one of the leveraging applications will get its own Xmir
22:42
bringing out a lot of communication between the apps. And with this step we successfully launched XApps in a non-X environment. So from now on, we can touch We can touch also on Xmir and it will can't run on XApps on its own.
23:01
Furthermore, users are prevented from using app with DPAX to install apps due to with DTAX being around its final system by default. So even apps that would work in a desktop Unity 8 session might not be installable and runnable in any of the touch sessions. However, in 3.0, XApps don't always run as well on with DTAX. But on some devices like the BQM10, we pre-install
23:22
a little bit of a container using essential classic apps such as LibreOps and XJaps. And we found that they work just fine, and it creates a truly convergent environment. Of course, we were looking at almost exactly the same way you were as you did in desktop Unity 8 launching on touch with an Xmir instance.
23:40
This is the older kernel in the Xmir that requires a T-shirt which is one of the reasons we don't get root-based containers around. Alright, so our end goal has always been to run classic apps and completely confine your system, which brings us to Snaps. As mentioned, Snaps are a packaging system to allow complete requirements. In which your personal
24:01
and system provides all new Snaps. In packaging, we can snap, we load the obtained source, you can see me and create one large package file. We need to update some padding and environment and variables to be more flexible for Snaps. And we need to make some architectural changes so those that retain the load can be. Snaps also encouraged us
24:21
to hurry the production of our little table back end using the Lexi snap available that we can use. In order to access the resources of Snaps, you have to connect to the interfaces, and we connect to quite a few. This is a short list currently that we use. Lexi obviously connects to Lexi.
24:41
Now we can network by connecting the sockets of the network, connect to the used home directories, and all the rest of your sound and graphics. Okay, live demo. This never looks great. Well, in plot twist, I've been running this presentation and Firefox ran a
25:01
limiting container in Snaps the whole time. So it works. But it might be interesting to also run some other applications real quick. Something like X to the left. Come up. And we can say
25:23
Firefox, and you can see the command that's being run within the container. And this is running really isolated in its own root file system, completely separate from the host. What else? You can run something like Leetpad to edit documents.
25:41
I can open something that already exists. Test file. And it all stays fine. And it doesn't crash, which is good. One more. How about some Minesweeper?
26:00
And there, well, pretty good. We've got 200. This is on Unity 8? This is all on Unity 8, yeah. So yeah. To be clear, this is everything running on Unity 8. Everything is a Snap running on XD containers. And with that, we can go ahead and finish up.
26:21
Oh, wrong. A few more of that. To very quickly review, we wanted to be able to run classic desktop applications in a fully-defined
26:40
and fully-defined X-List ecosystem in a packaging format. And we used Liberty to do just that, mostly through the containers. And there's a bit of work to do in the backend with Snaps. And we have been working on a lot of robotic machines. There's obviously a lot that we're still working on. All of these things, Unity 8, Mirror,
27:01
and Liberty and Snap are all moving a mile a minute, and also still broken. We're in trouble with a lot of apps that can keep us. Our Snaps are big. I recently think I might have made a regression where audio doesn't work in the Snap, but that's called working too hard
27:20
on that demo. And that's it for me. All my contact information might be out of time. I'm right on time, so
27:41
thanks to us once the setup probably can answer some questions, but it will be recorded because the recording is coming real soon. So are there any questions?
28:02
Yes. When will the app be running in the state container? So there is one being running in the state container? Right, so the system is made up such that you could create as many containers as you want. And then our underlying back end will know which container I needed to be, which container is the right one to run the application in.
28:20
All of these are running in the state container. And so if you want to go... Are you going against what you said in the beginning that it was like the modification tries to get the other ones to be presented on separate Xs? Right, so they all run, so we work around the problem of sneaking on each other, sort of interrupt. Does anyone have an HCI value to each of your models?
28:44
Yes. Yeah, so we prevent the apps from talking to each other through X11 by using individual instances of Xmir and I think you can do it. And so
29:01
each application will have its own Xmir that it's running on. But they can still all talk to the same cloud system. Yes. What about Cleveland? No comment. Although I'll say that it's nice to have a diverse ecosystem of new display servers and packaging formats on that horizon.
29:22
Anyone else? Okay, thank you.
Recommendations
Series of 5 media