We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Classic Applications in a Confined Ecosystem

00:00

Formal Metadata

Title
Classic Applications in a Confined Ecosystem
Subtitle
Using libertine to run deb-packaged, X11 applications in an all-snaps Ubuntu
Title of Series
Number of Parts
611
Author
License
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language
Production Year2017

Content Metadata

Subject Area
Genre
Abstract
Libertine is an application suite for installing and running classicapplications in a confined environment. Using libertine allows us to useeveryday applications which have not been ported over to a new packagingformat or don't work natively with a new display server. Libertine has allowedus to take advantage of classic apps such as Firefox and LibreOffice in UbuntuTouch, a version of Ubuntu for devices based on click packaging andMir/Unity8. Ubuntu developers have recently unveiled snappy, a new kind ofpackaging system which keeps an application confined to a readonly filesystemwith all of its dependencies managed internally. Since migrating applicationsto snaps is a manual process, we've been working on a libertine snap to giveus the ability to use non-snap applications in an all-snaps Ubuntu. This lecture will start with an overview of why libertine is a necessarycomponent for managing and launching deb-based applications on both a classicMir/Unity8 system and a completely confined all-snaps Ubuntu. Then there willbe a review of how libertine works in general through container management andapplication launching. Finally, these topics will come together with anexplanation of the infrastructure required for libertine to operate in an all-snaps environment. There will be light code and tooling examples throughout,as well as a live demonstration of deb-based X applications running in a snapenvironment.
17
Thumbnail
24:59
109
Thumbnail
48:51
117
Thumbnail
18:37
128
146
Thumbnail
22:32
162
Thumbnail
23:18
163
Thumbnail
25:09
164
Thumbnail
25:09
166
Thumbnail
24:48
171
177
181
Thumbnail
26:28
184
Thumbnail
30:09
191
Thumbnail
25:08
232
Thumbnail
39:45
287
292
Thumbnail
25:14
302
Thumbnail
26:55
304
Thumbnail
46:54
305
314
317
321
Thumbnail
18:50
330
Thumbnail
21:06
333
Thumbnail
22:18
336
Thumbnail
24:31
339
Thumbnail
49:21
340
Thumbnail
28:02
348
Thumbnail
41:47
354
Thumbnail
26:01
362
Thumbnail
18:56
371
Thumbnail
13:12
384
385
Thumbnail
25:08
386
Thumbnail
30:08
394
Thumbnail
15:09
395
411
Thumbnail
15:10
420
459
473
Thumbnail
13:48
483
501
Thumbnail
32:59
502
Thumbnail
14:48
511
518
575
Thumbnail
25:39
590
Thumbnail
25:00
592
Thumbnail
23:32
Percolation theoryThermodynamisches SystemSpacetimeRevision controlDifferent (Kate Ryan album)Cartesian coordinate systemLine (geometry)Statistical hypothesis testingPatch (Unix)SoftwarePhysical systemMathematicsDirectory serviceSlide ruleConnected spaceOperating systemColor confinementCanonical ensembleServer (computing)Differenz <Mathematik>Video gameMereologyVirtual machineProcess (computing)Mobile appDatabase normalizationContent (media)BitTouchscreenNumberIntegrated development environmentClassical physicsBinary codeMalwareRootkitElectronic visual displayComputer architectureChainMultiplication signFile systemSystem administratorVisualization (computer graphics)Loop (music)Uniform resource locatorData managementTask (computing)Interface (computing)Library (computing)Information securityResultantComputer fileElectronic mailing listComplete metric spaceUsabilityNeuroinformatikVector spaceQuicksortFlow separationKeyboard shortcutIterationContext awarenessExtreme programmingSuite (music)Radical (chemistry)1 (number)Form (programming)Hybrid computerSpectrum (functional analysis)Software frameworkAdditionMaxima and minimaMusical ensembleEinstein field equationsGoodness of fitDecision theoryComputer animation
File systemSet (mathematics)Physical systemComputer programmingData managementDirectory serviceCartesian coordinate systemWindowOutlierIntegrated development environmentProcedural programmingData dictionaryArray data structureDatabaseImplementationGoodness of fitDifferent (Kate Ryan album)CodeType theoryProcess (computing)NumberCASE <Informatik>GradientFlagSocial classLimit (category theory)MereologyLibrary (computing)Raw image formatLine (geometry)Mobile appComputer fileException handlingKernel (computing)NamespaceFunction (mathematics)Group actionFocus (optics)Server (computing)Keyboard shortcutTrailSoftware frameworkScripting languageUsabilityRootkitOnline helpDomain nameDefault (computer science)Binary codeAbstractionFunctional (mathematics)Link (knot theory)Utility softwareMultiplication signSampling (statistics)Patch (Unix)Instance (computer science)Traffic reportingMusical ensembleAddress spaceKey (cryptography)
Socket-SchnittstelleElectronic mailing listIntegrated development environmentMobile appPlotterDemo (music)Automatic differentiationCartesian coordinate systemException handlingClient (computing)Regular graphPresentation of a groupSystem callLibrary (computing)CASE <Informatik>Line (geometry)InformationEvent horizonPhysical systemMathematicsProduct (business)Structural loadOrder (biology)Information securityRootkit1 (number)Table (information)Computer iconFlow separationWrapper (data mining)Default (computer science)ImplementationFamilyClassical physicsInstance (computer science)Virtual machineMultiplication signSeries (mathematics)Data managementWindowGraphical user interfaceTelecommunicationKeyboard shortcutNumberElectronic visual displayFile systemVariable (mathematics)CodeThermodynamisches SystemRevision controlBitBinary codeCompilation albumDirectory serviceMappingServer (computing)Touch typingGraphics tabletComputer fileKernel (computing)Front and back endsIntrusion detection systemThread (computing)Interrupt <Informatik>Computing platformGroup actionAreaScheduling (computing)Computer programmingLetterpress printingMusical ensembleOcean currentWeb pagePower (physics)MereologyPressureView (database)State of matterSource code
Cartesian coordinate systemReal numberComputer fileRootkit
Computer fileSoftware testingState of matterGreen's functionPosition operatorProgram flowchart
1 (number)
Data recoveryState of matterWebsiteAreaMathematicsMobile appRow (database)RoboticsFront and back endsFlow separation1 (number)BitPhysical systemVirtual machineInterrupt <Informatik>QuicksortInformationLinear regressionInstance (computer science)Right anglePoint cloudCartesian coordinate systemClassical physics
Electronic visual displayFile formatServer (computing)HorizonComputer animation
Computer animation
Transcript: English(auto-generated)
Are we just starting? So hi, everyone. Bonjour à tous. Thanks for joining me. Like this slide says, my name is Larry Price. I work at Canonical on some of the new English Chinese
perks of popular living space operating systems of Bluetooth. This afternoon, I'm here to talk to you about how we run classic applications in a confined ecosystem in Ubuntu. So what do I mean by a classic application? And furthermore, what do I mean by a confined ecosystem?
Well, so let's start with a little bit of background. My job is focusing on the idea of convergence, bringing the same as Ubuntu on any platform, regardless of screen size, computing power, stuff like that. Convergence has driven us to build a lot of really cool software over the years, discovering new concepts of security and usability
from a computer spectrum. While we want to take advantage of these new architectural tool chains that we built, we also want to take advantage of a stable, mature software available in the archives. And we want the use of that software to be seamless to our existing users.
So a classic application in this context is any application that I can currently install with a traditional package manager, with a package or a software center, and then run using either the terminal or a graphical discovery interface, such as Unity Vetch. These applications are terminal-based for Graphi3 CDX11.
Their application we'll be using for years to make our Linux desktop experience complete. Actually, Firefox is your office, again, and D is your ID, PINGAS, and many others. Without these apps, it would never be your Linux desktop instead of every year being your Linux desktop.
And though some applications have already been migrated to new display servers and new packaging systems, this talk will be focused on those which have got to be migrated. Classic applications were designed to be installed and used in an unconfined ecosystem.
The way that applications have been traditionally installed and accessed resources has grown up organically alongside the tooling and other parts of system management. And throughout this process, applications can or must access parts of the system that they are never meant to access. And malicious software could easily snoop on the private business and deny applications.
This is not only a shown desktop where you're likely checking on the machine once a day or so, but also a potential threat to internet-enabled devices which are intricately outdated or checked for health issues and could be scattered throughout your home. Undependencies are another place where classic applications run the problem.
Typically, it acts to pin on several other packages and sometimes those packages can conflict with other versions of packages installed by other applications. Worse, sometimes applications need newer versions of packages that aren't available on the system or similar packaged versions like different patches on your distros. Putting the application and pulling back changes until a newer application version becomes available
which may not be until the distro makes a new release. This could be preventing users from running the best version of the PCOS software at any given time. On top of that, we also see an increase in the pollution of our root filesystem and this is hard to read but this graphic is a great visualization
of the way that the Linux filesystem looks. You may notice that it's pretty complicated and some of these things are a little bit redundant. For instance, this is the contents of my add variable. We mean they're user-runable binaries that's bin, sbin, userbin, user sbin for games,
user-local bin, and any number of doc directories in my home directory, as well as Heroku sneaking in some random user-local root event. Similarly, library and data files appear wherever they want these are the results I'm bringing out so I'm not considering this complete list
and data directories are kind of wherever they need to be. So, to generalize the goal of confinement, we want to move away from letting apps loop on each other without permission and we want to take back control of our filesystem. Putting these applications in a known location and giving each application the appropriate tendencies.
Our traditional package manager and definitely based operating systems is over 20 years old and appears to be one of those years on its shoulders and because of this, we thought it would be best to introduce a new packaging toolchain.
Historically, a couple of years ago Canonical announced click packages which were kind of a solution to this problem. An application packaged in click is bundled with any specific dependencies that it needs and it takes advantage of frameworks to access any shared resources
that it might need. It also uses AppArmor to create a confined application and specify any security feature that you need. Clicks were used to package applications for DNS where I would do test devices giving them a little bit of a headline. These devices ran here instead of X11 as well as UnityA
as a display server which sort of gave us our combined ecosystem but these toolchains were missing some important things and lack maturity. Fast forwarding to today, we've seen the emergence of snap vectoring
also referred to as snappy. Snappy is essentially another iteration on clicks giving better version management sleeker applications with finer definitions and new ways to interface with system resources. You can use snaps today on Modern Ubuntu or several other distros snaps will all live in a special root directory and can be completely confined when installed from stable channels
or unconfined when installed from unstable channels if you must. Snap apps come bundled with all their connections meaning that all the apps will have the connections they need and upgrade whenever they want. No longer do you need to wait on package containers to get them into the distro. Snaps are automatically updated once a day
taking the pressure off of the user to keep their system up to date. Snaps come with the option for delta updates which means that the toolchain can validate a diff instead of an entire package potentially making downloads smaller and faster. And as confinement is concerned snap apps are relegated to their root only piece of the process requiring interfaces to be connected to perform system tasks
such as reading the display, using audio and using the network. Instead of using a hybrid or classic environment in addition to a traditional package manager we can continue to pull in and use our classic applications while enjoying confinement security in our migrated apps. However, one exciting premise is
Snaps is the ability to run in an all-snaps environment which we, I think, call a bit diverse. But this puts us in a bind we can't run classic apps in this all-snaps environment. Our little classic applications are looking for an unconfined system and looking for the opposite of x11 and this all-snaps environment
is prepared to use in neither. We need a way to round this issue to allow users to run applications just like they would in a classic system. So to resolve this issue we've been working on a suite of tools which we call Libertine. Libertine is a person who acts without moral strength.
Libertine is an extreme form of hedonism living one's life for pleasure while only thinking of oneself. In a sense, our classic applications are practicing Libertines. And it's our job to pull them back and to more on my standing. These applications need a safe place to go while the rest of the world falls in line
and we provide the safe space as a Linux container. Containers are generally unrecognizable from a traditional Linux desktop experience. A Libertine container has some file systems to work with and binds to system resource directories to access things like home and some system resources. Internally, we continue to use apps and de-package to manage our packages
even when the host system is not allowed to use de-package, even if de-package is not installed on the host system. Libertine also connects to a program called Exmir which allows us to run an X11 window isolated on top of Mir. Every installation, removal, and system modification is done internally
protecting the outside world from whatever these classic applications may be doing to the classes. In the Libertine suite, it has many parts to extract away the underlying limitations of the classic printer as best as possible. C++ library including Python launchers
and command line interfaces and tools to control other aspects like watching Exmir and other people on your clipboard. I'll go over a few of these now. Again, a little difficult to see. This is just JSON dictionary and arrays
and all the good JSON stuff. So we use Libertine uses JSON as a database file keeping track of all installed packages creating containers modified bind mounts and we just use it as a database maintained individually for each user on a system
and using JSON we found lets us not handle any database frameworks which is good for our use because then we can also just ask users for the raw database file and they don't mind because 30 or 40 lines of JSON. The crux of our
container management is our Python library. Of course it contains an abstraction around the database that we just saw and utility functions but the primary use is to extract away the process of handling different implementations of Libertine containers of which we support three types JARU, LXC, and LexD.
JARU is a special case maintained only for use on devices with kernels too old to support LXC or LexD. For the JARU back then we used a finger cutting line tool to create a base domain system in the user's home directory as a JARU jail from which we'll run all of our commands and applications
and when running commands within a JARU we'll use a tool called help keyword and we'll generally try to run a keyword on privilege to run that command within our JARU but every now and then we have to run privilege keyword commands and so we prefer not to use root command containers instead of them completed though it's still useful.
But LXC containers have been our preferred container type for most of Libertine's lifetime. LXC combines the kernel C groups and support isolated namespaces to provide an isolated environment for applications.
We're capturing a song from the camera but it's just not perfect. We have a container inside the container that gives you access to home and system directories and setting up how to leverage LXC to integrate additional best of features and we've added great links to ensure that we can access things like audio through the system.
Also note that the LXC Python API is quite mature and works relatively well for our needs for instance. This is a code sample showing how we use the LXC container on the patch method to run our command into your Python but the newest shiny back-end
for Libertine is LXC containers. LXC is a new face on top of the LXC implementation, simplifying container management and frequently providing speed and usability improvements as well. Libertine LXC containers are outliers in that we depend on the LXC back-end to
maintain the root file system though we still maintain locally they use home directories and mobile applications directories. For the most part setting up LXC is just like every set of an LXC container we biomass it with devices mapping our own procedures in the system and it's also going to be okay.
I guess one weird thing about LXC is how we have to set up slash run slash user slash run slash user is a system directory for storing files and running processes when LXC finish starts. This directory unfortunately is overwritten completely when it starts, so we need to
remap it using the sound and stutter scripts. To work around this we create a custom script and copy it into the container. The script creates missing directories, faces and permissions and remaps things like slash run slash user since here the system is only reordered. We add the files to the container using the files API
which is back there which is part of the Python LXC package. But the Python API for LXC is not quite mature enough that we can use it for all of our use cases. So we use a hybrid solution that you can play the API and calls to be open to start processing manually. As an example here, we use the
Python API to update those devices. It's just a dictionary and we use the Python pylyc API to create this profile, update the dictionary and say using LXC exceptions to tell us when there's an error. On the other hand, when we run applications we have to use psc.py open
to launch applications because that's the best way for us to run LXC applications in the background while still getting all the output in the foreground. So we also have these services that are responsible for making sure the containers are
started and stopped appropriately, LXC and LXC. This is dialed with a safe battery life so that the containers aren't started all the time. And it ensures that when a user stops one container or stops one application within a container, the container doesn't stop, but we wait for all applications to be closed. Moving on,
we use the run with binaries Python library, or a Python binary called the original container manager, which allows you to essentially create and manage containers and excel packages and modify them by amounts.
This is a basic overview of using the original container manager to create a container. It's fairly straightforward. We just call them grade sub-command with any number of flags all of which have intelligent defaults except for ID so the user doesn't have to know what kind of container backing we're using. Here I've shown that as just a black box, and then it just
updates the database and adds all of the file systems using the given containers backend. And this is similarly just what installing a package would look like. Note that a package name may not line up with the binaries installed, but we're only
keeping track of what the user is supposed to be asked to install. Because the purpose of using the container manager is to prevent the user from having to deal with the low-level concepts of any particular container type or package management system and focus on the key concepts of creating and installing.
We also have U.S. servers within Python to replace the work being done by the last application we installed. All of this has been fully implemented yet, but many of the commands that we use to replace the container manager takes a very long time to arrive, and bad things will happen when those commands are interrupted. For instance, stopping bpack, which of course will cause, or can cause, broken
packages to appear in the system. Libertine being able to solve that by being a completely asynchronous method where we use Python threads and send updates to U.S. signals. And although we would like to replace the container manager backend with this in the future, we currently just use it to list container ads and application IDs.
Which brings us to libLibertine, which is the root of discovering container information for several external libraries. It's currently a wrapper around a client for Libertine-D, and many of the low-level ads libraries that access Libertine are written in C++, and Libertine gives them a convenient
way to access Libertine information without having to talk directly to D+. The code that we're listening to is just us making a simple D++ call from C++. We also have a GUI written in C++ with QOL. This is an easy way for us to manage containers and even allows us technical users to be able to
easily manage containers and install packages. It's mostly a series of lists showing you what you can do in a list of containers in which you click through the list of packages. For the most part, my family GUI is the same as it is in the mainline tool.
And now the real neat thing is that it can launch, which is a Python tool that we use to actually launch applications. What Libertine launch will do is connect the environment for running apps and then forward that environment to the container implementation to actually run the application. Libertine launch can be used for a terminal, but we generally prefer it to be used from a launching tool,
like in Ubuntu and Unity 8 we use at the draft launch. Hasty is a daemon. It's a convenient binary to help applications use the clipboard. We just use huge signals and slots to copy and paste events from the underlying Comp Hub and then we forward those events to our excellent maps.
Doing this, we can use copy and paste pretty much like we would in these applications in a classic desktop. So what actually happens when we launch applications through Libertine? By clicking our desktop icon, we'll call Libertine launch with a container ID and application event line
which was properly found using Libertine which found that information actually from Libertine. Libertine launch will then start utilities and paste it next year. It'll set up a session by coming up the environment and setting up a whole bunch of sockets to communicate between the two. And then it will launch a container instance
based on the Python library which will then do any number of things based on the container type. Update bind mounts, launch the window manager to connect to an instance with X, and finally launch the app. And that's most of the full picture. To review and build a container using one of the living machines and set it up to install applications through your app
even when the app is not a system. Libertine keeps your container and its classic application is confined to their root file system and a small folder of code to let them access things like the home directory and its resource directory. At which point, something or someone can launch the application by leveraging launch which will prepare the
container environment to mirror the host, show the window manager and set a container connected to the host display variable and launch applications. Given all of that, where can we actually use Libertine?
The answer is well in most of our modern movie systems. Unity 7 is current in the desktop session in the Bluetooth and using Libertine in Unity 7 is a little bit silly because Unity 7 is not a confined environment. Unity 7 already runs X11 and already installed the package to be packaged
but I think we are learning that we can use Libertine just as well here in order to allow users to run their applications in a confined container without clocking up their system. Because Unity 7 is already running X11 Libertine's classic app can use the running X server to connect to their window manager so we don't need any special tools for that.
And because Unity 7 is not a target platform applications in machine containers are automatically discovered by Unity Dash so we have to jump through some hoops to get them to show up either by writing some final desktop files or by manually launching Libertine launch.
Unity 8 is a new and improved version of Unity 7 running Mirrors display server instead of X and creating thousands of users. Desktop Unity 8 is also not particularly confined, but running Mirrors instead of X, we needed some of our previous security things.
Mirrors Unity 8 can run some applications natively, such as modern TDAX but are unable to run all of our classic apps natively just yet. And Libertine works well in Unity 8. And it's built into the application discovery logic. You guys can see here, search for pingas and it should come up. Click on icons to open classic app
and it's a regular version of launch except in this case there's no X11 so the system will also start instances with Xmir or there are four applications to attach to the window manager tool. Xmir allows us to run X11 apps in an X11 window on top of Mirrors. Each one of the leveraging applications will get its own Xmir
bringing out a lot of communication between the apps. And with this step we successfully launched XApps in a non-X environment. So from now on, we can touch We can touch also on Xmir and it will can't run on XApps on its own.
Furthermore, users are prevented from using app with DPAX to install apps due to with DTAX being around its final system by default. So even apps that would work in a desktop Unity 8 session might not be installable and runnable in any of the touch sessions. However, in 3.0, XApps don't always run as well on with DTAX. But on some devices like the BQM10, we pre-install
a little bit of a container using essential classic apps such as LibreOps and XJaps. And we found that they work just fine, and it creates a truly convergent environment. Of course, we were looking at almost exactly the same way you were as you did in desktop Unity 8 launching on touch with an Xmir instance.
This is the older kernel in the Xmir that requires a T-shirt which is one of the reasons we don't get root-based containers around. Alright, so our end goal has always been to run classic apps and completely confine your system, which brings us to Snaps. As mentioned, Snaps are a packaging system to allow complete requirements. In which your personal
and system provides all new Snaps. In packaging, we can snap, we load the obtained source, you can see me and create one large package file. We need to update some padding and environment and variables to be more flexible for Snaps. And we need to make some architectural changes so those that retain the load can be. Snaps also encouraged us
to hurry the production of our little table back end using the Lexi snap available that we can use. In order to access the resources of Snaps, you have to connect to the interfaces, and we connect to quite a few. This is a short list currently that we use. Lexi obviously connects to Lexi.
Now we can network by connecting the sockets of the network, connect to the used home directories, and all the rest of your sound and graphics. Okay, live demo. This never looks great. Well, in plot twist, I've been running this presentation and Firefox ran a
limiting container in Snaps the whole time. So it works. But it might be interesting to also run some other applications real quick. Something like X to the left. Come up. And we can say
Firefox, and you can see the command that's being run within the container. And this is running really isolated in its own root file system, completely separate from the host. What else? You can run something like Leetpad to edit documents.
I can open something that already exists. Test file. And it all stays fine. And it doesn't crash, which is good. One more. How about some Minesweeper?
And there, well, pretty good. We've got 200. This is on Unity 8? This is all on Unity 8, yeah. So yeah. To be clear, this is everything running on Unity 8. Everything is a Snap running on XD containers. And with that, we can go ahead and finish up.
Oh, wrong. A few more of that. To very quickly review, we wanted to be able to run classic desktop applications in a fully-defined
and fully-defined X-List ecosystem in a packaging format. And we used Liberty to do just that, mostly through the containers. And there's a bit of work to do in the backend with Snaps. And we have been working on a lot of robotic machines. There's obviously a lot that we're still working on. All of these things, Unity 8, Mirror,
and Liberty and Snap are all moving a mile a minute, and also still broken. We're in trouble with a lot of apps that can keep us. Our Snaps are big. I recently think I might have made a regression where audio doesn't work in the Snap, but that's called working too hard
on that demo. And that's it for me. All my contact information might be out of time. I'm right on time, so
thanks to us once the setup probably can answer some questions, but it will be recorded because the recording is coming real soon. So are there any questions?
Yes. When will the app be running in the state container? So there is one being running in the state container? Right, so the system is made up such that you could create as many containers as you want. And then our underlying back end will know which container I needed to be, which container is the right one to run the application in.
All of these are running in the state container. And so if you want to go... Are you going against what you said in the beginning that it was like the modification tries to get the other ones to be presented on separate Xs? Right, so they all run, so we work around the problem of sneaking on each other, sort of interrupt. Does anyone have an HCI value to each of your models?
Yes. Yeah, so we prevent the apps from talking to each other through X11 by using individual instances of Xmir and I think you can do it. And so
each application will have its own Xmir that it's running on. But they can still all talk to the same cloud system. Yes. What about Cleveland? No comment. Although I'll say that it's nice to have a diverse ecosystem of new display servers and packaging formats on that horizon.
Anyone else? Okay, thank you.