AV-Portal 3.23.3 (4dfb8a34932102951b25870966c61d06d6b97156)


Video in TIB AV-Portal: libcapsule

Formal Metadata

segregated loading of dynamic libraries
Title of Series
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Release Date

Content Metadata

Subject Area
libcapsule is a project that allows segregated dynamic linking: Access to the symbols of a library without being exposed to any of the dependencies of that library without requiring recompilation of the binary that pulls it in. libcapsule's goal is to improve portability of programs that are distributed with runtimes (cf flatpak) that still need access to [some] libraries from the host (eg libGL) while insulating said program from any system libraries outside the runtime other than those it directly requires.

Related Material

System call Run time (program lifecycle phase) Structural load Multiplication sign Sheaf (mathematics) Open set Parameter (computer programming) Mereology Machine code Computer programming Linker (computing) Electronic meeting system Object-oriented programming Aerodynamics Functional programming Library (computing) Stability theory Link (knot theory) Mapping Wrapper (data mining) Structural load Software developer Electronic mailing list Bit Control flow Digital rights management Process (computing) Linker (computing) Order (biology) Chain System programming Right angle Quicksort Procedural programming Arithmetic progression Row (database) Spacetime Reverse engineering Point (geometry) Dialect Vapor barrier Link (knot theory) Patch (Unix) Collaborationism Similarity (geometry) Portable communications device Revision control Term (mathematics) Computer programming Computer hardware System programming Spacetime Lie group Address space Default (computer science) Run time (program lifecycle phase) Computer program Machine code Cartesian coordinate system System call Symbol table Table (information) Personal digital assistant Network topology Statement (computer science) Game theory Library (computing)
Building Run time (program lifecycle phase) Thread (computing) Multiplication sign Set (mathematics) Bit rate Parameter (computer programming) Open set Electronic signature First-person shooter Semantics (computer science) Computer programming Heegaard splitting Mechanism design Malware Linker (computing) Core dump File system Flag Object-oriented programming Functional programming Resource allocation Library (computing) Simulation Link (knot theory) Mapping Namespace Wrapper (data mining) Structural load Point (geometry) Interior (topology) Fitness function Bit Instance (computer science) Open set Thread (computing) Electronic signature Process (computing) Exterior algebra Linker (computing) Auditory masking Order (biology) Phase transition System programming IRIS-T Normal (geometry) Right angle Lipschitz-Stetigkeit Quicksort Freeware Alpha (investment) Row (database) Reverse engineering Point (geometry) Frame problem Game controller Wrapper (data mining) Link (knot theory) Patch (Unix) Image resolution Real number Gene cluster Crash (computing) Deadlock Term (mathematics) Boundary value problem Directed set Lie group Booting Game theory Address space Plug-in (computing) Demo (music) Computer program Machine code Cartesian coordinate system System call 1 (number) Symbol table Compiler Uniform resource locator Pointer (computer programming) Integrated development environment Personal digital assistant Function (mathematics) Read-only memory Video game Game theory Hydraulic jump Routing Library (computing) Address space
Web page Frame problem Default (computer science) Wrapper (data mining) Mapping Namespace Server (computing) Bit Bit rate Machine code First-person shooter Pointer (computer programming) Event horizon Error message Linker (computing) Buffer solution Electronic meeting system Source code Cuboid Row (database) Default (computer science)
Frame problem Wrapper (data mining) Demo (music) Server (computing) Bit rate Open set Infinity First-person shooter Fluid statics Event horizon Error message Personal digital assistant Software testing Alpha (investment) Library (computing)
Asynchronous Transfer Mode Texture mapping Computer-generated imagery Analogy Complete metric space Open set Shader <Informatik> output Gamma function Default (computer science) Touchscreen Demo (music) Computer file Client (computing) Motion capture Machine code Disk read-and-write head Array data structure Network socket Function (mathematics) Computer hardware Endliche Modelltheorie Software testing Software cracking Window Library (computing)
Run time (program lifecycle phase) Confidence interval Direction (geometry) 1 (number) Set (mathematics) Open set Mereology Computer programming Different (Kate Ryan album) Semiconductor memory Oval Functional programming Data compression Error message Stability theory Area Bit Data mining Curvature Chain Order (biology) System programming Lipschitz-Stetigkeit Quicksort Video game console Point (geometry) Vapor barrier Link (knot theory) Real number Collaborationism Device driver High-level programming language Number Revision control Latent heat Profil (magazine) String (computer science) Computer hardware System programming Integer Demo (music) Surface Projective plane Interactive television Memory management Cartesian coordinate system Symbol table Pointer (computer programming) Personal digital assistant Hybrid computer Game theory Library (computing)
Link (knot theory) Touchscreen Texture mapping Collaborationism Client (computing) Bit Motion capture Machine code Machine code Bulletin board system Array data structure Computer hardware Function (mathematics) System programming Functional programming Gamma function Alpha (investment) Library (computing)
Linker (computing) Function (mathematics) System programming Collaborationism Address space
Link (knot theory) System programming Fitness function Counting Machine code Line (geometry) Alpha (investment) Library (computing)
System programming
hi my name's Vic I work for Calabria you've probably seen a couple of my
colleagues talk here already before and today I'm to talk about something that may not mean very much to which I'm calling segregated dynamic linking because naming things is hard and this will do until somebody comes up with something better okay so let's jump into it what is it you may be wondering so let's start with a statement of the problem that we're trying to solve these days a lot of applications are containerized I'm using that term kind of loosely there in some kind of wrapper they have libraries that come with them the libraries come from the runtime rather than from the hosts it's just a way of improving Portability and stability because you know we all know most people here know about the kind of portability problems we have well that's kind of mostly true it's a little bit of a lie some of them still need to come from the hosts notably your graphics stuff it's quite closely tied to the hardware there's no reasonable way of shipping it in a given run time and still having it work with whatever hardware you end up having to work with which is all fine for the most part you know you can sort of do that but then you've problem which we were trying to solve in the first place with our containerized applications which is that the host and runtime libraries may be incompatible this is kind of a really very primitive sketch of what the problem might look like you can see that we've got a dependency down there and we've got something in the runtime that uses it the main executable uses it and unfortunately we also need to have this library from the host probably Lib GL or something similar it's incompatible it's not going to run everyone's really unhappy because we haven't solved the problem that we set out to solve so what could a solution look like well we kind of want to have segregated linking we want the host library that needs its compatible version or incompatible version of the library link to it what we want the rest of everything else not to see that you know we want to be generally unaware that there's another incompatible version of this dependency over here and for those of you who are familiar with how linking works how many people is that by the way just so I know how many people I'm lying to it's a fair few okay so if you spot me telling any lies don't don't tell anyone else that's keep the illusion of competence alive so you'll know that this isn't how linking on Alf works I mean there are other systems which in which the link is a tree and that has its own problems but in general we have one copy of library and everything uses it which is kind of the problem that shared library set out to solve in the first place so what are our objectives here we want to expose only the library that that we we are pulling in so we want to expose Lib host but we don't want to expose this copy here which is incompatible it as it's a dependency it's the thing we're trying to protect ourselves from we don't want to see it we don't want to have to change the code because if we have to change the code of our application or our libraries that would kind of defeat the point you know we we could say to people hey you're gonna have to rewrite your programs to make them all portable well that's been tried several times we all know how that goes it doesn't happen we don't want to have to recompile the applications again same thing it's just going be nonsensical if we say and then you have to rebuild whatever the large application is that you wanted to run and we were very much like for there to be no performance hit so we also want it to be transparent and it should happen when someone loads their containerized application or game or whatever it should happen pretty much transparently the user should not know anything's happening here and just to make life easy for ourselves we want this to be done with minimal manual intervention you know like we want this to be as frictionless as possible because anytime you put a barrier in an application developers way they won't do it with reason you know we're asking them to do something and they should say well I shouldn't have to worry about this it's not my problem it's yours so well let's go through this and we'll see how many of these we can chief so what are the pieces of the puzzle that we're going to need for this first of all private dependencies a library should have isolated dependencies normally there's single linked list effectively of the dependencies in your in your program space everything will come from there you'll get anything that you mentioned the name of as your dependency you will see symbols from that and it's just one flat list there's no segregation going on there so we need to kind of defeat that somehow worst case scenario we're going to have to rewrite the linker I really really don't want to do that so let's hope I don't have to and the answer turns out to be DRM open so this is a new library call that was introduced in G Lib C and around I want to say 2.19 I think it was sort of it sort of existed before then but that's sort of the first version that works without immediately exploding in most cases it's it's not quite there yet it's a work in progress patches are still being fed upstream but from there and then on it kind of works and what does it do it is like DL open it adds a library after you've loaded up to your link map but it can also create a new secondary or tertiary link map and add libraries to that instead sounds kind of promising it's it's basically what we ordered for this and just like DL open it automatically loads dependencies so we have problem in that we want to pick the right libraries like if you remember from back we had two copies of the dependency we want to be able to pick the right one now by default it's going to go to your standard loading path and it's going to load the one that's coming from your runtime and your container with your containerized application c libraries are picked from the search path we kind of need to defeat that because we will still get the wrong copy for at least one side of our link chain if if we just let it do its thing so we want isolated libraries to come from the non-standard search path we must divert searches for dependencies of isolated libraries can we do that yeah sort of the linker loads all listed dependencies but it won't reload items that are already in the link map it's targeting so if libraries already there it looks at your dependencies and said you asked for live foo I have something called Lib fou my work here is done moving on to the next one so if we load the libraries for the isolated library by their full path not just by their SOE name not just by live food RSO we can say you know run host whatever wherever we've exposed the host libraries we do them in Reverse dependency order so we walk the dependencies backwards we say Oh Lib host needs Lib DEP I will go and get that from wherever I've mapped in the host libraries then by the time the linker gets to it it won't go and search the standard search path it'll look in the link map that it's loading to and it'll say I already have that my work here is done I'll get my symbols for you from here we can move on so yeah we can do it we can get what we wanted well sort of we're partway there we got partway there we haven't had to rewrite the linker which is always a plus in my book but what we've achieved is we've isolated the dlm open symbols completely they're not available by default to anything so we still haven't met our goal of all of this being frictionless and seamless so we need a caller of our library that's coming from the host to get those symbols automatically and in order to do that we need to understand how dynamic library calls work so we're going to do a little bit of a dive into that now don't worry about it because we're going to go into some detail but at the end of it what we will have figured out is that we don't actually need to know any of that detail because it's sort of works the way we need it to which is always nice so this is what jumping to a foreign function call of that a call from another elf object a library in this case looks like we start by pushing our arguments for the call onto the stack and then we look into something called the procedure linkage table so each foreign function that you have has an entry in the procedure linkage table and we look up its address we read it back from something called a relocation record and the things that are important to note here are the red and yellow sections are read-only they can't be touched once they've been loaded it's only the it's only the blue columns the stack and the RR written to everything else is read-only which sort of brings us some nice features of shared libraries for example they can be shared and between processes without any problems so this is the first step of calling a function so we've gotten address from the relocation record and then what happens is that the first time you call a function you don't actually get the address of the function you get something called fix-up code the
so the fix-up code pointed like the relocation record asks the linker for the real address the linker searches the core the dependencies of the calling object for where it should look for the symbol and then it writes that address back into the relocation record so this this chunk here in green so this is this blue this happens every time you call a foreign function this happens only the first time you call it and then this happens every other time we we've read the record back from the relocation record we jump to the text the function does whatever it needs to do whatever it was written to do the return value is pushed onto the stack and then we jump back to the caller so if we scribble on that relocation record before this first call ever happens the PLT fix-up will never be invoked so we sort of bypassed the normal symbol resolution entirely it's up to us where this goes the linker never resolves a symbol address we have total control over where the function call goes this is sort of the thing that a lot of people writing malware used to redirect your functions thing is they have a rather nice feature in that they need to only they only need to work about maybe one time out of a million and they've got ask us successful attack we need this to work every time you know which kind of makes things harder for us you know they're amateurs basically the this is an important bit at no point did we need to know the signature of the function we were calling the caller puts the arguments onto the stack in the expected way that was handled by the compiler because it had the signatures when the program was compiled the Cawley pulled the arguments off the stack it knew what it was expecting but even though we're diverting the call to a completely different location we didn't actually need to know the details it was taken over just before and just after we did anything so we didn't need to know that which is good because it makes it much easier for us to automate this whole process that's a huge amount less that we need to know we don't need to know the arguments we don't need to know how they are laid out on the stack or we need all we need to know which is guaranteed for us is that they are there on the stack when the call happens so the key question is can we find that relocation record and scribble on it reliably and the answer luckily for me again no need to rely clinker is yes we can the link map entry has a pointer to the elf data for each library and live elf can interrogate this so we've added one dependency Lib elf but I'm pretty sure that's not going to change in compatibly anytime soon so great we can now put these pieces together where you can make a shim library with the target libraries Esso name so that the len code when the linker loads it and we put it on the search path so for example if I'm want to load Lib GL there's no actual Lib GL inside my runtime in some with my containerized application but I can put a shim Lib GL there which will be loaded by the linker and during the init phase of that shim library I can deal ammo from the real library and Ovilus dependencies in reverse order as I've said I can search the alternate library path to do it so I know I'm getting the host one because I want the one that didn't come dependencies that didn't come with my runtime and then I walk the link map I'm making this sentence do a lot of work here I walk the link map and scribble on all the relevant relocation records so that all of the functions in my binary instead of going to the PLT fix up and then finding the location of the stub function my shim library they'll actually go to the real one they don't ever realize they've been diverted because I've I passed the normal symbol lookup code so yeah kind of we can basically do this now obviously I've simplified a lot there are some details and some problems the self shaving yak has not yet been invented so I'm just gonna establish a cup of term terms up front I've called an isolated set of libraries a capsule there may be a better name naming things is hard if you can think of a better name by all means let me know and then I've assumed that they come from file system mounted a slash house so within my chroot or container or a runtime or bubble-wrapped environment or whatever I have mapped in the actual host file system read-only at slash host for example it could actually be anywhere other problems deal open can't be called from inside ADL mo for namespace it causes see library to explode which is not fun for anyone so I need to use this relocation record scribbling mechanism which I've already established to replace the dl open inside a capsule with a wrapper that does the following it cause d LM open instead of dl open it does the path free mapping and symlink resolution relative to slash host or wherever I've told it the extra light the foreign libraries are located and it doesn't accept our TLD global because d LM open doesn't yet have semantics for that I'm discussing that with G Lipsy upstream when we've hashed that out it will but for the time being I need to mask that the reason I need to mask that is because lip GL does the it it DL opens itself our TLD global to expose symbols to its plugins which was fun and games when I figured it out but until I did was a puzzling crash as far as I was concerned what else ok deal sim which is how you get symbols from a DL open library now has to have a split personality needs a little bit of extra work remember we want this all to be seamless but now there are two namespaces not one and that means that when something in our application calls DL sim I need to transparently go up you may have meant this namespace over here and look in there but I only want it to find the symbols of the directly targeted library I don't want to look at any dependencies so I need to be quite careful so again similarly I will scribble on the relocation record for Neilsen and replace it with a wrapper that does these kind of safety checks for me another interesting thing so the usage of Lib GL which was the primary use case varies quite a lot some things linked statically against it some things DL open Lib GL there's a whole bunch of ways you can use it so anytime another library that's not necessarily one that I've exposed is DL opened I need to scribble on its relocation records as well which means that I similarly have to wrap DL o DL open outside the capsule in this you know in the same way so that anytime something is DL opened I scribble on its relocation records so if it needs I for example a Lib GL function it gets the right one extra problems currently each this is kind of a lie in my demo because this is fixed but the patches have not yet been reviewed and accepted upstream so currently each namespace is its own G Lipsy copy which works a lot better than you might think in that it actually works at all but it does make anything to do with threads quite it's not actually a deadlock but it looks like a deadlock deadlock prone Alec and free is interesting of course as long as all of your allocation a freeing of a particular pointer happens on the same side of the capsule boundary you're okay it works it's fine if you pass a pointer to across the boundary and expect it to be freed on the other side and you've got two G ellipses then yeah G Lib C immediately throws a hissy fit and says what are you asking to me to free I've never seen that pointer in my life corruption pull the ripcord bail out immediately we have a disaster on our hands so yeah there is going to be an extra flag called our TLD shared which is currently being reviewed and what that will do is allow me to put sort of the fundamental libraries like the G Lib C cluster have the same instance on both sides it's building on work that's there in India Lipsy anyway because as you can imagine there can only be one copy of the loader so it it's sort of half already does this but it needs to know to do it for gilepsy as well and that's kind of one of the things that I'm feeding upstream so the workaround I had before was to replace the ellic and free clusters inside the capsule which kind of been a hackish sort of way tried to dispatch the pointers to the right place to be freed it doesn't really come up much when using Lib GL because Lib GL doesn't really allow allocation to pass across the boundary it's API but for other libraries it might be relevant right so and finally does it actually work I'm gonna hostage to fortune here I'm going to attempt a live demo so these things have all been tried and worked I'm going to try GL x gears and then if that works I'm going to try open arena so where are we going so this so this is this is a route which has had the Lib GL surgically removed from it and I'm going to try and launch GL x gears you should see some spew from the capsule which is where you can see
it's mapped in the different things and we've got a GL x running and you can see there it says oh dear Levin wrapper cannot pass our tail D global so I know I'm intercepting these calls my points
are gone someone see my pointer others so if we
page up a bit you can see I've got this
is the default namespace you can see I've got some shim libraries maps in here which are stubs they don't contain out any actual code they're just a a placeholder so that the linker can map things in and then I can scribble on the relocation records later and then if I go up a bit there we go we can see there's a capsule there which has host libraries mapped in which is where the real libraries reside and these are isolated from the from GLX gears except where I've redirected them with relocation scribbling so that was
one test now that's a relatively simple
test because it has live GL LinkedIn not static LinkedIn but LinkedIn now s open arena uses SDL which is going to dlm ODL open Lib GL which is kind of a more complicated case because things are going to happen much later if I'm really lucky this will work and I've got about a 50% with us for this demo so let's see if we can get get that percentage up a little higher and oops let's put the
window here twice the window here let me I don't know if I can move the open arena window up damn it put it on the
wrong screen I mean I can just turn this around demos they never work right okay yeah tiny but you should be able to see oppa Rena actually running with I diverted Lib GL okay I think that's kind of the end of the talk and so the code is there it's
all kind of published already this talk as well if you're into that sort of thing is available there any questions I didn't understand then what makes the like your open arena demo only fifty percent reliable oh that way it was kind of unrelated to this it was a pulseaudio not being happy running a natural problem which I eventually tracked down but yeah it's like it's one of those things it breaks and quite often because there's a lot of moving parts so a colleague of mine so this work was originally well and still is sponsored by valve because they want to improve game stability is one of the things and so a colleague of mine Simon Girty is working on the other side of this which is assembling all of the right libraries within the runtime sort of hybridizing your runtime libraries with the bits from the host that you need you need to pick the newest Lipsy of the ones that are available and a couple of other things so he's working on that side of things but those pieces don't exist yet so I kind of had to hack it together my hand which is why it was unreliable there wasn't anything intrinsic to the project so I would like to ask if you have support for at the RT deep link to go to the next symbol in the chain sorry - support for deep linking when you load the symbol from the next library in the chain no I don't think we do it is pretty much just going to this stage so not yet I'm going in a different direction sure hey could you use this to make symbols like soft clinking where you you resolve a function if the library is installed it is library and if it is not installed just returns I don't know and error yeah yeah what happens when this one is actually I didn't demo it because my auto-generated stubs direct everything but yeah if you you can have what would happen there if a symbol wasn't available and you called it you would immediately get a back-trace saying you call the symbol that wasn't available in the target library here's the back trace were you called it from here so yes it's answer your question yeah I don't know to get a back trace I want to you know so for example I have a program and the program links to five different compression libraries sure and then if one of them is not installed I would just want to say no cannot decompress this you know like try a different one or something like that yeah you could do that okay I was wondering if you can speak a little about what valve is planning to
do longer-term with this where they're going okay sure I mean the idea is is I mean pretty much as I laid out at the beginning so you know you need lip GL or in the future you'll need your your Whalen drivers to come Vulcan drivers sorry to come from the actual system because the other thing that's closely tied to your hardware and there's there's no real sane way to put those crammed all the different ones into a runtime you know they would quite like a game that you bought 10 or whatever a number of years ago to keep running even though you've your on a completely different system so the idea is that the game will come tied to a runtime a version like pretty much similar to wear things like flat pack work you know you've got your application in the runtime you'll have a game and it will specify I want this flavor of runtime at this version but then you need to parachute in the most recent Lipsy out of the two that's available and need to parachute in the graphics drive the stuff that's tied to the hardware needs to be brought in from the host and so that's what it's going to be for and the idea is that when my colleagues work is done all of that will sort of be assembled into your sort of hybrid runtime so hopefully we'll the idea is that in the glorious flying car future we also improved game stability a lot right big if you have an old console you can plug a game in that you brought X number of years ago and it will work now they have the advantage that it's all baked into the hardware so they don't really have to deal with things changing but they would really like that to be the case because what their thing is that they want to sell you stuff electronically and they want you to have the confidence that whatever you paid for it goes on working so that's the goal here sounds great is there an ETA they're there they're sort of is but it's tied to other things that I can't talk about sir okay thank you so what if there is a different system that uses the same Mac for example a heap profiler that wants to replace all Melek sand in your entire program what if there is the I don't I don't know what would happen it might work it would depend on the order in which things happened is that was that we've saying would they clash or yeah I mean can those interact is there a way to make it work I I don't know the answer to that I'd have to try it I wonder if you could talk a little about the surface area of GL that's relevant to change because I'm in my memory there's like a couple of API is for you put a uniform in and then you like honk ajayan string of program text and that goes across a terrifying barrier into the drivers okay and then like but that is like a function that says int and like gives you a void star and stuff and so it doesn't really know versioning to be honest I don't actually know that much about how GL work I mean there was a possibility that I'd need to learn it to do this but I kind of got to the point where it worked and I just stopped looking because as you may know there's there's a whole number of different ways of using lib GL there's the simple GL acts gears approach of just linking against it there's the STL approach of DL opening it and finding the functions there's the one where you get a specific function from GL and then you ask it for the function pointers that you want to call but they all sort of work with this approach because it's only the small set of well-known functions that people can call that need to be proxied in this way and everything else just kind of works amazingly enough okay for the dynamically linked GL x gears can you just show what it looked like when you run through LD d oh yeah sure see ya
were stopped that's right don't seem to be able to get focused back where's he gone oh right sorry
giant screen I mean quit that may be out
there we go so you can see Lib GL is going to a Shem if we looked in that Shem we would just find stub functions with no actual code in them well there's a little bit of code this code that generates a back trace if you ever actually manage to call those functions so the the search path the Shem search path you added to the containers LD conf yep oh okay yes so let me know it's gonna work from here
to there oh isn't it oh okay I'll make make it public like you're the first person who actually asks oh yeah okay any other questions okay I saw that
you had so why then here is uh so you
see like the fit the fit to last line there why is ldd saying that GLX skiers which you didn't have to recompile or anything why is it dependent on lib capsule now let's count from the shim shim depends on lib capsule if capsule is kind of rather than making the code into every single shim they link again slip capsule which does that okay that seems three old questions so were there it is segregated dynamic linking [Applause]