Cloudlets: universal server images for the cloud
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 97 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/45686 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSDEM 201089 / 97
1
2
4
5
6
8
14
15
16
17
23
29
38
41
42
44
46
47
48
50
53
54
62
63
64
65
66
71
74
75
77
78
79
80
82
84
85
94
00:00
Server (computing)Computer-generated imagerySolomon (pianist)Data conversionSolomon (pianist)TwitterComputer animationXML
00:33
Point cloudInstallation artSoftware repositoryCodeModal logicWeb pageConfiguration spacePhysical systemComputer-generated imageryMiniDiscBootingVirtual machineWordBlogMathematicsCodeVirtualizationOpen sourceDynamical systemBootingOpen setMiniDiscComputer configurationMereologyCartesian coordinate systemPoint cloudRaw image formatMobile appAdditionConfiguration spaceOperator (mathematics)Proxy serverWritingFile formatServer (computing)CloningJust-in-Time-CompilerOvalPhysical systemSoftware developerSoftware development kitDifferent (Kate Ryan album)Computer animation
03:43
Manufacturing execution systemEntire functionRevision controlComputer-generated imageryConfiguration spaceMobile appServer (computing)MathematicsInterface (computing)Web 2.0CodeComputer animation
04:06
Normed vector spaceLink (knot theory)Virtual realityMathematical optimizationRule of inferenceComputer configurationConfiguration spaceLatent heatMathematicsComputer fileModule (mathematics)Computer animation
04:34
Software repositoryConfiguration spaceCodeTwitterLattice (order)Configuration spaceServer (computing)Block (periodic table)Focus (optics)Computer fileLattice (order)Multiplication signMatching (graph theory)Stack (abstract data type)Open setMereologyInformationCodeBinary codePoint cloudComputer hardwareResultantLevel (video gaming)Different (Kate Ryan album)File systemBootingComputer configurationNormal (geometry)MetadataOpen sourceLogicData storage deviceData conversionRaw image formatBitState of matterRight angleEquivalence relationCombinational logicRepository (publishing)Term (mathematics)Arithmetic progressionScripting languageEntire functionVirtual machineMathematicsTwitterPhysicalismPhysical systemFile formatEmailSoftware developerNeuroinformatikProxy serverNetwork topologyView (database)Hidden Markov modelLine (geometry)Computer animation
11:49
XML
Transcript: English(auto-generated)
00:07
Alright, thank you. Everybody hear me alright? So I'm Solomon Hykes, and let's talk about Cloudlit. Before I begin, let me just say this is a conversation. I'm here to talk about a problem that we're trying to solve.
00:22
It's not a problem that has been solved, so if you relate to the problem at all, if you have anything to contribute, please join the conversation. On IRC, Twitter, come see us at the bar, whatever you like. So the problem is sharing. Specifically, how do I share my code when I want it to run in the cloud?
00:43
So of course the cloud is just a fancy word to say anything server-side. So I'm writing an application, I want it to run on people's servers, how do I share that efficiently? So of course, as open source developers, we know how to share, we know how to share our code, we know how to collaborate with people
01:02
who want to contribute to our code, and we know how to package our code so that people can install it on their favorite OS. However, if we're writing applications for the cloud, in addition to all that, you also need to provide configurations. Typically, as an example, let's say I'm writing Apache,
01:24
people want to see the Apache code, other people want to install Apache on their systems, but there's a lot of interest in a great working configuration of Apache and all the necessary stuff to run Python apps or Ruby apps, a great stack that just works.
01:42
And if I want to share that, then that's when things get complicated. I can see three options here. Either I write a tutorial, write in a blog post, and people comment, edit it, or I package a virtual machine and put it up for people to download, or I use dynamic configuration engines, things like Chef, Puppet, etc.
02:05
We think that all three of these approaches have drawbacks, and the summary would be that we're good at sharing code, good at sharing packages, but we're still pretty bad at sharing configurations that just work. So what we tried to do is improve that situation
02:22
and provide a format to package a configuration and share it. So a Cloudlet is an image that you can use like a virtual machine. Basically, that means it's self-contained. You don't reference anything else. If I give you an image in the cloud that's format, you have everything you need to boot it,
02:41
and you have everything you need to boot it anywhere. From a Cloudlet image, you can convert that into a Xen image, a VMware image. You can get a raw disk image and then boot it on a physical machine. We've done it with open VZ containers, things like that. So the interesting part compared to virtual machines,
03:03
I don't know if everybody has ever tried to share a VMware virtual machine, something like that, or a Xen image, with people who don't actually use that virtualization technique. That's really tricky. And what we really like is the tools that everybody here, I think, knows, something like JIT, Mercurial, SVN.
03:23
I can clone someone's code. I can change it. I can commit the changes. I can merge. I can see the differences, all these cool operations that we're used to. We'd like to be able to do that on images as well, and that's basically what the format is made for. You can take someone else's Cloudlet image, fork it,
03:40
make changes, and contribute the changes back, things like that. Now, that's a quick example. If anybody here uses Mercurial, you'll recognize the actual Mercurial server interface. This is not our web code. This is an image that a friend of ours made. It's the bare WSTI server, so it's an Apache configuration to serve Python apps.
04:02
You can see the complete history of the image, and you can inspect a specific change. Let's say, oh, here I see changes in the Apache configuration, things like that. Here, this is a change set where, obviously, as the comment says, he installed the module, and you can see all the files that changed.
04:21
The interesting thing is we didn't replicate the work of a Mercurial. We use it. We take the image and put it in Mercurial. You can actually do ht pull the image. So that's the workflow we envision. The same way we say, here's the repository for my code,
04:42
you can say, here's the repository for my image. Do whatever you want with it. And I think that's a really interesting way of seeing things, because then you can start building on top of other people's work, just like you did with code. And I think that might accelerate the progress we make
05:03
in not just writing one piece of code here, another one there, but actually plugging them together and sharing entire cloud stacks. So I think that's as far as I'm going to go in terms of lecturing. I'm not going in any details on how it works.
05:21
I was worried I wouldn't have enough time. But I would love to either answer your questions on email, Twitter, IRC, whatever you'd like. Join us at the bar, like I said. We're going to go there right after this talk, and we'll be there again at five. Who here would like to see a demonstration later,
05:41
just to see it happen? All right, so either right after this at the bar or at five. Is that right? And I think I have time for a few questions.
06:00
Yes? Binaries and everything. Oh, I'm sorry. The question was, do you actually store the entire image in the repository? The answer is yes, we store everything. However, we don't work at the block storage level like VMware or Xen, for example. We don't store that. We work at the file system level.
06:21
So basically, we take a file system tree, and we say, OK, I'd love to put that in a repository, but what should it take out? So we add metadata so that the image is smart enough to say, if this file changes, you want to record that change. If this changes, you're not really interested in that.
06:41
Things like that. But yes. Any other question? All right. Well, this is it. Thank you. I guess I'm ahead of time. Yeah, right.
07:08
So the question was, you save the file system, but do you save any hardware information? The answer is no. We think in terms of moving from one cloud to the other, let's say you have a Xen image running,
07:21
and you want to share the behavior of that Xen image with someone who's using VMware. If any hardware information you share with him will be useless. So the interesting part is to look at all the information, all the bits you have in that image, and only take what's worth moving.
07:49
The question is, can you actually do that? Can you actually take information from a VMware image, move that to a physical machine, and is it enough? Will it work?
08:01
The answer is yes. So the way it works is you start from the file system. Let's say you don't have a cloud image. You have a normal VMware image, and you want to use cloudlets to move that. The way you do it is you start with the raw file system, and then you have to add the metadata the first time. So using the cloudlets format,
08:22
you'll add information saying, this is persistent, this is volatile, this is a template, things like that. And then once you've done that, then you can use the metadata to move the image. So that means there's an initial step of authoring, adding manual information,
08:40
and we've worked really hard to make that as simple and short as possible. So there's no giant XML file, no scripts to write, anything like that. Typically, the metadata is a 10-line JSON file. No, you don't have to describe the services.
09:01
You don't have to give any knowledge of how the code works inside, or how to construct the image, or what are the relationships inside the image. You just focus on the file system, on the final result. Because whatever the configuration logic is, the end result is always a certain file system.
09:22
So that's the big difference with Puppet or Chef, for example. We don't describe how to get to a certain behavior. We let you do that any way you want, and once it's done, we let you snapshot that. And it's actually a great combination with tools like Puppet or Chef, because who here uses something like Puppet, Chef, CF Engine,
09:43
or the equivalent homemade scripts? Typically, you describe steps to take from one state to the final step you want, but you still have the problem of the initial state. A Puppet script describes how to change a base image, but how do you get that base image in the first place?
10:02
So the typical scenario is, here's a base image in the Cloudlet format, and then here's a Puppet or Chef script to make it change its behavior dynamically once it's booted. So we do before boot, Chef Puppet, CF Engine do after boot.
10:20
Great match. Oh, yeah, the question is, does it only work on public Merkle repositories? I'm guessing you mean things like Bitbucket or open source?
10:48
Of course, yeah. Obviously, here, the focus is on sharing for open source developers, but the technology works for anybody who wants to share images with anyone. It's the same workflow options you have
11:01
as with Merkle in general. Sorry. Not on this computer, but yeah, I'd love to... This is basically why I'm here speaking, is I would like to interact with people
11:23
who are interested in this problem. So this is really just to get the conversation started. Any other question? Okay, thank you very much.
11:41
Thank you, too. Thanks.