An Overview of Aquilon
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 199 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/32682 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSDEM 20142 / 199
2
3
10
11
12
13
14
17
18
19
23
24
25
27
29
45
46
47
48
49
50
51
52
55
56
57
58
65
67
68
70
72
74
75
76
77
78
79
81
82
84
86
87
89
90
93
94
100
101
102
103
104
106
107
109
110
111
112
113
114
115
119
122
123
124
126
127
128
129
137
139
141
143
145
147
150
154
155
158
161
163
164
165
166
168
171
174
175
176
179
182
183
185
188
190
191
192
194
195
196
00:00
MathematicsAreaPresentation of a groupGoodness of fitTheoryFood energyData structureEvoluteMultiplication signView (database)CASE <Informatik>DemonElectric generatorProduct (business)Physical systemConfiguration spaceWeb 2.0WebsiteProjective planeProper mapMereologySoftware developerRule of inferenceMachine codeSystem administratorCuboidSoftware frameworkService-oriented architectureBitCodeConfiguration managementArchitectureInteractive televisionRelational databaseClient (computing)Lattice (order)Right angleStaff (military)Mobile appRevision controlServer (computing)Scripting languageSource codeDatabaseInterface (computing)FraktalgeometriePerspective (visual)InformationObject (grammar)NeuroinformatikComputer hardwareElectronic mailing listSupercomputerIntegrated development environmentPlanningSet (mathematics)GradientBranch (computer science)Multitier architectureCommunications protocolOnline helpSimilarity (geometry)Compilation albumExpert systemXMLUML
08:36
CodeMereologyScripting language2 (number)Online helpWeb pageType theoryOnline-AlgorithmusDemo (music)Service-oriented architectureBusiness modelKnowledge-based configurationConfiguration spaceBranch (computer science)Interface (computing)Power (physics)Absolute valueSinc functionProduct (business)Source codeData structure1 (number)Server (computing)Multiplication signUniform resource locatorForcing (mathematics)NumberRootINTEGRALRevision controlSoftwareAxiom of choiceDifferent (Kate Ryan album)MathematicsPhysical systemRule of inferenceVirtual machineObject (grammar)Web 2.0Integrated development environmentCompilerDemosceneBlock (periodic table)Self-organizationInstance (computer science)TrailMappingNeuroinformatikMixed realityUniverse (mathematics)BitSummierbarkeitTheoryInformationXMLUML
17:04
Set (mathematics)CASE <Informatik>View (database)QuicksortType theoryOnline helpCodeBusiness modelXMLUMLLecture/Conference
18:54
Business modelRight angleParameter (computer programming)CASE <Informatik>Web 2.0BuildingQuicksortData storage deviceServer (computing)Online helpCompilerTerm (mathematics)Asynchronous Transfer ModeWeb pageInformationSoftwareSuite (music)MathematicsMereologyVirtual machineDemonReal numberDirect numerical simulationType theoryPower (physics)Social classPhysical systemSource codeXML
22:34
Electric generatorView (database)Virtual machineFocus (optics)Computer animationLecture/ConferenceMeeting/Interview
Transcript: English(auto-generated)
00:13
Hello everyone. This is a follow-up to the previous talk, so if you weren't in the room for it, I'm not going to have room for you here.
00:21
So, I'm just going to tell you a little bit about me, so you can give some friendly information and some background information. I'll give you a complicated and basic design of how to use it for things, and then we'll wrap up. So, I work at the Science and Technology Research Council in the UK. I've decided to be in the computing department, and there's a high group
00:49
of computing, high performance computing, a couple of supercomputers in the top 500. There were three that we dropped off at the top 500. I've personally spent about seven years working on it.
01:02
So, I started working on the hardware panels, working out the middleware, and now I'm the designer for the installation system. So, that's the first of my perspective. I realise this is an empty room, so I'll try and get in the way to see as I go along.
01:24
So, this is told you about quite often, and what we've had so far. So, it originated from a good work. It originated from a good work in that it's a project. Can I speak up? Sure. I'll do my best.
01:40
Thanks. I'll just talk louder and louder and louder until you tell me not to. So, most of the usual kind of work for Fractal was done at CERN. They came up with an original configuration management system called CDB. As far as I can work out, C stands for CBS, because that's what the configuration code was storing.
02:01
It was integrated with some of the production systems at CERN for doing inventory and deployment. I tried to get used by a community, quickly abandoned it, and replaced it with something called SCDB, but it was cancelled as a version. And there are no entry system packages at all, it's just some anti-analysis of the entry to our stuff.
02:26
And some of the infrastructure that we used to make sure that both of these had scaling issues. So, CERN have recently moved away and have switched to Cupit. And some of their work throws involved, I believe, something like a two-hour wait for it to deploy for a change to go out.
02:42
I think the recent experience was first-hand at some point. Something like that. Quite honestly, if you're going to be doing that for 45 minutes, you should be wrong. Yeah, something really terrible. SCDB is what we're running in production at grade tier 1 in the Eberton Laboratory. We've got about 2,500 hosts in it now, and if we want to make things available in about 25 minutes, which is starting to be an outstanding issue for us.
03:06
Both of those systems have similar protocols of code, compile, commit, deploy, delete, or, in some people's cases, you code, compile, it doesn't compile. Code, compile, it doesn't compile. Go around that step until someone goes to fix a problem for you.
03:20
As I said, neither of them was much more than an environment for writing a plan and coding and pushing it out to hosts. For some, there are guidelines in the community, particularly the QWG template framework, which was an effort to try and share configuration. It still is an ongoing effort to project configuration between both sites in the area. It's a really great way of getting at starting and running, but the lack of any rules or
03:45
enforced rules for structure and configuration needs to do everyone in the site doing things in a different way. Every site does it differently, and admins within a site do it differently. Of course, there's more than one way of doing it right, but what you don't want is one codebase with seven different right ways of doing something.
04:01
What you want is everyone following the same coding guidelines. Coders understand work in the coding guidelines. Sysadmins, at large, and my experience, do not. It finds it in the works and goes on. This gets really, really messy. If you start deploying lots of, thousands of hosts, you're not going to want to write a code for each host individually.
04:23
Even if it is, this is a host that does this, it has this idea, and that's all. It gets really boring. So what's coming next is every single site designed its own imagery database and some scripts to write a code for that. Again, we ended up with a fragmented community. Now, these are not huge problems.
04:43
This is still a very powerful system, and everyone was still, and still is, mostly using this. And it's good enough. It's great. It does the job. It's posted. But, in 2007, the bank, Morgan Stanley, which is Louis and Louis too, turned up one of the community meetings and announced they wanted to deploy 20,000 hosts.
05:06
Everyone reeled and hollered at this because we knew our systems wouldn't scare them. And then they added lots of more requirements on top of this. Someone right there. Including giving it to people who aren't even sysapp'ing and step on my support staff. As in, they sit on the telephone at the help desk somewhere, and someone
05:22
tells them there's a problem, and they need to make a complex configuration thing. Everything pampered is not ready to do this. You need experts involved to do that. Obviously, all of us have been wanting to launch the configuration using some version of CVS.
05:41
Launch it, and you might not be able to merge it back ever again. And if you're going to start spreading it around hundreds of people, you're going to need to provide and enforce a structure for the configuration. So we need to provide something entirely new, which is where I've broken. So CDB was the first generation, and SCB was the second generation.
06:00
This makes Apple one of the third generation configuration management database for Crackle. So it is part of Crackle, which is not a set within two Crackle. It is a sub-component. It does incorporate many ideas from both CDB and SCB. It re-implements some of the things that SCB dropped from CDB. It also incorporates things from Morgan Stanley's previous configuration management system that they designed in the 90s.
06:24
Which was not designed for Linux at all. It was designed for a few hundred Unix boxes. And luckily for the community, Morgan Stanley did most of the framework work. Which has been great. They've released everything under the Apache 2 license, and it's now part of the project as well.
06:43
The first impressions we got were, Git. Wonderful. Git is everything we always wanted. Proper branching, proper merging. We can do development without breaking our production systems. Which is great. Also, we can give it to new people, and they can do their own learning without breaking our production systems.
07:02
The next thing we discovered was that there's no longer some code in compiler, in a web server, and it does push out systems. There's now this thing called Broker, whatever that is. It's a demon that owns parts of the configuration and writes code for you. Which is terrifying the first time you see it. And there's a command line.
07:21
We've documented commands in man pages. This was shocking. The architecture is the little box that you saw on the list of guys around, which is actually quite simple. Especially when you can see it without the rest of the mess around it. The two powerful things in the top are the user-facing.
07:40
So Git on the right. The AQ client. The left. That is your entire interaction with the system. It won't go into creative gear. You ask the both to do things using the client. So what this ends up is called ATDB, which is a relational database of your choice. There's only two of these four. It triggers the app to compile.
08:01
It triggers pan. It does all the build stuff. It builds up the web servers before. But now you actually have an interface for dealing with it. So this makes things really powerful. The Broker really is the source of all power. It provides the work code that you want to see. It writes you a pan code for your objects and relationships. I'll go on to the work that's in there.
08:22
Git codes are different, please. You ask it to do things on your behalf. So, for example, if you want to do some work on branch, you ask the Broker to create you a branch. That will then get turned out into a thing called a sandbox, where you can work on it. You can then attach your hosts, or some of your hosts, just to testing, to your sandbox.
08:41
So you can now work in a completely isolated environment with the production systems. You can throw that away at the end of the work. The Broker itself, if you don't like them, is used throughout the most force of power. As I said, you have a choice of your interface system, but we're only supported as an object, because those are the only ones that make sense.
09:02
And it has a kind of REST-ish API, which is used if you want to create things. And most of the time, we recommend using the online algorithm. So I talked about sandboxing, but they're really, really easy to straighten your way. They just give you a branch that you ask the Broker to do it for.
09:21
So you use the AP command. So, for example, I create a sandbox called new awesomeness, because I always implement new awesomeness. And it gets my code by the time it's set up for you. You can just see the interface start working. When you're finished, or when you want someone else to look around, you can come in and get an upstream code.
09:41
Because we're in here. We want code people. We are sysadmins. We want people to tell us what we've done as well. Yes, absolutely. Then, when everyone's happy with it, we'll merge it back into production. Now, you'll notice on the source here it has my name before the production merge.
10:00
This is because, really, someone else should be merging your sandbox back into production, not you. Once someone's happy with it, they'll immediately merge the sandbox into production. So, in this case, I'm merging my own back into production, because I've got everything that works for me. I meant to go a bit slow.
10:21
So, structure has always been about modeling your infrastructure and your configuration in code. Luis alluded to the fact that sometimes this gets messy and unstructured. So, Acron actually provides parts of your infrastructure as objects. These are then stored in machine-based scripts. You can change the relationships between them,
10:41
but things like buildings, rooms, rags, desks, machines, and such are good to hit with. These can all be defined using a cloud-like mix or together using a grime. Broker then writes the actual pando for you. You never see it. It's hidden behind the scenes. It's just made with compiler and compiler code, and everything happens as if by magic.
11:02
It also implements a few structures for Acron configuration. So, features, which are just blocks of pando. In fact, as the Broker sees them, the other one is something that you write. And then it has to be self-contained. You can include other things, and it is what you would expect when you're writing a pando. Personalities is what a machine has.
11:21
So, one machine has one personality. And it's just a collection of features. So, you have a search server, which requires DNS, HTTPD, PHP, perhaps, three different features, make one personality a web server, which is what we say is a web server. But that's personal to our site.
11:40
And a host is just a machine, plus its host name, plus its IP address, a personality, and what operating system it writes. So, the whole thing is still designed to be OS version agnostic. So, we run a scary number of different versions, pointly, since they're assigned to be Linux in production, with the same configuration tree, because most things need to be abstracted or abstracted away,
12:01
things that don't, aren't. The command line provides an add and delete and update command for these objects. And they're the huge help page that I'm going to show you, unless we do a demo after this talk. The next part that you can model is services, which is just the concept of a service, the fact that you have servers and clients,
12:20
and you want to keep track of them, you want to link them to each other. And the real power comes from these things, the service maps. So, you might have, for example, multiple services at the same time, but different types of services are connected. So, you can define rules based on which ones match, which services are different, convoluted, and slightly out of sight in a second.
12:43
But at the moment, you can create those rules based on what organization it is, what physical location it has, and what network it is. When we say network IP address, what we mean is the network root IP. So, whatever it is, that's what we want, for example. So, you can allocate a service to a server.
13:02
So, for example, you have two clusters. Those have two different types of computers. They each have an NFS server and have a service software cluster, which has exactly the same personality because they're just an NFS server. They each have a separate server. What you could do is define an NFS service with an instance reach cluster.
13:20
It's that simple. Add a server to each instance. Again, this assumes the server's already set up in the configuration system. Add the requirement for NFS to both of those who continue their personalities. So, that is to tell AquaLion that if you have a GPU cluster node or a ZM5 cluster node,
13:43
they both require NFS. As in, you can't build a NFS. This is not a valid configuration. And then we say, to get those two different servers, those instances use the rules for the integration.
14:01
So, if it's in one network, if it's in one server, and at this point,
15:40
as far as the community is concerned, they aren't able to get off. It integrates with the community.
16:27
This host is an app, and you can make that change.
17:49
Even if it was off the open overlook and is strong-willed enough.
18:31
Just to be honest, the hand code is still there, but the sort of help desk type user
18:43
you're talking about, I don't know if she sees that. They're just configuring the sort of the model out. Yes. I can't do any various things.
19:12
Each one of those has a piece of help page. So, I'm just going to see if you can find it.
19:32
So, these are all the things that the public web server has made out of. So, the personalities is an external web server, and it has a web server demon mine, and that's it.
19:41
Because it's part of our site, it also gets all these various things like DNS and LDP, and so on and so forth. So, every single person we have gets that. So, everything about it, but it's not actually a real machine, and it's in a fake building. So, it's currently a web server,
20:01
and that's made to be analyzed in search mode. This is the sort of thing that a help desk, or even a user could have,
20:27
So, the back door, they didn't ask you to break things.
20:45
I know it's an outcome in the last session, so I should probably undo that before anyone notices it. That means that the web server will probably get out of the store for a minute, and I'll ask the server to start something. So, for a help desk user, they can do that for excuses, probably pretty quickly.
21:01
Really, they shouldn't be changing the personality of the server. So, where you've got to, is it still breaks things, but not because human reasons like syntax errors? Because the compiler sits there and verifies where you're profiling, up front. You have that information before you've gone, broken everything. So, instead of, the way I think,
21:20
instead of moving class to break things, you're moving class to not break things. So, it's got to be better than that. Does it support something like roles? So, you can say you're a help desk user? Yes, yes. So, we don't have any roles in that because we all power users, and we trust them in one way. So, the roles are based on what commands
21:41
people can run. So, you might not want anyone to be able to make network plans, for example. So, you might be networking to have just that. You might not want people to be able to change personalities if they're in a certain subkey, or if they're in support. But you might want them to be able to add features and remove features to this system. Except you can find what the roles are.
22:01
At the moment, they're coded in Python, so you have to try to log them in and change the roles, but I believe that you haven't put in some kind of suit. They're very much a smooth and scary class, because it's a big team of developers, keeping it going. The weird team, are requesting for them.
22:25
Let's go to the next one. I know people have learned machines. People just treat machines as any type of managed machines. In terms of using it to trigger the generation of VMs, no, I don't think there has been any work done. I know that more than standing out
22:41
in the arena has broken out of it. At the moment, what they've done though is focus purely on things like VMware. So one of the concepts it has built in is the idea of a cluster of VMware machines, machines around the machine. And there's another view
23:01
that's alongside that.