Doomed are the dinosaurs!
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Title of Series | ||
| Number of Parts | 490 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/47462 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSDEM 2020242 / 490
4
7
9
10
14
15
16
25
26
29
31
33
34
35
37
40
41
42
43
45
46
47
50
51
52
53
54
58
60
64
65
66
67
70
71
72
74
75
76
77
78
82
83
84
86
89
90
93
94
95
96
98
100
101
105
106
109
110
116
118
123
124
130
135
137
141
142
144
146
151
154
157
159
164
166
167
169
172
174
178
182
184
185
186
187
189
190
191
192
193
194
195
200
202
203
204
205
206
207
208
211
212
214
218
222
225
228
230
232
233
235
236
240
242
244
249
250
251
253
254
258
261
262
266
267
268
271
273
274
275
278
280
281
282
283
284
285
286
288
289
290
291
293
295
296
297
298
301
302
303
305
306
307
310
311
315
317
318
319
328
333
350
353
354
356
359
360
361
370
372
373
374
375
379
380
381
383
385
386
387
388
391
393
394
395
397
398
399
401
409
410
411
414
420
421
422
423
424
425
427
429
430
434
438
439
444
449
450
454
457
458
459
460
461
464
465
466
468
469
470
471
472
480
484
486
487
489
490
00:00
Open sourceVideo gameForceOperator (mathematics)InformationOpen sourceNatural numberProcess (computing)MereologyReal numberCASE <Informatik>Multiplication signSlide ruleBasis <Mathematik>Computer animation
02:53
Latent heatMachine visionComputer networkData managementConfiguration spaceStack (abstract data type)Open setSoftware testingBroadcast programmingOpen sourceComponent-based software engineeringCodeSoftware developerNatural numberBitDigital rights managementGame theoryGame controllerPoint cloudTowerBuildingPower (physics)Real numberVideoconferencingHypermediaDomain nameCASE <Informatik>1 (number)MereologyOpen sourcePhase transitionSoftware testingSimilarity (geometry)Flow separationSelf-organizationIdeal (ethics)Universe (mathematics)Process (computing)Projective planeWeb serviceFreewareOperator (mathematics)Feasibility studyRevision controlConnectivity (graph theory)Data managementAnalytic setCodeStaff (military)Scheduling (computing)Series (mathematics)SoftwareRegular graphForcing (mathematics)ImplementationComputer programmingForm (programming)Message passingKey (cryptography)Instance (computer science)ResultantGateway (telecommunications)MicrocontrollerProfil (magazine)Virtual machineProper mapVideo projectorSystem callMultiplication signSet theoryComputer animation
12:06
NeuroinformatikLine (geometry)CuboidDifferent (Kate Ryan album)Data managementComputer architectureRevision controlGroup actionForcing (mathematics)Open sourceCivil engineeringSelf-organizationDiagramBitCartesian coordinate systemServer (computing)Projective planeMereologyGame controllerSoftware testingLattice (order)Entire functionSpectrum (functional analysis)Bit rateAxiom of choiceSoftwareProcess (computing)Greatest elementBuildingOnline chatConfiguration spaceContent (media)MiniDiscComputer animation
15:46
Curve fittingNeuroinformatikService (economics)Microcontroller1 (number)Latent heatHypermediaScripting languageMereologyForm factor (electronics)Content (media)Data managementVideo gameSoftwareInternet service providerFirmwareGame theoryDampingConfiguration spaceVideo projectorCuboidServer (computing)Gateway (telecommunications)Open setProcess (computing)Point (geometry)BootingTemplate (C++)Core dumpAddress spaceTouch typingGroup actionScheduling (computing)Module (mathematics)BuildingVirtual machineTowerEntire functionInformation securityComputing platformFlow separationInteractive televisionResultantVirtual realityThomas BayesDivisorAnnihilator (ring theory)MappingPoint cloudBitGoogolSoftware testingInternetworkingComputer animation
21:32
Game theoryAxonometric projectionOpen sourceAxiom of choiceService (economics)Integrated development environmentCodeProjective planeGame controllerNumberCodeOnline chatRevision controlService (economics)SequelIntegrated development environmentWindowAxiom of choiceScheduling (computing)Theory of relativityData managementProgramming paradigmBitPresentation of a groupSelf-organizationOpen sourceCASE <Informatik>Range (statistics)Entire functionConfiguration spaceBootingComputer animation
24:38
Point cloudFacebookOpen source
Transcript: English(auto-generated)
00:05
All right, we're about to get started. The next session is by Dennis. He's from the Naturalis Museum in Leiden, is Leiden? Yeah, in Leiden, and he will talk to us about the infrastructure they've built there.
00:22
So, enjoy. Yes, hello everyone. This talk is about something entirely different, about dinosaurs. And let's start with an introduction. My name is Tricks, which is an actual T. rex
00:40
that lived about 66 million years ago. And she's now one of the main attractions in our Natural History Museum in Leiden. Also introducing myself a bit. I'm living and working in Leiden as well.
01:01
I put it like, I identify as a Homo sapiens, a meager 38 years of age. And I moved to Leiden to study political science, and I ended up as a member of the IT crowd at Naturalis. Couple of them are sitting over there as well.
01:21
In my free time, I work as a volunteer in a social center called Freiplatz Leiden. So if you happen to be in Leiden, please come and visit us, or contact me if you want to organize an open source event, a hackathon or something. And talking about open source,
01:40
I'm an open source enthusiast ever since installing Debian for the first time in 2004 already. And I've been coming to FOSLAM ever since 2012, so this is my ninth edition with my first talk here, so that's quite nice. And it's always been really inspiring,
02:03
overwhelming sometimes, and for sure exhausting experience, two days of talks and talks and information. And as a result, the last couple of years, we've applied all kinds of tools and practices at Naturalis, inspired by talks at FOSLAM.
02:25
And I think that's also a big part of the job as IT operator is to actually try and to determine the right approach, the right tools, and how to use them best for your job.
02:43
And there's really a lot, like because of the 800 talks or something here at FOSLAM. So in this talk, I want to present a kind of a real world use case.
03:02
I skimmed through the program, and a lot of developers are talking about what they made for me, for example, as an operator. But this is the other perspective, and I hope it's useful. And I like to give you a glimpse of the way we dealt
03:22
with the challenge of applying all those wonderful, powerful open source tools to a domain that has been up until recently really fixated on proprietary solutions.
03:43
And the goal of this talk is not to try and paint as if we have the ideals universal solution or something. It's just, I hope it's interesting for you guys and girls to see how we managed with the problems
04:01
of building a museum. And we're actually quite proud, of course, of what we achieved with our implementation, but it's really far from perfect. And I want to basically, after explaining a bit about the use case, tell a bit more about the circumstances
04:24
we had to work in, the approach we took, the things we achieved, the end result, and at the end, I'll have some closing comments. So what was actually the use case?
04:42
We were asked to deploy and manage an entirely new natural history museum, consisting of 10 exhibitions and experiences with all kinds of technologies like media players, projectors, microcontrollers, interactives, all that kind of stuff.
05:04
A campus network, because it was a new building, and all the management tools around that. And to get a bit of an idea about the circumstances,
05:20
a bit about our institute, Naturales Biodiversity Center. First and foremost, we're the manager of the natural history collection. We have 40 million specimens stored in a big tower and also a new part of the building
05:42
with lots of old artifacts, small insects, so all the elephants, anything you can think of. Apart from that, we are a research institute as well. So we have at least 100 researchers doing all kinds of research related to biodiversity.
06:03
And we're a natural history museum, which is really popular with families and kids. But that poses a really fundamental challenge for us as a support organization, because basically the biodiversity in our institute is kind of the central theme.
06:21
And basically anything people can think of, what they can do related to biodiversity, they do. So and we have to support that. So building a museum, having a cloud for researchers doing their analysis, all that kind of stuff. So that is difficult to do everything really well.
06:43
Starting the project and start of the project, we already had quite a bit of technical expertise in house, so we have like an IT department of 30, 35 people with operators, developers, support.
07:01
And the operators were relatively well-furred. In conflict management already, we used Puppet and Foreman specifically for deploying web services. One of the infrastructure based on OpenStack and SEV. We've done some experiments, so to say, on Kubernetes.
07:25
We also canceled these. And for example, we've done analytics based on Senzu, the Algstech, Grafana. So that's also not normal for a regular museum,
07:41
doesn't have this IT staff in the Netherlands at least. As I mentioned, we built a new museum and what you can see here is the museum actually being constructed. The part on the left, that's the new part.
08:04
And that's also the museum part. It's completely new. And when actually running the project, this wasn't ready. So we had to start building stuff when we couldn't actually access the building. Or when we, it wasn't even completely finished
08:23
when we started building stuff in the building. And apart from that, we had to work together. Oh, we worked together with an internal museum department which was really used to working with suppliers
08:42
that just were fixated on proprietary solutions, media players, all kinds of show controllers. And in general, the museum building industry, if you can call it that, is, well, I'm a positive now, it's just starting to be influenced
09:01
by best practices from IT and DevOps. I put there cattle versus pets, it's like a big question mark for them and they didn't even consider it. It's like everything in the museum is considered like this special thing
09:20
and they basically are used to just making that thing. And then, well, here you have it and you maintain it or something. So also in a museum, you have to deal with a broad set of technologies. As I mentioned, audio and video players,
09:42
unity games in this instance, show controllers, KNX, gateways, microcontroller, all kinds of stuff. And of course, with a tight schedule, also known as no proper time for testing.
10:01
We used to have a testing phase, but it got squashed and basically we didn't have any testing phase. So in similar situations, in other museums, basically, a museum would hire an external company or several companies who would then build
10:22
and deliver a infrastructure. If you're lucky, according, you could set requirements as an internal support organization and then if you're actually really lucky, they would deliver something that integrates with the other things you do.
10:42
So in an effort to keep the diversity of technologies that we have to manage, to keep that down and limited, our approach was to actually build on the existing infrastructure and know how within the organization and to get involved really early in the process.
11:03
We had a bit of a struggle to get that message across internally. So there was quite a bit of politics involved, but we just hold on to our ideas.
11:21
And as a publicly funded institute, I think that's also really important. And just believing in the power of free and open source software, our aim was to use as much open source components as feasible.
11:41
And to combine this with a infra as code and DevOps practice. So ideally, our idea was that basically every variable that actually determines the workings of the museum, we would have under control, under version control and have it managed.
12:01
That was our ideal, to make everything like deployments repeatable. So quite early in the process, we made this kind of architecture diagram to make a bit of an overview. I'll go over it quickly.
12:20
Basically, the issue was that on the top, you see different groups. These are not the visitors, but the users of the management or the technical infrastructure, so to say. But also those are a diverse group.
12:40
And on the bottom, you see, let's say, an illustration of the diversity of all the equipment we had to manage. The blue line involves quite a bit of open source management tools. I won't go into too much details about those.
13:01
We use GitLab for version control, Metamos for kind of chat ops, YouGo for documentation, Nextcloud for content management, Sensium Prometheus for monitoring, and also we have to use top disks for some reason.
13:21
And of course, in the middle of it all, is Ansible ADWX. It's kind of, basically the design is put Ansible in the center and make it like the lingua franca of our automation. And it was,
13:40
this diagram didn't really change much during the project. It's kind of filling in the boxes. So it was a useful overview, also to explain to others in the organization, okay, this is roughly what we're gonna build. And also the choice for Ansible. We didn't have too much experience with Ansible
14:02
at the start of the project. But we have to, I'm gonna admit something here. We have worked for another museum in Leiden, Museum Boerhafen. And that was a bit of our guinea pig testing ground.
14:21
I'm not sure if they're watching. no, basically they asked us to help out with their new exhibition. But there we got involved really late in the process. So basically there was already a supplier and he had already installed Ubuntu Linux on computers by hand.
14:40
And then we had to deploy the applications. So what we did was just make a simple inventory of all those computers. And with some Ansible playbooks, we could actually manage that part. And I think that's an important feature. Although we didn't have control on the whole situation,
15:01
we could manage quite well that specific thing. And Ansible allowed us to do that. It wasn't an or nothing situation. And the choice for Ansible also was that it was really the most popular config management tool for network automation.
15:22
So the promise of being able to use the same config management tool for basically across the entire spectrum of the museum and maybe even the rest of our infrastructure that was really appealing for us. So I'm now gonna try to give you an impression
15:44
of how far we got with that. So starting with deployment, we have the network switches. We use Cumulus Linux for the switches. So those are the ideas that you have
16:01
white box switches like servers where you can install Linux on the switches. We use a process for deploying Cumulus Linux on those switches based on the only bootloader instead of ZTP, like zero touch provisioning to get Cumulus Linux on those switches. But as you remember,
16:22
we didn't have access to the building. And we also didn't have switches. So we started testing on virtual machines. Because it's just Debian basically, we could just build the entire campus network
16:42
in a virtual environment. For computers, so we have small form factor computers for all our interactive media players. We deploy those with Ubuntu Mask, Metal as a service. So we can, based on Ansible scripts from AWX,
17:01
we can commission and deploy computers from scratch. Config management. So for the switches, we have some base roles that are shared between all the switches. And then we use templating, Ansible templates
17:21
for the spines or the core switches and for the leaf switches. A lot of vendors have their specific Ansible modules. With Cumulus Linux, we could just use templates. The same for our computers. So we have several base roles,
17:40
like for all the interactive computers, museum computers. And then on top of them, we would deploy games basically we all run Unity 3D games because they don't make anything else. MPV for media players
18:01
and the Chromium-based digital signage. But also we do content provisioning. Well, not on the switches, but on the computers. We selected Nextcloud, so we would have a place for our content providers, software providers
18:22
to put the content on. And then with a small script, we have idempotent content updates on the computers. And based on that, we do also some kind of orchestration workflows. For example, that in one workflow,
18:42
we would configure a network port for a specific device in the museum and then actually deploy the computer from scratch, Ubuntu Linux, all the specific role of that computer, the content of it, and at the end of the process, you would have a functioning thing.
19:04
But yeah, we got started with automation, and then we thought, okay, we have also microcontrollers running Arduino. So we selected platform IO to actually deploy the firmware on the microcontrollers as well.
19:22
So we typically would have a computer with an Arduino microcontroller connected to it, mostly USB-based, and then basically as part of the deployment, it would also deploy the specific firmware for that setup. We manage projectors.
19:43
Unfortunately, the suppliers of the projectors don't have like open firmware or something like that on the projectors, but so we didn't implement that. But we did some basics, like the network configuration of those things, sends you checks on it,
20:00
turning as a part of the workflows, turning on and off the projectors when starting an exhibition, stuff like that. And also, for example, we have a KNX gateway to actually stop or start the power supply
20:20
in a exhibition, stuff like that. And we actually did that by mapping KNX data points or addresses to Ansible hosts, so you can use the groupings in Ansible just like you're used to. So as you can see, it's already quite a bit of scope.
20:43
And the nice thing is we use AWX Ansible Tower to actually delegate this whole package to personnel who isn't experienced in all this automation stuff. So for example, turning on the entire museum
21:03
is done from AWX by someone who works for the security every morning. We can also just schedule it, redeploy something entirely from scratch. It's six in the morning before the museum starts. We can do that.
21:22
And this is kind of the end result. So we have, this is one of the exhibitions about geology. We have a Ice Age exhibition with games and microcontrollers,
21:41
projections based on MPV, and plenty more. So to wrap things up, I think in general, I think what we learned at least is that our specific local circumstances
22:01
are really vital for the choices you make, technical and organizational choices we make. For us, a kind of relative simple and versatile tool is really ideal for an organization like us,
22:20
who has to support a really wide range of services. And also Ansible is really suitable for and forgiving for imperfect environments. So you don't have to fit your entire world into kind of the paradigm of the tool.
22:42
Ansible is so forgiving and simple that it can work in an imperfect environment as well. And of course, although we did not succeed using open source tools for every aspect of the museum, I think we came pretty far.
23:02
So to conclude, I think dinosaurs are definitely doomed, because even in these kind of challenging circumstances like tide schedule, a sector that's not really happy to change their ways in proprietary solutions,
23:23
I think our case shows that it's even in those circumstances possible to give the proverbial dinosaurs of your industry the boot. So if you want to know more, we can have a chat.
23:41
There's two more presentations about the museum. Tomorrow, I'll give a lightning talk about our usage of MPV. On Monday, I do a talk on config management camp. I'll go a bit, it's like the sequel of the dinosaurs are doomed one. And I'll go a bit more into detail about our workflows.
24:04
You can come and visit us in the museum, or check out our code. Thank you.