DBA Evolution: the Changing Role of the Database Administrator
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 542 | |
| Author | ||
| Contributors | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/61923 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSDEM 2023412 / 542
2
5
10
14
15
16
22
24
27
29
31
36
43
48
56
63
74
78
83
87
89
95
96
99
104
106
107
117
119
121
122
125
126
128
130
132
134
135
136
141
143
146
148
152
155
157
159
161
165
166
168
170
173
176
180
181
185
191
194
196
197
198
199
206
207
209
210
211
212
216
219
220
227
228
229
231
232
233
236
250
252
256
258
260
263
264
267
271
273
275
276
278
282
286
292
293
298
299
300
302
312
316
321
322
324
339
341
342
343
344
351
352
354
355
356
357
359
369
370
372
373
376
378
379
380
382
383
387
390
394
395
401
405
406
410
411
413
415
416
421
426
430
437
438
440
441
443
444
445
446
448
449
450
451
458
464
468
472
475
476
479
481
493
494
498
499
502
509
513
516
517
520
522
524
525
531
534
535
537
538
541
00:00
Menu (computing)System administratorMoment (mathematics)DatabaseEvoluteComputer animationLecture/Conference
00:34
Time evolutionDatabaseMathematicsTrailDatabasePoint (geometry)EvoluteBitVideo gameComputer animation
01:17
DatabaseMathematicsTime evolutionLevel (video gaming)OracleDemosceneSystem administratorSystem programmingOperations researchSoftwareInclusion mapType theoryTask (computing)Sheaf (mathematics)BitData miningSystem administratorDecimalDatabaseCountingMathematicsGoodness of fitFocus (optics)Software developerType theorySource codeIntegrated development environmentProcess (computing)Different (Kate Ryan album)Fitness functionFilm editingSoftware testingBackupBuildingSelf-organizationInformation securityCASE <Informatik>Web pagePhysical systemData warehouseMultiplication signCartesian coordinate systemSoftwareNeuroinformatikReplication (computing)WebsiteTask (computing)QuicksortVideo gameDegree (graph theory)ComputerCoprocessorPoint cloudDigital photographyProduct (business)Heegaard splittingReal numberDemosceneNumbering schemeEndliche ModelltheorieLecture/ConferenceComputer animation
07:33
Information securityData recoveryDatabaseSoftware maintenanceProcess modelingPlanningChannel capacityInstallation artSoftwareDesign by contractPermanentSystem programmingOperations researchHuman migrationBackupMathematicsPlastikkarteLocal ringMobile appCartesian coordinate systemBookmark (World Wide Web)Information securityType theoryObject-relational mappingSoftwareCuboidDatabaseSoftware developerPlastikkarteTask (computing)Electronic mailing listDependent and independent variablesInstallation artBackupData modelDifferent (Kate Ryan album)Software testingMessage passingSoftware maintenanceReplication (computing)Digital photographyChannel capacityMassFlowchartInformation privacyOperating systemNumberCustomer relationship managementPhysical systemProcess (computing)Structural loadCASE <Informatik>Integrated development environmentPoint cloudRelational databaseFlow separationWeb pageAddress spaceInformationHuman migrationINTEGRALProcedural programmingComputer animation
13:19
Maxima and minimaDistribution (mathematics)OracleMiniDiscContent (media)Computing platformElectronic program guideTape driveLibrary (computing)Physical systemView (database)Procedural programmingElectronic mailing listVirtual machineRelational databaseTape driveElectronic program guideAsynchronous Transfer ModeSystem administratorDatabaseProcess (computing)WebsiteServer (computing)Customer relationship managementTunisSoftwareInstallation artScripting languageGastropod shellData recoveryBackupComputer animation
15:23
DatabaseMathematicsTime evolutionRelational databaseNavigationData typeObject (grammar)Data modelRankingAbelian categoryGraph (mathematics)Series (mathematics)Exterior algebraDeep Blue (chess computer)Machine learningDifferent (Kate Ryan album)DatabaseWater vaporSoftwareData centerPoint cloudComputer chessContinuous integrationPoint (geometry)ConsistencyMiniDiscNP-hardComputing platformRelational databaseOvalPhysical systemCustomer relationship managementMassView (database)Strategy gameInternet der DingeSystem callFigurate numberPlanningPresentation of a groupType theoryZoom lensNormal (geometry)BitService (economics)Data warehouseImplementationDependent and independent variablesOffice suiteInternetworkingHypermediaSoftware bugAnalytic continuationCycle (graph theory)Computer programmingParallel portComputer animation
21:41
Variety (linguistics)Customer relationship managementType theorySelf-organizationPhysical systemRelational databaseData storage deviceDatabaseDifferent (Kate Ryan album)Flow separationCartesian coordinate systemMultiplicationGraph (mathematics)Multiplication signVariety (linguistics)
22:31
MathematicsDatabaseRootSphereCore dumpDigitizingSource codeMultiplication signDatabaseUniform resource locatorData storage deviceCASE <Informatik>Term (mathematics)Computer architectureCore dumpSoftware maintenanceSphereFigurate numberComputer animationDiagram
24:26
Lemma (mathematics)Kolmogorov complexityDistribution (mathematics)Self-organizationDatabaseDependent and independent variablesDiagramComputer architectureConnectivity (graph theory)Cartesian coordinate system
24:59
Integrated development environmentComputing platformMathematicsFlow separationInformation securityInformationSoftware developerBusiness IntelligenceDatabaseCollaborationismSelf-organizationFocus (optics)1 (number)Different (Kate Ryan album)Stack (abstract data type)DatabasePoint cloudIntegrated development environmentMathematicsType theoryWebsiteMultiplicationProcess (computing)SoftwareCloud computingOperating systemCartesian coordinate systemComputer hardwareWordInformation securityExpert systemEndliche ModelltheorieComputer animation
27:35
Broadcast programmingVacuumDatabaseSpacetimeScripting languageBackupExtension (kinesiology)Time evolutionMathematicsDatabaseAnalytic setComputer architectureScripting languageComputer fileBackupVacuumTask (computing)InternetworkingBounded variationComputer animationPanel painting
29:05
NumberFrequencyDesign by contractSoftware developerDatabaseVideoconferencingSystem administratorStatisticsTotal S.A.Electronic mailing listNumberLatent heatAverageHeegaard splittingProcess (computing)TwitterComputer scienceComputer animationDiagram
31:15
NumberMultiplication signDecision theoryDatabaseSimilarity (geometry)Video gamePlanningDialect
32:20
Task (computing)OracleStatement (computer science)DatabasePoint cloudVirtual machineWage labourPatch (Unix)Configuration spaceArchitectureMathematicsTime evolutionVolumeDatabaseQuery languageObject-relational mappingCodeNumberAutomationCASE <Informatik>Point cloudData modelExpert systemDependent and independent variables1 (number)Patch (Unix)Relational databaseTask (computing)Software maintenanceSimilarity (geometry)Figurate numberMathematicsComputer animation
35:40
Slide rulePhysical systemMultiplication signDatabaseOperator (mathematics)Rule of inferenceCausalitySelf-organizationComputer virusPlanningPoint (geometry)Information engineeringSoftware developerMereologyProcess (computing)Wave packetStress (mechanics)BlogQuery languageNeuroinformatikIntegrated development environmentQuicksortWeb-DesignerDemosceneField (computer science)Medical imagingWeb pageCodeComputer animation
43:25
Program flowchart
Transcript: English(auto-generated)
00:05
Thanks very much. I leave you in the capable hands of Karen Jacks. Hi, thanks, Jimmy. Hi, so I'm Karen Jacks and I'm going to talk to you today about DBA Evolution
00:22
or the changing role of the database administrator. So you can see that my title at the moment isn't database administrator but I was a DBA for over 20 years. I'm not going to talk through the agenda. I've just left it there so that you can keep track of where we're up to and when I might stop talking. So I promised that this would be a talk for everyone that's pondered
00:43
life's important questions. What does a DBA actually do? Why are DBAs always so grumpy? How has the role of the DBA changed over the last couple of decades? What's the DBA of the future going to do? And will autonomous databases finally have put us all out of work?
01:02
So this year marks the 25th anniversary of the start of my career as a DBA so I thought that I'd allow myself a bit of a self-indulgent retrospective. So first the evolution of this particular DBA and I'm going to apologize to Claire at this point because this is going to be a bit of a me, me, me section of my talk.
01:22
So I'm just going to have to get through that bit. Being a DBA definitely wasn't an ambition of mine at school. I mean anybody here that say database administrator when they were asked what they wanted to do when they grew up? I mean I didn't even know I was going to go into IT until about a year before I started my career
01:42
although with hindsight it's fairly obvious that that's what I was destined to do. My dad taught me to count in binary before I could even properly count in decimal. He built me my first computer, an Acorn Atom which he later replaced with a BBC Model B and obviously I used the computer for playing Chuckie Egg and other games
02:01
but I also learnt to code in BBC Basic. And I actually looked up the spec of the BBC Model B in case anyone's interested so it had a two megahertz processor and 64k of RAM. I then went on to do a maths degree
02:21
because I still had absolutely no idea what I wanted to do and maths was my favorite subject and after that I figured I probably ought to get some real world skills so I did a Masters in Software Development. Towards the end of that one of the graduate schemes I applied to did some aptitude tests and they came back to say that they thought I'd probably be good as a DBA.
02:43
So once I'd asked Jeeves what one of those was I thought that sounded like a fairly good fit and that I'd give it a go. Fortunately it turns out that Jeeves was right in the aptitude test and this is what my careers looked like so far
03:00
so I really haven't strayed very far from my DBA role. Obviously the things that I do in my job have changed over time because I've gradually moved into more senior roles and I've gradually changed the focus of what I do but I've tried not to let that influence what I talk about too much I'm trying to talk more about the DBA role in general.
03:26
So let's set the scene for the start of my career as a DBA in 1998. The 90s brought us grunge and Britpop they also brought us the first truly portable and affordable mobile phones for example the Nokia 3210
03:42
they brought us the first text message, the Palm Pilot and also for better or worse Amazon and Google. But what does a DBA actually do? Just so that we're all on the same page I've taken highlights of definitions of a DBA from various places
04:01
I looked at careers services, DBMS vendors and job sites and the general consensus is that a DBA manages and secures computer systems that store data using specialist software. So they're all pretty much agreed but that doesn't really tell us anything about what a DBA actually does.
04:26
So first of all one thing to note is that there are lots of different types of DBA and the different types of DBA will be different from organisation to organisation. So at Acme for example we might have the DBA role split into many different DBA roles
04:47
whereas at Wonka Industries we might have DBAs who are in one pool who are all responsible for all DBA tasks. A traditional split is between the production DBA and the development DBA
05:04
so the production DBA will be responsible for keeping all of the databases in the production environment up and running so looking at things like availability, performance, security. The development DBA would work more closely with the developers so they've got a focus on building and maintaining the database environment
05:22
that's supporting the application development life cycle. Just wait for the photos. In some organisations you might see a split of system versus application DBA. So here the application DBA will be more looking at the logical application related aspects of the database
05:46
whereas the system DBA is going to be responsible for the underlying software and the physical infrastructure. And the split is not clear cut. The roles often overlap, they may or they may not and you might see lots of different types of DBA as well.
06:04
So you might have a data warehouse DBA who will look after the data warehouses particularly ETL from various data sources into the organisation's data warehouse. A cloud DBA I think I've got in there as well who will be looking after cloud hosted databases including liaising with the cloud provider.
06:27
Also you might see database architects. I know it doesn't have DBA in the title but it's a very closely related role and often overlaps. It's a role that's existed for at least a couple of decades but it's gradually become more popular
06:42
and seems in a lot of cases to have split out from the traditional DBA role. And it's not an exhaustive list, you could see all sorts of different types of DBA. You might also get DBAs that look after just one particular task such as backup and recovery or replication
07:01
and any or all of those might be split into junior DBA and senior DBA with the junior DBA doing a lot of the day-to-day tasks and the senior DBA having more of a strategic focus.
07:23
So we saw the definition of a DBA role but as I said it didn't really tell us much about what a DBA actually does. So what does a DBA do in 2023? I looked at the different definitions that I showed you earlier and there were lists of responsibilities that went with those
07:41
and I've taken information from different job adverts and tried to pull together the list. So most people seem to agree that a DBA will do some or all of designing, implementing and managing backup and recovery policies.
08:02
Designing and implementing security policies and managing database access although in a lot of places the security role has been split out into a separate security team these days. Implementing monitoring and ongoing monitoring of the databases. Design and development of the database
08:20
including data modelling or maybe just reviewing data models that have been created by the development team. Support and troubleshooting including often being on a 24-7 on-call rotor. And database software install and upgrades listed in a lot of them as a specific task
08:43
but in a lot of places that's automated or the installation side of things is automated so the DBA would either be responsible for implementing that automation or another team would do the automation and the DBA wouldn't actually have anything to do there. Upgrades, I would say it's more of a strategic role
09:02
in working with the application team to make sure that upgrades are tested correctly so that they're successful. And there's a whole other page, sorry, a whole other list of responsibilities so providing database expertise to other teams, performance tuning, enhancement, often troubleshooting,
09:24
capacity planning, creating databases including for development and test environments, database maintenance and data protection and GDPR considerations. So that was a massive, sorry, before I move on, there's a massive long list of responsibilities
09:41
so clearly in most cases one person can't do all of those things. This chart shows some of the most requested skills for a DBA in the UK. I've taken this from IT jobs watch so it's for the six months up to January 2023 so we can see that SQL is the first requested skill
10:05
and I went up to number 23 because there we've got PostgreSQL listed there as well. On the next page I've summarised those different responsibilities, sorry, the different skills that are being asked for.
10:21
So consolidating the list of skills, it boils down to SQL, so writing SQL, making it more efficient, potentially knowing procedural languages, integration and everyone's favourite ORMs.
10:41
One or more database management systems usually still asking for relational. A knowledge of operating systems so this will be traditional operating systems but also various different cloud environments. Performance tuning, database migration seems to be quite a popular thing,
11:02
people are always moving from one database to another. Disaster recovery, social skills is listed most of the places as a separate thing, I don't know what they're trying to say there. HA replication clustering type skills
11:21
and then not listed in any of the things I found as specific skills but most DBAs I know are actually expected to know about different DevOps tools and methodologies and they're expected to know how to use automation tools like Ansible or Puppet to do things. So how does all that compare to what a DBA did in 1998?
11:44
On paper it actually looks very similar. So there I've just reproduced the same list of responsibilities as I had for what a DBA does in 2023. Although I've taken off GDPR because that didn't exist yet. Does that mean nothing's changed?
12:01
Well obviously although the responsibilities are pretty much the same, the skills that a DBA needs and the tools that a DBA is expected to be able to use have changed massively because a lot of the tasks that they do day to day are done very differently and obviously that's not limited to DBA tasks
12:22
or even IT tasks. We do loads of day to day tasks differently as well. So I've used send my sister a birthday card as an example. So that's the flow chart for 1998. I would have gone to the shop. I would have browsed the racks, chosen a card I liked. I would have paid for that card and probably bought a stamp at the same time,
12:42
taken it home, written a message in it, put my sister's address on it, stuck the stamp on and then walked it to my local post box to be collected by the post delivery. I can still use that method in 2023 if I want to but this is what I'm more likely to do. I'm going to open the app of my favourite online retail,
13:02
greetings card retailer. I'm going to upload a photo from my phone, type in a message and my sister's address, pay using my disposable online credit card and just get them to send it directly to my sister as soon as it's printed.
13:20
So if we go back to DBA tasks, I've taken installed database software as an example. I've not put this there to be read. This is an extract from the Oracle 7.3 installation guide which was my first database installation that I did. I can't imagine that many DBAs in 2023
13:41
need to first walk to the machine room and put the CD into the server before they start their installation. It used to take me I think two days to prepare and perform a single database install and we didn't have a repeatable process. There was a silent mode that was supposed to allow us to script it but because none of the servers were installed
14:02
using any kind of automation, they all have their own particularities so it didn't work and we didn't bother with it. In 2023, a DBA would be expected to know how to use an automation tool such as Ansible to do a database software install and they'd probably not just install a standalone database,
14:22
they can install the whole database infrastructure including high availability, monitoring, backup and recovery all in one command. So what kind of skills did a DBA need in 1998?
14:45
I can't reproduce the list of skills quite as easily because that has changed but a lot of the general skills were the same. So you'd be expected to know SQL and probably a procedural language. You'd be expected to know
15:01
one relational database management system. You'd probably be expected to know how to do Unix basic admin and shell scripting, database performance tuning and disaster recovery which was probably calling someone to bring the tapes back on site.
15:24
So that's what a DBA was doing in 98 and is doing today. What will a DBA do in another 25 years time? Well, I'm going to hedge that question for now. So before we start looking at what a DBA might do in 25 years time,
15:42
everyone likes a bit of buzzword bingo so I thought we'd have a look at the buzzwords, trends, hypes from the last few decades just to give us an idea of what's changed around our vintage DBA and what's still changing. So, the 1990s. The 90s brought us the World Wide Web,
16:03
first with painfully slow dial-up access and finally broadband internet which meant that remote work was finally a possibility for some DBAs even if it was just to avoid having to trek into the office when doing 24-7 on call. The concept of big data was introduced,
16:21
data warehouses became popular and we needed massively parallel processing, MPP, to analyse all of this data, so Teradata became popular. Netezza didn't come along until 2000s. Scrum started to creep into our normal everyday work process. Object-oriented programming and from their object-oriented databases
16:42
were the next big thing in programming methodology. Everybody was talking about AI and machine learning even though they weren't new concepts with Deep Blue beating Grandmaster Kasparov at chess. We were blessed with PostgreSQL and Linux. SSDs started to replace spinning hard drives,
17:03
spinning disk hard drives, but that's still a gradual, ongoing move. And a lot of us missed out on New Year's celebrations because we were on call to make sure that planes didn't fall out of the sky and fortunes weren't wiped out due to the Y2K bug. The 2000s, we were introduced to blockchain.
17:25
Cloud was and still is one of the biggest buzzwords with a move away from the traditional data centre. Everything started to be provided as a service. We had software as a service and platform as a service. We were all expected to be agile, to embrace DevOps methodologies
17:41
and to implement continuous integration and continuous deployment. The Internet of Things and social media contributed to a massive growth in data volumes. We got Hadoop, Exadata, NoSQL and JSON as new ways to store, process, analyse and describe data. Base was proposed as an alternative to ACID,
18:03
favouring availability over consistency for distributed systems. Access to data on the move started to be possible with Wi-Fi and containers were introduced, although they weren't at this point quite ready for databases. I'm going to take a deep breath and drink some water.
18:33
Okay, 2010s, infrastructure joined the as-a-service party. NFTs, non-fungible tokens, were introduced.
18:41
We started to store our data in data lakes, which sometimes became data swamps. Columna databases, even though they were actually a concept from the 1970s, became popular and joined the other NoSQL databases. We started to avoid putting all of our eggs in one basket with multi-cloud strategies. Oracle announced their autonomous databases,
19:00
which meant there was not going to be any need for DBAs anymore. Cloud Native was the next big thing with microservices, Docker and Kubernetes. And the EU caused DBAs all across Europe massive headaches by introducing the General Data Protection Regulations, GDPR.
19:23
The 2020s haven't all been a bed of roses in tech so far with a global pandemic and massive layoffs. On the other hand, remote work has started to become the norm for a lot of us. Access to data on the move is getting faster with the introduction of 5G.
19:41
Zoom became a household name. Everyone's now heard of Mastodon. And apparently we're all going to be living in the metaverse and using chat GPT to write all of our reports and presentations. And that's just the first few years. So who knows what's going to have happened by the end of the 2020s.
20:06
We can look at a few ways that database... I'm going to slow down a bit now. We can look at a few ways that databases have changed in response to all of those things that have been going on. This is a massively simplified view
20:21
of the history of database management systems. Please don't quote me on this. So in around 1980, the relational database pretty much... Well, it started to replace the old navigational databases. And it's been going strong since then and still is. In about 2000, various post-relational or NoSQL databases
20:48
became popular and are still popular. And of course, everything goes in cycles. The NoSQL databases were around before the relational databases, just that they weren't called NoSQL databases at that point.
21:06
It's easy to think that relational databases are going away when we hear about all the different NoSQL databases there are and how popular they are. This is a chart from dbengines.com from January this year. And it shows that actually relational databases are more than twice as popular
21:22
as all of the different types of database management system put together. And their figures also show that the popularity of relational database management systems has stayed fairly static over the last 10 years.
21:42
We've got more variety of databases. So in the 90s, there was a handful of database management systems and organizations would usually use one of those for all of their databases or maybe two. There are now over 400 different types of database management system. Most organizations don't restrict themselves to just one or two of those.
22:05
So you'll get not just within one organization multiple database types but also within one application. So that means a DBA might be expected to know not just several relational database management systems
22:21
but also various different NoSQL types. They might be expected to know about spatial databases, time series, graph databases or document store. Databases have got bigger. So the terms big data and very large database were introduced in the 90s but they don't actually tell us very much about the specific volume of data.
22:44
The definition in most places I found a very large database is a database that stores so much data that its maintenance and architecture need specialist tools and methodologies. So it's basically just a database that's so big
23:03
that we haven't yet figured out how to manage it. So in the 90s, most people I spoke to seem to think that a multi-gigabyte database was fairly big. These days, people have multi-terabyte databases and they're seen as fairly normal. Once you start to get multi-tens of terabytes, you might start to need some specialist methodologies.
23:30
In 2018, IDC said that the global data sphere was predicted to grow to 175 zettabytes by 2025. I haven't managed to find any later figures
23:42
so I don't know if that's on course to still be the case. So the global data sphere is apparently all of the data that's created, captured and replicated in core edge and endpoint locations. And one zettabyte is a trillion gigabytes. So for anyone like me who has difficulty visualizing
24:03
175 trillion gigabytes, apparently if you burnt that onto CDs, you'd have a stack of CDs that either took you to the moon 23 times or around the earth 222 times. So it doesn't look likely that there's going to be any shortage of data
24:21
to manage anytime soon. Databases have got more complex. I suspect we've all seen an application architecture diagram that looks approximately like that. We've got multiple database types, not just within one organization
24:42
but also often for a single application. We've got multi-cloud strategies, we've got containerized databases, we've got distributed systems and we've got many interrelated components. So how has the DBA adapted in response to all of these changes?
25:07
The DBA has had to develop a cloud mindset. I was really reluctant to embrace cloud. I can't remember when I first heard the word but I thought it was just a hype that was all going to blow over.
25:22
Whether we like it or not, cloud is here and it's growing so we have to work with it. So with the advent of cloud computing, many DBAs are no longer actually responsible for managing on-premise databases. The hardware and software is often managed by somebody else.
25:43
So the cloud DBA will need to have knowledge of probably multiple different cloud platforms and even organizations that keep their databases on premises are often moving to a cloud-like architecture,
26:01
a cloud-like model with self-service databases. The DBAs had to learn to be flexible so as I've said they would be expected often to know multiple different types of database, not just relational but also NoSQL. They need to be able to work in lots of different database environments
26:21
so the traditional operating systems but also cloud and things like Kubernetes. They're expected to know how to use DevOps and automation tools and methodologies and they've got to keep up to date with all the changes in all of these different types of tool and methodology.
26:44
The other side of the coin is that actually a lot of roles have been separated out into different teams so you might now have different teams that are responsible for infrastructure, for data, security and the application. We already looked at the different types of DBA
27:01
but actually there are a whole host of different roles related to database and the data in it and this is just a few of the ones that I've seen by scanning job sites. All of these changes mean that the DBA has to collaborate with other teams
27:21
so not just with other database experts but also with teams that support the entire application stack. I often say to people that I've been forced out of my little database bubble and the DBAs had to develop a strategic focus so the DBA is more likely to be looking at
27:42
the overall architecture of the database, analytics and data processing, improving automated processes, collaborating with other teams to make sure they've implemented business solutions, rather than looking at the day-to-day technical details of managing the database.
28:07
So as a reminder that change isn't all bad, I asked colleagues and contacts to tell me some of the DBA tasks that they used to have to do that they are glad to be rid of and these are just a few of the answers.
28:20
Vacuum cron jobs, data file reorganisations, commuting to work, that's a controversial one, some people probably do still have to do that. Writing complicated backup scripts and installing post GIS via scripts rather than create extension, so you've probably all got your own tasks
28:41
that you are glad to be rid of. So the million dollar question, a variation of the question is the DBA role obviously is asked constantly all over the internet and of course at database conferences and probably more so since Oracle announced
29:00
its autonomous database in 2017. Is the DBA the clock winder of the future or the lamp lighter, the switchboard operator, the video rental clock or probably more likely the COBOL developer because even if things change so much that modern databases don't require a DBA,
29:21
how many decades is it going to take before all databases are migrated over to this new DBA-less technology? Fortunately, data from the US Bureau of Labour Statistics says that there isn't a shortage of DBA roles.
29:45
So we can see this goes from 2003, so 2003 is the first year in which those stats listed database administrator as a separate role. Before that I think it was computer scientists
30:01
or I can't remember the exact role but there wasn't a specific DBA role. From 2020 it's listed as database administrators and architects so that supports what we saw before about the two roles splitting out from the same thing. So this is the total number of DBA's thousands
30:22
across the left there in the US. So we can see that it's going up slightly. It's not growing massively but it's definitely a slight upward trend and the Bureau actually projects a 9% growth from 2021 to 2031
30:42
which is higher than the average of all other occupations. So that's good. I tried to find similar stats for the UK but unfortunately the lists of job roles are more generic. They don't have database administrator listed. So instead I've gone to IT jobs watch
31:01
and I found the numbers of job adverts that have DBA or database administrator and those numbers are looking pretty healthy for 2022 and 2023. Well how about automation? Surely once everything's automated we don't need DBAs.
31:21
So one of my friends is a pilot and people like to ask him well the plane just flies itself doesn't it? You just put it in autopilot. You don't really need to be there as a pilot. And he says well yeah if everything's going well that's fine. You could just have someone with a lot less experience sitting there watching the dials. But as soon as something goes wrong
31:41
as soon as there's any kind of emergency situation you want someone there that knows what they're doing. Okay it's not usually such a life or death situation if a database has an emergency but you want someone on hand that knows what they're doing to fix that and get things up and running again. So you still need someone there to make the decisions.
32:02
And actually if each database takes less time to manage that means that one DBA can manage more databases. Well we've seen how much more data there is and a similar number of DBAs so that seems to probably work well.
32:20
What about the junior DBAs? I've not been able to find any figures but I wonder whether there are fewer DBA, fewer junior DBA roles. If automation's replacing a lot of the tasks, the day-to-day tasks that junior DBAs used to do do we still have roles for them? And if that's the case how can somebody become a senior DBA
32:43
if they haven't been a junior DBA? Does that mean that the career path's changing? And back to autonomous databases. I mean if we believe the headlines when Oracle announced autonomous databases back in 2017 DBAs were soon going to be a thing of the past.
33:01
There was absolutely no need for them. So even if all databases were moved over to autonomous databases we've got the day-to-day maintenance and patching is automated but you still need somebody to design the databases to do the data modeling to make sure that there's efficient code accessing the database.
33:23
And actually if you look at some of the reviews these are just extracts from a couple of recent ones. Apparently it's an extremely complicated thing to put in place. So you still need somebody who's an expert to be able to put that infrastructure in place in the first place.
33:41
So fortunately the answer to all of that was no, the DBA role isn't obsolete. So what conclusions can we draw from all of this? The DBA role has changed since 1998 and it will continue to change in response to changes in database technology and methodologies.
34:03
The DBA of 2023 has similar responsibilities to the DBA of 1998 but does them in a very different way and so they're expected to be able to do very different things. Relational databases are still going strong.
34:20
They're still the majority and we're not expecting them to go anywhere but DBAs will be expected to know about other databases as well. The amount of data is growing exponentially but being managed by a similar number of DBAs so that suggests that we really do still need those DBAs
34:42
and things are moving either to the cloud or to a cloud-like infrastructure on premises. So the main thing for me is don't panic as we've seen. Quantity of data is growing. A lot of that data is going to be stored in databases
35:02
and that's going to have to be managed somehow. Even if things are automated, you want someone on hand in an emergency. You want somebody to design those databases. You want somebody to write and improve the automation code. You want someone to fix the interesting queries
35:20
that have been created by ORMs. So it's a role that's evolving like any other role in IT, like any other role anywhere else but it's not going away. If you're happy to embrace automation and to try new things, you've got a long and happy career in front of you as a DBA. And even if you're a stick-in-the-mud, old-fashioned DBA
35:42
and you don't want to change, well, there are going to be enough legacy systems to keep you in work for a long time to come so you'll be fine until you retire.
36:05
Hey, I got through that faster than I expected so I've got time for questions. Hello. So just a note from my side. What I discovered is very interesting for me
36:22
is the operator concept in Kubernetes and instead of using a managed PostgreSQL database which is expensive, I just use an operator and the operator itself, it doesn't just install a database automatically for me, it also does piggy-bounds and it does switching from left to right and I think this will be the future.
36:42
So a lot of knowledge we have right now will be moved into operators, more central and it will be much better for us. Yeah, I agree. If you're going to have a containerized environment, operators are definitely the way to go. The problem that I've seen and I guess because of the role I do,
37:01
I see when there are problems is that people think, well, we've got an operator that does everything, we don't need DBAs. So you still need that DBA knowledge behind it definitely. So working in smaller companies, the DBA is basically if you're lucky, the DevOps team and if you're unlucky,
37:22
the developer team which means a lot of people don't have the knowledge or the interest or time. And things are done in a sort of not proper way. So if like somebody who is interested but has a day job as a developer, how can you like foster that environment where developers will start to understand
37:43
if they're making inefficient queries or if they're building the database wrong and get them to both know and be interested and also get the company to kind of care about those things rather than keeping them until it's an emergency like you said and the plane is crashing down?
38:02
That's a really good question, yeah. So I wish I had all of the answers to that because I've been in lots of organizations as a DBA or helping and we've had exactly that situation. So I would say one thing is make friends with, you know, make sure that all teams
38:21
are actually talking to each other. As a DBA, I was always kind of trying to integrate the other teams and actually be useful. I mean obviously sometimes I was that typical grumpy DBA. No, you can't do that. No, you can't have all these permissions. You know, I was enforcing rules but I tried really hard to be there
38:42
and helpful and show that I could actually help to make this query run faster or help to make that better. So I think that's, and I did training courses for developers as well to teach them about the basics of databases. So that's great because I found that most developers are really open to that
39:01
because they want to learn. As far as people higher up in the organization go, I'm not sure. I don't know what the answer is there but probably just showing them how it benefits the organization is the only way to go there. Hello, thank you for your talk.
39:21
I agree completely with your talk. I'm a DBA as well. I have a couple of questions. So first thing is the role of the DBA is still an all male role. So what can we do for making more attractive
39:41
for the woman? Because it's something very stressful. It's something that is very stressful. Another million dollar question. It's something that is not changing yet. It's still all male and I don't think it's a good idea to continue this way.
40:01
I know it's an horrible job but how can we make more attractive? And the other thing is a lot of companies, a lot of my customers struggle to find DBA. How can they make more attractive for getting either permanent DBA? Because I give advice of hiring DBA
40:22
because it's important to having in-house but they struggle to find DBA. Thank you. Yeah, so the first question, I wish I knew the answer to that. I have no idea how to get more women into being DBAs. That's something that's a problem across the whole of the IT community
40:42
and something that's not changed since I started 25 years ago. The only things that I do is I go into part of an organization that does coding workshops for girls to try and get people in early. I mean, I was so lucky because my dad was a self-taught coder. He loved IT.
41:01
Like I say, he built me my first computer and showed me how to play with it. That's what people need. They need to see that it's fun. I mean, well, we think it's fun. I know it's not everybody's idea of fun. I know. My husband still doesn't understand. I say, no, I know, but it's my hobby as well. So I think that's it. And that's probably the same as that first question
41:21
is that we need to make this look like fun to people and maybe we need to try and get rid of that image of the grumpy DBA. I don't know. Exactly. More flashing badges. Hi.
41:41
Hello, thank you very much for the talk. I wanted to ask you, I'm a data engineer myself. Okay, I'm a data engineer myself and I feel that it's complicated to get across what we do every day because we don't do any very visual things. We're not web developers so we can't show really nice pages or nice graphics.
42:01
Do you have any advice to give for somebody who's in the field of databases and behind the scenes to make their work more noticeable? Yeah, that's a very good question. So, I don't know. I mean, I suppose it depends which audience you're looking at.
42:23
Are you looking at kind of internally within an organization or outside to, you know, the general public? Yeah. Yeah, I think some of it is showing the value
42:42
that they bring. The problem with a lot of things, like, you know, I can speak for being a DBA, is that when everything's going right, you're not noticed. Because, you know, well, what's the DBA doing? Nothing's going on. I mean, we all need to start putting little things in the code to, you know, cause problems every so often.
43:02
So, maybe just try and bring attention to the value that you bring. Claire is probably a very good person to talk to later because I don't know if you saw Claire's talk earlier about how to write things in blogs so that you, you know, get your point across clearly
43:20
and that kind of thing, but Claire has some very good ideas on how to communicate.