The Democratization of Databases
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 637 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/52857 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSDEM 2021178 / 637
1
2
7
8
10
11
12
17
29
33
35
38
40
44
48
50
54
59
63
65
85
87
91
95
97
105
108
114
115
119
120
122
126
127
129
130
133
137
140
142
143
147
149
151
156
159
160
161
168
169
170
175
176
177
178
179
182
183
184
187
189
191
193
197
198
204
206
209
212
220
222
224
227
230
233
235
238
242
243
245
247
252
253
255
258
260
261
262
263
264
265
272
273
278
281
282
285
286
287
288
289
294
295
296
302
304
305
308
310
316
320
323
324
328
330
332
335
338
342
343
347
348
349
350
351
360
361
365
368
370
372
374
377
378
380
381
382
383
386
390
392
395
398
402
405
407
408
409
414
419
420
422
425
427
430
439
451
452
453
458
460
461
464
468
470
471
472
473
475
478
485
486
487
491
492
493
495
496
498
509
510
511
512
516
532
534
538
543
548
550
551
554
556
557
559
563
568
570
572
574
575
577
583
585
588
591
593
595
597
601
602
603
604
605
606
607
610
611
617
627
633
634
00:00
DatabaseOpen sourceRelational databaseData structureSoftwareGroup actionSingle-precision floating-point formatLocal GroupAbsolute valueVotingSampling (statistics)Multiplication signDatabasePerturbation theoryData structureGroup actionOpen sourceSlide ruleType theoryQuicksortUniform resource locatorForm (programming)Software developerRight angleSheaf (mathematics)Presentation of a groupMereologySoftwareHybrid computerNumberMedical imagingBitConnected spaceRepresentation (politics)Core dumpAbsolute valueTerm (mathematics)VotingLimit (category theory)Different (Kate Ryan album)Green computingSound effect1 (number)WebsiteMetropolitan area networkShared memoryDecision theoryEnterprise architectureExecution unitState of matterLevel (video gaming)Software testingCivil engineeringComputer animation
06:26
SpacetimeSoftwareRepresentation (politics)Branch (computer science)Classical physicsGame controllerImplementationSoftware developerLevel (video gaming)Decision theorySoftwareSpacetimeIntelligent NetworkData structureRevision controlArithmetic mean1 (number)Spectrum (functional analysis)Shooting methodMultiplication signQuicksortMetropolitan area networkGoodness of fitCollaborationismWeightVotingMereologyProcess (computing)RoboticsType theoryTask (computing)Analytic continuationHill differential equationComputer animation
13:44
MassQuicksortPredictabilitySoftware engineeringState of matterSystem callPhysical systemOffice suiteWave packetType theoryThermal expansionDecision theoryOrder (biology)19 (number)Discrete element methodBit rateBitPower (physics)Group actionDialectPropositional formulaReal numberProjective planeVideo gameMultiplication signWhiteboardWind tunnelNumberMereologyLatent heatRepresentation (politics)Complex (psychology)Closed setComputer animation
20:44
Surface of revolutionDecision tree learningNumberPosition operatorNegative numberTotal S.A.SoftwareQuicksortType theoryVotingRight angleProcess (computing)Letterpress printing19 (number)Service (economics)Multiplication signAuthorizationDemosceneAreaNumberStability theoryComputer iconGreatest elementSurface of revolutionArithmetic progressionArithmetic meanKey (cryptography)40 (number)Term (mathematics)Ideal (ethics)TwitterXINGMathematicsGreen's functionDynamical systemPrice indexFrequencyLecture/ConferenceComputer animation
25:35
SoftwareDecision theoryMatrix (mathematics)outputInsertion lossImplementationDirected setFeedbackOpen sourceInternetworkingSoftware testingSoftware developerInstallation artDataflowPatch (Unix)Beta functionGUI widgetSingle-precision floating-point formatDistribution (mathematics)Endliche ModelltheorieHybrid computerSoftwareBuildingCurveParallel portSlide ruleQuicksortTerm (mathematics)Open sourceFeedbackGame controllerEscape characterProjective planeDirection (geometry)Physical systemType theoryVotingDecision theoryNumberRepresentation (politics)Propositional formulaMatrix (mathematics)Mixed realityoutputImplementationRevision controlMultiplication signNegative numberSoftware developerInsertion lossDistribution (mathematics)Hybrid computerCASE <Informatik>Key (cryptography)Patch (Unix)Interactive televisionIterationLattice (order)InternetworkingShared memoryEmailScaling (geometry)Self-organizationOpen setGraph coloringBridging (networking)Order (biology)Software testingResultantGroup actionMathematicsFigurate numberVapor barrierEndliche ModelltheorieLevel (video gaming)Product (business)Charge carrierArithmetic meanRight angleWebsiteDean numberComputer animation
35:30
SoftwareDecision theoryMatrix (mathematics)FeedbackStaff (military)Open sourceClosed setTerm (mathematics)Line (geometry)Physical systemOpen sourceDecision theorySoftwareProjective planeBit rateSoftware developerStaff (military)Graph (mathematics)FeedbackRelational databaseCurveGastropod shellDemosceneForm (programming)SpeciesRight angleEndliche ModelltheorieComputer animation
37:44
Group actionPlot (narrative)BackupQuery languageScale (map)Parallel portType theoryPoint cloudEnterprise architectureDisintegrationSoftware frameworkComputer wormTwitterWrapper (data mining)Interface (computing)Reading (process)QuicksortInclusion mapExtension (kinesiology)Independence (probability theory)Physical systemGroup actionBit rateBuildingQuicksortOpen sourceSet (mathematics)DatabaseInterface (computing)Natural numberDecision theoryWaveDirection (geometry)Dynamical systemFlow separationMechanism designServer (computing)Point cloudPhysical systemProjective planeSoftware developerExtension (kinesiology)System administratorNegative numberSoftwareIndependence (probability theory)Repository (publishing)WorkloadUsabilityMereologyWrapper (data mining)AreaSmoothingEnterprise architectureWebsiteUniverse (mathematics)Cycle (graph theory)Matrix (mathematics)Data analysisDressing (medical)TwitterVideo gameComputer animationDiagram
42:59
Core dumpTask (computing)Focus (optics)Software developerVideo gameSummierbarkeitMotion blurAreaGroup actionCore dumpNP-hardVotingDatabaseQuicksortBitSoftwareHypermediaEvent horizonDot productTerm (mathematics)Device driverTask (computing)Open sourceDecision theoryPatch (Unix)Field (computer science)MereologySoftware developerSpacetimeCentralizer and normalizerMultiplication signProjective planeRelational databaseMusical ensembleFigurate numberCommitment schemeWeightLevel (video gaming)NeuroinformatikBlogRevision controlNumberFreezingTheory of relativityPlastikkarteCellular automatonRight angleComputer animation
49:04
Element (mathematics)Computer animation
Transcript: English(auto-generated)
00:07
Hello, it is a pleasure to be speaking to you today about the topic of democratization and databases. You may think that there's not a lot in common between those two, but I think by
00:20
the time this talk is over, you will find there actually is. My name is Bruce Momjian, I'm one of the Postgres core team members and an employee of EnterpriseDB for the past 14 years. These slides, along with dozens of other slide decks, are available at this URL here at the bottom, so if you'd like to follow along or look at the slides later, feel free to do that.
00:46
There are also videos of many of my presentations on that website as well. So what are we going to talk about today? We're going to talk about the democratization of databases. We're going to have to lay quite a bit of groundwork, however, before we get to that
01:02
specific connection between the two. We're going to have to talk about governance structures, we'll have to talk about the strength of different governance structures, we'll have to talk about the efficiency and messiness of democracy. It's not all roses, it's not all wonderful sometimes. We'll talk about the ultimate success of democracy, and then we'll start to talk about
01:23
software, software governance, how software is developed, what types of things drive software development in both open source and proprietary. Then we'll talk about Postgres, which I know very well as an example of a sort of democratic developed database, and finally talk about democracy in action.
01:43
So without further delay, let's get started. First, let's talk about governance structures. You know, I didn't realize that democracy actually had a start location. You know, I lived in democracy all my life, so I'm kind of familiar with it, but I just
02:04
thought it kind of happened or, you know, evolved. But in fact, the first democracy actually has a location, which is Pink's Hill in Athens. If you've, I don't know if you've been there, but it's, that's actually what it looks like, and it is where the Athenians first got together and decided to create a
02:25
democracy. I just didn't know that there was actually a place, a location that is associated with the start of democracy. It's kind of interesting. So let's talk about the type of governance that existed before democracy.
02:43
So autocracy is a much older form of government. You know, you think of basically before recorded civilization, most of that is basically some type of autocratic type of structure. There's a single person or a small group in power, and they're effectively calling
03:04
all the shots, and everyone basically has to fall in line. Now, that is not, you know, there's not a whole lot of pure autocracies now, but there are a number of heavily autocratic governments that still exist.
03:22
Syria is pretty clearly a dictatorship. Saudi Arabia is an absolute monarchy, and North Korea is communist, so those are pretty clear autocratic structures. And these are the first governing structures. Again, as I said before, Athens or Greece is sort of the dawn of democracy before
03:44
that you still had only autocracy, and even now you still have some autocracy that still exists. So it's not like democracy showed up and everything just like disappeared. You still have autocracy. And I'm going to talk later, there are actually some advantages to autocracy, and if there
04:05
were no advantages to it, you know, it probably would have disappeared by now. So I'm not saying it's better, certainly, than democracy, but I am saying that it does have advantages, and we even talk about this as we go forward.
04:22
A representative democracy, that was first established in Athens, as I said. Now, historically, up until the past, I guess, two centuries, or even less, actually, this was not the entire population that would be involved in this democracy.
04:46
Typically, up until maybe 100 years ago, I mean, you know, women weren't given the vote in the United States until the early 20th century, you know, certain minorities weren't given the right to vote in certain cultures.
05:03
So, and even in back in Athens, it was only the landowners who could vote, right? It wasn't everybody. So when you talk about democracy, you have to be kind of clear who's part of this democratic structure. Is it just the landowners?
05:21
Is it just the men? Is it just certain, you know, certain races are excluded? And so it's, even with democracy, you don't necessarily have a pure democracy. Everyone's voting. You may have a hybrid where a section of the population votes, and it could be even
05:41
a small section, as it was in Athens, or if it was in everybody. And then, in a representative democracy, you have representatives who vote on the issues. You don't actually have a direct democracy, which we can talk about, but it's not really relevant here. So having talked about autocracy as being the oldest form of government, and then
06:05
democracy from Athens going forward, and some of the limits of democracy in terms of who actually gets to vote in this democracy, we're going to talk about the strengths of each structure. And I've used this image specifically because it's going to foreshadow an image we're
06:22
going to see later. So just keep that in the back of your mind. This is obviously in Wales. So as I said a little earlier, I am going to talk about the strengths of autocracy. I'm not going to kind of show up and say, democracy is always better, autocracy is terrible, and, you know, the lecture's over, right?
06:43
There are some advantages. For example, autocracy is very good at focusing a fixed amount of resources on a clear goal. If you think of things like space exploration or the military, those are typically autocratic
07:03
structures. In fact, in the United States, one of the ironies is that the defense of the United States is sort of relegated to an autocratic structure. Now, of course, that military autocratic structure has to report and has to be approved
07:22
by the government. But you can't really get a much more of an autocratic structure than the military because, of course, you've got ranks and you've got people who have to, you know, go take that hill. You really aren't going to sit and vote on that, right? That's not something you do in the military. Now, there are some military branches that are more collaborative, but again, it's part
07:47
of the process of wartime that you don't have the luxury of doing. The kind of democratic voting that you normally have, it would be certainly very strange. Space exploration, that's a similar thing.
08:01
You know, you put a bunch of engineers together and throw a bunch of resources and, you know, you're going to go to the moon and you're going to vote on stuff. The engineers are going to vote on stuff. But again, it's not what you would think of as democracy. And if you think of, for example, the Soviet Union, which was certainly an autocratic style
08:24
of government, they sort of had strengths in things that had a clear goal and they were able to put a large amount of resources on it, specifically things like space exploration in the military. They did very well in things that required a lot of collaboration and a lot of adjustment
08:45
and a lot of agility, like consumer goods and things like that. Not so much, right? They didn't do very well there. But as you can see, there are certain tasks that actually thrive in autocratic structures,
09:02
even if they're autocratic structures inside of larger democratic structures, like the inside of a larger democratic government is kind of interesting. Now, democracy certainly has strengths. Probably the biggest one is its agility.
09:22
So it has the ability to make rapid adjustments for unclear goals. Because if you think of the way that the Soviet Union or autocratic governments do things, they're typically like, okay, everyone's going to go build the Great Wall.
09:42
Everyone's going to go, we're going to go to space or we're going to create a tank or something like that. But for things that have unclear goals and need to shift a lot, very, very hard to do in autocratic structure because everything's got to be kind of filtered up to the top and then come down again.
10:01
So it sort of gets set and it just kind of stays pretty stagnant as you kind of go through the actual implementation. The idea of adjusting, coming out with version two, continuous improvement. Autocratic governments don't do that very well. Democracies do that very well because, of course, if something doesn't go well,
10:23
something else replaces it. Most of the decision-making is pushed down in democracies to lower levels where either the free market or the democratic, the elected representatives effectively end up adjusting to the current goals that society needs.
10:42
Another great strength of democracy is emboldening talent. One of the classic problems of autocracy is that if you're a talented person but you're not in that autocratic leadership, there's really not a lot you can do. Because all the decisions are being made above you and you really don't have agency to kind
11:06
of take initiative to do stuff. Democracy has a tendency to embolden talent to act near where the problem is. So it effectively enables people to make decisions at much lower levels, at levels much
11:23
closer to the problem than you typically have in an autocratic structure. And that is a tremendous benefit if you've ever worked on a problem where you didn't have control of the situation and you had to wait for everything. That is incredibly inefficient and also tends to cause failure.
11:45
It expands the pool of talent because, as I said before, if you're in autocratic structure and you're not in the top leadership, then you're the listener, but you're just a doer. Democracy, again, has tendencies to spread out decision-making, push it down.
12:06
And some examples of where democracy is good in this sort of emboldening talent and expanding the talent pool is good. Things like consumer goods and software is another one where certainly it requires rapid
12:20
adjustment. There are often unclear goals. Even if version one has a clear goal, version two doesn't. So you're continually having to adjust. And, in fact, some of the newer development methods that we're using instead of waterfall, you know, using Scrum or using some type of more collaborative team-based, peer-based
12:41
development is, again, pushing even farther into that democracy, into that nimbledness away from the autocratic style of software development. So even in the software world, you have sort of a spectrum from autocracy to democracy.
13:01
And it's clear that the autocratic style that software development was done, for example, let's say in the 70s and 80s, you know, way back in time, or even, you know, in the 90s, certainly today, you're seeing much more democratically driven style of development.
13:20
And that leads to exactly the benefits I'm talking about, a better adjustment for unclear goals, emboldens talent near the problem, and expands the pool of talent. So, again, a great example, I think, of where democracy and software intersect. And that idea of democracy really gives a boost to the ability to create quality software
13:41
that meets the goals of today's users. However, democracy is messy. This is actually a picture from Madison, Wisconsin, during a sort of sit-in in the Capitol, where they're basically complaining about something, or protesting about something.
14:05
But it looks pretty messy, and it potentially is. Democracy is able to unleash a flurry of activity based on agency, based on personal power.
14:22
And that is something that is almost impossible to beat from an autocratic system. If you think of autocracies in the world, you can almost lay, you can almost think, picture the country or picture the region that's autocratic, and you can just feel
14:42
the lack of activity or the lack of personal agency or personal power, and how that really limits what that country or region is able to produce. It also is very good at producing a fluidity of solutions. So you have this sort of combined with capitalism, with its continually weeding out
15:03
producers that aren't producing useful solutions and boosting groups that do. Obviously, that's part of the same problem. But it is messy. And this is kind of hard to see, but it's certainly true with democracy.
15:26
It's really hard in democracy to predict behavior, because in an autocratic system, you typically have a small group of people who are controlling things. And if they say, we're going to the moon, or they say, we're going to produce
15:42
tractors tomorrow, you know that's what's going to happen. And typically, autocratic systems are very slow at change. So it's almost too easy to predict what's going to happen. Autocratic systems typically keep producing things even when they aren't needed anymore, because they can't adjust to the new needs.
16:03
But it's hard to predict behavior in a democracy. Problems can get stuck. And I can reel off a bunch of problems, at least in the United States, where we just kind of look at it. And every year, we kind of look at it, whether it's the national debt, or immigration,
16:23
or the patent system, or there's a bunch of these problems, and probably in every democracy, where it's a hard problem, there's not a real clear solution. And everyone's just kind of paralyzed.
16:42
They just can't seem to get the strength or the foresight. They're always thinking it's someone else's problem. And you just can't get around it. Another thing that lays on top of that is the difficulty of large projects that potentially span multiple elections.
17:02
So if you have a problem, and the solution isn't going to be implemented for 12 years, it's very hard for politicians in year one to make that decision, knowing exactly what is going to be acceptable to legislators 12 years in the future.
17:25
And you get a lot of the kick the can down the street. Well, it's not going to be a problem while I'm in office, or in my four years, or two years, or six years. So I'm just going to not worry about it and let Lynette work about it.
17:42
And it's kind of tragic. I'm going to give you a really bizarre example. In the United States, we have a tunnel, a train tunnel that goes between New Jersey and New York, it's called the Hudson Tunnels. And they were built in 1910 by the Pennsylvania Railroad.
18:06
So they were built 110 years ago, two tunnels. And every train that goes in and out of Manhattan has to go through those two tunnels. And those tunnels were built in 1910 by a railroad, not by government, but by a railroad.
18:24
Now, at that time, it was Pennsylvania Railroad was one of the biggest corporations in the world. But still, the tunnels were built by a railroad, two tunnels. The governments of New Jersey and New York have known, and the country overall has known
18:43
for several decades that those tunnels really need to be replaced. They're 110 years old. We had Hurricane Sandy, they ended up getting seawater in the tunnels, and they had to clean them out, but they left salt on the bricks and the mortar, and that's slowly
19:04
either way at the tunnel. So everyone knows that it needs to be replaced, but the governments can't seem, the federal government and the state governments, the two governments, they can't seem to come to an agreement on who's going to pay for the tunnel. And if they decide who's going to pay for the tunnel, they can't decide who's going
19:22
to pay for the cost overruns of the tunnel, because they assume that whatever they bid out, it's going to cost more than that. And they got that close. They were ready to pay for it. They agreed, but no government would pay for the cost overrun, and the tunnels are still being used. And someday, one of them is going to collapse, and then we're going to have one tunnel
19:42
in New Jersey, from New York to New Jersey, and the trains are going to be really messed up. But again, it's a type of problem. It's a big expense. The governments can't decide. They're always hoping somebody else will figure it out, and it's gone away for years. Direct democracy, I talked about a minute ago.
20:02
That's when you're not electing representatives, you're actually voting on specific bills. So the whole community is voting on individual bills. Now, you see a little bit of that in California with Proposition numbers, Proposition 13,
20:23
whatever, where they will take specific ballot initiatives to the population and have them vote on it. But in general, most of the decisions are not made directly by the community, but are made by representatives who are elected to make those decisions. So again, that's another aspect of democracy, another complexity of the issue.
20:46
So I've talked about autocracy. I've talked about democracy. I've talked about the positives and negatives of both of them, and I've tried to be as clear as I could, and I'm sort of honest about what the issues are. But there is an ultimate success of democracy.
21:04
This picture, obviously, being from Moscow, I believe this is a rally for Alexei Navalny, who was the gentleman who's in Germany now, who had the poisoning issue a couple of months ago.
21:22
So I've sort of talked about democracy and autocracy, and I've sort of talked about good and bad and so forth. But you know, democracy has a checkered history, as I've kind of indicated. First democracy in Athens, which is right, we talked about that, but even in Athens,
21:41
democracy was suspended when the country went to war. So a couple years or decades after they had democracy, they suspended it, because they're like, we're at war, we don't have time for democracy now. Roman democracy actually ended in dictatorship by the Caesars.
22:00
So originally, when Rome was set up in the first several centuries, it was a democracy run by patricians or landowners, and they voted and so forth. Again, not everyone voted, but you get the idea, but over time, the leader became more and more powerful, and you had really dictatorship at the end of the Roman period.
22:23
Middle Ages was kind of a mixed bag. You had Magna Carta in England, which was a very key document in terms of democracy. You had the Renaissance, obviously, was sort of a flourishing of democratic ideals. You had the American Revolution, which was sort of all about democracy, at least,
22:44
but black and women didn't have voting rights. So it took quite a while till they got voting rights. And the French Revolution, which is sort of the start internationally, I believe, of where certainly one of the big stakes for democracy eventually ended up in Napoleon and
23:04
monarchy. So I don't know how many of you studied the French Revolution, but it was pretty wacky, pretty crazy. Got out of hand very quickly, started to attack itself, and then monarchy came in and brought stability.
23:21
So hey, it's kind of interesting. Democracy has certainly a checkered history. But democracy today continues to make steady progress, but again, there are still setbacks. So this is a chart from 1916 to 2015, and it illustrates the number of people who are
23:47
living in various types of government. So this is total population is on the y-axis, years are on the x-axis there. The green is the number who live in democracy.
24:02
So as you can see, that number of people living in democracy continues to climb, to climb, to climb, although if you look at the population in autocracy, which is in red, surprisingly that is not shrinking. Number of people living in autocracy feels like it has remained the same at the same
24:23
time democracy has been growing in other countries. Looking at it more in a geographical region area, now of course this doesn't really represent population centers as much, but the bottom chart, 1977, you can see the blue is democracy here, and if you take a look at 2017 at the top there, obviously
24:44
you have a lot more democracy in, or non-authoritarian in that 40-year span there. So again, there is, democracy has setbacks, but there certainly is a steady trend toward
25:02
democratic institutions. For the reasons that I mentioned, for the benefits that I mentioned, society continues to get more, require greater agility. There's more changes going on, I think. We all feel that as the years go forward, and democracy certainly is much, much better
25:24
suited to a dynamic economy than autocracy ever will, and certainly software, I think the world of software benefits much more from a democratic style than autocracy. So, let's specifically get back into this. I mentioned I was going to reference another slide that looks similar to Wales, and this
25:43
is the one, this is Oracle's former headquarters, I believe, because they're moving to Texas, or will be moving to Texas soon, but again, sort of that curved building sort of turret thing, I thought it was too good to pass up as a parallel. So, let's look at software in terms of autocracy.
26:06
So, in the software world, there is autocratic styles of development, and particularly in proprietary development, it effectively is an autocratic system.
26:24
Now, there are some democratic aspects of it. There is some sense that customers have a control or some feedback to that autocratic system, but ultimately, in most of the proprietary development systems, they are autocratic.
26:43
So, what do I mean by that? So, typically, in proprietary software, executives make the decisions. Now, they have input from sales, and they've input from marketing, and indirectly, that input is coming from customers, but effectively, the executives are making the final decision on what to do with the software, how long to support it, what features to add, and
27:05
so forth, and the decision matrix actually is quite interesting. When somebody has to decide what to do with a piece of software, whether they should do something or not do something, add something or not add something, the questions really
27:21
become if we do this, what percentage of customers will we gain, and if we don't do this, what percentage of customers will we lose? That's pretty much the summary of how they make the decisions. It's not so much what is going to serve our customers necessarily.
27:40
It's typically, what will this do to our company as a funnel of income, as a funnel of revenue? Those are typically the way they are going to make these decisions. If you're curious more about this, if you go to my website, I have a slide deck called
28:04
Will Postgres Live Forever, and it goes into more detail about how this matrix works, but that's pretty much it. One interesting aspect about this is that gains are more sensitive to decision makers than losses, so something that will gain customers is usually considered to be more higher priority
28:26
than something that will lose customers, because the assumption is that your customers aren't going to disappear unless you do something really bad in the short term. Long term, that's different, but again, a lot of the executives are making decisions
28:44
based on quarters, not necessarily on the long term health of the organization. And again, the big question, do gains or avoided losses justify the implementation costs? And this is basically the matrix that goes in to decide what happens to that software
29:01
every quarter. Open source democracy has a different way of driving it. It's a mix of direct democracy and technically meritocracy, because it is really not necessarily every user gets the same vote, because the users aren't writing the software, right?
29:25
It's usually the people who are actually writing it and have been with an open source project for a long time typically have a little, certainly have a more weighted vote, although again, that can change.
29:40
You can have users demanding something and the community has to accept it. You have the ability of people to fork the software and create new versions that meet requirements if the leadership is currently not doing that, and that has happened to a number of software projects over the years, so it's not an idle thing. And I think that keeps things healthy.
30:02
I wouldn't necessarily want to have direct democracy, just as I think the California proposition number idea where you put things on the ballot works for certain decisions that have to be made by the population. But some decisions are really hard and really complex and very hard to explain and have
30:23
a whole population studying the intricacies of some of the stuff you're doing, so they almost have to defer to a meritocracy or some type of representative governance system to basically move that software forward. But of course, the escape hatch is that other people can create forked projects if
30:45
that leadership is not being responsive, and that is an absolute key aspect. Voting, even if you had voting in open source, it's kind of hard to figure out who can vote, like is it somebody on the email list, is it somebody who uses the software,
31:01
how do you know they use the software, it's so amorphous who a user is because they haven't paid anything, you just downloaded it, what is it, is it the whole world that votes, what do you do? Sometimes you get too much feedback from these democratic systems, you get this bike
31:22
shedding problem where everyone's complaining, discussing what color to make the bike shed, but nobody's talking about how to build it. So that can be an issue with direct democracy. So again, it's kind of a sliding scale exactly how democratic you want to be.
31:41
One nice thing about software is that bad decisions can be quickly corrected. It's easy to revert something. If you build a bridge and you build it in the wrong place, you're not going to pick it up and move it. With software, if you make a mistake, you can correct it fairly easily and hopefully it hasn't caused too much problem. One problem with open source democracy, and I've had to explain this to a number of
32:02
people, there is no roadmap. There's no roadmap of exactly where the project's going, exactly what we're going to produce. Some projects do have a roadmap, but many don't. Postgres particularly does not. So again, depending on how democratic you are, you may not have a roadmap similar
32:21
to the problem we talked about democracy, where it's kind of hard to get the whole agreement on a multi-year project because you can't really sometimes promise that all the governments are going to agree for that. Internet makes the democracy possible for us.
32:40
Certainly the ability to share ideas and iterate new solutions rapidly at virtually no cost is really what makes that democracy work. If we didn't have that, it would be really hard to do what we're doing as open source projects. Just to give you a visual of exactly how autocratic development works, this is autocratic
33:06
development. You have your developers on the left and the company. You have users on the right, and there's very little interaction between the two. You have releases, which happen down here. The releases get fed to the users, and then they have to report a problem.
33:23
It has to go back up here. It has to go through meetings again and then get released again. This is why it takes a year or two years to get something fixed often in proprietary software. Open source development is a case where users can be involved at any stage. They can discuss features.
33:41
They can review patches. They can test. They can debate testing. They can do production. All of this stuff happens simultaneously. Users and developers don't have a big barrier between them. And this is really the key of why Postgres and open source projects work so well. Linux and all the open source stuff is of such high quality and rapid development.
34:02
And it's because they're not doing this, which is incredibly slow, they're doing this. There is a hybrid model, which I hesitate to discuss, but I get a lot of questions about it. We've talked about proprietary. We've talked about open source development, but there is that hybrid in the middle.
34:23
A hybrid in the middle is where the software is open source, but the development is autocratic. So you've got this open source software that's distributed, but the development of the software is autocratic and you get all the negatives of the autocratic system. One company controls it.
34:42
The decision matrix is autocratic because all the developers are hired by one company. The democratic feedback really doesn't work unless you're a paying customer, for example. And it's a little more comfortable for some users because transitioning from proprietary to open source is maybe too much.
35:03
And going to a single company with one contact maybe is easy. Mongo and MySQL, MariaDB, all I think are in this kind of hybrid camp where you've got effectively a limited control of the development by a small group, but you've
35:21
got this open source distribution. Again, you get some of the advantages of democracy, but you also get a lot of the disadvantages of autocracy. So where does this leave Postgres? It's kind of a funny picture. We're not in COVID, but all these people sitting together like, wow, look at that.
35:42
So Postgres wins because democracy wins. Postgres uses a democracy to attract talent. We can effectively compete against proprietary staff. Just a stupid question. You know, how does an open source project with zero money compete against billion dollar
36:01
companies that make relational databases like IBM or Microsoft or Oracle? They do it because the democratic system of software development is just so efficient that even though they have billions of dollars to spend on their software, they really can't produce software of the same quality and the same advancing feature set that Postgres
36:24
and other open source projects have. We have a superior feedback and decision matrix, which leads to success. There still can be setbacks, right? Niche software is very hard to do in open source. You may not be able to get enough of an open source community around it. Sometimes that has to be done in a proprietary manner.
36:41
So again, you're still got autocratic around, right? You still have autocratic governments in the 21st century. You're still going to have proprietary software in the 21st century, right? It's just a question of how much and exactly which pieces are going to be in which governance
37:01
models, basically. When I started with Postgres in 96, we were kind of down here in terms of features, performance and reliability compared to closed source software. But we have a steeper curve of adoption.
37:20
We have a steeper curve. That red line is steeper than the blue one. So eventually I knew we were going to cross and we probably crossed a bunch of years ago. But again, it's kind of interesting. You may start very low as an open source project, but if your graph is steeper, your adoption rate is steeper, eventually you will cross.
37:41
People who are more popular than you. Democracy in action. This is actually, I believe, a picture from FOSTM. I think it was three years ago. I think two or three years. I think it might have been three years ago.
38:01
But again, you might recognize the building in the back. So it's kind of funny. Oh yeah, we got a FOSTM shirt on the guy right there in red. That's definitely FOSTM. I even remember, I think, where this was taken. But these are just two of our open source guys. And of course, Slonik, our elephant mascot in the middle. So why is Postgres sort of continued to sort of ride this popularity wave of democracy?
38:25
First, it was designed to be extendable. That extendability has allowed for side projects and extensions to Postgres for a whole bunch of workloads that weren't envisioned when Postgres was designed. Postgres was designed in 1986 at the University of California, Berkeley.
38:47
And fortunately, because it had an extendable design, we've been able to kind of add a whole bunch of 21st century requirements to the database in a very smooth manner. It's very hard if your database wasn't designed to be extendable to add extendability.
39:02
But because extendability came with it, it was actually really easy to add a whole bunch of capabilities to it that it didn't originally have. Another interesting aspect of Postgres is because it has a distributed nature of development, you don't have like a single direction.
39:20
So typically in proprietary software, people say, all right, this is going to be our cloud release. This is going to be our performance release. This is going to be our admin release. This is going to be our developer release. And they spend two years on that. And then they moved into something else. And they moved to something else. With Postgres, and we have major releases every year, we're improving Postgres, ease
39:42
of use, enterprise, new workloads, developers, cloud, data analytics, new workloads, all at the same time. Every major release has improvements in all these areas in one release. And that's because we have some people in Russia working on stuff, people in Germany,
40:01
people in Japan, people in the United States, people in Latin America, and people in Asia. China is sort of up and coming. So again, you've got a lot of things all moving. India, you've got Pakistan, you've got all these things kind of moving all over the place. And because it's distributed, you're getting the benefits of all this dynamism every year
40:25
that you don't have in an autocratic system where everyone says, let's go in this one direction for the next two years. One of the things we've added recently, part of our extendability is foreign data wrappers, the ability to have Postgres connect to other data sources like Oracle
40:45
or Mongo or Twitter, and effectively pull data or push data to those other sources. This is, again, an example of the flexibility that has sort of evolved through Postgres's lifecycle. And because of our democratic development, we're always kind of adding these new kind
41:04
of capabilities. Foreign data wrappers, we have over 100 different interfaces you can use. Most proprietary software don't want you to communicate to another data set because you may migrate to that set. You may get rid of them. We don't care. As an open source project, we're all about interoperability.
41:22
So when you think of that decision matrix, there's no negatives to us adding more data wrappers and making us more interoperable. So again, that's a great example, I think, of the interoperability and flexibility of the database. We have a PGXN, which is a repository for extensions for Postgres, all open source,
41:45
developed by the community, and again, a great opportunity to sort of showcase what other people are doing in that larger and larger and larger set of umbrellas around Postgres
42:01
that support what we do. PostGIS is a great example of an external project that is not really officially associated with the Postgres primary database server community. They developed just a GIS engine, which sits on top of Postgres. And because it uses the extension mechanisms of Postgres, it operates just like a natural
42:26
data type, where as though when you install PostGIS, it appears to be something that was always in the database, even though you just installed it. So it's developed by a separate team, a separate community. They have their own conferences.
42:42
But again, really, really popular. I've been to a couple of the conferences. They are really interesting, and some of the problems they were caught are really interesting. But again, a great example of a separate open source community working with the database community to create something that's bigger than both independently.
43:01
This is a great picture. I just kind of love it. This is our first big Postgres conference in 2006. In Toronto. And the reason I find this exciting is that it was 14 years ago. But when I look at these faces here, and I'm right over here, right here, when I look
43:25
at these faces, probably 85%, 90% of these people are still with the community. So that's a great example, I think, of longevity of the community, kind of showing them how useful it is to be part of this project.
43:42
And people really, they think their time spent in Postgres is a good use of their time. That's something that's helping them become better engineers and helping other people who need an open source database to do cool stuff. Obviously, we have great conferences. We have a lot of events.
44:01
So again, this is the team that, part of the team, certainly, that makes that possible. But again, the idea that they've been around for so long, really speaks to a dedication of what we're doing as an open source project, as a democracy, that's sort of moving the database field forward.
44:21
I travel quite a bit. These are all dots of where I've been. I didn't travel so much. I didn't travel much in 2020 last year at all. But I fortunately got about 20 travel days before it was shut down. But I still was able to do about 40 events. And again, I do about 40 events a year and travel about 90 days when there's no pandemic
44:46
around, you know, I get around and you can see the vast area that we've been able to cover with Postgres activities. One of the things I talked about earlier was voting. And voting is kind of hard in open source because it's hard to figure out who gets
45:04
to vote, who's sort of a member of the community. The way we've sort of set it up now is that we have a committers committee, which nominates new committers. So there's about 29, I think, committers. And they would nominate additional people to add to that group as necessary, or as
45:25
people become proficient enough to be committers. The core team, same policy, the core team nominates new core team members. We just added two core team members this last fall. So I think we're up to seven now, kind of cool.
45:40
Development is fortunately open to all. So even though we have committers and we have a core team, they're really more, those two groups are really more operational in terms of committing the patches and making decisions related to sort of project policies and so forth.
46:05
But development's open to all, pretty much anyone who shows up has pretty much the same voice as anyone else. Even occasional visitors will show up and they'll say, you know, something like, wow, that's a great idea, right? Like, let the best idea win. Sometimes I'm personally thinking, where did that person come from?
46:23
I've never heard of them before. And they did a great, they had a great idea on how to make, how to do this. So to me that the way I phrase it is that Postgres is great at focusing talent, like a lens on every task. And we just continue to do that.
46:41
And I really think that is the crux of what makes us successful. We don't really have a roadmap, like individuals and political parties have roadmaps, but democratic governments don't. So again, one of the problems is that because we don't have a central person or small
47:00
group that makes decisions for others, we are kind of at the mercy of whatever people want to work on. So that is one thing that is difficult for people to understand sometimes. Particularly companies who are used to proprietary development, they're kind of like, where's the roadmap? And I'm like, well, we have a to-do list, but that's not really the roadmap.
47:20
Like, how do you know what you're going to get this year? I'm like, we don't, we don't until it's done. So it's worked with us well. We've had really high quality releases for the past, what, 24 years. But that is something that people have to get used to, the lack of roadmap. If you're curious about the community, there is a website called PG Life right down
47:40
here, and it will show you the events, the IRC channel, the blog, the most recent commit, the most recent developer email, the least recent user email, the most recent announcement, so forth, the most recent mention in media. So again, if you're curious about what's going on, feel free to look at that.
48:03
And that does conclude my talk. I promised to show you how democracy and databases kind of have a commonality. And I think I've been able to explain that. It sort of opened my eyes in terms of understanding why is open source successful?
48:22
Why is Postgres successful? What is it that we do that makes us successful? And how do we do more of it? And how do we make sure we continue doing what is making us successful? And I think for me, this talk has sort of at least helped me to concretely see why is open source so much of a driver for innovation?
48:42
Why is it able to produce software better than billion dollar companies? And why will Postgres particularly be a database that will continue to be predominant in the relational database space for many years to come? Thank you for listening to my talk today.
49:01
And I hope you have a good rest of the conference.
Recommendations
Series of 14 media